Features¶

haystack-py provides a complete implementation of the Project Haystack protocol for both client and server use cases. This page summarises the library’s capabilities by category.

Transports¶

Two transport layers are supported, both fully asynchronous:

HTTP — aiohttp based with persistent connections, connection pooling, timeout configuration, and content-negotiated responses.
WebSocket (experimental) — websockets sans-I/O protocol with per-message deflate compression, batch request pipelining, and watch push delivery. The WebSocket API is subject to breaking changes in future releases.

Wire Formats¶

All four Haystack wire formats are implemented:

JSON — v3 and v4 encoding via orjson, with optional pythonic decode mode that converts values to native Python types. JsonVersion enum selects the format.
Zinc — The standard Haystack text grid format. encode_grid() / decode_grid() for full grids, plus scalar helpers.
Trio — Line-oriented tagged record format. parse_trio() / encode_trio().
CSV — Lossy comma-separated export (encode-only, per spec).

See Wire Formats for usage examples and format details.

Client¶

Client provides an async HTTP client with all 13 standard Haystack operations:

about — Server information
ops — Supported operations listing
formats — Content type negotiation
read — Filter-based and ID-based entity reads
nav — Site/equip/point tree navigation
hisRead / hisWrite — Time-series history access
pointWrite — Priority array writes for writable points
watchSub / watchUnsub / watchPoll — Real-time change subscriptions
invokeAction — Server-defined actions

WebSocketClient provides the same operations over a persistent WebSocket connection, plus:

Batch requests — Multiple ops in a single round-trip
Push delivery — Server-initiated watch updates
Auto-reconnection — ReconnectingWebSocketClient with exponential backoff
Connection pooling — WebSocketPool multiplexed channels over a single WebSocket

See HTTP Client and WebSocket Transport.

Server¶

A FastAPI-based server framework with content-negotiated routes, SCRAM middleware, and a WebSocket endpoint:

create_app() factory returns a ready-to-serve FastAPI application
HaystackOps dispatches all 13 ops to a pluggable StorageAdapter
WebSocketServer standalone WebSocket server with SCRAM handshake and batch dispatch

See HTTP Server.

Storage Backends¶

A StorageAdapter protocol decouples ops from data storage. A companion UserStore protocol handles user persistence. Three implementations ship with the library, each implementing both protocols:

Memory — MemoryAdapter for testing and prototyping
Redis — RediSearch full-text indexes + RedisTimeSeries for history
TimescaleDB — PostgreSQL JSONB entities, hypertable time-series, filter AST → SQL pushdown via asyncpg

See Storage Backends.

Authentication¶

SCRAM-SHA-256 — Full client and server implementation over both HTTP and WebSocket. Constant-time key derivation, RFC 5802 compliant.
PLAINTEXT — Fallback mode for development/testing.
Token-based — Bearer token authentication for WebSocket connections.
mTLS — CertAuthenticator validates client certificates for mutual TLS authentication.
Storage-backed — StorageAuthenticator reads SCRAM credentials from any UserStore backend. Disabled users are automatically denied.
Certificate generation — generate_test_certs() creates a CA, server, and client certificate chain for development.

See Authentication, Users & Permissions and TLS and mTLS.

User Management & Permissions¶

User — Frozen user model with SCRAM credentials (passwords never stored in plaintext).
Role — ADMIN, OPERATOR, VIEWER enum with strict ordering for permission checks.
UserStore — Protocol for user CRUD (get, list, create, update, delete) implemented by all three storage backends.
REST API — Admin-only CRUD endpoints at /api/users/ for creating, listing, updating, and deleting users.
Role enforcement on all Haystack ops: write ops require Operator+, user management requires Admin. Enforced on both HTTP and WebSocket.
Admin bootstrap: seeds an admin user from environment variables (HS_SUPERUSER_USERNAME / HS_SUPERUSER_PASSWORD) on first startup.

See Authentication, Users & Permissions.

TLS¶

TLS 1.3 enforced on all secure connections
TLSConfig frozen dataclass for certificate paths
build_client_ssl_context() / build_server_ssl_context() for SSL context construction
Peer certificate inspection: extract_peer_cn(), extract_peer_sans()

Data Model¶

All Haystack value types are implemented as frozen dataclasses:

Marker, Na, Remove — Singleton types
Number — Numeric value with optional unit
Ref — Entity reference with optional display name
Symbol — Def symbol
Uri — URI value
Coord — Geographic coordinate (lat/lng)
XStr — Typed string value

Grid is the universal immutable message format. GridBuilder provides fluent construction.

See Data Types and Grids.

Filters¶

Recursive descent parser with LRU-cached AST generation
Full filter syntax: has, missing, comparison operators, and/or, path traversal
Evaluation against dicts and grids
SQL pushdown for RediSearch and PostgreSQL JSONB

See Filter Expressions.