# Relay spawn tree (v3) The tree is declared at init and never grows. Every thread is its own domain; the only shared state between domains is a channel. There are two node kinds: a **manager** owns children and is the interchange between them, a **worker** does one job and talks only to its manager. ## Tree ``` root router + supervisor, no work beyond the gate ├── policy-manager write ACL/allowlist, mute, blacklist, spam, free-write limiter ├── sig-manager fan-out point for signature verification │ └── sig-worker[0..M] pure compute, one signature per request ├── network-manager the relay proper: listeners, connection pool, live registry │ ├── listener-worker[0..L] owns the listening socket AND the accepted sockets │ └── connection-worker[0..N] multiplexer: per-connection protocol state, no sockets └── database-engine single writer and reader, owns time and truth ``` `L` = one per bind address. `N` = fixed connection-worker pool, connections assigned by random selection. `M` = fixed signature pool, sized to saturate the cores. Nothing is spawned at runtime in response to load; a crashed child is respawned by its manager and its children come back with it. ## Who owns what - **listener-worker** owns the socket, including every accepted socket and its poll registration. A *connection* is identified by remote IP:port. The listener reads wire frames, sends them up with their connection, and writes frames the manager hands it. It does no protocol work. - **connection-worker** owns per-connection protocol state: frame parse and serialization, idempotence, per-connection limits. It holds no socket, no filter and no routing table. - **network-manager** is the single authoritative subscription registry (`conn_id -> {sub_id -> filter}`) and the only node that matches events against filters. It knows which listener and which connection-worker each connection belongs to and forwards between them: wire frames up from the listener to the owning worker, delivery down from the worker to the listener. It holds no sockets and does no crypto or disk I/O. - **root** routes and supervises. Inbound event: `Verify` to sig-manager first; on `VerifyResult{valid}` consult policy-manager, then `Persist` to database-engine and `Dispatch` to network-manager. It holds no durable state beyond the in-flight verification set. - **database-engine** is the only node that touches disk, assigns the monotonic `seq`, and decides replaceable / parameterized-replaceable / deletion semantics atomically with the write. It owns its own clock and does periodic work (checkpoint, flush, compaction) on that clock, not on a command. ## Backpressure A full channel stalls the sender until it clears: the buffer *is* the backpressure. There is no drop-oldest and no unbounded queue. If the peer stays unresponsive past the send retry window, the send fails; the manager treats that as a dead child, respawns it, and everything in flight is lost. Delivery is only ever attempted to a listener that is ready to write. ## Sequence and the history/live boundary `seq` is the database's monotonic record, assigned to every stored event, and is the seed of a Lamport-style clock for later sync protocols. `Dispatch` carries it (0 = ephemeral, not stored). A REQ registers its filter with `seq_floor = seq_high_water + 1`; stored events at or below that floor are already accounted for by the history result and must not be delivered twice. Ephemeral events are not stored, carry no seq, and are not subject to the floor. Persisted events are dispatched after `PersistAck`, so a client that queries immediately after publishing sees its own event. ## Flows Ingest: ``` listener -> network-manager -> connection-worker -> network-manager -> root root -> sig-manager -> sig-worker -> sig-manager -> root root -> policy-manager -> root root -> database-engine (Persist) root -> network-manager (Dispatch) network-manager -> connection-worker (match/format) -> listener (write) ``` Subscribe: ``` listener -> network-manager (register filter, seq floor) -> root -> database-engine database-engine -> root -> network-manager -> connection-worker -> listener ``` ## Invariants 1. No event is persisted or dispatched without a signature pass and a policy pass. 2. Only database-engine writes to disk. 3. Only network-manager owns live filter state. 4. Every domain has exactly one parent; crashes propagate down. 5. All children are spawned at init; the topology is static. 6. Connections are assigned to the worker pool by random selection. 7. One listener-worker per bind address. 8. Ephemeral events bypass persistence but not dispatch. 9. Storage is idempotent: the same event id is a no-op. ## Migration map (today -> v3) | v3 node | today | | --- | --- | | root | server domain poll loop (`pkg/relay/server/server.mx`), which also parses, ACLs and ingests | | policy-manager | inline in `pkg/relay/pipeline` and `pkg/relay/access` | | sig-manager / sig-worker | none natively; `pipeline.StageA` runs inline; the browser has a verify pool in `web/static/wasm-host.mjs` | | network-manager | `pkg/transport` + `Server.conns` + per-connection `c.subs` | | listener-worker | none; transport accepts inside the same domain | | connection-worker | none; one domain serves every connection | | database-engine | `pkg/store` engine owned by the server domain | | to delete | `wire.IngestWorker` (a second engine copy in a spawned domain), `wire.ProxyWorker`, `wire.BlossomWorker`, and the broadcast worker's ad-hoc fan-out | The browser app mirrors the same nodes: page = root, relay-proxy worker = network-manager, its client workers = connection-workers, the store worker = database-engine, the verify pool = sig-manager/sig-worker.