# iskra-server wire protocol

Binary request-response protocol over TCP or Unix domain sockets.

## Transport

- TCP default: `:4847`
- Unix default: `/tmp/iskra.sock`
- No TLS, no HTTP, no WebSocket
- Strictly synchronous: one request, one response, per connection at a time
- Connection lifetime unbounded; client closes when done

## Frame format

Every message (request or response) is length-prefixed:

```
[4 bytes] length (uint32 LE) - byte count of everything that follows
[N bytes] payload
```

Maximum frame size: 4 MiB.

## Request payload

```
[1 byte]  op code
[N bytes] op-specific fields
```

## Response payload

```
[1 byte]  status
[N bytes] op-specific fields
```

Status codes:

| Value | Name | Meaning |
|-------|------|---------|
| 0x00 | OK | Success, payload contains result |
| 0x01 | NOT_FOUND | Lookup returned no match, payload empty |
| 0x02 | ERROR | Server error, payload is UTF-8 error message |

## Encoding primitives

| Type | Encoding |
|------|----------|
| uint8 | 1 byte |
| uint16 | 2 bytes LE |
| uint32 | 4 bytes LE |
| uint64 | 8 bytes LE |
| string | uint16 length prefix + UTF-8 bytes |
| Key | 16 bytes (two uint64 LE) |

All multi-byte integers are little-endian.

---

## Operations

### 0x01 - BigramWeight

Exact bigram lookup at a specific domain and coord.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Domain byte (0x01=EN, 0x02=JA, 0x10=MoxieSRC, ...) |
| coord | uint64 | 64-bit packed coordinate |
| prev | string | Previous token |
| next | string | Next token |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| weight | uint32 | Observation count (0 = never observed) |

---

### 0x02 - BigramWeightRelaxed

Bigram lookup with full coord relaxation cascade. Strips coord axes one at a time (pragmatic -> register -> valency -> semantic -> grammatical -> cooccur -> morph) until a non-zero weight is found.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Domain byte |
| coord | uint64 | Starting coord (most specific) |
| prev | string | Previous token |
| next | string | Next token |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| weight | uint32 | First non-zero weight found |
| matched_coord | uint64 | Coord level that matched (0 = base/dictionary form) |

---

### 0x03 - IngestBigram

Record one observation of a (prev -> next) transition. Increments count if the bigram exists, inserts with count=1 if new.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Domain byte |
| coord | uint64 | Context coord for this observation |
| prev | string | Previous token |
| next | string | Next token |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| recIdx | uint32 | Record index of the bigram entry |

---

### 0x04 - IngestText

Bulk ingest a token sequence. Records bigrams for all adjacent pairs (tokens[0]->tokens[1], tokens[1]->tokens[2], ...).

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Domain byte |
| coord | uint64 | Base context coord for the sequence |
| count | uint16 | Number of tokens |
| tokens | string[] | Token sequence (count entries) |

**Response (OK):**

Empty payload.

---

### 0x05 - WalkStep

Compute candidate next positions from a current walk state. Returns continuations ordered by descending bigram weight.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Domain byte |
| coord | uint64 | Current context coord |
| word | string | Current word (prev token for bigram lookup) |
| maxCandidates | uint16 | Maximum candidates to return |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| count | uint16 | Number of candidates |
| candidates[] | variable | Repeated candidate entries |

Each candidate entry:

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Candidate domain |
| coord | uint64 | Matched coord |
| word | string | Next word |
| score | uint32 | Accumulated walk score |
| depth | uint8 | Steps taken |
| weight | uint32 | Bigram weight for this transition |

---

### 0x06 - Translate

Cross-domain translation via sub-lattice composition. Finds the best match for a word in dstDomain using coord-relaxation and cross-domain links.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| srcDomain | uint8 | Source domain byte |
| dstDomain | uint8 | Destination domain byte |
| coord | uint64 | Semantic/morph context in source domain |
| word | string | Word to translate |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| result | string | Translated word in destination domain |

Returns NOT_FOUND (0x01) if no translation exists at any relaxation level.

---

### 0x07 - Lookup

Raw record lookup by branch and key.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| branch | uint8 | Branch index (0-7) |
| key | Key | 128-bit lattice key |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| found | uint8 | 1 if found, 0 if not |
| form | string | Surface form of the record |
| count | uint32 | MetaEntry.Count |

---

### 0x08 - Insert

Insert a new record into the lattice.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| branch | uint8 | Branch index (0-7) |
| key | Key | 128-bit lattice key |
| form | string | Surface form to store |
| kind | uint16 | NodeKind bitfield |
| stage | uint8 | Stage tag |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| recIdx | uint32 | Record index of the new entry |

---

### 0x09 - BeamRun

Execute a full beam search walk. Starts from the given word and walks maxSteps iterations with the specified beam width and walk kind.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Domain byte |
| coord | uint64 | Starting context coord |
| word | string | Starting word |
| width | uint16 | Beam width (number of parallel hypotheses) |
| maxSteps | uint16 | Maximum walk iterations |
| kind | uint8 | Walk kind: 0=translate, 1=elaborate, 2=narrow |

Walk kinds:
- **translate (0)**: beam width stays constant each step
- **elaborate (1)**: beam width grows by 1 each step (up to 4x initial width)
- **narrow (2)**: beam width shrinks by 1 each step (down to 1)

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Final domain |
| depth | uint8 | Steps actually taken |
| coord | uint64 | Final coord |
| score | uint32 | Accumulated confidence score |
| word | string | Final word reached |

---

### 0x0A - Flush

Force immediate persistence of the lattice to disk.

**Request:**

Empty (just the op byte).

**Response (OK):**

Empty payload.

---

### 0x0B - IngestRef

Record a typed reference from src to target. References use the same storage as bigrams (Bcooccur branch) but with a RefKind byte set on the MetaEntry. The coord gets RefKind bits set at [15:13] so reference keys hash differently from sequential bigrams.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Domain byte |
| coord | uint64 | Context coord |
| src | string | Source symbol |
| target | string | Target symbol |
| kind | uint8 | Reference type (1-7) |

Reference types:

| Value | Name | Meaning |
|-------|------|---------|
| 1 | RefContains | src contains target (directory/scope) |
| 2 | RefRequires | src requires target (type constraint) |
| 3 | RefProduces | src produces target (return/output) |
| 4 | RefOperatesOn | src operates on target (parameter) |
| 5 | RefIsA | src is a target (subtype/hypernym) |
| 6 | RefBinds | src modifies relationship between neighbors |
| 7 | RefContext | src disambiguated by target (path segment) |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| recIdx | uint32 | Record index of the reference entry |

---

### 0x0C - GetRefs

Return all typed references originating from src in the given domain.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Domain byte |
| src | string | Source symbol to query |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| count | uint16 | Number of references |
| refs[] | variable | Repeated RefEntry |

Each RefEntry:

| Field | Type | Description |
|-------|------|-------------|
| recIdx | uint32 | Record index |
| target | string | Target symbol |
| kind | uint8 | Reference type (1-7) |
| weight | uint32 | Observation count |

---

### 0x0D - WalkOutward

Follow references from src outward, one depth level at a time, up to maxDepth. Returns the full reachable tree of symbols with their relationship types. Cost is proportional to ambiguity depth.

**Request:**

| Field | Type | Description |
|-------|------|-------------|
| domain | uint8 | Domain byte |
| src | string | Starting symbol |
| maxDepth | uint8 | Maximum depth levels to walk |

**Response (OK):**

| Field | Type | Description |
|-------|------|-------------|
| layerCount | uint8 | Number of depth layers |
| layers[] | variable | Repeated RefLayer |

Each RefLayer:

| Field | Type | Description |
|-------|------|-------------|
| count | uint16 | Number of entries in this layer |
| entries[] | variable | Repeated RefEntry (same format as GetRefs) |

---

## Coord layout - reference bits

Bits 15-13 of the coord encode the RefKind for reference records. When these bits are non-zero, the record is a typed reference rather than a sequential bigram. This ensures reference keys hash to different lattice positions than bigrams for the same word pair, naturally distributing them at higher depth where expansion has created room.

```
bits 15-13  refkind  (3 bits): reference type (0=bigram, 1-7=typed ref)
bits 12-8   position (4 bits): sequence order within a definition
bits  7-2   (available)
bits  1-0   register (2 bits): social register
```

---

## Domain bytes

| Value | Domain | Description |
|-------|--------|-------------|
| 0x01 | LangEN | English vocabulary |
| 0x02 | LangJA | Japanese vocabulary |
| 0x10 | MoxieSRC | Moxie source constructs |
| 0x11 | MoxieAST | Moxie AST nodes |
| 0x12 | MoxieIR | Moxie IR (LLVM) |
| 0x13 | MoxieASM | Assembly |
| 0x14 | MoxieBIN | Binary/bytecode |

## Branch indices

| Index | Name | Semantic axis |
|-------|------|---------------|
| 0 | Bsemantic | Ontological category |
| 1 | Bgrammatical (Bnoun) | Syntactic role |
| 2 | Bcooccur | Co-occurrence / bigrams |
| 3 | Bmorphology (Bverb) | Morphological state |
| 4 | Bpragmatic (Bmodifier) | Domain/register context |
| 5 | Bphonology | Phonological (reserved) |
| 6 | Bvalency | Argument count |
| 7 | Bregister | Social register |

## Coord layout (64-bit packed)

```
bits 63-48  semantic    (16 bits): 8 subject|object category pairs
bits 47-32  (reserved)
bits 31-29  grammatical  (3 bits): syntactic role
bits 28-25  cooccur      (4 bits): prev_type(2) + next_type(2)
bits 24-20  morphstate   (5 bits): tense/aspect/polarity/formality/evidential
bits 19-18  pragmatic    (2 bits): domain context
bits 17-16  valency      (2 bits): argument count
bits 15-2   (reserved)
bits  1-0   register     (2 bits): social register
```

coord=0 is the base key (dictionary form, context-free lookups).

## Auto-flush behavior

The server flushes to disk after every 64 write operations (IngestBigram, IngestText) or 5 seconds since last flush, whichever comes first. Clients can force an immediate flush with OpFlush.

## Connection semantics

- One request per frame, one response per frame
- No pipelining within a connection
- No authentication (local trust model)
- No compression
- Connections may be long-lived or per-request
- Server handles connections sequentially per listener (TCP and Unix independently)
- Multiple simultaneous connections queue at the OS accept backlog