# Plan: Moxie Allocation Instrumentation + Smesh Memory Fix

Version 3 (final). All review feedback resolved.

## Problem

Smesh web app has runaway memory growth. WASM target uses `gc_leaking` (bump allocator, never frees). ~15 unbounded maps in `web/wasm/app/main.mx`, DOM handle leaks in `jsruntime/dom.mjs`, no eviction anywhere. Every allocation is permanent for the lifetime of the WASM instance.

## Key Architecture Facts

- Smesh web app is **WASM** (`moxie build -target wasm`), NOT moxiejs. Makefile confirms all 4 modules use LLVM path.
- Canonical moxie compiler: `/home/mleku/s/moxie` (remote: `ssh://git@git.smesh.lol:2222/~/moxie.git`)
- moxiejs transpiler (separate repo): `/home/mleku/src/git.smesh.lol/moxie` (remote: `git@orly:moxie.git`) - NOT used for smesh web app
- `src/runtime/` is 100% `.mx` files (78 files, zero `.go`). Build tags work in `.mx`.
- WASM GC: `gc_leaking` - bump pointer, never frees. `runtime.ReadMemStats` tracks `gcTotalAlloc`, `gcMallocs`, `heapptr`.
- `wasm-host.mjs` runs on **main page thread** (has localStorage, DOM access). App WASM runs in a dedicated Worker.
- Firefox 150.0.1 on test machine. `credentialless` COEP and `measureUserAgentSpecificMemory()` both available.
- View-switch signal: `switchPage(name string)` at `main.mx:799`. Called on every view transition.

## Phase 1: Compiler Instrumentation (LLVM path)

### 1A: Wire `-print-allocs` into the compiler

Flag already exists (`main.go:453`, `compileopts/options.go:41`) but is not connected to the compiler backend.

**File**: `/home/mleku/s/moxie/compiler/compiler.go`

At each allocation site, when containing function matches `PrintAllocs` regex, emit LLVM IR call to `runtime.logAlloc(siteID, size)` after the `runtime.alloc` call.

Four sites:
1. `*ssa.Alloc` (line 2172) - struct/pointer heap allocation
2. `*ssa.MakeSlice` (line 2430) - slice allocation
3. `*ssa.MakeMap` (`compiler/map.go`) - map allocation
4. `*ssa.MakeChan` (`compiler/channel.go`) - channel allocation

### 1B: Runtime counter array

**New file**: `/home/mleku/s/moxie/src/runtime/alloc_trace.mx`

- Counter array sized at compile time (compiler counts sites during compilation, emits size as constant)
- Safety cap via `-print-allocs-max-sites` flag (default 8192). If exceeded, compile-time warning + saturating overflow bucket at last index.
- Site counts for smesh: ~1479 smesh-only, ~4000 with stdlib. Pattern `"web/wasm/app"` scopes to ~350 sites.

```
var allocCounters [N]uint64    // indexed by site ID, N set by compiler
var allocSizes [N]uint64       // cumulative bytes per site
var allocSiteNames [N]string   // populated at init by compiler-generated code
var nextSiteID int32

func logAlloc(siteID int32, size uintptr) {
    allocCounters[siteID]++
    allocSizes[siteID] += uint64(size)
}
```

Cost: one integer increment per allocation (~1 instruction). Zero string formatting, zero JS bridge crossing during execution.

### 1C: Jsbridge export for WASM target

- `__moxie_read_alloc_counters` - reads counter arrays from WASM memory, returns JSON
- `__moxie_mem_stats` - calls `runtime.ReadMemStats`, returns JSON `{totalAlloc, mallocs, heapInuse, sys}`
- Exposed on `globalThis` by `wasm-host.mjs`

### 1D: JS-side instrumentation (parallel with 1A-1C)

Guarded by `globalThis.__moxie_instrument = true`:
- `dom.mjs`: log `_elements.size` to `globalThis.__moxie_alloc_log` at thresholds (100, 500, 1000, 2000, 5000)
- `ws.mjs`: log `_conns.size`
- Ring buffer capped at 10000 entries, format: `{ts, source, metric, value}`
- `dom.mjs` addition: `releaseAll(ids)` batch function for bulk handle cleanup

## Phase 2: Instrumented Smesh Build

```sh
# Instrumented app WASM
MOXIEROOT=../moxie GOWORK=off ../moxie/moxie build -target wasm -print-allocs "web/wasm/app" \
  -o web/static/app.wasm ./web/wasm/app/

# Instrumented relay
MOXIEROOT=../moxie ../moxie/moxie build -print-allocs ".*" -o /tmp/smesh-instrumented .
```

Add `SMESH_TEST_HEADERS=1` env var to `main.mx` to emit COOP/COEP headers in test mode only.

## Phase 3: Selenium Test Harness

### 3A: Memory profiling fixtures in `test/conftest.py`

- `mem_stats(driver)` - reads WASM-internal `ReadMemStats` via `driver.execute_script("return window.__moxie_mem_stats()")`
- `alloc_counters(driver)` - reads per-site counters via `driver.execute_script("return window.__moxie_read_alloc_counters()")`
- `dom_handle_count(driver)` - reads `_elements.size` from JS side
- No `driver.get_log('browser')` (geckodriver doesn't support it)
- No `performance.memory` (Chrome-only). Primary signal is `ReadMemStats`.
- Supplementary: `performance.measureUserAgentSpecificMemory()` available with COOP/COEP in test mode.

### 3B: Test scenarios in `test/test_memory_profile.py`

1. **Baseline** - load app, record initial `totalAlloc` and `mallocs`
2. **Feed scroll 500 events** - measure bytes/event, mallocs/event
3. **Profile cycling x20** - verify rotate-map bounds working set
4. **Thread open/close x50** - verify `threadEvents` cleanup
5. **Idle 30 seconds** - subscriptions explicitly CLOSEd, 5s settling period, then measure rate
6. **DOM handle count** - assert `_elements.size < 3000` throughout all scenarios
7. **Heap threshold** - push past threshold, verify graceful restart

### 3C: Idle measurement definition

"Idle" = relay-proxy has no active subscriptions + 5s settling period since last EOSE. Test sends explicit CLOSE for all sub IDs ("feed", "ntf", etc.), waits 5s, then measures for 30s. Test relay is controlled (seeded dataset, no push after CLOSE). Assertion: `idle_rate < 1024 bytes/sec`.

### 3D: Output format

JSON per test run with initial/final heap, delta bytes, mallocs, top allocation sites by count/bytes, DOM handle count.

## Phase 4: Fix the Leaks

### 4A: Rotate-map on all unbounded caches

Uniform pattern, no LRU data structure needed. No reflect, no generics. ~15 lines per cache group.

| Cache group | Maps | Max entries | Rotate together |
|------------|------|------------|----------------|
| Event data | `eventCache`, `seenEvents`, `noteElements`, `noteReactionMap`, `noteRepostCounts`, `noteZapSeen`, `actionFetchedIDs` | 2000 | Yes (event ID keyed) |
| Author metadata | `authorNames`, `authorPics`, `authorContent`, `authorTs`, `authorRelays`, `fetchedK0`, `fetchedK10k` | 500 | Yes (pubkey keyed) |
| Embeds | `embedCallbacks`, `embedRequested`, `embedSubIDs` | 200 | Yes |
| Reply cache | `replyCache`, `replyAvatarCache`, `replyLineCache`, `replyPending`, `replyNeedName`, `replyQueue` | 500 | Yes |
| Profile wrappers | `profileWrappers`, `profileWrapOrder` | 10 | Yes |
| Thread | `threadEvents` | N/A | Cleared in `closeNoteThread()` already |

**Important**: All `*Old` map vars must be initialized to empty maps at startup (not nil). On first rotation, `range` over nil map is safe (zero iterations) but silently drops all current handles without releasing them. Initialize alongside their primary counterparts.

### 4B: DOM handle release on rotation

```
func rotateEventCaches() {
    // Release DOM handles from entries about to be dropped.
    // Iterate noteElementsOld, release handles NOT also in current noteElements.
    var ids []int32
    for id, el := range noteElementsOld {
        if _, ok := noteElements[id]; !ok {
            ids = append(ids, el)
        }
    }
    if len(ids) > 0 {
        dom.ReleaseAll(ids)  // single jsbridge call, bulk
    }
    // Rotate all event-keyed maps together.
    eventCacheOld = eventCache
    eventCache = map[string]*nostr.Event{}
    noteElementsOld = noteElements
    noteElements = map[string]dom.Element{}
    seenEventsOld = seenEvents
    seenEvents = map[string]bool{}
    // ... same for reaction/repost/zap maps
    eventCacheCount = 0
}
```

### 4C: switchPage teardown

Insert `teardownPage(activePage)` in `switchPage()` before `activePage = name` (line 816). Each page case sends CLOSE messages to relay-proxy and trims page-specific caches.

### 4D: Proactive WASM restart

**Strategy: option (c) - tune threshold high enough that restarts are rare.**

- Threshold: 128MB. At ~12MB/hour allocation rate (pre-fix), that's ~10 hours. Post-fix with rotate-maps the rate drops, pushing it further out.
- Revisit if Phase 3 shows allocation rate exceeds 30MB/hour post-fix.
- `wasm-host.mjs` polls `__moxie_mem_stats()` every 60s. If `heapInuse > 128MB`, save state to localStorage (main thread, directly available), `worker.terminate()`, `startWorker()`, send saved state via `postMessage`.
- State saved: current view (`activePage`), scroll position, logged-in pubkey. Event caches are lost - feed re-fetches from relay on restart. This is acceptable: a restart every 10+ hours is indistinguishable from opening the tab fresh.
- Options (a) accept cold-load and (b) persist event IDs for re-request are fallbacks if Phase 3 data shows restarts > 1/hour.

## Phase 5: Regression Gate

### 5A: Budget assertions

- After scrolling 500 events: `delta_bytes < 20MB`
- DOM handles < 3000 at all times
- Idle allocation rate < 1KB/s (subscriptions closed, relay quiesced)
- `heapInuse` after 5 minutes of use < 64MB

### 5B: CI integration

`test-memory` target in Makefile: builds instrumented WASM, runs memory tests.

## Execution Order

```
1A (wire -print-allocs in compiler/compiler.go)
  -> 1B (runtime counter array in alloc_trace.mx)
    -> 1C (jsbridge export)
      -> 2 (instrumented build)
        -> 3 (selenium measurement)
          -> 4A-4C (fixes based on data)
            -> 4D (proactive restart)
              -> 5 (regression gate)

1D (JS-side dom.mjs instrumentation) -- parallel with 1A-1C, independent
```

## Files Modified

| File | Change |
|------|--------|
| `/home/mleku/s/moxie/compiler/compiler.go` | Wire PrintAllocs: emit `logAlloc` call at 4 allocation sites |
| `/home/mleku/s/moxie/compiler/map.go` | Wire PrintAllocs for MakeMap |
| `/home/mleku/s/moxie/compiler/channel.go` | Wire PrintAllocs for MakeChan |
| `/home/mleku/s/moxie/src/runtime/alloc_trace.mx` | New: counter array + GetAllocCounters |
| `/home/mleku/s/moxie/jsruntime/dom.mjs` | Threshold logging on `_elements.size` + `releaseAll()` batch function |
| `/home/mleku/s/smesh/web/wasm/app/main.mx` | Rotate-map caches, switchPage teardown, jsbridge for mem_stats |
| `/home/mleku/s/smesh/web/static/wasm-host.mjs` | Expose `__moxie_mem_stats`, `__moxie_read_alloc_counters`; proactive restart |
| `/home/mleku/s/smesh/main.mx` | COOP/COEP headers in test mode |
| `/home/mleku/s/smesh/test/conftest.py` | Memory profiling fixtures |
| `/home/mleku/s/smesh/test/test_memory_profile.py` | New: memory measurement tests |
| `/home/mleku/s/smesh/Makefile` | `test-memory` target |