SYNC_CLIENT_MODE.md raw

ORLY Sync Client Mode

Overview

ORLY now supports two modes for the orly sync command, similar to how strfry's sync command works but with the added flexibility of gRPC-based IPC:

  1. Direct Mode (like strfry): Opens the database directly
  2. Client Mode (unique to ORLY): Connects to a running orly-sync-negentropy service via gRPC

Why Client Mode?

The Problem with Direct Database Access

Both strfry and ORLY's direct mode require:

The gRPC IPC Solution

ORLY's IPC architecture allows the sync command to:

Architecture Comparison

strfry sync (Direct Mode Only)

┌─────────────────┐         ┌─────────────────┐
│   strfry sync   │────────▶│   LMDB (file)   │
│   (CLI tool)    │  read   │                 │
└────────┬────────┘         └─────────────────┘
         │
         │ WebSocket
         ▼
┌─────────────────┐
│  Remote Relay   │
└─────────────────┘

ORLY sync (Direct Mode - same as strfry)

┌─────────────────┐         ┌─────────────────┐
│   orly sync     │────────▶│  Badger/Neo4j   │
│   (CLI tool)    │  read   │   (database)    │
└────────┬────────┘         └─────────────────┘
         │
         │ WebSocket
         ▼
┌─────────────────┐
│  Remote Relay   │
└─────────────────┘

ORLY sync (Client Mode - gRPC IPC)

┌─────────────────┐         ┌─────────────────────────────┐
│   orly sync     │────────▶│   orly-sync-negentropy      │
│   (CLI tool)    │  gRPC   │   (gRPC service)            │
│                 │         │                             │
│  No DB access   │         │  • Manages negentropy       │
│  needed!        │         │  • Opens database           │
└─────────────────┘         │  • Handles NIP-77 protocol  │
                            └──────────────┬──────────────┘
                                           │
                            ┌──────────────┴──────────────┐
                            │                             │
                   ┌────────▼────────┐         ┌─────────▼─────────┐
                   │  Badger/Neo4j   │         │   Remote Relay    │
                   │  (database)     │         │   (WebSocket)     │
                   └─────────────────┘         └───────────────────┘

Usage

Direct Mode (strfry-style)

# Sync using local database (must have filesystem access)
orly sync wss://relay.example.com --filter '{"kinds": [0, 3, 1984]}' --dir both

# Options:
#   --data-dir=PATH    Path to database (default: ~/.local/share/ORLY)
#   --filter=JSON      Nostr filter to limit sync scope
#   --dir=DIR          Direction: down, up, both (default: down)

Client Mode (gRPC IPC)

# Sync using a running ORLY service (no filesystem access needed!)
orly sync wss://relay.example.com --server 127.0.0.1:50064 --filter '{"kinds": [1]}'

# Options:
#   --server=ADDR      gRPC address of orly-sync-negentropy service
#   --filter=JSON      Nostr filter to limit sync scope
#   --dir=DIR          Direction: down, up, both (default: down)

Setting Up the Server

Option 1: Using orly-launcher (Recommended)

The launcher manages all services automatically:

# Enable negentropy service
export ORLY_LAUNCHER_SYNC_NEGENTROPY_ENABLED=true
export ORLY_LAUNCHER_SYNC_NEGENTROPY_LISTEN=127.0.0.1:50064

# Run launcher
./orly-launcher

Option 2: Standalone Service

Run the sync service manually:

# Start database first
ORLY_DB_LISTEN=127.0.0.1:50051 ./orly-db-badger &

# Start sync service
ORLY_LAUNCHER_SYNC_NEGENTROPY_LISTEN=127.0.0.1:50064 \
ORLY_DB_TYPE=grpc \
ORLY_GRPC_SERVER=127.0.0.1:50051 \
./orly-sync-negentropy

Real-World Scenarios

Scenario 1: Sync from Your Laptop to Production Relay

Problem: Your production ORLY relay runs on a server. You want to sync events from another relay using your laptop, but the database is on the server.

Solution:

# On server: ensure gRPC is accessible (bind to 0.0.0.0 or use SSH tunnel)
export ORLY_LAUNCHER_SYNC_NEGENTROPY_LISTEN=0.0.0.0:50064

# On laptop: sync via gRPC
orly sync wss://remote-relay.com \
  --server your-server.com:50064 \
  --filter '{"kinds": [0, 1, 3]}' \
  --dir both

Scenario 2: Multiple Admins Sharing a Relay

Problem: Multiple administrators need to sync content to the same relay, but you don't want to give them all filesystem access.

Solution:

Scenario 3: Automated Sync Scripts

Problem: You want to run sync scripts from a CI/CD pipeline or cron job.

Solution:

# Cron job that syncs daily
0 2 * * * /usr/local/bin/orly sync wss://backup-relay.com \
  --server 127.0.0.1:50064 \
  --filter '{"kinds": [0, 1, 3, 1984]}' \
  --dir both \
  >> /var/log/orly-sync.log 2>&1

Comparison with strfry

Featurestrfry syncORLY sync (Direct)ORLY sync (Client)
Direct DB accessYesYesNo
Network IPCNoNoYes (gRPC)
Remote syncNoNoYes
Multiple readersLMDBBadger/Neo4jYes (via service)
Requires relay runningNoNoYes

Technical Details

gRPC API

The client mode uses the NegentropyService.SyncWithPeer RPC:

rpc SyncWithPeer(SyncPeerRequest) returns (stream SyncProgress);

message SyncPeerRequest {
  string peer_url = 1;
  Filter filter = 2;
  int64 since = 3;
}

message SyncProgress {
  string peer_url = 1;
  int32 round = 2;
  int64 have_count = 3;
  int64 need_count = 4;
  int64 fetched_count = 5;
  int64 sent_count = 6;
  bool complete = 7;
  string error = 8;
}

Security Considerations

When exposing the gRPC service over the network:

  1. Bind to localhost only (default) for single-machine use
  2. Use SSH tunnels for remote access:

`bash ssh -L 50064:localhost:50064 user@relay-server orly sync wss://... --server 127.0.0.1:50064 `

  1. Use TLS/mTLS for production gRPC (future enhancement)
  2. Firewall rules to restrict access to trusted IPs

Troubleshooting

"Failed to connect to sync service"

"Sync service not ready"

"Permission denied" (Direct Mode)

Future Enhancements

  1. TLS/mTLS support for secure remote connections
  2. Authentication for client connections
  3. Rate limiting to prevent abuse
  4. Audit logging for compliance