aggregate_report.txt raw
1 ================================================================
2 NOSTR RELAY BENCHMARK AGGREGATE REPORT
3 ================================================================
4 Generated: 2025-11-20T15:53:41+00:00
5 Benchmark Configuration:
6 Events per test: 50000
7 Concurrent workers: 24
8 Test duration: 60s
9
10 Relays tested: 8
11
12 ================================================================
13 SUMMARY BY RELAY
14 ================================================================
15
16 Relay: next-orly-badger
17 ----------------------------------------
18 Status: COMPLETED
19 Events/sec: 17836.33
20 Events/sec: 6340.29
21 Events/sec: 17836.33
22 Success Rate: 100.0%
23 Success Rate: 100.0%
24 Success Rate: 100.0%
25 Avg Latency: 1.176626ms
26 Bottom 10% Avg Latency: 659.571µs
27 Avg Latency: 1.150109ms
28 P95 Latency: 1.79182ms
29 P95 Latency: 1.87572ms
30 P95 Latency: 870.11µs
31
32 Relay: next-orly-dgraph
33 ----------------------------------------
34 Status: COMPLETED
35 Events/sec: 16687.23
36 Events/sec: 6230.59
37 Events/sec: 16687.23
38 Success Rate: 100.0%
39 Success Rate: 100.0%
40 Success Rate: 100.0%
41 Avg Latency: 1.299973ms
42 Bottom 10% Avg Latency: 703.285µs
43 Avg Latency: 1.216351ms
44 P95 Latency: 2.203343ms
45 P95 Latency: 2.205777ms
46 P95 Latency: 869.669µs
47
48 Relay: next-orly-neo4j
49 ----------------------------------------
50 Status: COMPLETED
51 Events/sec: 17497.93
52 Events/sec: 6254.20
53 Events/sec: 17497.93
54 Success Rate: 100.0%
55 Success Rate: 100.0%
56 Success Rate: 100.0%
57 Avg Latency: 1.220061ms
58 Bottom 10% Avg Latency: 689.107µs
59 Avg Latency: 1.207729ms
60 P95 Latency: 1.873592ms
61 P95 Latency: 2.026464ms
62 P95 Latency: 860.711µs
63
64 Relay: khatru-sqlite
65 ----------------------------------------
66 Status: COMPLETED
67 Events/sec: 15692.37
68 Events/sec: 6031.64
69 Events/sec: 15692.37
70 Success Rate: 100.0%
71 Success Rate: 100.0%
72 Success Rate: 100.0%
73 Avg Latency: 1.434878ms
74 Bottom 10% Avg Latency: 773.12µs
75 Avg Latency: 1.438112ms
76 P95 Latency: 2.364988ms
77 P95 Latency: 2.530373ms
78 P95 Latency: 869.767µs
79
80 Relay: khatru-badger
81 ----------------------------------------
82 Status: COMPLETED
83 Events/sec: 15459.86
84 Events/sec: 6208.94
85 Events/sec: 15459.86
86 Success Rate: 100.0%
87 Success Rate: 100.0%
88 Success Rate: 100.0%
89 Avg Latency: 1.468719ms
90 Bottom 10% Avg Latency: 802.399µs
91 Avg Latency: 1.250479ms
92 P95 Latency: 2.396216ms
93 P95 Latency: 2.142422ms
94 P95 Latency: 869.166µs
95
96 Relay: relayer-basic
97 ----------------------------------------
98 Status: COMPLETED
99 Events/sec: 15191.51
100 Events/sec: 6144.49
101 Events/sec: 15191.51
102 Success Rate: 100.0%
103 Success Rate: 100.0%
104 Success Rate: 100.0%
105 Avg Latency: 1.494499ms
106 Bottom 10% Avg Latency: 790.923µs
107 Avg Latency: 1.322915ms
108 P95 Latency: 2.461731ms
109 P95 Latency: 2.255818ms
110 P95 Latency: 888.112µs
111
112 Relay: strfry
113 ----------------------------------------
114 Status: COMPLETED
115 Events/sec: 16583.98
116 Events/sec: 5979.92
117 Events/sec: 16583.98
118 Success Rate: 100.0%
119 Success Rate: 100.0%
120 Success Rate: 100.0%
121 Avg Latency: 1.325163ms
122 Bottom 10% Avg Latency: 732.389µs
123 Avg Latency: 1.467778ms
124 P95 Latency: 2.114188ms
125 P95 Latency: 2.793392ms
126 P95 Latency: 878.634µs
127
128 Relay: nostr-rs-relay
129 ----------------------------------------
130 Status: COMPLETED
131 Events/sec: 15250.43
132 Events/sec: 6286.54
133 Events/sec: 15250.43
134 Success Rate: 100.0%
135 Success Rate: 100.0%
136 Success Rate: 100.0%
137 Avg Latency: 1.477342ms
138 Bottom 10% Avg Latency: 760.393µs
139 Avg Latency: 1.167307ms
140 P95 Latency: 2.527756ms
141 P95 Latency: 2.003086ms
142 P95 Latency: 868.365µs
143
144
145 ================================================================
146 DETAILED RESULTS
147 ================================================================
148
149 Individual relay reports are available in:
150 - /reports/run_20251120_152640/khatru-badger_results.txt
151 - /reports/run_20251120_152640/khatru-sqlite_results.txt
152 - /reports/run_20251120_152640/next-orly-badger_results.txt
153 - /reports/run_20251120_152640/next-orly-dgraph_results.txt
154 - /reports/run_20251120_152640/next-orly-neo4j_results.txt
155 - /reports/run_20251120_152640/nostr-rs-relay_results.txt
156 - /reports/run_20251120_152640/relayer-basic_results.txt
157 - /reports/run_20251120_152640/strfry_results.txt
158
159 ================================================================
160 BENCHMARK COMPARISON TABLE
161 ================================================================
162
163 Relay Status Peak Tput/s Avg Latency Success Rate
164 ---- ------ ----------- ----------- ------------
165 next-orly-badger OK 17836.33 1.176626ms 100.0%
166 next-orly-dgraph OK 16687.23 1.299973ms 100.0%
167 next-orly-neo4j OK 17497.93 1.220061ms 100.0%
168 khatru-sqlite OK 15692.37 1.434878ms 100.0%
169 khatru-badger OK 15459.86 1.468719ms 100.0%
170 relayer-basic OK 15191.51 1.494499ms 100.0%
171 strfry OK 16583.98 1.325163ms 100.0%
172 nostr-rs-relay OK 15250.43 1.477342ms 100.0%
173
174 ================================================================
175 End of Report
176 ================================================================
177