aggregate_report.txt raw
1 ================================================================
2 NOSTR RELAY BENCHMARK AGGREGATE REPORT
3 ================================================================
4 Generated: 2025-11-22T19:37:27+00:00
5 Benchmark Configuration:
6 Events per test: 50000
7 Concurrent workers: 24
8 Test duration: 60s
9
10 Relays tested: 9
11
12 ================================================================
13 SUMMARY BY RELAY
14 ================================================================
15
16 Relay: rely-sqlite
17 ----------------------------------------
18 Status: COMPLETED
19 Events/sec: 15903.28
20 Events/sec: 6308.59
21 Events/sec: 15903.28
22 Success Rate: 100.0%
23 Success Rate: 100.0%
24 Success Rate: 100.0%
25 Avg Latency: 1.399274ms
26 Bottom 10% Avg Latency: 746.992µs
27 Avg Latency: 1.174853ms
28 P95 Latency: 2.34974ms
29 P95 Latency: 1.933092ms
30 P95 Latency: 897.528µs
31
32 Relay: next-orly-badger
33 ----------------------------------------
34 Status: COMPLETED
35 Events/sec: 16607.66
36 Events/sec: 5941.60
37 Events/sec: 16607.66
38 Success Rate: 100.0%
39 Success Rate: 100.0%
40 Success Rate: 100.0%
41 Avg Latency: 1.338951ms
42 Bottom 10% Avg Latency: 757.49µs
43 Avg Latency: 1.490934ms
44 P95 Latency: 2.047963ms
45 P95 Latency: 2.961357ms
46 P95 Latency: 928.904µs
47
48 Relay: next-orly-dgraph
49 ----------------------------------------
50 Status: COMPLETED
51 Events/sec: 16030.75
52 Events/sec: 6221.38
53 Events/sec: 16030.75
54 Success Rate: 100.0%
55 Success Rate: 100.0%
56 Success Rate: 100.0%
57 Avg Latency: 1.395117ms
58 Bottom 10% Avg Latency: 759.404µs
59 Avg Latency: 1.256755ms
60 P95 Latency: 2.2327ms
61 P95 Latency: 2.095959ms
62 P95 Latency: 890.448µs
63
64 Relay: next-orly-neo4j
65 ----------------------------------------
66 Status: COMPLETED
67 Events/sec: 16565.07
68 Events/sec: 6026.51
69 Events/sec: 16565.07
70 Success Rate: 100.0%
71 Success Rate: 100.0%
72 Success Rate: 100.0%
73 Avg Latency: 1.32858ms
74 Bottom 10% Avg Latency: 724.65µs
75 Avg Latency: 1.392811ms
76 P95 Latency: 2.11453ms
77 P95 Latency: 2.568976ms
78 P95 Latency: 910.826µs
79
80 Relay: khatru-sqlite
81 ----------------------------------------
82 Status: COMPLETED
83 Events/sec: 13273.11
84 Events/sec: 6204.61
85 Events/sec: 13273.11
86 Success Rate: 100.0%
87 Success Rate: 100.0%
88 Success Rate: 100.0%
89 Avg Latency: 1.732057ms
90 Bottom 10% Avg Latency: 803.833µs
91 Avg Latency: 1.263843ms
92 P95 Latency: 3.370931ms
93 P95 Latency: 2.195471ms
94 P95 Latency: 905.805µs
95
96 Relay: khatru-badger
97 ----------------------------------------
98 Status: COMPLETED
99 Events/sec: 15590.07
100 Events/sec: 6139.02
101 Events/sec: 15590.07
102 Success Rate: 100.0%
103 Success Rate: 100.0%
104 Success Rate: 100.0%
105 Avg Latency: 1.451694ms
106 Bottom 10% Avg Latency: 793.994µs
107 Avg Latency: 1.324245ms
108 P95 Latency: 2.351317ms
109 P95 Latency: 2.291241ms
110 P95 Latency: 901.036µs
111
112 Relay: relayer-basic
113 ----------------------------------------
114 Status: COMPLETED
115 Events/sec: 15076.33
116 Events/sec: 6071.70
117 Events/sec: 15076.33
118 Success Rate: 100.0%
119 Success Rate: 100.0%
120 Success Rate: 100.0%
121 Avg Latency: 1.517087ms
122 Bottom 10% Avg Latency: 821.229µs
123 Avg Latency: 1.385607ms
124 P95 Latency: 2.48546ms
125 P95 Latency: 2.478156ms
126 P95 Latency: 916.474µs
127
128 Relay: strfry
129 ----------------------------------------
130 Status: COMPLETED
131 Events/sec: 16279.08
132 Events/sec: 6097.81
133 Events/sec: 16279.08
134 Success Rate: 100.0%
135 Success Rate: 100.0%
136 Success Rate: 100.0%
137 Avg Latency: 1.369757ms
138 Bottom 10% Avg Latency: 764.155µs
139 Avg Latency: 1.369895ms
140 P95 Latency: 2.13361ms
141 P95 Latency: 2.341095ms
142 P95 Latency: 894.733µs
143
144 Relay: nostr-rs-relay
145 ----------------------------------------
146 Status: COMPLETED
147 Events/sec: 14836.18
148 Events/sec: 6111.29
149 Events/sec: 14836.18
150 Success Rate: 100.0%
151 Success Rate: 100.0%
152 Success Rate: 100.0%
153 Avg Latency: 1.545053ms
154 Bottom 10% Avg Latency: 829.94µs
155 Avg Latency: 1.336805ms
156 P95 Latency: 2.562666ms
157 P95 Latency: 2.417039ms
158 P95 Latency: 936.832µs
159
160
161 ================================================================
162 DETAILED RESULTS
163 ================================================================
164
165 Individual relay reports are available in:
166 - /reports/run_20251122_190700/khatru-badger_results.txt
167 - /reports/run_20251122_190700/khatru-sqlite_results.txt
168 - /reports/run_20251122_190700/next-orly-badger_results.txt
169 - /reports/run_20251122_190700/next-orly-dgraph_results.txt
170 - /reports/run_20251122_190700/next-orly-neo4j_results.txt
171 - /reports/run_20251122_190700/nostr-rs-relay_results.txt
172 - /reports/run_20251122_190700/relayer-basic_results.txt
173 - /reports/run_20251122_190700/rely-sqlite_results.txt
174 - /reports/run_20251122_190700/strfry_results.txt
175
176 ================================================================
177 BENCHMARK COMPARISON TABLE
178 ================================================================
179
180 Relay Status Peak Tput/s Avg Latency Success Rate
181 ---- ------ ----------- ----------- ------------
182 rely-sqlite OK 15903.28 1.399274ms 100.0%
183 next-orly-badger OK 16607.66 1.338951ms 100.0%
184 next-orly-dgraph OK 16030.75 1.395117ms 100.0%
185 next-orly-neo4j OK 16565.07 1.32858ms 100.0%
186 khatru-sqlite OK 13273.11 1.732057ms 100.0%
187 khatru-badger OK 15590.07 1.451694ms 100.0%
188 relayer-basic OK 15076.33 1.517087ms 100.0%
189 strfry OK 16279.08 1.369757ms 100.0%
190 nostr-rs-relay OK 14836.18 1.545053ms 100.0%
191
192 ================================================================
193 End of Report
194 ================================================================
195