aggregate_report.txt raw
1 ================================================================
2 NOSTR RELAY BENCHMARK AGGREGATE REPORT
3 ================================================================
4 Generated: 2025-11-20T20:32:25+00:00
5 Benchmark Configuration:
6 Events per test: 50000
7 Concurrent workers: 24
8 Test duration: 60s
9
10 Relays tested: 9
11
12 ================================================================
13 SUMMARY BY RELAY
14 ================================================================
15
16 Relay: rely-sqlite
17 ----------------------------------------
18 Status: COMPLETED
19 Events/sec: 17507.10
20 Events/sec: 6243.12
21 Events/sec: 17507.10
22 Success Rate: 100.0%
23 Success Rate: 100.0%
24 Success Rate: 100.0%
25 Avg Latency: 1.223582ms
26 Bottom 10% Avg Latency: 698.877µs
27 Avg Latency: 1.178662ms
28 P95 Latency: 1.87223ms
29 P95 Latency: 2.046981ms
30 P95 Latency: 883.507µs
31
32 Relay: next-orly-badger
33 ----------------------------------------
34 Status: COMPLETED
35 Events/sec: 16840.34
36 Events/sec: 6128.23
37 Events/sec: 16840.34
38 Success Rate: 100.0%
39 Success Rate: 100.0%
40 Success Rate: 100.0%
41 Avg Latency: 1.297596ms
42 Bottom 10% Avg Latency: 722.094µs
43 Avg Latency: 1.265918ms
44 P95 Latency: 2.027536ms
45 P95 Latency: 2.302166ms
46 P95 Latency: 894.834µs
47
48 Relay: next-orly-dgraph
49 ----------------------------------------
50 Status: COMPLETED
51 Events/sec: 16563.45
52 Events/sec: 6132.86
53 Events/sec: 16563.45
54 Success Rate: 100.0%
55 Success Rate: 100.0%
56 Success Rate: 100.0%
57 Avg Latency: 1.32589ms
58 Bottom 10% Avg Latency: 726.176µs
59 Avg Latency: 1.340819ms
60 P95 Latency: 2.152481ms
61 P95 Latency: 2.37338ms
62 P95 Latency: 904.165µs
63
64 Relay: next-orly-neo4j
65 ----------------------------------------
66 Status: COMPLETED
67 Events/sec: 14622.22
68 Events/sec: 6182.48
69 Events/sec: 14622.22
70 Success Rate: 100.0%
71 Success Rate: 100.0%
72 Success Rate: 100.0%
73 Avg Latency: 1.559545ms
74 Bottom 10% Avg Latency: 795.698µs
75 Avg Latency: 1.269605ms
76 P95 Latency: 2.658118ms
77 P95 Latency: 2.293256ms
78 P95 Latency: 867.888µs
79
80 Relay: khatru-sqlite
81 ----------------------------------------
82 Status: COMPLETED
83 Events/sec: 16872.81
84 Events/sec: 6219.91
85 Events/sec: 16872.81
86 Success Rate: 100.0%
87 Success Rate: 100.0%
88 Success Rate: 100.0%
89 Avg Latency: 1.294206ms
90 Bottom 10% Avg Latency: 724.237µs
91 Avg Latency: 1.28288ms
92 P95 Latency: 2.011193ms
93 P95 Latency: 2.16732ms
94 P95 Latency: 868.521µs
95
96 Relay: khatru-badger
97 ----------------------------------------
98 Status: COMPLETED
99 Events/sec: 15204.92
100 Events/sec: 6277.98
101 Events/sec: 15204.92
102 Success Rate: 100.0%
103 Success Rate: 100.0%
104 Success Rate: 100.0%
105 Avg Latency: 1.485679ms
106 Bottom 10% Avg Latency: 768.979µs
107 Avg Latency: 1.216531ms
108 P95 Latency: 2.501619ms
109 P95 Latency: 2.028348ms
110 P95 Latency: 862.271µs
111
112 Relay: relayer-basic
113 ----------------------------------------
114 Status: COMPLETED
115 Events/sec: 17272.97
116 Events/sec: 6207.90
117 Events/sec: 17272.97
118 Success Rate: 100.0%
119 Success Rate: 100.0%
120 Success Rate: 100.0%
121 Avg Latency: 1.255956ms
122 Bottom 10% Avg Latency: 712.498µs
123 Avg Latency: 1.21703ms
124 P95 Latency: 1.909735ms
125 P95 Latency: 2.233521ms
126 P95 Latency: 871.278µs
127
128 Relay: strfry
129 ----------------------------------------
130 Status: COMPLETED
131 Events/sec: 15745.79
132 Events/sec: 6264.53
133 Events/sec: 15745.79
134 Success Rate: 100.0%
135 Success Rate: 100.0%
136 Success Rate: 100.0%
137 Avg Latency: 1.415908ms
138 Bottom 10% Avg Latency: 739.523µs
139 Avg Latency: 1.153768ms
140 P95 Latency: 2.340716ms
141 P95 Latency: 2.007502ms
142 P95 Latency: 855.87µs
143
144 Relay: nostr-rs-relay
145 ----------------------------------------
146 Status: COMPLETED
147 Events/sec: 17638.66
148 Events/sec: 6241.74
149 Events/sec: 17638.66
150 Success Rate: 100.0%
151 Success Rate: 100.0%
152 Success Rate: 100.0%
153 Avg Latency: 1.18563ms
154 Bottom 10% Avg Latency: 646.954µs
155 Avg Latency: 1.182584ms
156 P95 Latency: 1.847889ms
157 P95 Latency: 2.120267ms
158 P95 Latency: 866.51µs
159
160
161 ================================================================
162 DETAILED RESULTS
163 ================================================================
164
165 Individual relay reports are available in:
166 - /reports/run_20251120_200202/khatru-badger_results.txt
167 - /reports/run_20251120_200202/khatru-sqlite_results.txt
168 - /reports/run_20251120_200202/next-orly-badger_results.txt
169 - /reports/run_20251120_200202/next-orly-dgraph_results.txt
170 - /reports/run_20251120_200202/next-orly-neo4j_results.txt
171 - /reports/run_20251120_200202/nostr-rs-relay_results.txt
172 - /reports/run_20251120_200202/relayer-basic_results.txt
173 - /reports/run_20251120_200202/rely-sqlite_results.txt
174 - /reports/run_20251120_200202/strfry_results.txt
175
176 ================================================================
177 BENCHMARK COMPARISON TABLE
178 ================================================================
179
180 Relay Status Peak Tput/s Avg Latency Success Rate
181 ---- ------ ----------- ----------- ------------
182 rely-sqlite OK 17507.10 1.223582ms 100.0%
183 next-orly-badger OK 16840.34 1.297596ms 100.0%
184 next-orly-dgraph OK 16563.45 1.32589ms 100.0%
185 next-orly-neo4j OK 14622.22 1.559545ms 100.0%
186 khatru-sqlite OK 16872.81 1.294206ms 100.0%
187 khatru-badger OK 15204.92 1.485679ms 100.0%
188 relayer-basic OK 17272.97 1.255956ms 100.0%
189 strfry OK 15745.79 1.415908ms 100.0%
190 nostr-rs-relay OK 17638.66 1.18563ms 100.0%
191
192 ================================================================
193 End of Report
194 ================================================================
195