aggregate_report.txt raw
1 ================================================================
2 NOSTR RELAY BENCHMARK AGGREGATE REPORT
3 ================================================================
4 Generated: 2025-12-03T22:47:18+00:00
5 Benchmark Configuration:
6 Events per test: 50000
7 Concurrent workers: 24
8 Test duration: 60s
9
10 Relays tested: 8
11
12 ================================================================
13 SUMMARY BY RELAY
14 ================================================================
15
16 Relay: rely-sqlite
17 ----------------------------------------
18 Status: COMPLETED
19 Events/sec: 16694.18
20 Events/sec: 6270.43
21 Events/sec: 16694.18
22 Success Rate: 100.0%
23 Success Rate: 100.0%
24 Success Rate: 100.0%
25 Avg Latency: 1.339216ms
26 Bottom 10% Avg Latency: 779.552µs
27 Avg Latency: 1.281976ms
28 P95 Latency: 1.894111ms
29 P95 Latency: 2.087148ms
30 P95 Latency: 910.529µs
31
32 Relay: next-orly-badger
33 ----------------------------------------
34 Status: COMPLETED
35 Events/sec: 17987.95
36 Events/sec: 6246.39
37 Events/sec: 17987.95
38 Success Rate: 100.0%
39 Success Rate: 100.0%
40 Success Rate: 100.0%
41 Avg Latency: 1.16914ms
42 Bottom 10% Avg Latency: 675.419µs
43 Avg Latency: 1.301155ms
44 P95 Latency: 1.605171ms
45 P95 Latency: 2.260728ms
46 P95 Latency: 911.513µs
47
48 Relay: next-orly-neo4j
49 ----------------------------------------
50 Status: COMPLETED
51 Events/sec: 17437.04
52 Events/sec: 6060.82
53 Events/sec: 17437.04
54 Success Rate: 100.0%
55 Success Rate: 100.0%
56 Success Rate: 100.0%
57 Avg Latency: 1.250926ms
58 Bottom 10% Avg Latency: 735.971µs
59 Avg Latency: 1.493295ms
60 P95 Latency: 1.757814ms
61 P95 Latency: 2.404304ms
62 P95 Latency: 896.796µs
63
64 Relay: khatru-sqlite
65 ----------------------------------------
66 Status: COMPLETED
67 Events/sec: 17531.15
68 Events/sec: 6335.87
69 Events/sec: 17531.15
70 Success Rate: 100.0%
71 Success Rate: 100.0%
72 Success Rate: 100.0%
73 Avg Latency: 1.233875ms
74 Bottom 10% Avg Latency: 707.713µs
75 Avg Latency: 1.239192ms
76 P95 Latency: 1.713051ms
77 P95 Latency: 1.880869ms
78 P95 Latency: 918.848µs
79
80 Relay: khatru-badger
81 ----------------------------------------
82 Status: COMPLETED
83 Events/sec: 15958.89
84 Events/sec: 6352.78
85 Events/sec: 15958.89
86 Success Rate: 100.0%
87 Success Rate: 100.0%
88 Success Rate: 100.0%
89 Avg Latency: 1.414398ms
90 Bottom 10% Avg Latency: 791.675µs
91 Avg Latency: 1.183812ms
92 P95 Latency: 2.191322ms
93 P95 Latency: 1.80172ms
94 P95 Latency: 903.25µs
95
96 Relay: relayer-basic
97 ----------------------------------------
98 Status: COMPLETED
99 Events/sec: 17757.23
100 Events/sec: 6227.33
101 Events/sec: 17757.23
102 Success Rate: 100.0%
103 Success Rate: 100.0%
104 Success Rate: 100.0%
105 Avg Latency: 1.193531ms
106 Bottom 10% Avg Latency: 679.232µs
107 Avg Latency: 1.304ms
108 P95 Latency: 1.679267ms
109 P95 Latency: 2.155365ms
110 P95 Latency: 930.632µs
111
112 Relay: strfry
113 ----------------------------------------
114 Status: COMPLETED
115 Events/sec: 17794.50
116 Events/sec: 6252.01
117 Events/sec: 17794.50
118 Success Rate: 100.0%
119 Success Rate: 100.0%
120 Success Rate: 100.0%
121 Avg Latency: 1.192717ms
122 Bottom 10% Avg Latency: 676.594µs
123 Avg Latency: 1.270724ms
124 P95 Latency: 1.645564ms
125 P95 Latency: 2.251457ms
126 P95 Latency: 915.623µs
127
128 Relay: nostr-rs-relay
129 ----------------------------------------
130 Status: COMPLETED
131 Events/sec: 17174.61
132 Events/sec: 6311.06
133 Events/sec: 17174.61
134 Success Rate: 100.0%
135 Success Rate: 100.0%
136 Success Rate: 100.0%
137 Avg Latency: 1.281647ms
138 Bottom 10% Avg Latency: 742.249µs
139 Avg Latency: 1.260479ms
140 P95 Latency: 1.836808ms
141 P95 Latency: 1.893887ms
142 P95 Latency: 922.647µs
143
144
145 ================================================================
146 DETAILED RESULTS
147 ================================================================
148
149 Individual relay reports are available in:
150 - /reports/run_20251203_222024/khatru-badger_results.txt
151 - /reports/run_20251203_222024/khatru-sqlite_results.txt
152 - /reports/run_20251203_222024/next-orly-badger_results.txt
153 - /reports/run_20251203_222024/next-orly-neo4j_results.txt
154 - /reports/run_20251203_222024/nostr-rs-relay_results.txt
155 - /reports/run_20251203_222024/relayer-basic_results.txt
156 - /reports/run_20251203_222024/rely-sqlite_results.txt
157 - /reports/run_20251203_222024/strfry_results.txt
158
159 ================================================================
160 BENCHMARK COMPARISON TABLE
161 ================================================================
162
163 Relay Status Peak Tput/s Avg Latency Success Rate
164 ---- ------ ----------- ----------- ------------
165 rely-sqlite OK 16694.18 1.339216ms 100.0%
166 next-orly-badger OK 17987.95 1.16914ms 100.0%
167 next-orly-neo4j OK 17437.04 1.250926ms 100.0%
168 khatru-sqlite OK 17531.15 1.233875ms 100.0%
169 khatru-badger OK 15958.89 1.414398ms 100.0%
170 relayer-basic OK 17757.23 1.193531ms 100.0%
171 strfry OK 17794.50 1.192717ms 100.0%
172 nostr-rs-relay OK 17174.61 1.281647ms 100.0%
173
174 ================================================================
175 End of Report
176 ================================================================
177