aggregate_report.txt raw
1 ================================================================
2 NOSTR RELAY BENCHMARK AGGREGATE REPORT
3 ================================================================
4 Generated: 2025-11-26T08:04:35+00:00
5 Benchmark Configuration:
6 Events per test: 50000
7 Concurrent workers: 24
8 Test duration: 60s
9
10 Relays tested: 9
11
12 ================================================================
13 SUMMARY BY RELAY
14 ================================================================
15
16 Relay: rely-sqlite
17 ----------------------------------------
18 Status: COMPLETED
19 Events/sec: 16298.40
20 Events/sec: 6150.97
21 Events/sec: 16298.40
22 Success Rate: 100.0%
23 Success Rate: 100.0%
24 Success Rate: 100.0%
25 Avg Latency: 1.360569ms
26 Bottom 10% Avg Latency: 746.704µs
27 Avg Latency: 1.411735ms
28 P95 Latency: 2.160818ms
29 P95 Latency: 2.29313ms
30 P95 Latency: 916.446µs
31
32 Relay: next-orly-badger
33 ----------------------------------------
34 Status: COMPLETED
35 Events/sec: 16698.91
36 Events/sec: 6011.59
37 Events/sec: 16698.91
38 Success Rate: 100.0%
39 Success Rate: 100.0%
40 Success Rate: 100.0%
41 Avg Latency: 1.331911ms
42 Bottom 10% Avg Latency: 766.682µs
43 Avg Latency: 1.496861ms
44 P95 Latency: 2.019719ms
45 P95 Latency: 2.715024ms
46 P95 Latency: 914.112µs
47
48 Relay: next-orly-dgraph
49 ----------------------------------------
50 Status: COMPLETED
51 Events/sec: 14573.58
52 Events/sec: 6072.22
53 Events/sec: 14573.58
54 Success Rate: 100.0%
55 Success Rate: 100.0%
56 Success Rate: 100.0%
57 Avg Latency: 1.571025ms
58 Bottom 10% Avg Latency: 802.953µs
59 Avg Latency: 1.454825ms
60 P95 Latency: 2.610305ms
61 P95 Latency: 2.541414ms
62 P95 Latency: 902.751µs
63
64 Relay: next-orly-neo4j
65 ----------------------------------------
66 Status: COMPLETED
67 Events/sec: 16594.60
68 Events/sec: 6139.73
69 Events/sec: 16594.60
70 Success Rate: 100.0%
71 Success Rate: 100.0%
72 Success Rate: 100.0%
73 Avg Latency: 1.341265ms
74 Bottom 10% Avg Latency: 760.268µs
75 Avg Latency: 1.417529ms
76 P95 Latency: 2.068012ms
77 P95 Latency: 2.279114ms
78 P95 Latency: 893.313µs
79
80 Relay: khatru-sqlite
81 ----------------------------------------
82 Status: COMPLETED
83 Events/sec: 16775.48
84 Events/sec: 6077.32
85 Events/sec: 16775.48
86 Success Rate: 100.0%
87 Success Rate: 100.0%
88 Success Rate: 100.0%
89 Avg Latency: 1.316097ms
90 Bottom 10% Avg Latency: 743.925µs
91 Avg Latency: 1.448816ms
92 P95 Latency: 2.019999ms
93 P95 Latency: 2.415349ms
94 P95 Latency: 915.807µs
95
96 Relay: khatru-badger
97 ----------------------------------------
98 Status: COMPLETED
99 Events/sec: 14573.64
100 Events/sec: 6123.62
101 Events/sec: 14573.64
102 Success Rate: 100.0%
103 Success Rate: 100.0%
104 Success Rate: 100.0%
105 Avg Latency: 1.582659ms
106 Bottom 10% Avg Latency: 849.196µs
107 Avg Latency: 1.42045ms
108 P95 Latency: 2.584156ms
109 P95 Latency: 2.297743ms
110 P95 Latency: 911.2µs
111
112 Relay: relayer-basic
113 ----------------------------------------
114 Status: COMPLETED
115 Events/sec: 16103.85
116 Events/sec: 6038.31
117 Events/sec: 16103.85
118 Success Rate: 100.0%
119 Success Rate: 100.0%
120 Success Rate: 100.0%
121 Avg Latency: 1.401051ms
122 Bottom 10% Avg Latency: 788.805µs
123 Avg Latency: 1.501362ms
124 P95 Latency: 2.187347ms
125 P95 Latency: 2.477719ms
126 P95 Latency: 920.8µs
127
128 Relay: strfry
129 ----------------------------------------
130 Status: COMPLETED
131 Events/sec: 16207.30
132 Events/sec: 6075.12
133 Events/sec: 16207.30
134 Success Rate: 100.0%
135 Success Rate: 100.0%
136 Success Rate: 100.0%
137 Avg Latency: 1.381579ms
138 Bottom 10% Avg Latency: 760.474µs
139 Avg Latency: 1.45496ms
140 P95 Latency: 2.15555ms
141 P95 Latency: 2.414222ms
142 P95 Latency: 907.647µs
143
144 Relay: nostr-rs-relay
145 ----------------------------------------
146 Status: COMPLETED
147 Events/sec: 15751.45
148 Events/sec: 6163.36
149 Events/sec: 15751.45
150 Success Rate: 100.0%
151 Success Rate: 100.0%
152 Success Rate: 100.0%
153 Avg Latency: 1.442411ms
154 Bottom 10% Avg Latency: 812.222µs
155 Avg Latency: 1.414472ms
156 P95 Latency: 2.22848ms
157 P95 Latency: 2.267184ms
158 P95 Latency: 921.434µs
159
160
161 ================================================================
162 DETAILED RESULTS
163 ================================================================
164
165 Individual relay reports are available in:
166 - /reports/run_20251126_073410/khatru-badger_results.txt
167 - /reports/run_20251126_073410/khatru-sqlite_results.txt
168 - /reports/run_20251126_073410/next-orly-badger_results.txt
169 - /reports/run_20251126_073410/next-orly-dgraph_results.txt
170 - /reports/run_20251126_073410/next-orly-neo4j_results.txt
171 - /reports/run_20251126_073410/nostr-rs-relay_results.txt
172 - /reports/run_20251126_073410/relayer-basic_results.txt
173 - /reports/run_20251126_073410/rely-sqlite_results.txt
174 - /reports/run_20251126_073410/strfry_results.txt
175
176 ================================================================
177 BENCHMARK COMPARISON TABLE
178 ================================================================
179
180 Relay Status Peak Tput/s Avg Latency Success Rate
181 ---- ------ ----------- ----------- ------------
182 rely-sqlite OK 16298.40 1.360569ms 100.0%
183 next-orly-badger OK 16698.91 1.331911ms 100.0%
184 next-orly-dgraph OK 14573.58 1.571025ms 100.0%
185 next-orly-neo4j OK 16594.60 1.341265ms 100.0%
186 khatru-sqlite OK 16775.48 1.316097ms 100.0%
187 khatru-badger OK 14573.64 1.582659ms 100.0%
188 relayer-basic OK 16103.85 1.401051ms 100.0%
189 strfry OK 16207.30 1.381579ms 100.0%
190 nostr-rs-relay OK 15751.45 1.442411ms 100.0%
191
192 ================================================================
193 End of Report
194 ================================================================
195