aggregate_report.txt raw
1 ================================================================
2 NOSTR RELAY BENCHMARK AGGREGATE REPORT
3 ================================================================
4 Generated: 2025-11-19T12:08:43+00:00
5 Benchmark Configuration:
6 Events per test: 50000
7 Concurrent workers: 24
8 Test duration: 60s
9
10 Relays tested: 8
11
12 ================================================================
13 SUMMARY BY RELAY
14 ================================================================
15
16 Relay: next-orly-badger
17 ----------------------------------------
18 Status: COMPLETED
19 Events/sec: 17949.86
20 Events/sec: 6293.77
21 Events/sec: 17949.86
22 Success Rate: 100.0%
23 Success Rate: 100.0%
24 Success Rate: 100.0%
25 Avg Latency: 1.089014ms
26 Bottom 10% Avg Latency: 552.633µs
27 Avg Latency: 749.292µs
28 P95 Latency: 1.801326ms
29 P95 Latency: 1.544064ms
30 P95 Latency: 797.32µs
31
32 Relay: next-orly-dgraph
33 ----------------------------------------
34 Status: COMPLETED
35 Events/sec: 17627.19
36 Events/sec: 6241.01
37 Events/sec: 17627.19
38 Success Rate: 100.0%
39 Success Rate: 100.0%
40 Success Rate: 100.0%
41 Avg Latency: 1.103766ms
42 Bottom 10% Avg Latency: 537.227µs
43 Avg Latency: 973.956µs
44 P95 Latency: 1.895983ms
45 P95 Latency: 1.938364ms
46 P95 Latency: 839.77µs
47
48 Relay: next-orly-neo4j
49 ----------------------------------------
50 Status: COMPLETED
51 Events/sec: 15536.46
52 Events/sec: 6269.18
53 Events/sec: 15536.46
54 Success Rate: 100.0%
55 Success Rate: 100.0%
56 Success Rate: 100.0%
57 Avg Latency: 1.414281ms
58 Bottom 10% Avg Latency: 704.384µs
59 Avg Latency: 919.794µs
60 P95 Latency: 2.486204ms
61 P95 Latency: 1.842478ms
62 P95 Latency: 828.598µs
63
64 Relay: khatru-sqlite
65 ----------------------------------------
66 Status: COMPLETED
67 Events/sec: 17237.90
68 Events/sec: 6137.41
69 Events/sec: 17237.90
70 Success Rate: 100.0%
71 Success Rate: 100.0%
72 Success Rate: 100.0%
73 Avg Latency: 1.195398ms
74 Bottom 10% Avg Latency: 614.1µs
75 Avg Latency: 967.476µs
76 P95 Latency: 2.00684ms
77 P95 Latency: 2.046996ms
78 P95 Latency: 843.455µs
79
80 Relay: khatru-badger
81 ----------------------------------------
82 Status: COMPLETED
83 Events/sec: 16911.23
84 Events/sec: 6231.83
85 Events/sec: 16911.23
86 Success Rate: 100.0%
87 Success Rate: 100.0%
88 Success Rate: 100.0%
89 Avg Latency: 1.187112ms
90 Bottom 10% Avg Latency: 540.572µs
91 Avg Latency: 957.9µs
92 P95 Latency: 2.183304ms
93 P95 Latency: 1.888493ms
94 P95 Latency: 824.399µs
95
96 Relay: relayer-basic
97 ----------------------------------------
98 Status: COMPLETED
99 Events/sec: 17836.39
100 Events/sec: 6270.82
101 Events/sec: 17836.39
102 Success Rate: 100.0%
103 Success Rate: 100.0%
104 Success Rate: 100.0%
105 Avg Latency: 1.081434ms
106 Bottom 10% Avg Latency: 525.619µs
107 Avg Latency: 951.65µs
108 P95 Latency: 1.853627ms
109 P95 Latency: 1.779976ms
110 P95 Latency: 831.883µs
111
112 Relay: strfry
113 ----------------------------------------
114 Status: COMPLETED
115 Events/sec: 16470.06
116 Events/sec: 6004.96
117 Events/sec: 16470.06
118 Success Rate: 100.0%
119 Success Rate: 100.0%
120 Success Rate: 100.0%
121 Avg Latency: 1.261656ms
122 Bottom 10% Avg Latency: 566.551µs
123 Avg Latency: 1.02418ms
124 P95 Latency: 2.241835ms
125 P95 Latency: 2.314062ms
126 P95 Latency: 821.493µs
127
128 Relay: nostr-rs-relay
129 ----------------------------------------
130 Status: COMPLETED
131 Events/sec: 16764.35
132 Events/sec: 6300.71
133 Events/sec: 16764.35
134 Success Rate: 100.0%
135 Success Rate: 100.0%
136 Success Rate: 100.0%
137 Avg Latency: 1.245012ms
138 Bottom 10% Avg Latency: 614.335µs
139 Avg Latency: 869.47µs
140 P95 Latency: 2.151312ms
141 P95 Latency: 1.707251ms
142 P95 Latency: 816.334µs
143
144
145 ================================================================
146 DETAILED RESULTS
147 ================================================================
148
149 Individual relay reports are available in:
150 - /reports/run_20251119_114143/khatru-badger_results.txt
151 - /reports/run_20251119_114143/khatru-sqlite_results.txt
152 - /reports/run_20251119_114143/next-orly-badger_results.txt
153 - /reports/run_20251119_114143/next-orly-dgraph_results.txt
154 - /reports/run_20251119_114143/next-orly-neo4j_results.txt
155 - /reports/run_20251119_114143/nostr-rs-relay_results.txt
156 - /reports/run_20251119_114143/relayer-basic_results.txt
157 - /reports/run_20251119_114143/strfry_results.txt
158
159 ================================================================
160 BENCHMARK COMPARISON TABLE
161 ================================================================
162
163 Relay Status Peak Tput/s Avg Latency Success Rate
164 ---- ------ ----------- ----------- ------------
165 next-orly-badger OK 17949.86 1.089014ms 100.0%
166 next-orly-dgraph OK 17627.19 1.103766ms 100.0%
167 next-orly-neo4j OK 15536.46 1.414281ms 100.0%
168 khatru-sqlite OK 17237.90 1.195398ms 100.0%
169 khatru-badger OK 16911.23 1.187112ms 100.0%
170 relayer-basic OK 17836.39 1.081434ms 100.0%
171 strfry OK 16470.06 1.261656ms 100.0%
172 nostr-rs-relay OK 16764.35 1.245012ms 100.0%
173
174 ================================================================
175 End of Report
176 ================================================================
177