aggregate_report.txt raw

   1  ================================================================
   2  NOSTR RELAY BENCHMARK AGGREGATE REPORT
   3  ================================================================
   4  Generated: 2025-11-19T12:08:43+00:00
   5  Benchmark Configuration:
   6    Events per test: 50000
   7    Concurrent workers: 24
   8    Test duration: 60s
   9    
  10  Relays tested: 8
  11  
  12  ================================================================
  13  SUMMARY BY RELAY
  14  ================================================================
  15  
  16  Relay: next-orly-badger
  17  ----------------------------------------
  18  Status: COMPLETED
  19  Events/sec: 17949.86
  20  Events/sec: 6293.77
  21  Events/sec: 17949.86
  22  Success Rate: 100.0%
  23  Success Rate: 100.0%
  24  Success Rate: 100.0%
  25  Avg Latency: 1.089014ms
  26  Bottom 10% Avg Latency: 552.633µs
  27  Avg Latency: 749.292µs
  28  P95 Latency: 1.801326ms
  29  P95 Latency: 1.544064ms
  30  P95 Latency: 797.32µs
  31  
  32  Relay: next-orly-dgraph
  33  ----------------------------------------
  34  Status: COMPLETED
  35  Events/sec: 17627.19
  36  Events/sec: 6241.01
  37  Events/sec: 17627.19
  38  Success Rate: 100.0%
  39  Success Rate: 100.0%
  40  Success Rate: 100.0%
  41  Avg Latency: 1.103766ms
  42  Bottom 10% Avg Latency: 537.227µs
  43  Avg Latency: 973.956µs
  44  P95 Latency: 1.895983ms
  45  P95 Latency: 1.938364ms
  46  P95 Latency: 839.77µs
  47  
  48  Relay: next-orly-neo4j
  49  ----------------------------------------
  50  Status: COMPLETED
  51  Events/sec: 15536.46
  52  Events/sec: 6269.18
  53  Events/sec: 15536.46
  54  Success Rate: 100.0%
  55  Success Rate: 100.0%
  56  Success Rate: 100.0%
  57  Avg Latency: 1.414281ms
  58  Bottom 10% Avg Latency: 704.384µs
  59  Avg Latency: 919.794µs
  60  P95 Latency: 2.486204ms
  61  P95 Latency: 1.842478ms
  62  P95 Latency: 828.598µs
  63  
  64  Relay: khatru-sqlite
  65  ----------------------------------------
  66  Status: COMPLETED
  67  Events/sec: 17237.90
  68  Events/sec: 6137.41
  69  Events/sec: 17237.90
  70  Success Rate: 100.0%
  71  Success Rate: 100.0%
  72  Success Rate: 100.0%
  73  Avg Latency: 1.195398ms
  74  Bottom 10% Avg Latency: 614.1µs
  75  Avg Latency: 967.476µs
  76  P95 Latency: 2.00684ms
  77  P95 Latency: 2.046996ms
  78  P95 Latency: 843.455µs
  79  
  80  Relay: khatru-badger
  81  ----------------------------------------
  82  Status: COMPLETED
  83  Events/sec: 16911.23
  84  Events/sec: 6231.83
  85  Events/sec: 16911.23
  86  Success Rate: 100.0%
  87  Success Rate: 100.0%
  88  Success Rate: 100.0%
  89  Avg Latency: 1.187112ms
  90  Bottom 10% Avg Latency: 540.572µs
  91  Avg Latency: 957.9µs
  92  P95 Latency: 2.183304ms
  93  P95 Latency: 1.888493ms
  94  P95 Latency: 824.399µs
  95  
  96  Relay: relayer-basic
  97  ----------------------------------------
  98  Status: COMPLETED
  99  Events/sec: 17836.39
 100  Events/sec: 6270.82
 101  Events/sec: 17836.39
 102  Success Rate: 100.0%
 103  Success Rate: 100.0%
 104  Success Rate: 100.0%
 105  Avg Latency: 1.081434ms
 106  Bottom 10% Avg Latency: 525.619µs
 107  Avg Latency: 951.65µs
 108  P95 Latency: 1.853627ms
 109  P95 Latency: 1.779976ms
 110  P95 Latency: 831.883µs
 111  
 112  Relay: strfry
 113  ----------------------------------------
 114  Status: COMPLETED
 115  Events/sec: 16470.06
 116  Events/sec: 6004.96
 117  Events/sec: 16470.06
 118  Success Rate: 100.0%
 119  Success Rate: 100.0%
 120  Success Rate: 100.0%
 121  Avg Latency: 1.261656ms
 122  Bottom 10% Avg Latency: 566.551µs
 123  Avg Latency: 1.02418ms
 124  P95 Latency: 2.241835ms
 125  P95 Latency: 2.314062ms
 126  P95 Latency: 821.493µs
 127  
 128  Relay: nostr-rs-relay
 129  ----------------------------------------
 130  Status: COMPLETED
 131  Events/sec: 16764.35
 132  Events/sec: 6300.71
 133  Events/sec: 16764.35
 134  Success Rate: 100.0%
 135  Success Rate: 100.0%
 136  Success Rate: 100.0%
 137  Avg Latency: 1.245012ms
 138  Bottom 10% Avg Latency: 614.335µs
 139  Avg Latency: 869.47µs
 140  P95 Latency: 2.151312ms
 141  P95 Latency: 1.707251ms
 142  P95 Latency: 816.334µs
 143  
 144  
 145  ================================================================
 146  DETAILED RESULTS
 147  ================================================================
 148  
 149  Individual relay reports are available in:
 150    - /reports/run_20251119_114143/khatru-badger_results.txt
 151    - /reports/run_20251119_114143/khatru-sqlite_results.txt
 152    - /reports/run_20251119_114143/next-orly-badger_results.txt
 153    - /reports/run_20251119_114143/next-orly-dgraph_results.txt
 154    - /reports/run_20251119_114143/next-orly-neo4j_results.txt
 155    - /reports/run_20251119_114143/nostr-rs-relay_results.txt
 156    - /reports/run_20251119_114143/relayer-basic_results.txt
 157    - /reports/run_20251119_114143/strfry_results.txt
 158  
 159  ================================================================
 160  BENCHMARK COMPARISON TABLE
 161  ================================================================
 162  
 163  Relay                Status     Peak Tput/s     Avg Latency     Success Rate   
 164  ----                 ------     -----------     -----------     ------------   
 165  next-orly-badger     OK         17949.86        1.089014ms      100.0%         
 166  next-orly-dgraph     OK         17627.19        1.103766ms      100.0%         
 167  next-orly-neo4j      OK         15536.46        1.414281ms      100.0%         
 168  khatru-sqlite        OK         17237.90        1.195398ms      100.0%         
 169  khatru-badger        OK         16911.23        1.187112ms      100.0%         
 170  relayer-basic        OK         17836.39        1.081434ms      100.0%         
 171  strfry               OK         16470.06        1.261656ms      100.0%         
 172  nostr-rs-relay       OK         16764.35        1.245012ms      100.0%         
 173  
 174  ================================================================
 175  End of Report
 176  ================================================================
 177