QA Knowledge Hub

Benchmarks Overview

How benchmarks should be presented without overclaiming.

Benchmarks Overview

Benchmarks should be presented as questions, not only as numbers:

  • what was the test trying to show
  • which data path was measured
  • with what data size
  • what was not tested

Recommendation

Each benchmark should keep these separate:

  1. assumptions
  2. results
  3. limitations

Core lesson

The most important benchmark rule is now clear:

bytesRead alone is not enough. In R2 latency, rangeReads is equally important and often more important.

Strongest current result

According to the report, the 10M-row JRI4 remote R2 + Worker object produced six working query shapes from a real 346 MiB object:

materialized_top   rangeReads=4   bytesRead=2279    durationMs=176
indexed_top        rangeReads=6   bytesRead=10627   durationMs=238
source_filter      rangeReads=4   bytesRead=2259    durationMs=142
timestamp_range    rangeReads=4   bytesRead=2459    durationMs=148
source_index       rangeReads=7   bytesRead=14543   durationMs=261
timestamp_index    rangeReads=8   bytesRead=16791   durationMs=284

What this shows

  • object size can grow large without bytes read growing linearly
  • query shape strongly affects range-read count
  • covering and indexed paths must be treated separately

On this page