Methodology

What counts as a source

A source is a URL that reports a measurement, together with the author who measured it and the date they did. A source that does not state the quantisation and the context length it used is not usable here, because a tok/s figure without those is not comparable to anything.

Two sources are the same source when they share an author, regardless of which host carries the write-up: one person cross-posting a single run is one measurement, not two independent confirmations. Keying on the author alone can under-count independence, but it cannot manufacture agreement out of a repost. A press article restating a vendor’s own number is recorded as a vendor source, not an independent one.

How confidence is computed

Confidence is never typed by hand. It is derived from the sources and checked in CI, which fails the build when a committed value disagrees with the computed one.

The rule is applied per metric, not once per record, because a record can be well-supported on one number and thin on another:

• Any metric whose independent sources disagree by more than 25% makes the whole record disputed, and both figures are shown.
• Otherwise, if a primary metric — decode_tps or prefill_tps — has only one source behind it, the record is single-source.
• A supplementary metric with only one source — ttft_ms, power_w — does not demote the record. A benchmark with two agreeing decode-speed sources and a single power reading is still verified.

A disputed record also carries a spread_cause where one is known — for example, “differing backend versions” — so a reader can tell explained disagreement from unexplained disagreement.

The published figure for a row is the median of the independent source values, not a pick.

Why dates matter, and the 270-day rule

The same silicon gets faster as backends improve, so a measurement is only meaningful with a date attached. Sources inside one record must be measured within 270 days of each other; further apart, they are measuring different software and are split into separate records rather than being allowed to “confirm” each other. This is why a hardware page can show speed rising over time on unchanged hardware.

Memory footprint

weights_gb  = params_b x (bits_per_weight / 8)
kv_cache_gb = kv_bytes_per_token x context_length     (halved at Q8 KV)
overhead_gb = 0.8
footprint   = weights_gb + kv_cache_gb + overhead_gb

Fit is judged with 10% headroom on top of that.

What this dataset cannot tell you

Almost every measurement here is short-context. Most of the corpus comes from llama-bench, and the builds used to produce it have no context-size flag at all — its runs are a fixed 512-token prompt and 128-token generation. Records derived from it record context_len: 512 by that convention, not because any source stated a context length. That means this site is strong on short-context throughput and close to silent on long-context behaviour. Treat every number here as “at 512 tokens of context” even where the record does not say so out loud.

Laptop numbers rarely state a power profile. The same laptop GPU at 80W and at 175W performs very differently, and most sources this site draws from do not say which power limit was active. Where a source does state one, the power limit is recorded on the record and each limit gets its own row, because they are different runs. Treat every laptop figure without a stated power limit as measured under an unstated, possibly favourable, one.

Some KV-cache figures are deliberate upper bounds, not measurements. Gemma 3 and gpt-oss use sliding-window attention, and the recorded kv_bytes_per_token for them is a conservative over-estimate rather than the true windowed figure — so their memory footprints on this site are pessimistic, and a real deployment may fit in less. DeepSeek V3 and R1 use Multi-head Latent Attention, where the standard per-token KV formula overstates the cache by roughly 25x; a model-specific figure is used for them instead.

Partial-offload records are not memory-checked. offload: partial means some layers live in system RAM, and this project does not model system RAM capacity, so there is no honest bound to check a partial-offload record against. The physics ceiling on decode speed still applies, but the fit-versus- footprint check does not run on them.

Quantisation bits-per-weight are measured, not nominal. Q4_K_M is recorded at 4.90 bits per weight on this site, not the commonly quoted 4.5. The recorded figure is measured directly — GGUF file size in bytes, times 8, divided by the exact parameter count — rather than assumed from the format name, which ignores the higher-precision tensors k-quant formats mix in.

Some hardware classes have no verified record at all. CPU-only builds and multi-GPU builds are, in this dataset, entirely single-source: every such configuration was measured by exactly one person. That is a gap in coverage, not a claim that those classes are unreliable.

What we rejected

Nine records were removed from this dataset after re-verification against raw source text showed they were fabricated or misattributed — including figures attributed to a named author on a date when no comment by that author existed at all. One source site was excluded entirely after its claims contradicted a verifiable primary source. Vendor capability claims — a TOPS rating on a spec sheet, “runs 200B models” in marketing copy — are never recorded as benchmarks here.

Estimates

Decode speed is bounded by reading the active weights from memory once per token:

ceiling_tps = memory_bandwidth / (active_params x bits_per_weight / 8)

Real backends reach some fraction of that ceiling. That fraction is fitted from the verified records by least squares through the origin, separately for each backend and hardware class, and published below with its mean residual. A bucket with fewer than three distinct verified configurations produces no estimate at all.

The fitted efficiencies rest on a narrow base. The table above is recomputed from the live dataset on every build. As of this build it holds 4 buckets: llama.cpp|gpu at an efficiency of 0.548, fitted on 17 verified records covering 15 distinct configurations, with a 20.6% mean residual; llama.cpp|soc at an efficiency of 0.443, fitted on 5 verified records covering 3 distinct configurations, with a 17.9% mean residual; llama.cpp|mac at an efficiency of 0.629, fitted on 8 verified records covering 8 distinct configurations, with a 12.4% mean residual; and llama.cpp|apu at an efficiency of 0.505, fitted on 3 verified records covering 3 distinct configurations, with a 22.7% mean residual.

The llama.cpp|gpu fit rests on 2 model and quantisation combinations: Llama 2 7B at Q4_0 and Qwen3.8 27B at Q4_K_M. The llama.cpp|soc fit rests on 2 model and quantisation combinations: gpt-oss-20b at MXFP4 and gpt-oss-120b at MXFP4. The llama.cpp|mac fit rests on 3 model and quantisation combinations: Llama 2 7B at Q4_0, Llama 2 7B at F16 and Llama 2 7B at Q8_0. The llama.cpp|apu fit rests on 3 model and quantisation combinations: Llama 2 7B at Q4_0, gpt-oss-120b at F16 and gpt-oss-120b at MXFP4. Applying an efficiency fitted on that to a very different model, or to a long context, is extrapolation past the evidence that produced it, not interpolation within it.

So estimates are produced only inside the fitted range, on both inputs. A fitted efficiency is applied only within the span of active parameter counts and the span of memory bandwidths it was actually measured over, each widened by a factor of two on each end. Both inputs are guarded by the same rule because both are what the ceiling is computed from: how many bytes must be read per token, and how fast they can be read. Extrapolating on either one carries the fit into a regime it was never measured in — below roughly a couple of billion active parameters batch-1 decode is dominated by per-layer kernel launch latency rather than by memory bandwidth, and a card several times faster than anything in the fit is no more inside the evidence than a model several times larger. Outside these bounds this site publishes nothing, rather than a number that could be wrong by an order of magnitude.

• llama.cpp|gpu: models with 3.37–55.56 billion active parameters (fitted over 6.74 to 27.78 B), on hardware with 144–3870 GB/s of memory bandwidth (fitted over 288 to 1935 GB/s).
• llama.cpp|soc: models with 1.8–10.2 billion active parameters (fitted over 3.6 to 5.1 B), on hardware with 136.5–546 GB/s of memory bandwidth (every fitted point sits at 273 GB/s, so this bound comes entirely from the assumed factor of two and not from any spread in the data).
• llama.cpp|mac: models with 3.37–13.48 billion active parameters (every fitted point sits at 6.74 B, so this bound comes entirely from the assumed factor of two and not from any spread in the data), on hardware with 200–820 GB/s of memory bandwidth (fitted over 400 to 410 GB/s).
• llama.cpp|apu: models with 2.55–13.48 billion active parameters (fitted over 5.1 to 6.74 B), on hardware with 128–512 GB/s of memory bandwidth (every fitted point sits at 256 GB/s, so this bound comes entirely from the assumed factor of two and not from any spread in the data).

A bucket that has not reached three distinct verified configurations produces no estimate at all, by design.

Estimated rows do not know about context length at all. The bandwidth model predicts the same decode rate no matter what context you ask it about. Real long-context performance is worse than that, on every backend. Every estimated row says this on the row itself.

An estimate never outranks a measurement anywhere on this site.

Fitted efficiency, current dataset

Dataset built 2026-08-24T14:21:55.479Z.