Intel Xeon 6980P (single socket of a dual-socket box, SNC=Disable, DDR5-6400)
Reference: Xeon on Wikipedia
- Class
- CPU
- Memory
- 768 GB DDR5-6400 RDIMM (Micron MTC40F2046S1RC64BDY), 24 x 64GB = 1.5 TB across two sockets
- Runs (Q4_K_M)
- —
- Bandwidth
- 564.2 GB/s
- Released
- 2024-09-24
Sourcing and disambiguation notes
Represents ONE socket of a dual-socket 6980P server with 128 P-cores per socket and 1.5TB of RAM, because every CPU-only run on this box was pinned to a single socket with `numactl -N 0 -m 0` and SNC disabled. The bandwidth is MEASURED with Intel MLC on this exact machine rather than taken from spec: 564.2 GB/s per socket, cross-checked against half the whole-system figure of 1126.0 GB/s. Twelve channels of DDR5-6400 would be 614.4 GB/s theoretical, so the measurement is 92% of spec. The memory size is half the box's 1.5TB and the per-socket channel count follows from dividing 24 DIMMs across two sockets -- both are inference rather than anything the source states, which is why the channel count is kept out of this entry's id and name. No price researched.
Specification sources
- https://github.com/ggml-org/llama.cpp/discussions/12088 — forum, 2025-03-18
Measurements
Decode is token generation — the speed you feel while an answer streams. Prefill is prompt processing — the wait before it starts. Why bandwidth predicts decode speed.
No records yet for this hardware. Know of a published benchmark on it? Submit the link.
What verified, single-source and estimated mean, and the same rows with every filter and sort in the benchmarks explorer.
Speed over time
What it can run
Fit is arithmetic, not a measurement — how it is computed. Against 768 GB; models that fit are listed largest first.
No modeled quant fits in 768 GB.