NVIDIA H100 NVL

Compare NVIDIA H100 NVL against another machine in the Arena

Reference: Hopper (microarchitecture) on Wikipedia

Class: GPU
Memory: 94 GB HBM3
Runs (Q4_K_M): —
Bandwidth: 3938 GB/s
TDP: 400 W
Released: 2023-03-21

Sourcing and disambiguation notes

Sold only as a factory NVLink-bridged pair of PCIe cards for large-model inference; the spec fields here are per-GPU (94GB HBM3, ~3.9TB/s each), matching how NVIDIA documents it. Deliberately left unpriced. Aggregator quotes came back at roughly $29,000 and $35,000, and earlier reseller quotes spanned $28k-$65k, but none of them states whether the figure covers one card or the bridged pair this part is only sold as -- a 2.4x swing depending on which is meant. The price stays out until a source states the unit.

Specification sources

https://www.nvidia.com/en-us/data-center/h100/ — vendor, 2023-03-21

Measurements

Decode is token generation — the speed you feel while an answer streams. Prefill is prompt processing — the wait before it starts. Why bandwidth predicts decode speed.

What verified, single-source and estimated mean, and the same rows with every filter and sort in the benchmarks explorer.

Speed over time

What it can run

Fit is arithmetic, not a measurement — how it is computed. Against 94 GB; models that fit are listed largest first.