Two figures decide most of what a GPU can do with a language model: its memory, which bounds the model and the context it can hold, and its memory bandwidth, which bounds how fast it generates tokens. Tensor throughput matters when many requests are served at once.
The table lists only GPUs with an official vendor source. Tensor figures are dense, without sparsity, so GPUs from different datasheets can be compared; each GPU's page says how its published figures were turned into these.
Every GPU in the dataset
Ordered by memory. A figure read more than 180 days ago is marked stale. Each GPU links to its page.
| GPU | Memory | Bandwidth | BF16, dense | FP8, dense | Board power | Source |
|---|---|---|---|---|---|---|
| NVIDIA L4 | 24 GB GDDR6 | 300 GB/s | 121 TFLOPS | 242.5 TFLOPS | 72 W | |
| NVIDIA GeForce RTX 4090 | 24 GB GDDR6X | 1,008 GB/s | 165.2 TFLOPS | 330.3 TFLOPS | 450 W | |
| NVIDIA GeForce RTX 5090 | 32 GB GDDR7 | 1,792 GB/s | 209.5 TFLOPS | 419 TFLOPS | 575 W | |
| NVIDIA L40S | 48 GB GDDR6 | 864 GB/s | 362.05 TFLOPS | 733 TFLOPS | 350 W | |
| NVIDIA A100 80GB PCIe | 80 GB HBM2e | 1,935 GB/s | 312 TFLOPS | No FP8 Tensor Cores | 300 W | |
| NVIDIA A100 80GB SXM | 80 GB HBM2e | 2,039 GB/s | 312 TFLOPS | No FP8 Tensor Cores | 400 W | |
| NVIDIA H100 SXM | 80 GB HBM3 | 3,350 GB/s | 989.5 TFLOPS | 1,979 TFLOPS | 700 W | |
| NVIDIA H100 NVL | 94 GB HBM3 | 3,900 GB/s | 835.5 TFLOPS | 1,670.5 TFLOPS | 400 W | |
| NVIDIA H200 SXM | 141 GB HBM3e | 4,800 GB/s | 989.5 TFLOPS | 1,979 TFLOPS | 700 W | |
| NVIDIA B200 | 180 GB HBM3e | 8,000 GB/s | 2,250 TFLOPS | 4,500 TFLOPS | Not published per GPU |
Reading the table
An H100 SXM and an H200 SXM have the same Tensor throughput, but the H200 SXM has 141 GB at 4,800 GB/s against 80 GB at 3,350 GB/s: it holds larger models and generates faster on the same model.
Limits
- Datasheet figures, not measurements: real throughput depends on the engine, the kernels and the workload.
- NVIDIA GPUs only, as the dataset holds them today; a GPU without an official source has no page.
- No price, availability or offer is part of these pages.
Sources
- NVIDIA L4 Tensor Core GPU: Product Specifications
- NVIDIA Ada GPU Architecture, Appendix A: GeForce RTX 4090 Full Specifications
- NVIDIA RTX Blackwell GPU Architecture, Appendix A: GeForce RTX 5090
- NVIDIA L40S GPU Specifications
- NVIDIA A100 Tensor Core GPU: Specifications
- NVIDIA H100 Tensor Core GPU: Product Specifications
- NVIDIA H200 Tensor Core GPU: Specifications
- NVIDIA HGX Platform: HGX B200 Specifications
- NVIDIA DGX B200 Specifications
GPU engine @anaroq/gpu 1.0.0 · GPU data 2026.10.0 · model data 2026.10.0
How the estimates are worked out: Methodology