עב
עב

NVIDIA GeForce RTX 5090: specs and which LLMs fit

The NVIDIA GeForce RTX 5090 as its official source states it, which open models fit on it, and how it compares with the GPUs next to it in memory. Every figure links to its source and the day it was read.

Specifications

From the vendor's own source. Tensor figures are dense, without sparsity.

Memory
32 GB GDDR7Source ·
Memory bandwidth
1,792 GB/sSource ·
FP16 Tensor, dense
209.5 TFLOPSSource ·
BF16 Tensor, dense
209.5 TFLOPSSource ·
FP8 Tensor, dense
419 TFLOPSSource ·
INT8 Tensor, dense
838 TOPSSource ·
Max board power
575 WSource ·
Architecture
BlackwellSource ·
Segment
Consumer (GeForce)Source ·
Form factor
PCIe cardSource ·

How the figures were stored

The whitepaper lists dense and sparse figures; the dense ones are here, FP16, BF16 and FP8 with FP32 accumulate, comparable to data-center figures. Board power is the total graphics power (TGP).

Which open models fit

On one NVIDIA GeForce RTX 5090, 4 of the 10 models fit in bf16 as published and 8 in int4, with this workload.

Estimate An estimate from the GPU Planner's calculator: one request of 8,192 tokens, KV cache in bf16, served with vLLM, with 90% of each GPU's memory usable. Each cell is how many of this GPU the model needs with tensor parallelism; tight means only the optimistic estimate fits.

Open models and the GPUs they need, by weight format
Modelbf16 (as published)fp8int4
Qwen2.5 7B InstructFits on 1Fits on 1Fits on 1
Qwen2.5 14B InstructFits on 2Fits on 1Fits on 1
Qwen2.5 32B InstructFits on 4Fits on 2Fits on 1
Qwen2.5 72B InstructFits on 8Fits on 4Fits on 2
Qwen3 8BFits on 1Fits on 1Fits on 1
Qwen3 32BFits on 4Fits on 2Fits on 1
DeepSeek-R1-Distill-Llama 8BFits on 1Fits on 1Fits on 1
DeepSeek-R1-Distill-Llama 70BFits on 8Fits on 4Fits on 2
Phi-4Fits on 2Fits on 1Fits on 1
Mistral 7B Instruct v0.3Fits on 1Fits on 1Fits on 1

Assumptions

Every number the result uses that you didn't enter. Change any of them to match your case.

  • %

    vLLM's default gpu_memory_utilization; the rest is left to the driver and other processes.

    Source ·
  • Bits per weight in bf16Assumption
    16 bits (Fixed definition)

    16 bits a weight: the format the models are published in.

    No source: this is our own assumption. Change it if you know your case.
  • Bits per weight in fp8Assumption
    8 bits (Fixed definition)

    8 bits a weight; the per-channel or per-block scales add under 0.1 bit.

    No source: this is our own assumption. Change it if you know your case.
  • Bits per weight in int4Assumption
    4.25 bits (Fixed definition)

    4 bits a weight plus a 16-bit scale and a 4-bit zero per 128 weights (AWQ / GPTQ), about 4.16, rounded up to 4.25 for packing.

    No source: this is our own assumption. Change it if you know your case.
  • Bits per embedding weight, cautiousAssumption
    16 bits (Fixed definition)

    Published quantized checkpoints keep the embedding and output matrices at 16 bits; the cautious end counts them so.

    No source: this is our own assumption. Change it if you know your case.
  • Runtime overhead per GPU, fixed, optimisticAssumption
    0.75 GB (Fixed definition)

    CUDA context, the serving framework and activation workspace on each GPU: a fixed part and a part that grows with the weights it holds. A declared range, not a measurement.

    No source: this is our own assumption. Change it if you know your case.
  • Runtime overhead per GPU, fixed, cautiousAssumption
    1.5 GB (Fixed definition)

    CUDA context, the serving framework and activation workspace on each GPU: a fixed part and a part that grows with the weights it holds. A declared range, not a measurement.

    No source: this is our own assumption. Change it if you know your case.
  • Runtime overhead, share of weights, optimisticAssumption
    0.03 (Fixed definition)

    CUDA context, the serving framework and activation workspace on each GPU: a fixed part and a part that grows with the weights it holds. A declared range, not a measurement.

    No source: this is our own assumption. Change it if you know your case.
  • Runtime overhead, share of weights, cautiousAssumption
    0.1 (Fixed definition)

    CUDA context, the serving framework and activation workspace on each GPU: a fixed part and a part that grows with the weights it holds. A declared range, not a measurement.

    No source: this is our own assumption. Change it if you know your case.

What you enter and the assumptions you change are kept in this page's link, after the #.

Compared with the GPUs next to it

The nearest GPUs in the dataset by memory, with the same figures from their own sources.

Compared with the GPUs next to it
GPUMemoryBandwidthBF16, denseFP8, denseBoard powerSource
NVIDIA GeForce RTX 409024 GB GDDR6X1,008 GB/s165.2 TFLOPS330.3 TFLOPS450 WSource ·
NVIDIA GeForce RTX 5090 This GPU32 GB GDDR71,792 GB/s209.5 TFLOPS419 TFLOPS575 WSource ·
NVIDIA L40S48 GB GDDR6864 GB/s362.05 TFLOPS733 TFLOPS350 WSource ·

Plan your own case

More requests at once, a longer context or another model change the answer. The GPU Planner works it out with this GPU already picked, and writes the vLLM or Ollama command.

Open the GPU Planner with the NVIDIA GeForce RTX 5090 picked

Limits

  • The figures are the vendor's, not measurements; the fit is an estimate of the site's GPU engine, not a benchmark.
  • The fit table covers dense open models of the dataset only; mixture-of-experts models are not in it.
  • No price, availability or offer is part of this page.

Sources

Model configurations

GPU engine @anaroq/gpu 1.0.0 · GPU data 2026.10.0 · model data 2026.10.0

How the estimates are worked out: Methodology

All GPUs compared