What this guide adds
The GPU Planner says which cards a model fits on, the LLM cost calculator says whether renting beats an API, and the GPU Price Index lists what single cards rent for. This guide puts them in the sequence a decision is made in, and adds what none of them covers: interconnect, the software stack, billing and data transfer.
Before you compare
The question is rarely which GPU is best. It is which is the smallest card the model fits on, at the context and concurrency you need, and whether renting it costs less than an API. That question has a sequence: model, memory, cost, then provider.
Every figure here comes from the site's data with its source and read date. The prose names no price and ranks no provider.
What to compare
- GPU memory comes first
- The weights, the key-value cache of the context and the runtime's own overhead all have to fit in GPU memory. A card with more compute and too little memory does not run the model at all.
- Weight format
- Quantized weights need less memory at some loss of quality. Decide the format before choosing the card; the GPU Planner compares formats.
- One card or several
- A model split across cards needs a fast link between them. Offers with several GPUs in one machine state their interconnect; a split across machines is slower still.
- Software stack
- Check the driver, the CUDA or ROCm version and the image your model server needs against what the offer provides, before you rent.
- Billing and stopping
- On-demand billing runs while the machine exists. Check the billing increment, whether a stopped machine still bills for its storage, and how fast it starts again.
- Moving data
- Model weights and datasets are large. Check what bringing them in and moving results out costs at the provider.
Measure with the tools
In this sequence. Each step is answered by a free tool on this site that works in your browser.
- Fit the model
Pick the model, the weight format, the context and the concurrent requests. The GPU Planner gives the memory needed, which cards it fits on and how many of each.
- Compare with an API
Enter your token volume, the API you would otherwise pay and the card you would rent. The calculator gives both monthly costs and the volume from which running the model yourself stops costing more.
- Check a CPU baseline
For a small model or light use, the Stack Sizer sizes Open WebUI with Ollama on CPU only. If that is enough, you need no GPU at all.
Tool: Stack Sizer
- Price the data transfer
Enter what you would send out of the provider, results and backups included, and what you would move out if you left. Weights and datasets make the exit cost real.
Tool: Cloud egress calculator
Dated prices from the Price Index
Single-GPU on-demand offers from the GPU Price Index sample, as listed in the dataset, with each card's memory. Read them only after the GPU Planner has told you how much memory the model needs; the full index adds multi-GPU offers and the price per GB of memory.
| Provider | GPU | Region | GPU memory (GB) | Listed price per GPU-hour |
|---|---|---|---|---|
| Lambda | NVIDIA H100 SXM | United States | 80 | $4.29 |
| Lambda | NVIDIA A100 PCIe 40 GB | United States | 40 | $1.99 |
| Lambda | NVIDIA B200 SXM6 | United States | 180 | $6.99 |
| DigitalOcean | NVIDIA HGX H100 | One price everywhere | 80 | $4.41 |
| DigitalOcean | NVIDIA HGX H200 | One price everywhere | 141 | $4.47 |
| DigitalOcean | NVIDIA L40S | One price everywhere | 48 | $1.57 |
| DigitalOcean | NVIDIA RTX 4000 Ada Generation | One price everywhere | 20 | $0.76 |
| Runpod | H100 SXM | One price everywhere | 80 | $3.99 |
| Runpod | H100 PCIe | One price everywhere | 80 | $2.89 |
| Runpod | A100 SXM | One price everywhere | 80 | $1.79 |
| Runpod | L40S | One price everywhere | 48 | $1.09 |
| Runpod | L4 | One price everywhere | 24 | $0.59 |
| Hyperstack | NVIDIA H200 SXM | One price everywhere | 141 | $3.99 |
| Hyperstack | NVIDIA H100 SXM | One price everywhere | 80 | $3.2 |
| Hyperstack | NVIDIA H100 | One price everywhere | 80 | $2.5 |
| Hyperstack | NVIDIA A100 SXM | One price everywhere | 80 | $1.6 |
| Hyperstack | NVIDIA B200 | One price everywhere | 192 | $6 |
| UpCloud | NVIDIA L4 | One price everywhere | 24 | €0.58 |
Prices as each provider lists them, in its currency, excluding taxes unless the row's source says otherwise. The full index converts them with dated rates.
The full index: GPU cloud price index
Common mistakes
- Choosing the newest card
- A newer card is often more than a model needs. The smallest card the model fits on, with room for the context, is where to start measuring.
- Forgetting the context
- Memory for a long context and many concurrent requests can exceed the weights themselves. Size with the context you will actually serve.
- Leaving it running
- An idle on-demand GPU bills like a busy one. Stop it when nothing runs, and check what a stopped machine still costs.
- Same name, different card
- Offers name cards differently, and the PCIe and SXM versions of one chip differ in memory bandwidth and interconnect. Read the variant, not only the name.
- Comparing hourly prices only
- An offer that starts slower or bills in larger increments can cost more for short jobs. Compare with your own pattern of use.
Limits
- The sample covers a few providers and on-demand offers only: no reserved, spot or committed prices.
- Throughput was not measured here: the GPU Planner's figure is an estimate, and only your own prompts give a real one.
- Prices change: every row shows the day it was read and is marked stale after the refresh window. The provider's page is authoritative.
On the /cloud page
The /cloud page for this kind of compute lists what to check before you commit, and describes the principles ANAROQ is building to, with its In development status.