עב
עב

LLM cost calculator: self-host vs API

Enter your tokens a month, the API you would pay and the open model and GPU you would rent. You get both monthly costs, the volume from which self-hosting stops costing more, and every price with its source and the day it was read.

Workload
Prompt tokens, including system prompt and retrieved documents.
Tokens the model generates.
Model API
A dated price from the provider's page, or your own. Pick the API that serves the same open model to compare like with like.
Self-host on rented GPUs
An open model of ANAROQ's GPU engine, the same list as the LLM VRAM explainer.
Lower formats need fewer GPUs and decode faster, with some loss of quality.
A dated on-demand price from the provider's page, or a GPU and price of your own.
More sequences at once raise the total tokens a second, and the KV cache to hold.
Prompt + output one sequence holds; the KV cache is reserved for it.
How the server runs
Always on: whole servers billed all month. On demand: you pay only the hours the work needs.
0 leaves the upkeep hours out of the money.

Estimate An estimate from dated prices and stated assumptions, not a quote: the throughput is a range from datasheet figures, not a benchmark.

Result

API per month
$272/month
Self-host per month
$3,151.7/month
API minus self-host
-$2,879.7/month
Tokens a month
2,400 million tokens/month
GPUs per server
1
Servers always on
1
GPU-hours paid a month
730 hours/month
Output throughput per server
625 tokens/s to 1,037 tokens/s
Input (prefill) throughput per server
9,061 tokens/s to 15,101 tokens/s
Breakdown
LineValueAssumptions used
API: input tokens$160/month
  • DeepInfra Qwen3-32B: input price
API: output tokens$112/month
  • DeepInfra Qwen3-32B: output price
GPU memory per GPU (weights, KV cache, overhead)51.67 GB to 56.43 GB
    Busy hours: prefill + decode143.94 hours/month to 239.21 hours/month
    • Memory bandwidth reached, cautious (Assumption)
    • Memory bandwidth reached, optimistic (Assumption)
    • Compute reached, cautious (Assumption)
    • Compute reached, optimistic (Assumption)
    Server hours paid, at the utilization730 hours/month
    • Memory bandwidth reached, cautious (Assumption)
    • Memory bandwidth reached, optimistic (Assumption)
    • Compute reached, cautious (Assumption)
    • Compute reached, optimistic (Assumption)
    • Average GPU utilization (Assumption)
    • Hours in a month (Assumption)
    Self-host: GPUs$3,131.7/month
    • Memory bandwidth reached, cautious (Assumption)
    • Memory bandwidth reached, optimistic (Assumption)
    • Compute reached, cautious (Assumption)
    • Compute reached, optimistic (Assumption)
    • Average GPU utilization (Assumption)
    • Hours in a month (Assumption)
    • Lambda NVIDIA H100 SXM (1× instance): price per GPU-hour
    Self-host: storage and network$20/month
    • Storage and network a month (Assumption)
    Self-host: upkeep time$0/month
    • Upkeep hours a month (Assumption)
    Self-host per month$3,151.7/month
    • Memory bandwidth reached, cautious (Assumption)
    • Memory bandwidth reached, optimistic (Assumption)
    • Compute reached, cautious (Assumption)
    • Compute reached, optimistic (Assumption)
    • Average GPU utilization (Assumption)
    • Hours in a month (Assumption)
    • Lambda NVIDIA H100 SXM (1× instance): price per GPU-hour
    • Storage and network a month (Assumption)
    • Upkeep hours a month (Assumption)
    API per month$272/month
    • DeepInfra Qwen3-32B: input price
    • DeepInfra Qwen3-32B: output price

    Calculator llm-cost-self-host-vs-api 1.0.0 · engine @anaroq/calc 1.4.0 · data snapshot 2026.10.4 · How the calculators work

    No break-even: at these prices the API costs less at any volume, because a server's tokens cost more than the API's.

    Assumptions

    Every number the result uses that you didn't enter. Change any of them to match your case.

    • Assumption
      hours/month

      8,760 hours a year divided by 12, the month GPU clouds bill an always-on server by; ÷ 24 gives the days for requests a day.

      No source: this is our own assumption. Change it if you know your case.
    • Assumption
      %

      Traffic is uneven and a server is sized for its peaks, so on average it runs below the throughput it can reach; 50% is our assumption for a steady workload.

      No source: this is our own assumption. Change it if you know your case.
    • Assumption
      USD/month

      Disk for the weights and logs, and outbound traffic, around the GPU price; our assumption, change it to your provider's.

      No source: this is our own assumption. Change it if you know your case.
    • Assumption
      hours/month

      Updating the serving stack and the model, monitoring and incidents; an API has none of these. Our assumption; it costs money only with an hourly cost.

      No source: this is our own assumption. Change it if you know your case.
    • Memory bandwidth reached, cautiousAssumption
      0.5 (Fixed definition)

      Share of the GPU's datasheet memory bandwidth real decode kernels reach: @anaroq/gpu's range, the one the GPU Planner shows.

      No source: this is our own assumption. Change it if you know your case.
    • Memory bandwidth reached, optimisticAssumption
      0.8 (Fixed definition)

      Share of the GPU's datasheet memory bandwidth real decode kernels reach: @anaroq/gpu's range, the one the GPU Planner shows.

      No source: this is our own assumption. Change it if you know your case.
    • Compute reached, cautiousAssumption
      0.3 (Fixed definition)

      Share of the GPU's peak tensor throughput reached when compute-bound (prefill, large batches): @anaroq/gpu's range.

      No source: this is our own assumption. Change it if you know your case.
    • Compute reached, optimisticAssumption
      0.5 (Fixed definition)

      Share of the GPU's peak tensor throughput reached when compute-bound (prefill, large batches): @anaroq/gpu's range.

      No source: this is our own assumption. Change it if you know your case.
    • USD/million tokens

      The price per million input tokens as the provider lists it. Change it to the price you pay.

      Source · · data 2026.10.4
    • USD/million tokens

      The price per million output tokens as the provider lists it. Change it to the price you pay.

      Source · · data 2026.10.4
    • USD/GPU-hour

      The on-demand price per GPU-hour of a 1-GPU instance, as the provider lists it; other sizes may be priced differently.

      Source · · data 2026.10.4

    What you enter and the assumptions you change are kept in this page's link, after the #.

    Prices used

    Each price with the page it was read from and the day; a price read more than 90 days ago is marked stale.

    • DeepInfra Qwen3-32B: input, per million tokens: $0.08/million tokensSource ·
    • DeepInfra Qwen3-32B: output, per million tokens: $0.28/million tokensSource ·
    • Lambda NVIDIA H100 SXM: per GPU-hour: $4.29/GPU-hourSource ·

    What the money does not show

    • Quality: the API and the open model you run may not answer equally well; this page does not compare them.
    • Latency and limits: an API has rate limits and its own queue; a server of your own has its own peaks.
    • Data: where prompts are processed and kept, which matters for privacy and compliance.
    • Discounts: prompt caching, batch APIs, reserved or spot GPUs change both sides.

    Take the result with you

    The result as a Markdown file: what you entered, the breakdown, every price with its source and date, every assumption, and the link that reproduces it.

    llm-cost.md
    # LLM cost calculator: self-host vs API

An estimate from the assumptions listed below, not a measurement or a guarantee.

No source: this is our own assumption. Change it if you know your case.

## What you entered

- Count the workload as: Tokens a month
- Input tokens a month (millions): 2,000
- Output tokens a month (millions): 400
- API price: DeepInfra: Qwen3-32B
- Open model: Qwen3 32B
- Weight format: fp8
- GPU price: Lambda: NVIDIA H100 SXM (1× instance)
- Sequences at once: 16
- Context per sequence (tokens): 4,096
- Schedule: Always on (24/7)
- Cost of an hour of upkeep (USD): 0 USD

## Result

- API per month: $272/month
- Self-host per month: $3,151.7/month
- API minus self-host: -$2,879.7/month
- Tokens a month: 2,400 million tokens/month
- GPUs per server: 1
- Servers always on: 1
- GPU-hours paid a month: 730 hours/month
- Output throughput per server: 625 tokens/s to 1,037 tokens/s
- Input (prefill) throughput per server: 9,061 tokens/s to 15,101 tokens/s

> Estimate: An estimate from dated prices and stated assumptions, not a quote: the throughput is a range from datasheet figures, not a benchmark.

> No break-even: at these prices the API costs less at any volume, because a server's tokens cost more than the API's.

## What the money does not show

- Quality: the API and the open model you run may not answer equally well; this page does not compare them.
- Latency and limits: an API has rate limits and its own queue; a server of your own has its own peaks.
- Data: where prompts are processed and kept, which matters for privacy and compliance.
- Discounts: prompt caching, batch APIs, reserved or spot GPUs change both sides.

## Breakdown

| Line | Value | Assumptions used |
| --- | --- | --- |
| API: input tokens | $160/month | DeepInfra Qwen3-32B: input price |
| API: output tokens | $112/month | DeepInfra Qwen3-32B: output price |
| GPU memory per GPU (weights, KV cache, overhead) | 51.67 GB to 56.43 GB | – |
| Busy hours: prefill + decode | 143.94 hours/month to 239.21 hours/month | Memory bandwidth reached, cautious, Memory bandwidth reached, optimistic, Compute reached, cautious, Compute reached, optimistic |
| Server hours paid, at the utilization | 730 hours/month | Memory bandwidth reached, cautious, Memory bandwidth reached, optimistic, Compute reached, cautious, Compute reached, optimistic, Average GPU utilization, Hours in a month |
| Self-host: GPUs | $3,131.7/month | Memory bandwidth reached, cautious, Memory bandwidth reached, optimistic, Compute reached, cautious, Compute reached, optimistic, Average GPU utilization, Hours in a month, Lambda NVIDIA H100 SXM (1× instance): price per GPU-hour |
| Self-host: storage and network | $20/month | Storage and network a month |
| Self-host: upkeep time | $0/month | Upkeep hours a month |
| Self-host per month | $3,151.7/month | Memory bandwidth reached, cautious, Memory bandwidth reached, optimistic, Compute reached, cautious, Compute reached, optimistic, Average GPU utilization, Hours in a month, Lambda NVIDIA H100 SXM (1× instance): price per GPU-hour, Storage and network a month, Upkeep hours a month |
| API per month | $272/month | DeepInfra Qwen3-32B: input price, DeepInfra Qwen3-32B: output price |

## Assumptions

- Hours in a month: 730 hours/month. 8,760 hours a year divided by 12, the month GPU clouds bill an always-on server by; ÷ 24 gives the days for requests a day. **Assumption**
- Average GPU utilization: 50%. Traffic is uneven and a server is sized for its peaks, so on average it runs below the throughput it can reach; 50% is our assumption for a steady workload. **Assumption**
- Storage and network a month: $20/month. Disk for the weights and logs, and outbound traffic, around the GPU price; our assumption, change it to your provider's. **Assumption**
- Upkeep hours a month: 4 hours/month. Updating the serving stack and the model, monitoring and incidents; an API has none of these. Our assumption; it costs money only with an hourly cost. **Assumption**
- Memory bandwidth reached, cautious: 0.5. Share of the GPU's datasheet memory bandwidth real decode kernels reach: @anaroq/gpu's range, the one the GPU Planner shows. **Assumption**
- Memory bandwidth reached, optimistic: 0.8. Share of the GPU's datasheet memory bandwidth real decode kernels reach: @anaroq/gpu's range, the one the GPU Planner shows. **Assumption**
- Compute reached, cautious: 0.3. Share of the GPU's peak tensor throughput reached when compute-bound (prefill, large batches): @anaroq/gpu's range. **Assumption**
- Compute reached, optimistic: 0.5. Share of the GPU's peak tensor throughput reached when compute-bound (prefill, large batches): @anaroq/gpu's range. **Assumption**
- DeepInfra Qwen3-32B: input price: $0.08/million tokens. The price per million input tokens as the provider lists it. Change it to the price you pay. Source: https://deepinfra.com/pricing (read 16 Oct 2026)
- DeepInfra Qwen3-32B: output price: $0.28/million tokens. The price per million output tokens as the provider lists it. Change it to the price you pay. Source: https://deepinfra.com/pricing (read 16 Oct 2026)
- Lambda NVIDIA H100 SXM (1× instance): price per GPU-hour: $4.29/GPU-hour. The on-demand price per GPU-hour of a 1-GPU instance, as the provider lists it; other sizes may be priced differently. Source: https://lambda.ai/pricing (read 13 Oct 2026)

Calculator llm-cost-self-host-vs-api 1.0.0 · engine @anaroq/calc 1.4.0 · data snapshot 2026.10.4

Open this calculation: https://anaroq.com/tools/llm-cost-self-host-vs-api#k=eyJhc3N1bXB0aW9ucyI6e30sImNhbGMiOiJsbG0tY29zdC1zZWxmLWhvc3QtdnMtYXBpIiwiaW5wdXQiOnsiYXBpIjoiYXBpLWRlZXBpbmZyYS1xd2VuMy0zMmIiLCJiYXRjaCI6MTYsImNvbnRleHRUb2tlbnMiOjQwOTYsImdwdSI6ImdwdS1sYW1iZGEtaDEwMC1zeG0tMXgiLCJob3VybHlSYXRlIjowLCJpbnB1dE10b2siOjIwMDAsImlucHV0VG9rZW5zUGVyUmVxdWVzdCI6MjAwMCwibWFudWFsR3B1IjoicnR4LTQwOTAiLCJtYW51YWxHcHVIb3VybHkiOjAuNSwibWFudWFsSW5wdXRQcmljZSI6MSwibWFudWFsT3V0cHV0UHJpY2UiOjQsIm1vZGVsIjoicXdlbjMtMzJiIiwib3V0cHV0TXRvayI6NDAwLCJvdXRwdXRUb2tlbnNQZXJSZXF1ZXN0Ijo0MDAsInF1YW50aXphdGlvbiI6ImZwOCIsInJlcXVlc3RzUGVyRGF5IjoyMDAwMCwic2NoZWR1bGUiOiJhbHdheXMtb24iLCJ2b2x1bWUiOiJ0b2tlbnMifSwidiI6MX0
How the calculators work: https://anaroq.com/methodology#calculators

    What it does

    It sets the monthly cost of a token workload on a model API next to running an open model on GPUs you rent by the hour. The API side is the tokens × the listed prices. The self-host side is the hours the GPUs need for those tokens, from ANAROQ's GPU engine, × the price per GPU-hour, plus storage, network and upkeep, which are stated as assumptions.

    How it works

    1. Tokens a month: typed, or requests a day × tokens per request × 730 ÷ 24 days.
    2. API = input millions × input price + output millions × output price, each price from the provider's own page with the day it was read, or your own.
    3. GPUs per server: the smallest number (1, 2, 4 or 8) on which the model's weights, the KV cache of the sequences at once and the runtime overhead fit, as ANAROQ's GPU engine estimates it (90% of each GPU's memory usable).
    4. Busy hours = output tokens ÷ decode throughput + input tokens ÷ prefill throughput. Decode is the GPU engine's estimate (50 to 80% of the memory bandwidth); prefill is compute-bound (30 to 50% of the peak tensor FLOPS ÷ 2 FLOPs per parameter). Both are ranges, so the cost is a range.
    5. At the average utilization (50% by default): always on, whole servers of 730 hours each; on demand, busy hours ÷ utilization. × GPUs × price per GPU-hour, + storage and network (20 USD by default) + upkeep hours × your hourly cost.
    6. Break-even: the tokens a month, at the same input:output mix, from which self-hosting costs no more than the API. None when a fully used server's tokens cost more than the API's.

    Example

    Qwen3-32B: 2,000 million input and 400 million output tokens a month on DeepInfra's API, against the same model in fp8 on an H100 from Lambda, always on:

    2,000 M input · 400 M output tokens/month · DeepInfra Qwen3-32B · Qwen3 32B fp8 · Lambda H100 SXM · 16 sequences × 4,096 tokens · always on

    API per month
    $272/month
    Self-host per month
    $3,151.7/month
    API minus self-host
    -$2,879.7/month
    Tokens a month
    2,400 million tokens/month
    GPUs per server
    1
    Servers always on
    1
    GPU-hours paid a month
    730 hours/month
    Output throughput per server
    625 tokens/s to 1,037 tokens/s
    Input (prefill) throughput per server
    9,061 tokens/s to 15,101 tokens/s

    No break-even: at these prices the API costs less at any volume, because a server's tokens cost more than the API's.

    Limits

    • An estimate, not a quote or a benchmark: real throughput depends on the serving software, the prompt lengths and the traffic pattern. Measure yours before you commit.
    • Prices are in USD as the providers list them, before tax, and change; each is dated, and marked stale after 90 days.
    • A price per GPU-hour is the one read for the instance size in its label; a provider may price or rent other sizes differently.
    • No quality comparison, and no ANAROQ price: ANAROQ does not sell GPUs or tokens.

    Methodology: how the numbers are worked out

    Sources

    1. ANAROQ calculator methodology: assumptions, units and rounding
    2. DeepInfra: Pricing
    3. Together AI: Pricing
    4. Anthropic: Pricing
    5. Lambda: GPU Cloud pricing

    Updated

    Next step

    How to choose a GPU server

    A free public tool by ANAROQ, a company building compute infrastructure. What ANAROQ is building