GPU Count for Throughput Calculator
Find how many GPUs you need for a target tokens/sec.
Results update automatically as you type.
Result
7 GPU(s) needed
for 500 tok/s at 80 tok/s each
- Target throughput
- 500 tok/s
- Per-GPU throughput
- 80 tok/s
- GPUs needed
- 7
Enter your target total throughput and the tokens/sec a single GPU delivers to see how many GPUs your inference workload needs.
How this is calculated
This calculator uses the infrastructure formulas described in our methodology. All figures are estimates for planning; verify current pricing with each provider before relying on them.
Worked example (defaults)
With the default inputs above, here is the result:
Result
7 GPU(s) needed
for 500 tok/s at 80 tok/s each
- Target throughput
- 500 tok/s
- Per-GPU throughput
- 80 tok/s
- GPUs needed
- 7
Frequently asked questions
How do I find per-GPU throughput?+
Benchmark your model on one GPU, or estimate it with our local LLM throughput calculator.
Does batching change this?+
Yes — batching raises aggregate tokens/sec per GPU, reducing the count needed. Measure under realistic load.