Local LLM Throughput Calculator
Estimate tokens/sec for a model on your GPU.
Results update automatically as you type.
Result
~250 tokens/sec
8B model at int4 on 1,000 GB/s
- • Rough upper bound; real speed is lower due to compute and batching effects.
- Model size
- 8 B
- Precision
- int4
- Memory bandwidth
- 1,000 GB/s
- Estimated speed
- 250 tok/s
Estimate the token generation speed of a model on your hardware using a memory-bandwidth-bound approximation — a quick sanity check before running models locally.
How this is calculated
This calculator uses the infrastructure formulas described in our methodology. All figures are estimates for planning; verify current pricing with each provider before relying on them.
Worked example (defaults)
With the default inputs above, here is the result:
Result
~250 tokens/sec
8B model at int4 on 1,000 GB/s
- • Rough upper bound; real speed is lower due to compute and batching effects.
- Model size
- 8 B
- Precision
- int4
- Memory bandwidth
- 1,000 GB/s
- Estimated speed
- 250 tok/s
Frequently asked questions
Why memory bandwidth?+
Single-stream token generation is usually memory-bound: each token reads all model weights, so tok/s ≈ bandwidth ÷ model bytes.
Is this exact?+
No — it's an optimistic upper bound. Real speed is lower due to compute, attention and framework overhead.