How Much GPU VRAM Do You Need to Run an LLM?
VRAM is the hard constraint on self-hosting: if a model plus its context does not fit, it will not run. This guide shows how to size VRAM and how quantization lowers the bar.
What eats VRAM
Model weights dominate, but the key-value cache for context also grows with sequence length and batch size. Sizing only for weights and forgetting the cache is a common way to run out of memory under real load.
Quantization changes the math
Running weights at lower precision can cut VRAM substantially, often letting a model fit on smaller, cheaper GPUs with modest quality loss. It is usually the fastest way to make a model affordable to host.
Calculators for this topic
Frequently asked questions
How much VRAM do I need for an LLM?+
Enough for the model weights plus the key-value cache for your context and batch size — size for both, not just weights.
Does quantization reduce VRAM?+
Yes — lower-precision weights use less memory, often enough to fit a model on a smaller GPU with modest quality loss.