Home/FAQ & Glossary/How much VRAM do I need for 8B, 32B, and 70B models?
FREQUENTLY ASKED QUESTIONhardware
← Back to FAQ & Glossary

How much VRAM do I need for 8B, 32B, and 70B models?

To run a local model at full speed, the entire model and your active chat history must fit inside your GPU's VRAM. If any part spills over into system RAM, token generation slows to a crawl.

Using the standard 4-bit (Q4_K_M) quantization format, plan on ~0.6GB of VRAM per billion parameters, plus 1GB to 4GB of extra headroom for your conversation window (the KV cache) and basic display overhead.

Model SizeModel File Size (Q4_K_M)Recommended VRAMExample Hardware
8B (e.g., Llama 3.1 8B, Qwen 3 8B)~4.6 – 4.9 GB8 GBRTX 4060, RTX 3070, or a 16GB Apple Silicon Mac
32B (e.g., DeepSeek R1 32B, Qwen 32B)~18.5 – 19.2 GB24 GBRTX 3090, RTX 4090, or a 36GB+ Apple Silicon Mac
70B (e.g., Llama 3.3 70B)~39.5 – 42 GB48 GB+Dual RTX 3090/4090s, RTX 6000 Ada, or a 64GB+ Mac Studio

What each tier actually feels like

  • 8B Models (8GB VRAM): Fast, cheap to run, and plenty capable for summarizing emails, quick Q&A, and basic scripting. Just expect occasional logic slip-ups when you push them on multi-step instructions.
  • 32B Models (24GB VRAM): The current sweet spot for local AI. Stepping up to 32B brings a noticeable jump in reasoning, data extraction, and coding reliability—and a Q4_K_M file fits cleanly onto a single 24GB card (like a used RTX 3090) with room left over for a solid chat history.
  • 70B Models (48GB+ VRAM): Heavyweight performance for complex writing and deep analysis. Because no single consumer gaming GPU ships with 48GB of VRAM, running a 70B model at full speed means linking two 24GB cards, buying a workstation GPU like an RTX 6000 Ada, or using a 64GB+ Apple Silicon Mac (where macOS reserves roughly 25–30% of unified memory by default, leaving ~46–48GB available to the GPU).

Watch out for the context trap

A model's file size is only its static footprint. Every message you send and every document you paste expands the KV cache inside your VRAM.

If a 32B model takes up 19GB on a 24GB GPU, you have roughly 5GB left for context—more than enough for everyday conversations. Paste a 20-page PDF into that same chat, however, and you can easily blow past that 5GB buffer. If you regularly work with massive documents, either budget for extra VRAM or drop down a model size.

Looking for verified hardware rigs?

Check our real-time £/GB VRAM index to find systems optimized for local model inference.

Explore VRAM Index →
Rent Cloud GPU Power

Not Ready to Buy a Full Rig?

Don't want to spend £3,500+ upfront? Rent the GPU power instead.

Buying a dedicated AI workstation is a serious investment—and if you aren't ready to spend that much right now, you don't have to. Renting cloud GPUs gives you instant access to high-VRAM hardware like RTX 4090s and H100s on demand. Spin up an instance for fine-tuning or running massive 70B+ models, pay strictly for the minutes you use, and avoid the upfront hardware cost, electricity bills, and maintenance.

Vast.ai Global GPU Marketplace

Lowest $/hr
Unbeatable $/hr rates on community-hosted RTX 4090 & 3090 rigs
Deploy Pod →

RunPod Community & Secure Cloud

Community & Secure Cloud
RTX 4090 from $0.34/hr • H100 SXM from $1.99/hr (Billed in USD)
Deploy Pod →

FindBestRig independently benchmarks cloud GPU providers to compare rental economics against physical hardware.