LLM System Requirements Calculator

Your System

OS: …CPU cores: …RAM:GB

Configure your model

Model

Apache 2.0 · released 2024-05 · native context up to 32K tokens · HuggingFace

Quantization

Standard community default — best size/quality tradeoff

Context length (tokens)

Typical: 4K–8K chat, 32K–128K long-doc work. Longer context = larger KV cache.

Quick presets

Estimated VRAM

9.42 GB

weights 3.92 GB + KV 4.00 GB + overhead 1.50 GB

System RAM (CPU inference)

13.9 GB

Minimum headroom for CPU-only inference. GPU inference needs half of this.

Disk space

3.92 GB

Download size for a single GGUF/safetensors file

Will it run on my hardware?

Hardware	Memory	Fit	Headroom	Note
NVIDIA RTX 3060 12GB	12 GB	Fits	+2.58 GB	Budget consumer GPU
NVIDIA RTX 4060 Ti 16GB	16 GB	Fits	+6.58 GB	Mid-range with extra VRAM
NVIDIA RTX 3090 24GB	24 GB	Fits	+14.6 GB	Enthusiast, great for local LLMs
NVIDIA RTX 4090 24GB	24 GB	Fits	+14.6 GB	Top consumer GPU
NVIDIA RTX 5090 32GB	32 GB	Fits	+22.6 GB	Current-gen flagship
NVIDIA A100 40GB	40 GB	Fits	+30.6 GB	Datacenter-class
NVIDIA A100 80GB	80 GB	Fits	+70.6 GB	Datacenter-class, large
NVIDIA H100 80GB	80 GB	Fits	+70.6 GB	Current datacenter flagship
Mac M2 16GB	12 GB(unified)	Fits	+2.58 GB	Unified memory; ~12GB usable after OS
Mac M3 Max 36GB	30 GB(unified)	Fits	+20.6 GB	Unified memory; ~30GB usable
Mac M3 Max 64GB	56 GB(unified)	Fits	+46.6 GB	Unified memory; ~56GB usable
Mac M3 Ultra 128GB	115 GB(unified)	Fits	+105.6 GB	Unified memory; ~115GB usable
Mac M4 Max 128GB	115 GB(unified)	Fits	+105.6 GB	Unified memory; ~115GB usable

How the math works:

Weights: total params × bytes/param. MoE routing reduces per-token compute, but normal resident memory still includes all expert weights.
KV cache: a conservative FP16 estimate that scales with context and batch size. Cache precision and attention layout can reduce it.
Overhead: activations, framework workspace, CUDA kernels — typically 1.5–3 GB or ~10% of weights, whichever is larger.
Disk: the full quantized checkpoint. MoE models store every expert on disk even if only a few are active per token.

Sources: Architecture numbers (num_hidden_layers, hidden_size, max_position_embeddings) come directly from each model's published HuggingFace config.json — click the HuggingFace link next to the selected model to see the exact config. Quantization byte/param ratios come from the llama.cpp GGUF k-quants spec. Treat the result as a planning estimate because cache precision, offload, and framework buffers differ by runtime.

Quantization availability: Mathematically any model can be quantized to any level — but community GGUF releases don't always ship every variant. Q4_K_M, Q5_K_M, Q8_0 and FP16 are near-universal; Q2_K and Q3_K_M are often skipped on smaller models where quality loss is noticeable. To find a specific quant for a specific model, search bartowski, TheBloke, or the official repo on HuggingFace.

Context length: Each model has a native max context defined in its config. Going beyond it (via RoPE/YaRN scaling) works mechanically but degrades quality — the warning banner above flags this whenever the current selection exceeds the model's native limit.

How to estimate LLM requirements

Pick the model

Select the open-source model you plan to run — Llama 3.1, Mistral, Qwen 2.5, DeepSeek R1, Gemma 2, Phi, or Code Llama. The dropdown is grouped by family and shows license and release date.
Choose a quantization level

Q4_K_M is the community default: about 0.58 bytes per parameter and nearly indistinguishable from FP16 on most tasks. FP16 doubles the size but matches the original training precision. Q2_K is the smallest, with visible quality loss.
Set the context length

Context length drives the KV cache, which scales linearly with tokens. A 128K-context session on Llama 3.1 70B adds over 10 GB to VRAM on top of the weights. Use the preset buttons for common sizes.
Read the verdict

The three cards show estimated total VRAM, a system RAM planning value, and estimated download size. The hardware table compares each target with the same conservative estimate.

Who this is for

Choosing a GPU for local inference

Before you buy a 4090 or an M3 Max, check whether the model you care about actually fits after accounting for the KV cache at your real context length. A 70B model at 128K context needs far more memory than the base weights suggest.

Sizing a cloud instance

Compare a quantization and context setting with common GPU memory sizes before you choose a hosted instance. Keep extra room for runtime-specific buffers and concurrent requests.

Picking a quantization for your hardware

If Q4_K_M fits on your GPU but Q5_K_M is tight, the calculator shows the delta so you can make an informed quality-vs-memory tradeoff. Swap between quantization levels and watch the VRAM estimate update instantly.

Budgeting disk space

Download sizes for models are not obvious from their name. A Q4_K_M quant of Llama 3.1 70B is about 40 GB on disk; the 405B version exceeds 200 GB. MoE models store every expert on disk even if only a few are active per token.

About this tool

This calculator estimates the VRAM, system RAM, and disk space required to run a selected open-weight language model on local hardware. It covers models across the Llama, Mistral, Qwen, DeepSeek, Gemma, Phi, and Code Llama families, with quantization choices from FP32 through Q2_K.

The estimate separates quantized model weights, a conservative KV-cache value, and runtime overhead. Model weight memory uses total parameter count and the selected bytes-per-parameter value. KV cache grows with context length and batch size. The final runtime can use more or less memory because attention layout, cache precision, GPU offload, framework buffers, and concurrent requests differ.

For MoE models such as Mixtral and DeepSeek V3, active parameters describe per-token compute. Normal resident memory still needs all expert weights unless the runtime explicitly offloads experts to system memory or storage. The calculator therefore uses total parameters for its default memory estimate and keeps active parameters as model information.

Pair this calculator with the AI Model Picker to choose a model, or use the linked model card to confirm architecture details before downloading weights.

How it compares

Memory estimator blog posts and Hugging Face Space demos exist, but most are model-specific, skip the KV cache entirely, or ignore runtime overhead — which is exactly the memory that makes a model OOM instead of OOM-near-miss. This calculator is model-agnostic across 30+ curated checkpoints, always includes KV cache and overhead, and lets you see the headroom on 13 specific hardware targets side by side.

The alternative — trying to load the model and watching nvidia-smi or macOS Activity Monitor — works but wastes a 40 GB download when the answer is no. This tool gives you the answer before you download anything, and runs entirely in your browser: the model list and math are static data, so your hardware choices and model preferences are never transmitted anywhere.

Tips for running local LLMs

Q4_K_M is the community default for a reason — it produces the best quality per gigabyte on almost every model and saves roughly 75% vs FP16.
On unified-memory Macs (M1/M2/M3/M4), the whole RAM pool doubles as VRAM, minus about 4 GB for macOS. A 36 GB M3 Max gives you ~30 GB usable for model weights.
KV cache memory grows with context length and batch size. Some runtimes support lower-precision caches, so check the settings for your runtime.
Mixture-of-Experts models route each token through only some experts, but a normal resident-memory estimate still includes all expert weights. Expert offloading is a separate runtime choice.
For a 7B–8B model at 8K context, Q4_K_M fits comfortably in 8 GB of VRAM and runs well on a mid-range laptop GPU or a base-model M2 Mac.

Frequently Asked Questions

How much VRAM do I need to run Llama 3.1 70B?

At Q4_K_M with an 8K context, Llama 3.1 70B needs about 42 GB of VRAM (around 40 GB of weights plus 1 GB of KV cache and 3 GB of overhead). That fits a 2×3090 or 2×4090 setup, an A100 40GB with tight headroom, or an M3 Max with 64+ GB of unified memory. At FP16 it doubles to about 150 GB, requiring an H100 80GB pair or A100 80GB multi-GPU.

What does Q4_K_M mean, and why is it the default?

Q4_K_M is a 4-bit group-wise quantization format from llama.cpp. It stores most weights in 4-bit blocks with per-block scale factors, reaching about 0.58 bytes per parameter. The 'K' means k-quants (improved block layout) and the 'M' means medium mix — important weights stay in higher precision. It's the default because quality loss is typically under 1% on reasoning benchmarks while size drops 75% vs FP16.

Can I run DeepSeek V3 or DeepSeek R1 locally?

The full 671B MoE models are extremely large. Only part of the network is active for each token, but all expert weights must still reside somewhere. A consumer system normally needs a smaller distill model unless its runtime uses extensive CPU or storage offload.

Why does my VRAM estimate jump when I increase context length?

The KV cache is 2 × num_layers × hidden_size × context_length × 2 bytes, and it sits in VRAM alongside the weights. On a 70B model with 80 layers and 8192 hidden size, each 1K of context adds ~2.5 GB. Going from 8K to 128K adds over 30 GB. Shortening context is usually the fastest way to reclaim VRAM without changing models.

Is this calculator accurate for vLLM, TensorRT, and MLX?

It is a planning estimate, not a runtime guarantee. Cache precision, grouped-query attention, GPU offload, framework buffers, and concurrent sequences can change the final memory use. Confirm the result with your runtime documentation.

Does the calculator handle Apple Silicon / M-series Macs?

Yes. Apple Silicon uses unified memory, so the model shares one pool with the operating system and other applications. The hardware table keeps operating-system headroom in its planning values.

What's the difference between VRAM and RAM in the results?

VRAM is dedicated GPU memory. System RAM serves CPU inference and GPU offload. Unified-memory systems share one physical pool, but the operating system and applications also use it.

Are these download sizes accurate for GGUF files?

The disk value estimates quantized tensor size from parameter count. Real GGUF files also contain metadata and can use mixed tensor types, so check the published file size before downloading.

Run it from code

$ curl -X POST \
  api.findutils.com/api/tools/llm-requirements-calculator/execute \
  -d '{"model":"llama-3.1-8b","quantization":"q4_k_m","context_length":8192}'

$ claude mcp add findutils \
  --transport http https://mcp.findutils.com/
# then call findutils:llm_requirements_calculator

API reference Add to Claude