New: AI API Cost Calculator — compare GPT, Claude, Gemini, Grok & Llama pricing live
Free Tool

Can I Run It? — Local LLM Hardware Calculator

Pick a model, a quantization and your hardware; get a straight verdict with the memory math shown. No signup, runs in your browser.

Model & your hardware

—
—
—
total memory needed (weights + KV cache + 12% runtime overhead)
—
weights
—
KV cache
—
total
Honest assumptions. Weights use total parameters, because every expert in an MoE model must be resident in memory — the router can send any token to any expert. Speed, by contrast, is estimated from active parameters. The KV-cache figures are planning estimates (at FP16 KV) and vary by architecture — grouped-query attention and sparse attention lower them, sometimes a lot. The 12% overhead covers the runtime, activations and buffers. Memory-bandwidth and tok/s figures are rough classes, not benchmarks. Always confirm the real checkpoint size on the model's Hugging Face page before you download 200 GB on faith.

How the math works

Three numbers decide whether a model runs locally. First the weights: weightsGB = totalParams × bytesPerParam — an INT4 copy of a 32B model is roughly 32 × 0.55 ≈ 17.6 GB, while the FP16 original is about 64 GB. Then the KV cache, which stores the attention state for every token in your context window: kvGB = kvGBper1K × (contextTokens ÷ 1000). It grows linearly, which is why jumping from 32K to 1M context can add more memory than the quantization saved you. Finally: totalGB = (weightsGB + kvGB) × 1.12, with 12% for the runtime. If the total fits in VRAM (or in the single unified-memory pool on Apple Silicon and similar systems, where RAM and VRAM are the same number — never double-counted), it runs at full speed. If it only fits in VRAM + system RAM combined, the overflow is offloaded to the CPU and generation slows sharply.

Going further

This calculator is part of our offline AI coverage. Start with the Offline AI hub for the full picture, read the offline AI stack guide for the software side, and if local turns out not to fit your hardware, price the cloud alternative in our AI API Cost Calculator.

FAQ

Does quantization hurt quality?

A little, and it depends on the level. FP8 and INT8 are close to lossless for most chat and coding work. INT4 has a small but measurable cost — most noticeable on math, multi-step reasoning, and in smaller models, which have less redundancy to spare. Larger models generally tolerate INT4 better. If you have the memory headroom, INT8 is the safe default; INT4 is what makes big models fit on consumer hardware at all.

Why do MoE models still need so much memory?

Because every expert has to live somewhere. A mixture-of-experts model only activates a few experts per token — that's why speed is driven by the active parameter count — but the router can send any token to any expert, so all of them must be resident in memory. Memory is set by total parameters; speed is set by active parameters. That split is the whole reason MoE models are fast but not small.

Is system RAM as good as VRAM?

No. Generation speed is mostly memory-bandwidth-bound, and VRAM is far faster than system RAM — and anything offloaded to RAM also has to cross the PCIe bus every token. CPU offload works and is fine for batch jobs or overnight runs, but interactive chat usually drops to a few tokens per second. Think of RAM as overflow parking, not a second GPU.

What context length should I pick?

Match it to the job, because the KV cache grows linearly with context. 8K–32K covers most chat and coding. Pick 128K for long documents or large RAG contexts. Only choose 1M if your workload genuinely needs it — at long contexts the KV cache can rival the weights themselves, especially on dense models.

Why does the calculator say a 220GB model won't fit my 256GB machine?

Because weights are not the whole bill. Add the KV cache for your context length, then add roughly 12% runtime overhead for activations and the inference engine — and your operating system and other apps are using some of that 256GB too. A 220GB checkpoint plus 10GB of KV cache, times 1.12 overhead, is already about 258GB. The checkpoint size on the download page is the floor, not the total.