Memory bandwidth is your token speed

For dense LLMs generating text, the machine reads essentially every weight in the model for every token. That single fact makes local inference a memory problem before it is a compute problem: your tokens-per-second ceiling is approximately memory bandwidth ÷ model size in memory. Cores and clocks barely move it; the memory bus is the speed limit.

The one formula, applied

Dual-channel DDR5-6000 moves about 96 GB/s (2 channels × 8 bytes × 6,000 MT/s). An 8-billion-parameter model quantized to ~5 GB therefore tops out near 19 tokens/sec — before any real-world overhead. Same platform, 70B model at ~40 GB: ~2.4 tokens/sec, which is why nobody enjoys running 70B dense models from system RAM.

Memory system	~Bandwidth	8B @ ~5GB	70B @ ~40GB
DDR4-3200, dual channel	51 GB/s	~10 tok/s	~1.3 tok/s
DDR5-6000, dual channel	96 GB/s	~19 tok/s	~2.4 tok/s
DDR5-8000, dual channel	128 GB/s	~26 tok/s	~3.2 tok/s
Unified-memory laptop/desktop SoCs	~400–800+ GB/s	80+ tok/s	10–20 tok/s
Discrete GPU (fits in VRAM)	~1,000 GB/s	100+ tok/s	needs 2×24GB+

Theoretical ceilings — real throughput lands below them, but the ratios hold. Numbers assume dense models; mixture-of-experts models read only active experts per token, which is friendlier to modest bandwidth.

What this means when you spec a machine

Capacity decides what you can load; bandwidth decides how it feels. 192GB of slow RAM runs a 70B model at reading pace. Know which you're buying.
Faster kits pay off linearly. DDR5-6000 → 8000 is a real ~33% token-speed bump when CPU-bound — rare in PC upgrades. See the DDR5 buyer's page.
Prompt processing is different. Reading a long prompt is compute-parallel and benefits from cores/GPU; it's generation that rides the memory bus. If your use is long-document Q&A, a GPU for prefill + RAM for generation is a legitimate split.

Why this page lives on a DDR6 site

Because DDR6's ~2× bandwidth target is the first memory-generation jump that maps directly onto a mainstream workload people feel: local token speed. Doubling the bus roughly doubles the table above — in 2027+, on new platforms (timeline). Until then, this math is the reason to buy DDR5 well rather than wait: the decision page.