DeepSeek 284B Runs on a CPU With 64 GB of RAM

How 284B parameters fit into ordinary memory

The key result is this: llama-modchip claims to run DeepSeek-V4-Flash with 284.33B parameters on a 16-core AMD Ryzen 9 9955HX, 64 GB of DDR5 memory, and no GPU at all. The project repository and benchmark table list a model size of 56.56 GiB, while peak memory use is roughly 57.6–58.9 GiB with a 256K context.

The point is not that someone suddenly made a dense 284B model inexpensive to compute on a CPU. DeepSeek-V4-Flash uses a Mixture-of-Experts architecture: it has many parameters overall, but only about 13.17B are active for each token. The full expert set still has to remain in memory, yet the compute required per token is substantially lower.

The second part of the trick is aggressive quantization and a specialized GGUF format. llama-modchip documentation also attributes savings of 20.31 GiB to its SwiGLU-H2 implementation. Combined with memory mapping and tight allocation control, this leaves room for runtime state and the KV cache, although the margin on a nominal 64 GB machine remains extremely narrow.

Published figures show 43.16–44.36 tokens per second during prompt processing and 9.06–10.74 during regular generation. With Stockfish-PVS configurations for speculative decoding, the project reports 11.50–14.69 tokens per second and roughly 94.5% draft-token acceptance. These are the project's own measurements at publication time, not an independent reproduction.

GPUs are not obsolete, but the equation has changed

This is not a replacement for accelerators used in training or high-volume serving. It is, however, a realistic path to local inference for a large MoE model. It is especially relevant when autonomy, access to a large set of weights, and avoiding dedicated VRAM matter more than maximum throughput.

The first things to verify would be quality after quantization, stability at long context lengths, and how strongly speed depends on memory. For this kind of inference, the bottleneck is often not CPU arithmetic but RAM bandwidth, thread placement, and moving tens of gigabytes of weights. Results therefore may not transfer to another processor or slower memory configuration.

A specific Hacker News thread cannot be confirmed reliably from the available materials, so conclusions should be kept separate from community discussion. The experiment remains notable: parameter count is becoming a weaker measure of actual inference cost when the architecture is sparse and the runtime is optimized alongside the weight format. The main question is no longer whether the model runs, but how well this compressed version retains its original quality.

We previously covered Rust LocalGPT, a local assistant delivered as a single binary with memory and an HTTP API. This example complements the DeepSeek 284B run on standard hardware and shows which local LLM infrastructure options are available without relying on the cloud.