Kimi K3 Fits on One B300 x8, But Not All

What Actually Fits on the B300 x8

Kimi K3 on a single server with 8 NVIDIA B300 looks like a beautiful moment, but the keyword here is MXFP4. In Fixstars' blog, in the Deploying Kimi K3 on Day 0 case, the focus is precisely on running a 2.8-trillion-parameter model on a single B300 x8 node, not on freely choosing any weight format.

According to materials from vLLM and technical breakdowns, a single B300 x8 node handles the basic scenario for MXFP4. For BF16 and MXFP8, the picture is different: one B300 server can no longer hold the full K3 model at that precision. That's when hype turns into engineering memory accounting.

The hard truth is simple: the MXFP4 checkpoint for 2.8T parameters is estimated at around 1.4 TB just for weights. The 8-GPU B300 node discussed features 2304 GB of HBM3e, so there is headroom, but it's not infinite. KV-cache, activations, and overhead don't disappear.

The vLLM recipe for Kimi K3 also states quite plainly: 8 NVIDIA B300 or 8 AMD MI355X, tensor parallelism 8, Kimi-specific parsers for tool calling and reasoning, fastsafetensors, prefix caching. This isn't a toy notebook run but a careful distribution of one giant MoE across eight accelerators.

Why This Changes the Game

The main shift is that local deployment of a model this size no longer automatically means a full rack of hardware. For MXFP4, a single B300 x8 already looks like a practical minimum, not a fantasy from a presentation.

But I wouldn't confuse loading the model with good production serving. There is no formal reproducible benchmark for tokens per second or latency for Kimi K3 on a single B300 x8 in the sources. So for now, what's proven is capacity and the basic launch scheme, not comfortable high concurrency.

For an engineer, the sharpest question isn't whether the process starts. According to this data, for MXFP4 it does start. The question is how much context, how many concurrent requests, and what latency tail this configuration can withstand, especially when enabling long context like max-model-len 1048576 mentioned in the guides.

The winner here is obvious: Blackwell-class hardware with large HBM gets a real argument for local SOTA LLM inference. The illusion that a 2.8T model can simply be dumped onto any modern eight-GPU server and forget about the weight format is the loser. Kimi K3 on a single B300 x8 is interesting precisely because the boundary has become visible, not disappeared.

We previously reviewed Rust LocalGPT — a compact local assistant that runs entirely on your hardware. This shows that the trend of deploying AI without clouds encompasses both small and giant models.