Local Jev Runs 2.4x Faster on DiffusionGemma

What the local Jev run showed

The straightforward result here is compelling: local Jev was roughly 2.4 times faster than its cloud counterpart once network latency was factored in. The post by user 144406 also reports an approximately 4% quality gap on the Banking77 dataset. This is a user-run test, not a benchmark validated by NVIDIA or Google.

Under the hood is DiffusionGemma with NVFP4 quantization. NVIDIA's Hugging Face model card lists 25.2 billion total parameters and 3.8 billion active parameters, while Google's model overview uses the rounded figures of 26B total and 4B active. Its MoE architecture activates only part of the model for each pass.

DiffusionGemma also differs in how it generates output: it processes parallel blocks of 256 tokens rather than relying on strictly sequential left-to-right decoding. The model card also advertises a 256K context window, function calling, and an adjustable reasoning mode. For local inference, the NVFP4 release is prepared for vLLM, with both weights and activations quantized.

The cost estimate is about $30 per month using a Spark GPU under a typical workload profile. At the time of discussion, that was a third-party calculation rather than an official NVIDIA or Google price. The actual figure will depend on workload, batching, and how long the hardware is kept running.

As of September 22, 2026, I would treat these numbers as a strong preliminary signal. A fair comparison still needs identical prompts, hardware details, batch size, latency distribution, and a full explanation of the Banking77 evaluation.

Why local inference is genuinely interesting here

The main benefit is not just average speed: it removes the network from the critical path. For interactive classifiers and agents, that can mean more predictable latency, stronger data control, and less dependence on an external API.

NVFP4 does not turn every RTX card into an ideal platform, however. The format's native advantage is tied to Blackwell, while older RTX GPUs may need a different format, strict memory management, or a throughput compromise. I would start by checking VRAM, memory bandwidth, and quality on the actual target dataset.

A roughly 4% Banking77 deficit may be acceptable or critical depending on errors within individual classes. The real question is not the attractive 2.4x number, but whether this balance of speed, cost, and quality holds outside one user scenario.

We previously covered Rust LocalGPT, a single-binary local assistant built for self-hosted inference with persistent memory and an HTTP API. Its deployment model provides useful context for evaluating faster local inference at cloud-level performance.