Raptor: 3D-DRAM with 100+ TB/s Bandwidth

What d-Matrix put into Raptor

What stands out here is not just the headline figure of more than 100 TB/s, but how d-Matrix intends to achieve it. In its partnership announcement with Alchip, the company describes Raptor as the first commercial data-center inference accelerator based on its 3DIMC architecture: three-dimensional DRAM with computation performed directly in memory.

A logic die sits directly above custom DRAM and connects through vertical micro-bumps on a 36-micron pitch. This layout removes the conventional PHY and shortens the data path. The company describes configurations with up to four DRAM layers.

According to published material covering its Hot Chips 2026 presentation, a single Raptor chiplet carries roughly 4 GB of DRAM and delivers around 12.5 TB/s. Aggregate accelerator bandwidth exceeds 100 TB/s, while the claimed data-transfer energy is about 0.37 pJ per bit. This is not a cosmetic memory upgrade; it is an attempt to physically redesign the link between memory and compute blocks.

d-Matrix targets roughly a tenfold advantage in bandwidth and energy efficiency versus HBM4-class solutions. Coverage of the presentation also cites about 20 times more bandwidth per unit area than NVIDIA Rubin and 13.5 times lower power per transferred gigabyte. For now, however, these comparisons come from company material rather than independent testing.

The boldest scenario from the presentation is about 1,000 tokens per second per user for a 3T-class model with a 1M context. For GLM 5.2, the company claims roughly 2,121 tokens per second per user at 32 concurrent users; for Kimi K3 at the same concurrency, about 785. Reported effective bandwidth utilization was 83–85%.

The memory wall may genuinely be getting lower

If the claimed characteristics hold in production systems, Raptor could change the equation for long-context inference and highly concurrent workloads. In these cases, performance is often limited less by arithmetic than by moving weights and KV cache, making local bandwidth more important than another increase in peak compute throughput.

The trade-off is visible as well: a specialized accelerator exchanges general-purpose flexibility for dense bandwidth and low latency. I would first look at sustained rather than peak performance, scaling across chiplets, and whether the software stack can reproduce the claimed utilization across different models.

The collected material contains no public Raptor code examples, and the key numbers remain architectural claims and presentation results. So 100+ TB/s is already engineering-wise compelling, but the central question remains: how much of that bandwidth will survive contact with real model graphs and operational constraints?

We previously examined how confidential computing is reshaping AI inference infrastructure, its cost, and privacy requirements. Memory delivering more than 100 TB/s adds a hardware-level piece to that picture by removing one of the key constraints on generative models.