NVIDIA's New NVHBM Boosts AI Chip Speed by 30%

NVIDIA has unveiled a fresh memory solution called NVHBM that promises to lift AI chip speeds by roughly 30 percent. The new tech is meant to give XPUs—any kind of processor—more bandwidth while sipping less power. It also frees up space on the silicon die, which could help designers pack more features into smaller chips.

The foundation for this advance is NVLink Fusion, an open‑door program that lets outside chipmakers borrow NVIDIA’s high‑speed NVLink fabric. Partners can embed the interconnect into their own silicon and mix it with NVIDIA GPUs in a single rack. Big names such as Amazon, Marvell, MediaTek, Fujitsu, Arm, Cadence and Samsung have already signed up to the ecosystem.

NVHBM takes a different approach to high‑bandwidth memory (HBM). Instead of tucking the memory controller onto the processor die, NVIDIA stacks the HBM itself on a separate die that sits atop the MCU. This layout is similar to what AMD did with its Radeon RX 7900 XTX, but NVIDIA places the memory layer directly on the controller. The result, according to the firm, is up to 30 percent more bandwidth, 15 percent less power draw and roughly 25 percent more usable die area compared with standard HBM4E. The improvements are said to translate into a 30 percent overall end‑to‑end performance gain for any XPU that uses the design.

Amazon’s Annapurna Labs, an AWS division, is already on board, planning to use NVLink Fusion in its Trainium4 processors and collaborating with NVIDIA on NVHBM, though it has not yet committed to a specific implementation. The effort is part of a broader push to make NVHBM a multi‑vendor standard, so several memory makers will eventually offer the technology. Zak Killian, a 30‑year veteran of PC building, penned the piece, offering a layperson’s view of the hardware shift.