NVIDIA expands NVLink Fusion with new NVHBM memory

NVIDIA has expanded NVLink Fusion with a new high-bandwidth memory technology called NVHBM, extending its semi-custom AI infrastructure offering into memory design.

NVHBM moves the memory controller from the XPU die into the HBM base die, changing the long-established arrangement in which the controller sits on the processor itself. NVIDIA says that shift can free more silicon area for compute while improving memory throughput and reducing power use.

The technology is based on the same approach NVIDIA plans to use in future GPUs. It will be validated and supplied by multiple memory partners, with the aim of giving NVLink Fusion customers a standard implementation that can be sourced from more than one provider.

According to NVIDIA, the design delivers up to 30% greater memory bandwidth and 15% lower HBM power consumption than standard HBM4E. It can also free up to 25% more area on the XPU compute die by removing the memory controller from that chip.

AWS link

Amazon's Annapurna Labs will be the first group to work on NVHBM under its broader collaboration with NVIDIA on NVLink Fusion. The work will also cover the NVLink scale-up architecture for AI workloads.

Annapurna Labs is already part of Amazon's Trainium effort, and support for NVLink Fusion is planned for the next generation of those chips, starting with Trainium4. That would allow Amazon chips and NVIDIA GPUs to operate together within a shared rack-scale architecture.

"NVHBM represents a new architectural approach to advancing high-bandwidth memory performance and efficiency," said Nafea Bshara, Vice President of Annapurna Labs at Amazon.

"We look forward to this technology collaboration to benefit future AWS infrastructure designs," Bshara said.

Memory shift

High-bandwidth memory has become central to AI system design as model sizes grow and data movement puts more strain on system architecture. In conventional designs, memory controllers occupy space on the XPU die, limiting how much of that silicon is available for compute functions.

NVIDIA's proposal is to embed its custom memory controller in the three-dimensional HBM stack instead. That should reduce the engineering work needed to integrate and qualify memory from different suppliers because the NVHBM implementation will be standardised across several providers.

That matters for cloud companies and chip designers seeking more customised AI systems without having to build every element themselves. NVLink Fusion is NVIDIA's framework for linking custom XPUs and CPUs to its broader rack-scale platform, including chiplets, interconnect, switches, systems and racks.

Broader strategy

The move shows how NVIDIA is trying to extend its influence beyond GPUs into the surrounding architecture of AI data centres. Rather than offering only finished processors, it is packaging interconnect, system design and now memory architecture into a platform that outside partners can use with their own silicon.

For hyperscalers and AI-focused infrastructure developers, that model offers a route to building semi-custom systems while staying aligned with NVIDIA's networking and rack design. It also gives NVIDIA a way to remain at the centre of AI infrastructure decisions even when customers develop chips in-house.

NVLink Fusion gives partners access to NVIDIA NVLink chiplets, NVLink-C2C, NVLink switches, and MGX systems and racks, alongside a wider ecosystem of CPU partners, ASIC designers, system manufacturers, and other technology providers. NVIDIA positions the platform as a way for customers to focus on XPU design while relying on an existing stack for scale-up and scale-out networking, rack systems, and software.

Adding memory architecture strengthens that proposition, particularly as AI customers look for gains not just from processor performance but from the balance between compute, memory and system efficiency. As AI agents and very large models place heavier demands on infrastructure, control over those design trade-offs is becoming more important across the supply chain.

Amazon's participation gives the effort an early high-profile user with a history of building its own AI silicon.