# NVHBM

> Source: https://aiwiki.ai/wiki/nvhbm
> Updated: 2026-08-27
> Fact-checked: 2026-08-27
> Categories: AI Hardware, AI Infrastructure, NVIDIA
> License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/) - attribute to "AI Wiki (aiwiki.ai)"
> Cite as: AI Wiki. "NVHBM." aiwiki.ai, 27 Aug 2026. https://aiwiki.ai/wiki/nvhbm
> From AI Wiki (https://aiwiki.ai), the free encyclopedia of artificial intelligence. Reuse freely with attribution.

**NVHBM** is an announced custom [high-bandwidth memory](https://aiwiki.ai/wiki/high_bandwidth_memory) architecture from [NVIDIA](https://aiwiki.ai/wiki/nvidia) for custom AI accelerators that participate in the company's [NVLink Fusion](https://aiwiki.ai/wiki/nvlink_fusion) platform. NVIDIA introduced it on August 26, 2026. The design places an NVIDIA-developed memory controller in the HBM base die rather than leaving that controller on the accelerator's main compute die.[1][2][3]

NVIDIA presents NVHBM as a way to give custom XPUs more memory bandwidth while reducing the area and power associated with their HBM interfaces. Its published figures are projections against standard HBM4E, not independent benchmark results. The announcement named Amazon's Annapurna Labs as the first collaborator, but it did not identify a shipping product, a general-availability date, or a memory supplier.[1][4][5]

| Item | Publicly documented detail |
|---|---|
| Announcement date | August 26, 2026[1][2] |
| Developer | NVIDIA[1][3] |
| Form | Custom HBM base die with an NVIDIA-designed memory controller and physical interface[1][2][3] |
| Platform relationship | Offered to custom-XPU developers through NVLink Fusion[1][3] |
| First named collaborator | Amazon's Annapurna Labs[1][4][5] |
| Comparison used by NVIDIA | Standard HBM4E[1][2] |
| Availability stated in the announcement | Collaboration and future integration; no public shipping date or generally available product was identified[1][4][5] |

## Architecture

HBM places stacked DRAM close to a processor and connects it through a wide interface. For context, the JEDEC HBM4 standard specifies a 2,048-bit interface and up to 2 TB/s of bandwidth per stack at its standard data rate.[6] Those HBM4 figures do not describe NVHBM. NVIDIA has published relative comparisons for NVHBM but has not disclosed its absolute interface width, data rate, capacity per stack, or number of channels.

In NVIDIA's account of a conventional HBM implementation, the memory controller and supporting physical interface occupy part of the XPU's main die. NVHBM moves the NVIDIA-designed controller into the logic base die at the bottom of the 3D memory stack. NVIDIA's technical description also refers to a custom PHY and a narrower connection between the memory stack and the XPU.[1][2] This change is meant to reduce the main die area devoted to memory access and simplify routing through the package.

The public description concerns placement and integration, not a new type of DRAM cell. NVIDIA says memory providers will validate and offer NVHBM, while NVIDIA supplies a common controller and base-die implementation for NVLink Fusion customers.[1][3] The company calls this a standard implementation, but the announcement does not identify a JEDEC specification for NVHBM. It is therefore more precise to describe it as an NVIDIA-defined architecture intended for multiple suppliers.

## Relationship to NVLink Fusion

NVHBM and NVLink Fusion address different levels of an accelerator system. NVHBM is the local memory architecture attached to an XPU. NVLink Fusion supplies the chiplets, interface intellectual property, switches, and rack architecture that connect custom XPUs or CPUs with NVIDIA systems.[1][2][3]

This distinction matters when interpreting system claims. A higher local HBM bandwidth can feed compute engines more quickly when a workload is limited by memory traffic. NVLink handles communication among processors across a scale-up domain. NVHBM does not itself connect accelerators across a rack, and NVLink Fusion does not replace the XPU's local memory interface.[2][3]

NVIDIA's product page describes a common rack design in which custom-XPU systems and GPU systems such as [NVIDIA Vera Rubin](https://aiwiki.ai/wiki/nvidia_vera_rubin) can share rack footprints, networking, cooling, power delivery, and management infrastructure.[3] That example explains the role of NVLink Fusion. It does not establish that a current Vera Rubin product uses NVHBM.

## NVIDIA performance and efficiency projections

NVIDIA compares NVHBM with standard HBM4E. The figures below are vendor projections. The announcement and technical article do not provide independent measurements, a complete package configuration, or a reproducible benchmark method.[1][2]

| Metric | NVIDIA's stated comparison with standard HBM4E | What the figure does not establish |
|---|---|---|
| Memory bandwidth | Up to 30 percent more bandwidth per stack[1][2] | An absolute bandwidth figure or a 30 percent application-speed increase |
| HBM power | Up to 15 percent lower HBM power use[1][2] | A 15 percent reduction in total XPU, rack, or data-center power |
| XPU compute-die area | Up to 25 percent more area available on the compute die[1][2] | A 25 percent increase in compute performance |

The area figure needs an additional qualification. NVIDIA's corporate announcement and the comparison table in its technical article say up to 25 percent more compute-die area. A later paragraph in the same technical article says the design provides up to 30 percent more main-die silicon.[1][2] The post does not reconcile the two figures or publish package dimensions, so they should not be treated as interchangeable measurements.

The technical article also projects a 30 percent overall end-to-end performance increase per XPU by combining its assumptions for bandwidth, die area, and power.[2] It does not report a workload, test system, sample count, or third-party result for that figure. The same article gives an illustrative calculation in which HBM power savings create headroom for up to 15,000 additional 2,000 W XPUs within a 1 GW data-center envelope.[2] This is a design projection, not a report that a facility has deployed that many extra processors.

The practical benefit of additional bandwidth depends on the workload and software. Research on distributed large-language-model serving has identified HBM reads for model weights and key-value caches as a bottleneck under some configurations.[7] Other measurements of batch-1 decode found that peak HBM bandwidth did not translate proportionally into lower latency when software and launch overheads became limiting.[8] Neither study evaluated NVHBM, but together they show why a memory-bandwidth percentage alone cannot predict end-to-end application performance.

## AWS and Annapurna Labs collaboration

[Amazon Web Services](https://aiwiki.ai/wiki/amazon_web_services) and NVIDIA announced that Annapurna Labs would be the first organization to collaborate on NVHBM. The work extends an earlier plan to add NVLink Fusion support to next-generation Trainium accelerators, beginning with Trainium4.[1][4][5]

The companies describe NVHBM as part of future heterogeneous infrastructure. Their joint release says Annapurna Labs is working with NVIDIA and memory suppliers on the technology, which would give Trainium access to faster and more power-efficient memory. It also says NVLink Fusion would place Trainium and NVIDIA GPUs in a common rack-scale architecture.[4][5]

Those statements describe an engineering plan. They do not say that Trainium4 or any available Amazon EC2 instance already contains NVHBM, and they give no date for an NVHBM-equipped Trainium service.[1][4][5] Trainium4 is the stated starting point for NVLink Fusion support, while the NVHBM work is described separately as an expansion of that collaboration.

## Integration and supplier model

NVIDIA says leading memory partners will validate and offer NVHBM. Its stated goal is to give custom-XPU designers one controller and base-die implementation that can be qualified across multiple memory suppliers, reducing the amount of supplier-specific integration work.[1][2] No memory provider was named in the August 26 materials, so the breadth of that supply model had not been demonstrated publicly.

The controller's placement also shifts part of the memory-interface design away from the XPU main die. In principle, that lets an accelerator designer assign more of the main die to compute, cache, control logic, on-chip networking, or other functions. NVIDIA lists these as possible uses of the recovered area rather than fixed features of NVHBM.[2] The actual allocation remains a choice made in each XPU design.

NVIDIA says NVHBM is built on technology intended for its future GPUs.[1] The announcement does not identify the first NVIDIA GPU generation that will use it. It also does not say whether the partner implementation and NVIDIA's own future implementation will have identical bandwidth, capacity, stack organization, or packaging.

## Status and evidence limits

The August 2026 publication is an architecture announcement rather than a complete memory specification. It provides a design concept, comparative projections, and the first collaboration, but leaves several implementation facts undisclosed. These include memory capacity, absolute bandwidth, stack height, process technology, package footprint, supported XPU designs, named memory vendors, sampling dates, qualification milestones, pricing, and production availability.[1][2][3][4][5]

No independent NVHBM benchmark was included in the public materials. All performance, power, and area percentages come from NVIDIA. The `up to` wording also means the figures are upper-bound claims under conditions that the company did not fully document in the announcement. Results for a finished accelerator will depend on its compute design, memory configuration, package, cooling, firmware, software, and workload.

NVHBM should therefore be distinguished from both a shipping memory part and an industry-wide HBM generation. It is a named NVIDIA architecture for a custom base die and controller, offered within the NVLink Fusion program and intended for validation by multiple suppliers. Its first announced partner activity is collaborative work with Annapurna Labs, not a generally available AWS instance or a confirmed product shipment.[1][3][4][5]

## References

1. Jesse Clayton. "NVIDIA NVLink Fusion Expands With NVHBM Custom High-Bandwidth Memory." NVIDIA Blog, August 26, 2026. https://blogs.nvidia.com/blog/nvlink-fusion-nvhbm-custom-high-bandwidth-memory/
2. Farshad Ghodsian, Jesse Clayton, and Varun Nanda Kumar. "NVIDIA NVLink Fusion Brings NVHBM to Next-Generation AI Infrastructure." NVIDIA Technical Blog, August 26, 2026. https://developer.nvidia.com/blog/nvidia-nvlink-fusion-brings-nvhbm-to-next-generation-ai-infrastructure/
3. NVIDIA. "NVIDIA NVLink Fusion." Accessed August 27, 2026. https://www.nvidia.com/en-us/data-center/nvlink-fusion/
4. NVIDIA Newsroom. "AWS and NVIDIA to Deliver 2 Million Additional GPUs and Next-Generation Infrastructure for Agentic and Physical AI." August 26, 2026. https://nvidianews.nvidia.com/news/aws-and-nvidia-to-deliver-2-million-additional-gpus-and-next-generation-infrastructure-for-agentic-and-physical-ai
5. Amazon Press Center. "AWS and NVIDIA to Deliver 2 Million Additional GPUs and Next-Generation Infrastructure for Agentic and Physical AI." August 26, 2026. https://press.aboutamazon.com/aws/2026/8/aws-and-nvidia-to-deliver-2-million-additional-gpus-and-next-generation-infrastructure-for-agentic-and-physical-ai
6. JEDEC Solid State Technology Association. "JEDEC and Industry Leaders Collaborate to Release JESD270-4 HBM4 Standard: Advancing Bandwidth, Efficiency, and Capacity for AI and HPC." Business Wire, April 16, 2025. https://www.businesswire.com/news/home/20250416843598/en/JEDEC-and-Industry-Leaders-Collaborate-to-Release-JESD270-4-HBM4-Standard-Advancing-Bandwidth-Efficiency-and-Capacity-for-AI-and-HPC
7. Ahmet Caner Yüzügüler, Jiawei Zhuang, and Lukas Cavigelli. "PRESERVE: Prefetching Model Weights and KV-Cache in Distributed LLM Serving." arXiv:2501.08192, revised May 26, 2025. https://arxiv.org/abs/2501.08192
8. Josef Chen. "Memory-Bound but Not Bandwidth-Limited: The Physical AI Inference Gap in Batch-1 LLM Decode." arXiv:2605.30571, submitted May 28, 2026. https://arxiv.org/abs/2605.30571