AI Wiki

HBM3e

AI Hardware
19 min read
Updated
Suggest editHistory
RawGraph

Last reviewed

Sources

No citations yet

Review status

Needs citations

Revision

v3 · 3,727 words

HBM3e (also written HBM3E and HBM3 Extended) is the extended-bandwidth variant of HBM3 high bandwidth memory, a stacked DRAM technology used as on-package memory for AI accelerators and high-performance computing devices.[1][2] Standardized under JEDEC's JESD238 HBM3 specification with vendor-specific speed extensions up to 9.6 Gb/s per pin, HBM3e raised per-stack capacity to 24 GB in an 8-Hi configuration and 36 GB in a 12-Hi configuration, with per-stack bandwidth above 1.2 TB/s.[1][3][4] SK hynix delivered the first 8-Hi HBM3e for mass production in March 2024 to support NVIDIA's H200, the HBM3-equipped H100 successor, followed by Micron in February 2024 and Samsung after a delayed qualification cycle.[5][6][7] When SK hynix first announced the technology in August 2023 it described HBM3e as "the extended version of HBM3 which delivers the world's best specifications," able to "process data up to 1.15 terabytes (TB) a second."[5] HBM3e powers the dominant AI training and inference accelerators of the 2024 to 2025 cycle, including nvidia h200, nvidia b200, the nvidia gb200 nvl72 platform, nvidia gb300 nvl72 with 288 GB per GPU, amd instinct mi300x, amd instinct mi325x, amd instinct mi355x, and aws trainium2.[8][9][10][11][12][13][14] The technology is succeeded by HBM4, which doubles the interface width to 2048 bits and reaches 2 TB/s per stack under JEDEC JESD270-4 (April 2025).[15]

What is HBM3E and how did it evolve from HBM3?

The High Bandwidth Memory family was first standardized by JEDEC as JESD235 in 2013 and reached commercial deployment in AMD's Fiji GPU in 2015.[16] Successive generations, HBM2 (JESD235A, 2016), HBM2e (JESD235C, 2018), and HBM3 (JESD238, 2022), increased per-pin data rates and per-stack capacity while keeping the wide 1024-bit interface common to the family.[1][16] HBM3 doubled the per-pin data rate of HBM2 to 6.4 Gb/s, supported 8 Gb to 32 Gb per memory layer with stack heights up to 16-Hi, and reached 819 GB/s per device at peak data rate.[1] JEDEC published the JESD238 specification on January 27, 2022.[1]

HBM3e was not released as a separate JEDEC specification but rather as a vendor-driven extension of HBM3 with higher pin speeds, taller stacks (12-Hi), and higher-capacity 24 Gb DRAM dies.[4][17] SK hynix referred to the technology as the fifth generation in the HBM lineage when describing initial samples in August 2023.[5] The 'e' designation signals an extended revision of HBM3, with the performance gain driven by higher per-pin data rates that scale from 9.2 Gb/s in early Micron and Samsung silicon to 9.6 Gb/s in SK hynix's 12-Hi product.[4][7][18][19]

Motivation: the bandwidth wall in AI accelerators

Large transformer models are memory-bandwidth bound during inference: every generated token requires reading the model's parameters from memory, and for attention layers the key-value cache scales with sequence length.[20] GDDR-style memory bolted to a wide PCB-trace bus, even at the high pin speeds of GDDR6X and GDDR7, could not match the bandwidth or energy efficiency required by trillion-parameter models.[20] HBM addresses the bandwidth wall by mounting stacked DRAM dies directly onto the same silicon interposer or substrate as the compute die, exposing a 1024-bit per stack interface and packing multiple stacks (typically 4, 6, or 8) around the accelerator.[21] Each individual HBM die exposes 256 bits of I/O, roughly 16 times the bus width of an LPDDR chip.[21] SemiAnalysis estimates HBM3e cost at roughly 5 to 10 times the per-bit price of commodity DDR5, but the additional bandwidth and lower energy per bit make it the only viable choice for high-end AI accelerators.[20][22] By 2024, HBM revenue accounted for more than 30 percent of total DRAM value, up from a single-digit share in 2020.[41]

What are HBM3E's specifications?

HBM3e is implemented as a vertical stack of DRAM dies bonded to a base logic die using Through Silicon Vias (TSVs) and microbumps. The stack connects to the host accelerator through a 1024-bit channel arranged as 16 independent 64-bit channels with two pseudo-channels per channel, a structure inherited from the HBM3 JEDEC specification.[1][4]

ParameterHBM3 baseline (JESD238)HBM3e 8-Hi (Micron, SK hynix)HBM3e 12-Hi (Micron, SK hynix)HBM3e 12-Hi (SK hynix Sep 2024)
Interface width per stack1024 bits1024 bits1024 bits1024 bits
Pin speedup to 6.4 Gb/sgreater than 9.2 Gb/sgreater than 9.2 Gb/s9.6 Gb/s
Per-stack bandwidthup to 819 GB/sgreater than 1.2 TB/sgreater than 1.2 TB/sup to 1.22 TB/s
Stack height4-Hi to 16-Hi8-Hi12-Hi12-Hi
Per-die capacity8 Gb to 32 Gb24 Gb (3 GB)24 Gb (3 GB)24 Gb (3 GB)
Per-stack capacityup to 64 GB24 GB36 GB36 GB
Independent channels16161616
Pseudo-channels per channel2222

The standard HBM3 specification limited total stack height to 720 micrometers, a constraint preserved in early HBM3e 8-Hi parts.[21] To fit 12 dies in the same physical envelope, SK hynix made each DRAM die roughly 40 percent thinner than the 8-Hi version and used its Advanced MR-MUF (Mass Reflow Molded Underfill) process to provide thermal dissipation and warpage control across the taller stack; the company stacked twelve 3 GB chips at the same thickness as the eight-layer product to raise capacity by 50 percent.[6] Micron's 12-Hi 36 GB product targeted similar height envelopes while reporting "significantly lower power consumption" than competing 8-Hi 24 GB devices, and Micron states its 8-high HBM3e consumes about 30 percent less power than competing offerings.[7][18]

Three differences distinguish HBM3e from baseline HBM3 in shipping products. First, pin speed rises from a 6.4 Gb/s ceiling to roughly 9.2 to 9.6 Gb/s, increasing per-stack bandwidth by 44 to 50 percent.[1][4][6] Second, per-die capacity doubles from 16 Gb to 24 Gb, allowing 24 GB in 8-Hi and 36 GB in 12-Hi configurations.[7][18] Third, the technology is positioned exclusively as an AI accelerator memory: every shipping HBM3e product is paired with a TSMC CoWoS-S or CoWoS-L package containing an NVIDIA, AMD, or hyperscaler accelerator die.[23][24]

Who manufactures HBM3E?

The HBM3e supply chain is concentrated among three DRAM vendors: SK hynix, Micron, and Samsung. Each shipped products to NVIDIA and AMD on overlapping but distinct timelines.

SK hynix

SK hynix announced HBM3e samples on August 21, 2023, claiming a 1.15 TB/s per-stack bandwidth at a 9 GT/s data rate and a 40 percent speed improvement over its HBM3 product.[5] The company began volume production of 8-Hi HBM3e on March 19, 2024, becoming the first vendor to mass-produce the technology.[25] On September 26, 2024, SK hynix announced volume production of the world's first 12-layer HBM3e product with 36 GB capacity and 9.6 Gb/s per-pin speed, equivalent to 1.22 TB/s per stack.[6][26] SK hynix held an estimated 54 percent share of the HBM market in 2024, rising to 62 percent in Q2 2025 according to industry trackers.[27]

Micron

Micron began volume production of HBM3e on February 26, 2024, announcing 24 GB 8-Hi modules running at pin speeds above 9.2 Gb/s and bandwidth above 1.2 TB/s per stack.[18] The company targeted NVIDIA's H200 platform for Q2 2024 shipments.[18] On September 9, 2024, Micron announced 12-Hi 36 GB HBM3e sampling to customers, claiming over 1.2 TB/s of memory bandwidth at a pin speed greater than 9.2 Gb/s, and asserted "significantly lower power consumption" than 8-Hi 24 GB competing offerings.[7] Micron's HBM share rose from 7 percent in 2024 to roughly 21 percent in Q2 2025, overtaking Samsung in that quarter.[27]

Samsung

Samsung introduced its HBM3e product line under the brand name "Shinebolt," which the company describes as offering up to 1,180 GB/s per stack at 9.2 Gb/s in a 12-layer configuration.[28] Samsung's path to NVIDIA qualification was extended: the company demonstrated 12-Hi HBM3e at GTC 2024 in March 2024 but did not reach full NVIDIA qualification at that time.[29] Reuters and KED Global reported that Samsung passed NVIDIA's qualification tests for the 12-layer HBM3e product in September 2025, roughly 18 months after the company completed development of the chip, making Samsung the third supplier after SK hynix and Micron to clear NVIDIA's HBM3e validation.[30] Samsung's HBM market share declined from 39 percent in 2024 to 17 percent in Q2 2025 before recovering to 22 percent in Q3 2025 as additional qualifications closed.[27]

Which AI accelerators use HBM3E?

HBM3e was the dominant on-package memory for AI training and inference accelerators announced from late 2023 through 2025. Five accelerator families consumed the majority of HBM3e supply: NVIDIA's Hopper-refresh H200, NVIDIA's Blackwell B100, B200, and B300, AMD's Instinct MI325X and MI355X (the MI300X used HBM3), AWS Trainium 2, and Etched's Sohu.[8][9][10][11][12][14][31][32]

AcceleratorHBM3e capacityHBM3e bandwidthStack count and heightAnnounced or shipped
NVIDIA H200141 GB4.8 TB/ssix 24 GB 8-Hi stacksannounced November 13, 2023; shipping Q2 2024
NVIDIA B100 / B200192 GB8 TB/seight 24 GB 8-Hi stacksannounced March 18, 2024 at GTC
NVIDIA B300 (Blackwell Ultra)288 GBup to 8 TB/seight 36 GB 12-Hi stacksannounced GTC 2025; shipping mid-2025
AMD Instinct MI300X192 GB HBM3 (not HBM3e)5.2 TB/seight 24 GB 8-Hi stackslaunched December 2023
AMD Instinct MI325X256 GB HBM3e6 TB/seight 32 GB 8-Hi stackslaunched Q4 2024
AMD Instinct MI355X288 GB HBM3e8 TB/seight 36 GB 12-Hi stacksannounced June 2025
AWS Trainium296 GB HBM3e2.9 TB/sfour stacksannounced Dec 2023; GA late 2024
Etched Sohu144 GB HBM3enot disclosedsix 24 GB 8-Hi stacksannounced June 2024

NVIDIA H200

The H200 was announced on November 13, 2023 at the Supercomputing 23 (SC23) conference and is the first GPU to ship with HBM3e.[8] It uses the same GH100 die as the H100 with the memory subsystem replaced: six stacks of 24 GB HBM3e for a total of 141 GB at 4.8 TB/s, compared to five stacks of 16 GB HBM3 for 80 GB at 3.35 TB/s on the H100 SXM5.[8][33] NVIDIA states the H200 offers nearly double the memory capacity and 1.4 times the memory bandwidth of the H100, and reported the configuration delivers up to 1.9 times the inference throughput of the H100 on Llama 2 70B and 1.6 times on GPT-3 175B.[33][46] H200-powered systems from server manufacturers and cloud providers began shipping in Q2 2024.[33]

NVIDIA Blackwell B100, B200, GB200, B300, GB300

NVIDIA announced the Blackwell architecture at GTC on March 18, 2024 with the B100 and B200 data center accelerators and the GB200 Grace Blackwell Superchip.[9] The B200 is a dual-die package totaling 208 billion transistors and ships with eight HBM3e stacks for 192 GB at 8 TB/s of memory bandwidth on an 8192-bit aggregate interface.[9][34] The GB200 NVL72 rack-scale system contains 72 B200 GPUs and 36 Grace CPUs delivering 13.5 TB of HBM3e at 576 TB/s of aggregate memory bandwidth.[9][35] NVIDIA announced the Blackwell Ultra B300 at GTC 2025; the part raises per-GPU HBM3e to 288 GB by replacing the 24 GB 8-Hi stacks with 36 GB 12-Hi stacks, while keeping the same eight-stack layout.[10] The B300 reaches dense FP4 performance of 15 PFLOPS, compared to 9 PFLOPS on the B200, with GB300 NVL72 systems shipping in mid-2025.[10]

AMD Instinct MI300X, MI325X, MI355X

AMD's first-generation Instinct MI300X uses HBM3 (not HBM3e), shipping with 192 GB across eight 24 GB 8-Hi stacks at 5.2 TB/s and launching in late 2023.[11][36] AMD upgraded to HBM3e with the MI325X, announced at the Advancing AI event in October 2024 and shipping in Q4 2024: 256 GB of HBM3e at 6 TB/s, structured as eight 32 GB stacks.[12][37] The MI355X, announced in June 2025 with shipments later that year, increased on-package memory to 288 GB of HBM3e at 8 TB/s using eight 36 GB 12-Hi stacks, and added native FP4 and FP6 precision.[13][38]

AWS Trainium 2

Amazon Web Services announced Trainium 2 at re:Invent 2023; the accelerator integrates 96 GB of HBM3e per chip with 2.9 TB/s of memory bandwidth, organized in four HBM stacks.[14][39] SemiAnalysis reported that Trn2 instances initially operated the HBM3e at HBM3 speeds through firmware, with the option to clock the memory to roughly 3.2 TB/s in custom configurations.[39] Trainium 2 entered general availability in late 2024 in the Trn2 instance family, and the larger UltraServer configuration provides 6 TB of HBM3e at 185 TB/s aggregate bandwidth across 64 chips.[14][40]

Other adoption

Etched's Sohu inference ai chip (transformer-only ASIC), unveiled in June 2024, uses six stacks of 24 GB HBM3e for 144 GB of on-package memory.[32] cerebras wse 3 uses on-die SRAM rather than HBM. Various startups including Tenstorrent, MatX, and Groq have published HBM3e-using designs at conferences and in marketing materials, but production volumes remain small compared to NVIDIA and AMD.[20]

How is HBM3E manufactured and why is supply constrained?

HBM3e is integrated with the host accelerator through 2.5D advanced packaging, almost exclusively using TSMC's CoWoS (Chip-on-Wafer-on-Substrate) technology. CoWoS-S, which uses a monolithic silicon interposer, was the packaging path for NVIDIA Hopper and AMD MI300 generations; CoWoS-L, which uses local silicon interconnect bridges in a larger organic substrate, is used for NVIDIA Blackwell and Blackwell Ultra to support the larger dual-die compute layout with eight HBM stacks.[23][24] TSMC scaled CoWoS capacity from approximately 35,000 wafers per month in late 2024 to a target of 130,000 wafers per month by the end of 2026, with NVIDIA reportedly securing roughly 60 percent of the global CoWoS allocation.[24]

Supply of HBM3e was constrained throughout 2024 and 2025. TrendForce reported HBM3e supply was tight enough that suppliers raised contract prices 5 to 10 percent in 2024 alone.[41] SemiAnalysis reported that TSV yields for HBM3e ranged between 40 and 60 percent, lower than commodity DRAM yields, which compounded the production bottleneck.[22] By late 2025, Samsung and SK hynix reportedly negotiated HBM3e contract price increases of nearly 20 percent for 2026 supply as demand from NVIDIA's H200 refresh inventory and the Blackwell Ultra ramp exceeded available capacity.[42] SK hynix's dominance of the supply pool reached an estimated 62 percent of HBM revenue in Q2 2025 and 57 percent in Q3 2025, with Samsung's recovery driven by the late qualification of its 12-Hi product for NVIDIA.[27][30]

How much does HBM3E cost?

Direct per-GB pricing for HBM3e is not publicly disclosed by SK hynix, Samsung, or Micron because the parts ship under accelerator-vendor contracts rather than spot DRAM channels. Industry sources triangulate the cost. SemiAnalysis reports that the HBM stacks on a single high-end AI accelerator represent 40 to 50 percent of the bill of materials, with six to eight HBM3e stacks adding $700 to $1,500 to the manufacturing cost of a single GPU.[22] Industry analysts including IntuitionLabs estimate HBM3e at approximately $15 to $20 per GB in 2024 to 2025, roughly 5 to 10 times the per-bit price of DDR5.[43] Channel pricing for the resulting accelerators reflects this premium: NVIDIA H200 NVL cards retail in the $30,000 to $40,000 range through authorized resellers, with H200 SXM5 modules priced higher within OEM bundles.[44]

How does HBM3E compare to HBM3 and HBM4?

Compared to baseline HBM3 (JESD238), HBM3e increases per-pin data rate from 6.4 Gb/s to roughly 9.2 to 9.6 Gb/s, expands per-stack capacity from 16 to 36 GB through the introduction of 24 Gb dies and 12-Hi stacking, and lifts per-stack bandwidth from 819 GB/s to 1.2 TB/s.[1][4][6][7] The interface remains 1024 bits per stack with 16 channels and 32 pseudo-channels, preserving backward compatibility with HBM3 memory controllers.[4]

HBM4 was finalized by JEDEC as JESD270-4 in April 2025.[15] Its principal change is doubling the per-stack interface width to 2048 bits, allowing per-stack bandwidth above 2 TB/s even with conservative per-pin data rates of 8 Gb/s.[15] HBM4 also normalizes 12-Hi and 16-Hi stack heights and targets capacities up to 64 GB per stack with 32 Gb dies.[15] Initial NVIDIA Rubin and AMD MI400-series accelerators are expected to use HBM4 starting in 2026.[15][45] Compared to HBM3e, HBM4 roughly doubles bandwidth at the cost of a wider interposer footprint and tighter alignment tolerances during 2.5D packaging.[15]

What are HBM3E's limitations?

HBM3e has three principal limitations. First, capacity remains modest relative to HPC memory pools: even at 36 GB per stack and eight stacks, a single GPU tops out at 288 GB, far below the parameter count of trillion-parameter models when stored in BF16.[10] Second, the technology is supply-constrained, with TSV yields and CoWoS packaging capacity as the binding limits across 2024 and 2025.[22][24] Third, HBM3e's higher per-pin data rates push thermal density: SK hynix's MR-MUF process and Samsung's advanced TC-NCF technology aim to dissipate the additional heat across 12-Hi stacks, but stack thermals become a more pronounced constraint at 16-Hi.[6][28] These limitations motivate the move to HBM4 and its wider interface, which delivers more bandwidth per stack without further increasing per-pin data rates.[15]

See also

References

  1. JEDEC, "JEDEC Publishes HBM3 Update to High Bandwidth Memory (HBM) Standard", JEDEC, 2022-01-27. https://www.jedec.org/news/pressreleases/jedec-publishes-hbm3-update-high-bandwidth-memory-hbm-standard. Accessed 2026-05-26. ↩
  2. Servethehome, "JEDEC Publishes JESD238 HBM3 Spec For Next-Gen Accelerators", Serve The Home, 2022-01-27. https://www.servethehome.com/jedec-publishes-jesd238-hbm3-spec-for-next-gen-accelerators/. Accessed 2026-05-26. ↩
  3. VideoCardz, "JEDEC announces official HBM3 (High Bandwidth Memory) specs, up to 819 GB/s", VideoCardz, 2022-01-27. https://videocardz.com/press-release/jedec-announces-official-hbm3-high-bandwidth-memory-specs-up-to-819-gb-s. Accessed 2026-05-26. ↩
  4. Wevolver, "What is High Bandwidth Memory 3 (HBM3): Complete Engineering Guide", Wevolver, 2025-06-01. https://www.wevolver.com/article/what-is-high-bandwidth-memory-3-hbm3-complete-engineering-guide-2025. Accessed 2026-05-26. ↩
  5. SK hynix Newsroom, "SK hynix Develops World's Best Performing HBM3E", SK hynix, 2023-08-21. https://news.skhynix.com/sk-hynix-develops-worlds-best-performing-hbm3e/. Accessed 2026-05-26. ↩
  6. SK hynix Newsroom, "SK hynix Begins Volume Production of the World's First 12-Layer HBM3E", SK hynix, 2024-09-26. https://news.skhynix.com/sk-hynix-begins-volume-production-of-the-world-first-12-layer-hbm3e/. Accessed 2026-05-26. ↩
  7. Micron Technology, "Micron continues memory industry leadership with HBM3E 12-high 36GB", Micron, 2024-09-09. https://www.micron.com/about/blog/applications/ai/micron-continues-memory-industry-leadership-with-hbm3e-12-high-36gb. Accessed 2026-05-26. ↩
  8. NVIDIA Newsroom, "NVIDIA Supercharges Hopper, the World's Leading AI Computing Platform", NVIDIA, 2023-11-13. https://nvidianews.nvidia.com/news/nvidia-supercharges-hopper-the-worlds-leading-ai-computing-platform. Accessed 2026-05-26. ↩
  9. Network World, "Nvidia launches Blackwell GPU architecture", Network World, 2024-03-18. https://www.networkworld.com/article/2066534/nvidia-launches-blackwell-gpu-architecture.html. Accessed 2026-05-26. ↩
  10. NVIDIA Developer, "Inside NVIDIA Blackwell Ultra: The Chip Powering the AI Factory Era", NVIDIA Technical Blog, 2025-03-18. https://developer.nvidia.com/blog/inside-nvidia-blackwell-ultra-the-chip-powering-the-ai-factory-era/. Accessed 2026-05-26. ↩
  11. AMD, "AMD Instinct MI300X Accelerators", AMD, 2023-12-06. https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html. Accessed 2026-05-26. ↩
  12. AMD, "AMD Instinct MI325X", AMD, 2024-10-10. https://www.amd.com/en/products/accelerators/instinct/mi325.html. Accessed 2026-05-26. ↩
  13. AMD, "AMD Instinct MI355X GPUs", AMD, 2025-06-12. https://www.amd.com/en/products/accelerators/instinct/mi350/mi355x.html. Accessed 2026-05-26. ↩
  14. Amazon Web Services, "AI Accelerator AWS Trainium", AWS, 2024-12-03. https://aws.amazon.com/ai/machine-learning/trainium/. Accessed 2026-05-26. ↩
  15. JEDEC, "JEDEC and Industry Leaders Collaborate to Release JESD270-4 HBM4 Standard", JEDEC, 2025-04-15. https://www.jedec.org/news/pressreleases/jedec%C2%AE-and-industry-leaders-collaborate-release-jesd270-4-hbm4-standard-advancing. Accessed 2026-05-26. ↩
  16. Synopsys, "What is High Bandwidth Memory 3 (HBM3)?", Synopsys, 2022-04-01. https://www.synopsys.com/glossary/what-is-high-bandwitdth-memory-3.html. Accessed 2026-05-26. ↩
  17. Introl Blog, "HBM evolution: from HBM3 to HBM4 and the AI memory war", Introl, 2025-07-15. https://introl.com/blog/hbm-evolution-hbm3-hbm3e-hbm4-memory-ai-gpu-2025. Accessed 2026-05-26. ↩
  18. TrendForce, "Micron Begins Mass Production of HBM3e for NVIDIA's H200", TrendForce, 2024-02-27. https://www.trendforce.com/news/2024/02/27/news-micron-begins-mass-production-of-hbm3e-for-nvidias-h200/. Accessed 2026-05-26. ↩
  19. TweakTown, "SK hynix starts mass production of 12-layer HBM3E memory: 36GB capacity per module @ 9.6Gbps", TweakTown, 2024-09-26. https://www.tweaktown.com/news/100730/sk-hynix-starts-mass-production-of-12-layer-hbm3e-memory-36gb-capacity-per-module-9-6gbps/index.html. Accessed 2026-05-26. ↩
  20. SemiAnalysis, "Scaling the Memory Wall: The Rise and Roadmap of HBM", SemiAnalysis, 2024-09-25. https://newsletter.semianalysis.com/p/scaling-the-memory-wall-the-rise-and-roadmap-of-hbm. Accessed 2026-05-26. ↩
  21. Wevolver, "What is HBM (High Bandwidth Memory)? Deep Dive into Architecture, Packaging, and Applications", Wevolver, 2024-08-01. https://www.wevolver.com/article/what-is-hbm-high-bandwidth-memory-deep-dive-into-architecture-packaging-and-applications. Accessed 2026-05-26. ↩
  22. SemiAnalysis, "AI Capacity Constraints, CoWoS and HBM Supply Chain", SemiAnalysis, 2023-12-15. https://newsletter.semianalysis.com/p/ai-capacity-constraints-cowos-and. Accessed 2026-05-26. ↩
  23. Tom's Hardware, "TSMC's CoWoS packaging capacity reportedly stretched due to AI demand", Tom's Hardware, 2024-04-22. https://www.tomshardware.com/tech-industry/semiconductors/intel-gains-ground-in-ai-packaging-as-cowos-capacity-remains-stretched. Accessed 2026-05-26. ↩
  24. TweakTown, "NVIDIA ordering more CoWoS-L advanced packaging from TSMC: ready for more Blackwell AI GPUs", TweakTown, 2024-09-04. https://www.tweaktown.com/news/102653/nvidia-ordering-more-cowos-advanced-packaging-from-tsmc-ready-for-blackwell-ai-gpus/index.html. Accessed 2026-05-26. ↩
  25. SK hynix Newsroom, "SK hynix Begins Volume Production of Industry's First HBM3E", SK hynix, 2024-03-19. https://news.skhynix.com/sk-hynix-begins-volume-production-of-industry-first-hbm3e/. Accessed 2026-05-26. ↩
  26. PR Newswire, "SK hynix Begins Volume Production of the World's First 12-Layer HBM3E", PR Newswire, 2024-09-26. https://www.prnewswire.com/news-releases/sk-hynix-begins-volume-production-of-the-worlds-first-12-layer-hbm3e-302259251.html. Accessed 2026-05-26. ↩
  27. Astute Group, "SK hynix holds 62% of HBM, Micron overtakes Samsung, 2026 battle pivots to HBM4", Astute Group, 2025-10-15. https://www.astutegroup.com/news/general/sk-hynix-holds-62-of-hbm-micron-overtakes-samsung-2026-battle-pivots-to-hbm4/. Accessed 2026-05-26. ↩
  28. Samsung Semiconductor, "HBM3E Shinebolt", Samsung, 2024-10-01. https://semiconductor.samsung.com/dram/hbm/hbm3e-shinebolt/. Accessed 2026-05-26. ↩
  29. TweakTown, "Samsung expected to unveil world's first HBM3e 12-High memory at NVIDIA GTC 2024", TweakTown, 2024-03-15. https://www.tweaktown.com/news/96694/samsung-expected-to-unveil-worlds-first-hbm3e-12-high-memory-at-nvidia-gtc-2024/index.html. Accessed 2026-05-26. ↩
  30. KED Global, "Samsung clears Nvidia hurdle for 12-layer HBM3E supply, setting stage for HBM4 battle", KED Global, 2025-09-19. https://www.kedglobal.com/korean-chipmakers/newsView/ked202509190008. Accessed 2026-05-26. ↩
  31. AMD Hot Chips 2024 Presentation, "AMD Instinct MI300X Generative AI Accelerator and Platform Architecture", IEEE Hot Chips, 2024-08-17. https://hc2024.hotchips.org/assets/program/conference/day1/23_HC2024.AMD.MI300X.ASmith(MI300X).v1.Final.20240817.pdf. Accessed 2026-05-26. ↩
  32. Tom's Hardware, "Etched announces Sohu transformer-only chip", Tom's Hardware, 2024-06-26. https://www.tomshardware.com/tech-industry/artificial-intelligence/etched-claims-its-sohu-chip-runs-llms-20x-faster-than-an-nvidia-h100. Accessed 2026-05-26. ↩
  33. ServeTheHome, "NVIDIA H200 Launched with 141GB of HBM3e at SC23", Serve The Home, 2023-11-13. https://www.servethehome.com/nvidia-h200-launched-with-141gb-of-hbm3e-at-sc23/. Accessed 2026-05-26. ↩
  34. Wccftech, "NVIDIA Blackwell GPU Architecture Official: 208 Billion Transistors, 5x AI Performance, 192 GB HBM3e Memory, 8 TB/s Bandwidth", Wccftech, 2024-03-18. https://wccftech.com/nvidia-blackwell-gpu-architecture-official-208-billion-transistors-5x-ai-performance-192-gb-hbm3e-memory/. Accessed 2026-05-26. ↩
  35. NVIDIA, "DGX B200: The Foundation for Your AI Factory", NVIDIA, 2024-03-18. https://www.nvidia.com/en-us/data-center/dgx-b200/. Accessed 2026-05-26. ↩
  36. AMD, "AMD Instinct MI300X Platform Data Sheet", AMD, 2024-02-01. https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/data-sheets/amd-instinct-mi300x-platform-data-sheet.pdf. Accessed 2026-05-26. ↩
  37. TweakTown, "AMD details Instinct MI300X MCM GPU: 192GB of HBM3 out now, MI325X with 288GB HBM3E in October", TweakTown, 2024-06-04. https://www.tweaktown.com/news/100187/amd-details-instinct-mi300x-mcm-gpu-192gb-of-hbm3-out-now-mi325x-with-288gb-hbm3e-in-october/index.html. Accessed 2026-05-26. ↩
  38. AMD, "AMD Instinct MI355X GPU Brochure", AMD, 2025-06-12. https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/product-briefs/amd-instinct-mi355x-gpu-brochure.pdf. Accessed 2026-05-26. ↩
  39. SemiAnalysis, "Amazon's AI Self Sufficiency, Trainium2 Architecture and Networking", SemiAnalysis, 2024-12-03. https://newsletter.semianalysis.com/p/amazons-ai-self-sufficiency-trainium2-architecture-networking. Accessed 2026-05-26. ↩
  40. Amazon Web Services, "Amazon EC2 Trn2 Instances", AWS, 2024-12-03. https://aws.amazon.com/ec2/instance-types/trn2/. Accessed 2026-05-26. ↩
  41. TrendForce, "HBM Prices to Increase by 5-10% in 2025, Accounting for Over 30% of Total DRAM Value, Says TrendForce", TrendForce, 2024-05-06. https://www.trendforce.com/presscenter/news/20240506-12125.html. Accessed 2026-05-26. ↩
  42. TrendForce, "Samsung, SK hynix Reportedly Plan ~20% HBM3E Price Hike for 2026 as NVIDIA H200, ASIC Demand Rises", TrendForce, 2025-12-24. https://www.trendforce.com/news/2025/12/24/news-samsung-sk-hynix-reportedly-plan-20-hbm3e-price-hike-for-2026-as-nvidia-h200-asic-demand-rises/. Accessed 2026-05-26. ↩
  43. IntuitionLabs, "HBM vs. DDR: Key Differences in Memory Technology Explained", IntuitionLabs, 2025-03-15. https://intuitionlabs.ai/articles/hbm-vs-ddr-memory-comparison. Accessed 2026-05-26. ↩
  44. IntuitionLabs, "NVIDIA AI GPU Prices: H100 ($27K-$40K) and H200 ($315K/8-GPU) Cost Guide", IntuitionLabs, 2025-05-15. https://intuitionlabs.ai/articles/nvidia-ai-gpu-pricing-guide. Accessed 2026-05-26. ↩
  45. Tom's Hardware, "JEDEC finalizes HBM4 memory standard with major bandwidth and efficiency upgrades", Tom's Hardware, 2025-04-16. https://www.tomshardware.com/pc-components/ram/jedec-finalizes-hbm4-memory-standard-with-major-bandwidth-and-efficiency-upgrades. Accessed 2026-05-26. ↩
  46. NVIDIA, "NVIDIA H200 Tensor Core GPU", NVIDIA, 2024-03-18. https://www.nvidia.com/en-us/data-center/h200/. Accessed 2026-06-23. ↩

Improve this article

Add missing citations, update stale details, or suggest a clearer explanation.

Suggest edit