Loading history…
Loading history…
|
COMPUTER MEMORY
High Bandwidth Memory
HBM
| |||||||||||
| Type | Stacked DRAM[1] | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Standardised | October 2013 (JESD235)[2] | ||||||||||
| First product | AMD Radeon R9 Fury X, June 2015[3] | ||||||||||
| Current standard | HBM4 (JESD270-4, 2025)[4] | ||||||||||
| Used in | AI accelerators, supercomputers, some graphics cards[4] | ||||||||||
[ More details ▾ ]
| |||||||||||
High Bandwidth Memory, or HBM, is a form of DRAM built by stacking memory chips on top of one another and wiring them together vertically. The stack sits beside the processor inside the same package, rather than out on the motherboard, and connects to it over an interface thousands of wires wide.[1][4] It is the memory that artificial-intelligence accelerators are built around, and the most valuable product the memory industry makes.
The reason for all this trouble is bandwidth. A modern accelerator can perform arithmetic far faster than ordinary memory can supply numbers for it to work on, so the memory, not the processor, sets the pace. HBM attacks the problem from the opposite direction to conventional memory: instead of driving a narrow connection at a very high clock rate, it drives an extremely wide one comparatively slowly.[2] That turns out to use far less energy for each bit moved, which matters enormously when a data centre's electricity bill is the limiting factor.[2][3]
HBM is an open standard, published by JEDEC, the industry body that also defines ordinary DDR memory.[4] Work on it began around 2010, the first version was adopted in October 2013, and SK hynix developed the first product with AMD.[2][6] For its first decade it was a niche part for graphics cards and supercomputers. The AI boom changed that completely, and only three companies — SK hynix, Samsung Electronics and Micron Technology — can make it.[5]
Ordinary computer memory is connected by a narrow path driven very fast. A GDDR7 graphics memory chip, for instance, presents a 32-bit interface and pushes it to 32 gigabits per second on each pin, delivering about 128 gigabytes per second per chip.[7] Driving signals that fast across a circuit board is expensive in power, and the faster you go the worse it gets.
HBM inverts the trade. An HBM4 stack presents 2,048 bits at a comparatively sedate 8 gigabits per second per pin, which yields up to two terabytes per second — roughly fifteen times a GDDR7 chip.[4][7] The original design was measured at around 6 to 7 picojoules for each bit moved, against 18 to 22 for the GDDR5 memory of the day.[2] When AMD shipped the first HBM graphics card it claimed 60% more bandwidth than GDDR5, more than three times the performance per watt, and 94% less board area.[3]
The catch is that 2,048 wires cannot be routed across a circuit board. That constraint is what forces everything else about HBM: the stacking, the special packaging, and the cost.
The dies in a stack are joined by through-silicon vias, copper connections etched straight down through a thinned wafer so that a chip can pass signals to the one above it.[1] There are a great many of them: Samsung's twelve-layer packaging used more than 60,000 in a single stack while holding the whole package to 720 micrometres tall.[8] Fitting more dies into that budget means grinding each one thinner. For its sixteen-layer HBM4, SK hynix thins the wafers to 30 micrometres, about a third the width of a human hair.[9]
At the bottom of every stack sits a base die, sometimes called the logic die, which handles the traffic between the memory above it and the processor beside it. It used to be made on a memory process, but HBM4 changed that. SK hynix agreed with TSMC in April 2024 to have its HBM4 base die built on a logic foundry process, and the part now uses TSMC's 12-nanometre technology.[10][9]
Samsung took the opposite route, building its HBM4 base die on its own four-nanometre foundry process, which makes it the only supplier that controls the entire stack.[11] Micron has contracted TSMC to make the base die for its next generation.[12] A logic base die also opens the door to customisation: SK hynix has said it intends to move beyond standard parts to memory tailored to individual customers.[13]
The stacks and the processor are mounted side by side on a slab of silicon called an interposer, which carries the thousands of connections between them. This arrangement is known as 2.5D packaging. The dominant version is TSMC's CoWoS, short for chip on wafer on substrate, which places logic chips and HBM stacks together over a large silicon interposer.[14]
Interposer capacity has been a hard limit on how many AI accelerators the world can build. TrendForce put the gap between demand and supply for TSMC's packaging at about 20% in mid-2026, narrowing to roughly 10% by the end of the year as capacity comes online.[15] JEDEC published a variant in June 2026 with a narrower 512-bit interface, meant to let cheaper designs use an ordinary organic substrate instead of silicon.[16]
How the dies are joined matters more than it sounds. SK hynix uses a method it calls MR-MUF, which heats and connects every chip in the stack at once and fills the gaps with a moulding compound in the same step. The alternative, TC-NCF, lays a film between chips and bonds them one layer at a time.[17] SK hynix credits its method with better heat dissipation, and says a refined version raised the thermal conductivity of the compound by more than half.[17][18]
Heat is a real constraint, because a stack of memory sits millimetres from a processor running near its thermal limit and the dies in the middle have nowhere to send their heat. Researchers at imec simulated putting HBM directly on top of a graphics processor rather than beside it and found the assembly reached 140 °C, hot enough to stop working.[19] Hybrid bonding, which would do away with the solder bumps between dies altogether, has been expected as the answer for the tallest stacks. It was not used in HBM4, and reports in July 2026 suggest sixteen-layer HBM4E is now the earliest likely adoption.[20]
All of this is expensive. A gigabyte of HBM consumes more than three times the wafer area of a gigabyte of ordinary DRAM, so making it eats capacity that would otherwise produce far more memory.[21] TrendForce estimated HBM4 wafers at US$7,000 to US$8,000, three to four times the cost of conventional DRAM wafers.[22]
Each generation has roughly doubled the bandwidth of the one before, at first by raising the signalling rate and, with HBM4, by widening the interface.[2][23][24][25][4]
| Generation | JEDEC standard | Interface | Peak bandwidth per stack |
|---|---|---|---|
| HBM | JESD235, October 2013 | 1,024-bit | 128 GB/s |
| HBM2 | JESD235A, announced January 2016 | 1,024-bit | 256 GB/s |
| HBM2E | JESD235B, announced December 2018 | 1,024-bit | 307 GB/s |
| HBM3 | JESD238, January 2022 | 1,024-bit | 819 GB/s |
| HBM3E | extension of JESD238 | 1,024-bit | about 1.2 TB/s |
| HBM4 | JESD270-4, April 2025 | 2,048-bit | 2 TB/s |
The first version reached 128 gigabytes per second per stack, and AMD put four stacks on the Radeon R9 Fury X in June 2015 to reach 512 gigabytes per second in total.[2][3] HBM2 doubled that and introduced the pseudo-channel arrangement still used today; the first product built on it was NVIDIA's Tesla P100 accelerator in 2016.[23][26] A 2018 revision raised capacity to 24 gigabytes and allowed twelve-high stacks, and SK hynix began making 16-gigabyte parts at 3.6 gigabits per second in July 2020.[24][27]
HBM3, published in January 2022, doubled the pin rate again to 6.4 gigabits per second and 819 gigabytes per second per device. It also cut the core voltage to 1.1 volts and added error correction on the die itself.[25] The faster parts sold as HBM3E are not a separate JEDEC standard but an industry extension of HBM3; manufacturers were shipping them before the higher speed grades were formally settled.[28] SK hynix began volume production of a twelve-layer, 36-gigabyte HBM3E running at 9.6 gigabits per second in September 2024.[29]
HBM4, published on 16 April 2025, is the biggest change since the format was invented. It doubles the interface to 2,048 bits and the channel count to 32, allows stacks of up to sixteen dies and 64 gigabytes, and adds a mechanism for defending against the Rowhammer attack.[4] SK hynix announced it had finished developing HBM4 and was ready to manufacture it in September 2025, running above 10 gigabits per second — faster than the standard requires — with more than 40% better power efficiency.[30] Samsung shipped the first commercial HBM4 in February 2026, at 11.7 gigabits per second and 3.3 terabytes per second per stack.[11] A further generation, HBM5, is in development at JEDEC; no specification has been published.[31]
HBM began in graphics. AMD used it on the Fury X and then the Vega cards, and NVIDIA put it in the Tesla P100 for scientific computing.[3][26] It never reached mainstream consumer graphics cards, which still use GDDR, because the packaging costs too much for the price bracket.[22]
Machine learning made it essential. Training and running large models means moving vast quantities of numbers between memory and arithmetic units, and an accelerator starved of data sits idle no matter how fast it can calculate. Every serious AI accelerator now uses HBM, and demand from that one application reshaped the memory industry around it.[4][32] In June 2026 NVIDIA certified all three memory makers to supply HBM4 for its Vera Rubin platform, the first time every supplier had been cleared for a flagship product at launch.[33]
Only three companies make HBM, and SK hynix has led since the market became important. Counterpoint Research put its share of HBM revenue at 58% in the first quarter of 2026, with Samsung Electronics and Micron Technology on 21% each. That is a narrower lead than the 69% it held a year earlier.[5]
Getting there took the others a long time. Samsung spent roughly eighteen months and several attempts qualifying its twelve-layer HBM3E with NVIDIA, finally passing in September 2025.[34] Both it and Micron have moved faster on HBM4. Samsung passed a billion dollars of HBM4 revenue within about four months of starting production, and Micron reported shipping more than a billion dollars' worth by June 2026.[35][36]
HBM now consumes a large and growing slice of the world's DRAM production. TrendForce estimated it at about 18% of DRAM wafer input at the end of 2025, rising to roughly 22% by the end of 2026. That buys only about 9% of the bits produced, the gap being the wafer area each HBM bit costs.[32] Squeezing ordinary memory out of the fabs this way is a large part of why conventional DRAM prices rose so sharply in 2026.[32]
That created an odd situation in pricing. HBM is sold on annual contracts, so its price did not move when conventional memory spiked; TrendForce reported that revenue per wafer for HBM was overtaken by ordinary server memory in the first quarter of 2026, and expects HBM contract prices to rise substantially in 2027 as suppliers reprice.[32] Meanwhile SK hynix told investors in April 2026 that it expects HBM demand to run ahead of supply for at least three years.[37]
HBM has also become an instrument of trade policy. In December 2024 the United States imposed export controls covering high bandwidth memory, restricting sales to China both of American-made parts and of foreign-made parts caught by the rules on advanced computing.[38] Chinese manufacturers are working on their own: ChangXin Memory Technologies is reported to be targeting HBM3 production, though the country's makers remain several generations behind.[39]
Content available under CC BY-SA 4.0