AI computing power continues to expand, but the growth rate of memory bandwidth is far behind. Even HBM (High Bandwidth Memory) faces problems such as soaring power consumption, high packaging costs, and increasing heat dissipation. Qualcomm officially unveiled its new structure at the 2026 Investor Day HBC（High-Bandwidth Compute）, trying to fundamentally change the relationship between AI accelerators and memory, no longer relying on HBM, but stacking computing units directly under LPDDR memory.
Qualcomm releases HBC near memory architecture: putting AI accelerator under DRAM
Memory Wall: The Biggest Bottleneck of AI Inference
The so-called “Memory Wall” refers to the fact that the increase in memory bandwidth cannot keep up with the increase in computing power demand. This problem is particularly serious in large-scale language model inference scenarios: the generation of each token requires a large amount of memory access. When bandwidth is insufficient, no matter how powerful the computing power is, it cannot be fully utilized. At the same time, power consumption and TCO (total cost of ownership) continue to rise.
HBM is currently the standard equipment of AI accelerators, but HBM’s packaging process is complex (requiring a silicon interposer), design cost is high, and power consumption and heating problems are becoming increasingly serious. Qualcomm’s HBC architecture has chosen a different path: replacing HBM with LPDDR as the memory choice, and tightly integrating computing units with memory through 3D stacking.
How HBC works
The core design of HBC is to separate the dedicated near-memory accelerator from the SoC and stack it below the LPDDR memory stack, with a TSV (Through Silicon Via)Direct connection. TSV is a conductive channel that vertically penetrates the silicon wafer. It can realize high-speed signal transmission between chips in a very small space. It is the core process of 3D packaging technology.
This design brings several key advantages:
- Latency down to SRAM levels: Due to the extremely close physical distance between the computing unit and the memory, the data transmission path is greatly shortened, and the delay can be reduced to the level of traditional SRAM.
- High density and large capacity: Retains the capacity advantage of stacked memory, and the unit power consumption capacity is more than 200 times that of on-chip SRAM.
- Avoid the pain points of HBM: No silicon interposer is required, the packaging process is simpler, the cost is lower, and the power consumption and heat generation are also lower
- LPDDR mature supply chain: Choosing LPDDR instead of HBM as the memory medium can make use of the existing mature process and supply system to reduce mass production risks.
Qualcomm builds the HBC architecture on four technology foundations: 3D integration leadership, system-level design, LPDDR technology leadership, and power efficiency expertise.
Official data: 6 times energy efficiency, 200 times capacity
According to data released by Qualcomm, the HBC architecture significantly surpasses existing solutions in key indicators:
- Bandwidth per unit power consumption: 5 to 7 times that of HBM
- Unit power consumption capacity: More than 200 times that of on-chip SRAM
- HBC Gen1 (AI250 accelerator): The effective bandwidth of each accelerator card reaches 133 TB/s, 18 times higher than the previous generation AI200 with LPDDR5X
- HBC Gen2 (AI300 accelerator): The effective bandwidth is 54 times higher than that of AI200, and the bandwidth per unit power consumption is 7 times that of HBM.
Product Roadmap: Gen1 2027, Gen2 2028
Qualcomm has announced a clear iteration schedule:
- HBC Gen1 + AI250: Commercial sample testing is expected to start in mid-2027
- HBC Gen2 + AI300: Expected to be launched in 2028 to enhance expansion capabilities
The HBC architecture is an important part of Qualcomm’s “Dragonfly” data center product system. The entire platform is oriented to the computing power needs of generative AI and intelligent agents, integrating CPU, AI accelerator, and near-memory architecture, with the goal of continuously reducing the unit computing power cost of AI inference.
Not an original concept, but Qualcomm was the first to give a timetable
The architectural idea of Near-Memory Computing is not the first of its kind by Qualcomm. Many storage manufacturers and chip design companies are researching similar technologies, but most have failed to implement them on a large scale.
For example, ASIC manufacturer GUC recently launched DRAM-on-Logic（DoL）technology, stacking 1 to 4 layers of DRAM on a logic chip, the bandwidth can reach about 5 TB/s, which is even better than some HBM3E solutions. In addition, SanDisk is also exploring options to stack NAND and computing on the same die in an attempt to solve the HBM shortage. Samsung Electronics also recently launched UFS 5.0 flash memory, which has twice the read and write bandwidth compared to UFS 4.1, showing that the entire industry is looking for ways to break through the memory bottleneck.
Qualcomm’s difference lies in that it doesn’t just show a technology prototype, but gives a specific product roadmap and mass production schedule. HBC Gen1 with AI250 accelerator is expected to be sample tested in mid-2027, and Gen2 with AI300 is expected to be launched in 2028. This takes HBC from a “laboratory concept” to the “expected product” stage.
It is also worth noting that JEDEC has recently approved the SPHBM4 standard, trying to replace HBM’s expensive silicon interposer packaging with standard packaging and retain the speed of HBM4 level. This means that the industry is simultaneously breaking through the memory wall problem from multiple directions.
Advantages and limitations
The advantages of the HBC architecture are clear:
- Avoid HBM’s high-cost packaging and high power consumption issues
- LPDDR’s mature supply chain reduces mass production risks
- 3D stacking enables tight coupling of computing and memory
- Solve the memory wall problem at the architectural level rather than relying solely on process scaling
But there are also obvious limitations to observe:
- Absolute bandwidth and capacity are not as good as HBM: Qualcomm did not announce specific values, but only emphasized the advantage of “unit power consumption”. In scenarios that require extreme absolute bandwidth (such as very large-scale model training), HBM may still be the first choice
- The ecosystem has not yet been established: There is currently no software ecosystem or third-party support, and developers need time to adapt to the new architecture.
- Mass production time is still far away: Gen1 will not start sample testing until mid-2027, and actual large-scale mass production may have to wait until after 2028
- 3D Stacked Cooling Challenge: With the computing unit sandwiched between the memory and the substrate, thermal management will be a major engineering challenge.
- Compatibility with existing ecology: Existing AI frameworks and software stacks are designed around HBM, and HBC needs to prove its software compatibility.
What it means to the AI industry
The emergence of HBC represents an important trend: the bottleneck of AI computing is shifting from “insufficient computing power” to “insufficient memory bandwidth”. While the computing power of GPUs and AI accelerators continues to grow exponentially, memory bandwidth increases far behind, forming a so-called memory wall. Qualcomm chose to solve this problem from the architectural level rather than the process level, using 3D stacking to redefine the spatial relationship between computing units and memory.
If HBC can be delivered as scheduled in 2027 and verify its performance commitments, it will provide a different technology route for the AI inference market than HBM. This also has profound implications for the entire supply chain: SK Hynix and Samsung are currently the dominant players in the HBM market, and HBC, based on LPDDR, could change the competitive landscape of the memory industry. SK Hynix has recently announced that it will give priority to the production of DDR5 general-purpose DRAM, showing that HBM’s era of high profits is facing structural changes.
Source: KOCPC Chinese