Tech companies are exploring a major shift in HBM design seeking to embed GPU cores directly into next-generation memory stacks. According to Korean industry reports, Meta and NVIDIA are evaluating " custom HBM " architectures that integrate GPU cores into the base die of future HBM devices, with SK Hynix and Samsung said to be involved in early discussions. HBM stacks multiple DRAM dies on top of a base die that handles external I/O. HBM4 is expected to reach mass production next year and will include an onboard controller to improve bandwidth and efficiency. Integrating GPU cores would take this concept several steps further, distributing compute into the memory itself to reduce data movement and cut power usage. Industry sources say the approach could boost both performance and energy efficiency for AI workloads by cutting the distance between compute and memory. But the design still faces major struggles such as limited die area in Through-Silicon Vias (TSV) based stacks, power delivery constraints, and the difficulty of cooling those compute-heavy GPU logic inside the base die.

Kim Joung-ho, a professor in the School of Electrical Engineering at KAIST, said, "The speed of technological transition where the boundary between memory and system semiconductors collapses for AI advancement will accelerate," and added, "Domestic companies must expand their ecosystem beyond memory into the logic sector to preempt the next-generation HBM market." AMD's recently unveiled Instinct MI430X accelerator built on the next-generation AMD CDNA architecture supports 432 GB of HBM4 memory and 19.6 TB/s of memory bandwidth. NVIDIA's "Vera Rubin" Superchip takes a different approach. Each Rubin GPU integrates two reticle-sized compute chiplets paired with eight HBM4 stacks, delivering around 288 GB of HBM4 per GPU and roughly 576 GB of HBM4 across the full Superchip. From those potential changes companies that have strong packaging and logic capabilities will benefit, while pure memory vendors may need to expand into system-level semiconductor technologies to stay competitive.