NewsNTech
Memory bandwidth is the hard physical ceiling on AI inference speed.
For every token a large-language model generates, the accelerator must repeatedly read billions of model weights from memory at high frequency, and the only memory architecture fast enough to keep pace with modern GPU compute is high-bandwidth memory (HBM), a stacked DRAM design built to maximize throughput at the die level.
The companies that design those accelerators and manufacture that memory have taken the sharpest losses in the current chip selloff.
Nvidia, SK Hynix, and Samsung Electronics have led the sector since Friday's market close, with combined chip-stock losses now exceeding $1 trillion.
Keep reading