Chatbot frenzy driving HBM demand, CoWoS shortage
Leading cloud service providers have been buying AI servers in 2023 driving demand for HBM memory and advanced chip packaging, according to TrendForce.
The market research firm said Microsoft, Google, Amazon Web Services, Baidu and ByteDance, have all invested heavily in high-end AI servers. As a result, advanced packaging capacity is expected to increase by 30 to 40 percent in 2024 compared with 2023 but may still not meet demand.
The leading suppliers of AI chips for the datacenter, such as Nvidia, AMD and Intel, have opted to incorporate the HBM interface standard. Nvidia’s A100 and H100 chips each boast up to 80 Gbyte of HBM2e and HBM3 memory, In Nvidia’s integrated CPU and GPU, the Grace Hopper chiplet component, the company expanded a single chip’s HBM capacity by 20 percent to hit 96Gbyte.
AMD’s MI300 chip uses HBM3, with the MI300A capacity remaining at 128Gbyte but the more advanced MI300X ramped up to 192Gbyte. Google is expected to broaden its partnership with Broadcom in late 2023 to produce the AISC AI accelerator chip TPU, which will also incorporate HBM memory, in order to extend AI infrastructure.
The demand for HBM memory is escalating demand for advanced packaging, with TSMC’s Chip on Wafer on Substrate (CoWoS) being the predominant choice for AI server chips. The demand for CoWoS has increased by almost by 50 percent since the onset of 2023, driven by Nvidia’s A100 and H100 and associated AI server sales, TrendForce said.
CoWoS capacity is likely to be strained in 2H23 as demand from customers and AMD, Google, and other companies intensifies. This strong demand is anticipated to continue into 2024, with a projected growth of 30 to 40 percent in advanced packaging capacity.
Other bottlenecks could occur in Through Silicon Via packaging, the generation of interposer circuit boards and availability of associated wet processing equipment. As a result Nvidia may explore advanced packaging alternatives from Amkor and Samsung, TrendForce said.
Related links and articles:
News articles:
SK Hynix capitalizes on investment in HBM
Rapid Silicon lets engineers use GPT for FPGA design
ChatGPT leaking Samsung chip secrets is iceberg’s tip
SpiNNaker2 spiking neural network project gets €2.5 million
Microsoft announces multibillion investment in OpenAI
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
