Sandisk proposes HBF to replace HBM, enable AI at the edge
Cette publication existe aussi en Français
Sandisk Corp. is pursuing an innovation in 3D-NAND flash memory that the company claims could replace DRAM-based HBM (high-bandwidth memory) for AI inference applications.
When Sandisk, was spun off from data storage company Western Digital in February 2025, the company said it intended to deliver flash memory products while pursuing the development of emerging disruptive memory technologies.
At the Sandisk Investor Day held on February 11, shortly before the spin off, Alper Ilkbahar, the incoming senior vice president of memory technology, described high bandwidth flash as well as something he called 3D Matrix Memory.
At the same presentation Ilkbahar said that by optimizing NAND flash for bandwidth – rather than for die area and cost – the company had come up with an architecture it calls high bandwidth flash (HBF).
The approach is to divide the NAND memory array into many mini arrays and access each of these arrays in parallel. These multiple mini-arrays can be stacked in the vertical dimension using the Kioxia BICS 3D-NAND technology. This has been used to produce 16-layer R&D memories with 8 to 16x the capacity of HBM at a similar price point, Ilkbahar said.
“We are developing this with input from major AI players,” Ilkbahar told the investment analyst audience. HBF has the potential to replace HBM in datacenter GPUs and extend their use to AI-capable smartphones and other edge equipment he added.
At present a typical AI-GPU comprises two GPU logic die and eight HBMs. An upcoming AI-GPU uses these eight HBMs to provide 192Gbytes of DRAM. Ilkbahar said that using HBF could provide a component with 4Tbytes of non-volatile memory.
A demanding contemporary LLM such as GPT4 has 1.8 trillion parameters and uses 16bit weights and requires 3.6Tbytes, Ilkbahar said. “This means the whole model can be put on a single GPU and avoids a lot of data movement,” he observed. This efficiency will be important for coming multimodal models that combine text with audio and video.
For AI on smartphones the focus has been on reducing LLM sizes because of memory, performance and power limitations – but with underwhelming results. This has delayed the development of AI at the edge, Ilkbahar said. But a more advanced LLM, or one that is based on mixture-of-experts models, could have 64 billion parameters with 8bit weights requiring 64Gbytes of memory. “A single HBF die could contain the model,” said Ilkbahar.
Ilkbahar confessed that HBF could not be a drop-in replacement for HBM but said that Sandisk has decided to drive an open-standard interface based on the same electrical interface with minimal protocol changes. To that end Sandisk is forming a technical advisory board made of industry luminaries and representatives from partner companies.
Ilkbahar did not disclose the names of partners nor provide a timeline for the introduction of HBF. He did show a roadmap showing a doubling of capacity and read bandwidth and improvement in energy efficiency of 36 percent compared with the first generation of HBF.
Related links and articles:
News articles:
Sandisk to pursue emerging memory technology after split
Intel’s Optane memory business lost more than $500 million in 2020
3D flash memory hits 4.8 Gbit/s NAND interface speed
Startup Neo says its 3D DRAM offers 8x 2D capacity
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
