Nvidia’s first HBM3e processor targets generative AI
Cette publication existe aussi en Français
Nvidia has announced the next generation of its Grace Hopper processor technology, supporting high performance HBM3e memory for the first time for generative AI applications.
The GH200 platform will be based on a new Grace Hopper chip with ARM-based CPU and next generation GPU and HBM3e memory. A dual chip platform configuration will be a single server with 144 ARM Neoverse cores, eight petaflops of AI performance and 282GB of HBM3e. This is 3.5x more memory capacity and 3x more bandwidth than the current offering.
The company is seeing huge demand for its GPUs to run generative AI frameworks in data centres, many of which are memory-bound.
- Generative AI for chip manufacturing
- Keeping Generative AI on track
- $1.3bn to build world’s largest AI supercomputer
“To meet surging demand for generative AI, data centres require accelerated computing platforms with specialized needs,” said Jensen Huang, founder and CEO of Nvidia at the SIGGRAPH conference this week. “The new GH200 Grace Hopper Superchip platform delivers this with exceptional memory technology and bandwidth to improve throughput, the ability to connect GPUs to aggregate performance without compromise, and a server design that can be easily deployed across the entire data centre.”
The Grace Hopper chips are connected together using the NVLInk interconnect to deploy the giant models used for generative AI. This high-speed, coherent technology gives the GPU full access to the CPU memory, providing a combined 1.2TB of fast memory when in dual configuration.
HBM3e memory, which is 50% faster than current HBM3, delivers a total of 10TB/sec of combined bandwidth, allowing the new platform to run models 3.5x larger than the previous version while improving performance with 3x faster memory bandwidth.
- NVIDIA launches DGX Cloud, an AI supercomputing cloud
- Cerebras delivers 4exaFLOP AI supercomputer
- Generative AI helps prototype electric racecar smash world
Leading server makers are already offering systems based on the previously announced Grace Hopper chip. The GH200 will be fully compatible with the current Nvidia MGX server specification unveiled at COMPUTEX earlier this year. With MGX, any system manufacturer can quickly and cost-effectively add Grace Hopper into over 100 server variations.
Leading system manufacturers are expected to deliver systems based on the platform in Q2 of 2024.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
