Wafer-scale chip designer Cerebras Systems has developed an AI supercomputer with 13.5 million cores capable of exacale processing.
The Andromeda supercomputer consists of 16 Cerebras CS-2 systems and uses Cerebras MemoryX and SwarmX technologies to provide over 1 Exaflop of AI compute and 120 Petaflops of dense compute at 16-bit half precision.
The CS-2 system is a 15U rack with s 850,000 AI optimized compute cores, 40GB of on-chip SRAM, 20 PB/s memory bandwidth and 220Pb/s interconnect. Each rack consumes a peak power of 23kW, giving a total power usage of 368kW. These are fed by 18,176 3rd Gen AMD x86 EPYC processor cores in 284 chips.
The SwarmX fabric, which links the MemoryX parameter storage solution to the 16 CS-2s, provides more than 96.8 terabits of bandwidth.
Andromeda has 1.6 times as many cores as the largest supercomputer in the world, Frontier, which has 8.7 million cores.
The integration of hundreds of thousands of cores on a single wafer enables the AI supercomputer to demonstrate near-perfect linear scaling on large language model workloads relying on simple data parallelism alone, unlike supercomputers that use GPUs.
Near-perfect scaling means that as additional CS-2s are used, training time is reduced in near perfect proportion. This includes large language models with very large sequence lengths, a task that is impossible to achieve on GPUs.
One of Andromeda’s first users achieved near perfect scaling on the GPT-J AI large language model at 2.5 billion and 25 billion parameters with long sequence lengths.
Access to Andromeda is available now, and customers and academic researchers are already running real workloads and deriving value from the leading AI supercomputer’s extraordinary capabilities, including:
“AMD is investing in technology that will pave the way for pervasive AI, unlocking new efficiency and agility abilities for businesses. The combination of the Cerebras Andromeda AI supercomputer and a data pre-processing pipeline powered by AMD EPYC-powered servers, together will put more capacity in the hands of researchers and support faster and deeper AI capabilities,” said Kumaran Siva, corporate vice president, Software & Systems Business Development, AMD.
Andromeda can be used simultaneously by multiple users. Users can easily specify how many of Andromeda’s CS-2s they want to use within seconds. This means Andromeda can be used as a 16 CS-2 supercomputer cluster for a single user working on a single job, or 16 individual CS-2 systems for sixteen distinct users with sixteen distinct jobs, or any combination in between.
Andromeda is deployed in Santa Clara, California, in 16 racks at Colovore, a leading high performance data centre.
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.