FuriosaAI unveils power-efficient AI processor at Hot Chips
FuriosaAI (Seoul, South Korea) has unveiled its RNGD (aka Renegade) data center inference accelerator at Hot Chips 2024, claiming superior power efficiency to GPU-based AI solutions.
The accelerator is described as a Tensor Contraction Processor that performs high-performance large language model (LLM) and multimodal model inference.
The chip is implemented in TSMC 5nm manufacturing process and is designed to operate at a clock frequency of 1.0GHz. This produces a performance of 256TFLOPS with a BF16 data type and 512TFLOPS at FP8 and 512TOPs with INT8 data type. The RNGD has 256Mbytes of on-chip SRAM and can be linked to 48Gbytes of external HBM3 DRAM. The bandwidth of the chip is 1.5Tbytes per second.
SemiFive helps FuriosaAI ‘Warboy’ processor get to market
RNGD has been tested running large language models such as GPT-J and Llama 3.1. A single RNGD PCIe card delivers 2,000 to 3,000 tokens per second throughput performance (depending on context length) for models with around 10 billion parameters, the company said. Further improvements are expected with software compiler optimizations.
The RNGD PCIe card has a thermal design profile (TDP) of 150W, which compares with more than a kilowatt required for GPU-based solutions, the company said.
“RNGD is a sustainable and accessible AI computing solution that meets the industry’s real-world needs for inference,” said June Paik, co-founder and CEO of FuriosaAI, in a statement.
“The Furiosa RNGD AI Inference solution drives the adoption of green computing with Supermicro. By integrating Furiosa’s technology, Supermicro systems can reduce power consumption per card while still delivering exceptional inference performance,” said Vik Malyala, senior vice president for technology and AI at Supermicro, in the same statement.
The chip is currently sampling to early access customers, with broader availability expected in early 2025.
Furiosa was founded in 2017 by three engineers with experience gained at AMD, Qualcomm, and Samsung.
Related links and articles:
News articles:
Merger terms agreed to form Korean AI unicorn
Korean AI startup Rebellions gets US$15 million Saudi backing
Korean AI chipmaker Rebellions closes $124 million funding round
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
