Chiplet-base generative AI platform raises LLM performance
Cette publication existe aussi en Français
Generative AI technology provider d-Matrix Inc. (Santa Clara, Calif.), has announced Jayhawk II, a second generation of its generative AI compute platform.
The silicon provides an enhanced version of the digital in-memory-compute (DIMC) engine through the use of chiplet interconnect.
The silicon delivers a 40x improvement in memory bandwidth when compared to the state-of-the-art high-end GPUs the company said. d-Matrix also claimed that this allows Jayhawk II to handle between 10x and 20x more generative inferences per second for large language model (LLM) sizes ranging from 3 billion to 40 billion parameters compared to state-of the-art GPU solutions. This translates into a 10x to 20x better total cost of ownership for generative inference when compared to these GPU solutions, the company states.
The silicon demonstrates a DIMC architecture coupled with the OCP Bunch of Wires (BoW) PHY interconnect standard for low-latency AI inference on large language models (LLMs) from data center scale LLMs like ChatGPT to more focused models like Meta’s Llama2 or Falcon from the Technology Innovation Institute.
The DIMC engine that scales from 30 TOPS per watt to150 TOPS per watt and is implemented in a 6nm manufacturing process technology. The engine supports floating point and block floating point data types across a range of precisions. It supports compression and sparsity approaches enabling prompt caching for generative AI models.
Jayhawk II is now available for demos and evaluation.
Related links and articles:
News articles:
d-Matrix delays chiplet processor to better address generative AI
SemiFive helps bring up Nvidia-beating Chatbot processor
South Korean startup Rebellions launches AI processor
Nvidia hobbles A100 chip to meet US export control rules
Chinese chiplet-based GPU claims performance record
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
