Axelera AI plans transformer chip variants
European edge AI chip designer Axelera AI is planning variants of its chip optimised for transformer models as it starts shipping its Metis chip in volume.
“Metis is targeting video and supports LLMs with billions of parameters,” said Fabrizio del Maffeo, CEO of Axerela AI. “We want to show that the chip delivers what we promised with the highest performance per dollar for a video stream with real time YOLO for less than $200 rather than a $1000 card,” he said.
“Full production of the chip will start in Q2 and access to more customers with general availability by by the end of the year. It’s in the fab,” he told eeNews Europe as the chip benchmarks were shown at the CES 2024 show in Las Vegas.
The chip is built on a 12nm process at TSMC and has been benchmarked at 480frame/s handling YOLO AI video analysis on 16 HD streams simultaneously running at 30frame/s each.
- Axelera launches Metis in memory computing AI edge
- RISC-V AI platform for edge vision
- Series A funding round increases to $50m
“Now we are working on the next generation to target that market domain. 99% of our customers are in the vision market and are looking at LLMs on how they can use this. There is a high interest. Metis supports vision and vision transformers and on the LLMs we can run them as the requirements are different with bandwidth and access to memory so in the future we will have something more specific as a separate product line,” he said.
The company also announced several lead customers in surveillance and factory automation for image processing and video at the CES 2024 show in the US today.
Coesia is a German-Italian group developing machines for quality control and logistics while XXII is a French pure software company that specialises in surveillance. Realy2 is designing an intelligent access point with Metis that will give access to more than the Internet and can be the brain of a building. “They are using us as they need more performance,” he said.
The full production-ready platform will be available in mid-2024 and has a projected performance up to 800 frames per second. The Metis chip matches 99% of the original model’s precision, indistinguishable from GPU-based inference models while offering 4-5 times the energy efficiency and cost savings using PCIe and M.2 Edge AI accelerator cards that are sampling now.
“Integrating YOLO into our Metis AI Platform marks a significant milestone in Edge AI inference,” said Del Maffeo. “As more and more customers harness the advantages of AI, technologies like YOLO and the Metis AI Platform enable applications previously constrained by computational power.”
Running transformer models at the edge is a key driver, he says.
“Running LLMs at the edge will be more efficient and the beauty of our architecture is we already have a lot of computing power with 200TOPS so its not so difficult to add the bandwidth,” he said.
“Once we can balance the larger memory bandwidth the problem can be solved and we have simulated that with very good results,” he said.
Competing edge AI chip developers Ambarella, Hailo and Blaize are all also working on supporting transformer models on their chips.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
