Blaize to add generative AI to edge development tool

Blaize to add generative AI to edge development tool

Technology News |
By Nick Flaherty

Edge AI chip and software developer Blaize is to add generative AI frameworks such as Llama 2 to is development tools to run on its chips.

Blaize has already used generative AI to help develop applications, but is now looking to support a wide range of generative AI based on transformer networks.

Generative AI can be particularly useful for automotive and IoT applications, for example blurring facing to preserve the anonymity of pedestrians or airport passengers. This is an area where other edge AI chip developers such as Hailo, Ceva and Quadric are also working to port and support transformer-based networks.

“Long term we have significant automotive partnerships with Denso and other OEMs in their programmes with chips with AECQ100 qualification in development,” said Dinakar Munagala, CEO and Co-founder of Blaize, which is setting up an edge AI joint venture in the United Arab Emirates. “We’ve been working on automotive customers for six and a half years but for industrial grade devices on the factory floor, in retail, in airports the low hanging fruit was video analytics.”

Embedded generative AI applications

Generative AI can be used for a wide range of applications at the edge in embedded chips. “This can be used for blurring and anonymity but also for sentiment prediction, combining gesture recognition and facial expressions to identify missed business or for incident management,” said Val Cook, Chief Software Architect and co-founder at Blaize.

“We demonstrated transformers in early days and we are working on the newer transformers,” said  Dmitry Zakharchenko, VP of research and product at Blaize.

“What we are seeing from customer conversations, the generative AI we know today with large language models is facing the same challenges of anonymization, as people are refusing to upload data to those models so the smaller models are becoming more efficient and effective. Some of the models we are seeing with less than 7bn are delivering higher accuracy in specific domains so enterprises are realising they need more specialised models and those are very good fit for our software platform,” he said.

“The challenge today is that there will be hundreds of thousands of smaller models running at the edge and being to get those to perform is a huge challenge and with Studio AI we can get the smaller models up and running. Some will be airgapped, [without network connections for security reasons] and this is where we think our platform will be more effective at getting genAI where it needs to be. Where we are really seeing this is with customers in federal applications for example processing satellite feeds narrowly trained and locked to that domain. “

“We can get those models to perform and run,” he said. “We have Llama 2 deployed at the edge and when we saw the framework drifting we could retrain and redeploy it.”

AI Studio development tool

“Blaize AI Studio already has non-linear unsupervised learning and we are adding the ability to use ChatGPT and as new technologies come out we can swap those out,” said Zakharchenko. “Studio will become a huge portal, with example code that developers can put into production with two clicks and inspect the code from the interface or conversational interface.”  

“With non linear systems you are in the wild west but as we process a lot of meta data we have a clearly defined generative system,” he said. “Llama 2 is very promising, but there are some others, it is part of the roadmap and we are working on that as we speak and we expect a big release in Q4.”

If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :    eeNews on Google News


Linked Articles