AndesAIRE core and tools target edge AI

AndesAIRE core and tools target edge AI

Technology News |
By Nick Flaherty

Andes Technology has developed a scalable AI core aimed at machine learning at the edge that can be used alongside its RISC-V processor cores.

AndesAIRE (Andes AI Runs Everywhere) includes a first generation 8bit integer AI/ML hardware accelerator intellectual property (IP), the AnDLA I350 (Andes Deep Learning Accelerator), shown above, and the neural network software tools and runtimes, the AndesAIRE NN SDK

The core is aimed at low power AI inference at the edge of the network and at sensor end points for the Internet of Things (IoT) with added sensor fusion and analysis capabilities.

The AnDLA I350 supports the popular deep learning frameworks, including TensorFlow Lite, PyTorch, and ONNX, and performs versatile neural network operations such as convolution, fully-connect, element-wise, activation, pooling, channel padding, upsample, concatenation, etc. in the int8 data type.

The accelerator has been designed with an internal Direct Memory Access (DMA) and local memory to address the memory bandwidth issues of AI inference as well as four 64bit AXI bus interfaces..

The key configurable parameters of the core include the number of multiple accumulate units (MACs) which can scale from 32 to 4096, and the SRAM can scale from 16KB to 4MB. Overall, the core can provide 64 GOPS to 8 TOPS (at 1 GHz) for a wide range of applications.

Tools are a vital part of custom edge AI development. The AndesAIRE NN SDK is a comprehensive set of software tools and runtimes for end-to-end development and deployment. It includes a neural network optimization tool suite called NNPilot and an AnDLA-optimized inference framework running on a host based on TensorFlow Lite for Microcontrollers (TFLM) as well as the AnDLA driver and runtime for bare metal implementations.

NNPilot can automatically analyze input NN models, apply model pruning and quantization, and generate AnDLA executable based on its configuration to perform inference together with the TFLM framework. This also generates sample host C code to invoke the AnDLA driver in the bare metal environment.

This allows the core to support AlexNet, VGG-16/19, MobileNet-v1/v2/v3, ResNet-8/50, Tiny YOLO v1/v2, YOLO v1/v2/v3/v4/v5, SSD MobileNet v1/v2, Inception v2, EfficientNet-lite, MobileFaceNet and BlazeNet for image and video inference, and LSTM, RNN and GRU networks for Speech/Voice and audio.

Andes sees this as part of an extensible AI subsystem, integrating AndesAIRE, AnDLA, the AndesCore RISC-V CPU, and the Andes Custom Extension (ACE). In such a subsystem, most structural and time-consuming parts of the AI workloads can be computed efficiently in the AnDLA while less structural computations such as non-linear functions can be offloaded to the powerful and flexible RISC-V CPUs with DSP/SIMD or Vector extensions.

The ACE plays a key role for efficient data movement between the CPU and the AnDLA to reduce significant memory bandwidth and power consumption while increasing hardware utilization. The ACE can use custom instructions for domain-specific applications to provide faster processing at lower power consumption.

Andes says it will continue to develop the AndesAIRE NN SDK and AndesAIRE NN Library for the mass-production SoCs with future AI algorithms. It has already added over a hundred compute library APIs yearly since 2021.

“Andes Technology is thrilled to introduce our new product line for AI/ML solutions, the AndesAIRE AnDLA I350 and the AndesAIRE NN SDK, an exceptional performance-efficient hardware and an end-to-end software solution for edge and end-point inference,” said Dr. Charlie Su, CTO and President of Andes Technology in Taiwan.

“AndesAIRE product line projects our vision for the AI/ML market,” said Simon TC Wang, Senior Technical Marketing Manager of Andes Technology. “By combining the advantages from RISC-V CPUs, the AnDLA™, and the ACE into an AI subsystem, the performance, power consumption, and area can be well-balanced for customers to deliver competitive solutions. Furthermore, the flexibility is guaranteed by RISC-V CPUs and NN software stack, and the extensibility is affluent for customers to differentiate their unique value for target AI/ML applications.”

The AndesAIRE AnDLA I350 and the AndesAIRE NN SDK are available for licensing now in an early-adopter program.

If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :    eeNews on Google News


Linked Articles