Esperanto runs generative-AI on RISC-V

Technology News | April 24, 2023

By Peter Clarke

Esperanto Technologies Inc. (Mountain View, Calif.) has announced it has ported a range of generative AI models to its RISC-V hardware.

Initial work includes running a range of large language models (LLMs), including Meta’s open pre-trained transformer generative AI model. Power consumption can be as low as 25W for AI inferencing on the ET-SoC-1 chip.

The ET-SoC-1 features:

1088 energy-efficient ET-Minion 64-bit RISC-V in-order cores, each with a custom vector/tensor unit optimized for ML applications
4 high-performance ET-Maxion 64-bit RISC-V out-of-order cores for running an OS in self-hosted mode
Over 160 million bytes of on-chip SRAM

Several versions of Meta’s Open Pre-Trained Transformer (OPT) model are now running on Esperanto’s hardware at multiple precision levels and context sizes with power levels as low as 25W per chip for inferencing.

Esperanto said it plans to provide access to researchers in the RISC-V community to help accelerate development of generative AI technology on RISC-V.

Generating research

“Generative AI is one of the latest advancements in machine learning, and we are pleased to contribute elements of our efforts in the area of large language models to the RISC-V research community,” said Art Swift, CEO of Esperanto, in a statement.

“RISC-V offers unparalleled opportunities for collaboration and customization, making it ideally suited for this next wave of AI innovation,” said Calista Redmond, CEO of RISC-V International. “Esperanto is one of the companies leading the charge in this space, pushing the limits of performance and power-efficiency to make generative AI development more accessible.”

Esperanto is currently shipping AI evaluation servers in a standard 2U-high form factor, each Esperanto evaluation server includes dual Xeon host processors and either 8 or 16 ET-SoC-1 PCIe cards. Thus a 2U server can contain more than 16,000 RISC-V CPUs.