Edge AI SDK leverages LLMs to reduce reliance on GPUs
Advantech has announced GenAI Studio for its Edge AI SDK, addressing the growing demand for cost-effective, on-premises large language models (LLMs).
This initiative aims to make adopting LLMs more accessible for developers and small—to medium-sized businesses (SMBs) by significantly reducing reliance on GPUs, lowering costs, and enabling broader use of generative AI technologies.
As one of the Advantech Edge AI SDK software offerings, GenAI Studio addresses industry pain points, such as reducing factory operator wait times for critical information and easing the documentation workload of healthcare professionals. Its no-code, cost-effective platform streamlines LLM adoption, enabling businesses to deploy AI quickly and efficiently, thereby improving productivity and operational efficiency. GenAI Studio leverages a versatile LLM platform with exceptional integration capabilities for local and cloud-based LLMs, including OpenAI, Gemini, Anthropic, and Ollama. It also introduces full-parameter fine-tuning functionality, optimised for environments with limited GPU resources, thus enabling broader accessibility and performance enhancements.
GenAI Studio integrates fine-tuning and inference capabilities to maximize hardware utilization, allowing for more flexible and efficient resource allocation. Meanwhile, advanced GPU resource management and task scheduling enable users to optimise AI hardware performance, enhancing the cost-effectiveness of high-value equipment.
The rapid rise of AI has underscored the need for accessible LLMs, but limited resources hinder many companies. The Advantech Edge AI SDK addresses this challenge by offering a toolset that enables efficient evaluation, development, and deployment of edge AI applications. For instance, fine-tuning a 70-billion-parameter LLM, which traditionally requires over 30 GPUs with 48GB memory each, can be achieved with just 4 GPUs using the Advantech Edge AI SDK. This represents an 87% reduction in resource requirements, dramatically lowering costs and making LLMs more attainable.
Advantech’s AIR-520 edge AI server complements GenAI Studio and provides a robust hardware platform equipped with NVIDIA RTX GPUs and Phison AI SSDs. This integration provides reliable, high-efficiency computing capabilities tailored to meet the demands of AI applications in industries such as manufacturing, healthcare, and retail.
The Edge AI SDK is a fully integrated platform for seamless edge AI development. With pre-configured hardware and optimally tuned software, it delivers a plug-and-play experience that enables cost-effective LLM customisation, seamless toolkit compatibility, and effortless management of large-scale edge deployments. Designed with reliability, scalability, and user-friendliness, it simplifies the path to AI innovation. It now includes three core components:
- GenAI Studio: Facilitates cost-effective creation, evaluation, and integration of custom LLMs on-premises.
- Inference Kit: Enables rapid optimisation and assessment of efficient AI runtimes that are compatible with embedded operating systems.
- Orchestration Platform: Provides efficient management of AI models and application updates across large-scale edge deployments, integrating MLOps for streamlined operations.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
