Mediatek adds Llama 3 generative AI to smartphone chips
Mediatek has added the latest open source transformer-based generative AI model, Llama 3, to its Dimensity smartphone chips.
Meta has just introduced Llama 3 Large Language Model (LLM) for AI applications and Mediatek has ported the model, with 8bn parameters, to the Dimensity 9300, Dimensity 8300, and future Gen-AI enabled platforms.
- MediaTek looks to generative AI to replace touchscreens
- MediaTek drops ‘big-little’ strategy for AI-capable mobile
The 9300 is the first to market with the latest ARM Cortex-X4 and Cortex-A720 processors, using four of each with a custom AI processor unit (APU) architecture with a hardware generative AI engine that speeds up transformer-based generative AI by a factor of 8. It uses a hardware-accelerated memory compression technology to handle the massive memory requirements of transformer AI, and Mediatek says it will be able to handle models with up to 33bn parameters. The chips is also the first with LPDDR5T running at 9600Mbps, currently the world’s fastest smartphone memory.
The port builds on the framework MediaTek established with Meta Llama 2 last year, and Mediatek says the Lama 3 implementation will provide more responsiveness, better privacy, security and reliability for language nuances, contextual understanding, and complex tasks like translation and dialogue generation.
Llama 3 can handle multi-step tasks effortlessly, while its refined post-training processes significantly lower false refusal rates, improve response alignment, and boost diversity in model answers. Additionally, it drastically elevates capabilities like reasoning, code generation, text understanding, instruction following, and world knowledge.
Developers can use Llama 3 through MediaTek’s NeuroPilot SDK. NeuroPilot is a toolkit that enables and optimizes on-device GenAI inference capabilities across MediaTek’s range of chips, including mobile platforms and edge-AI capable devices.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
