VeriSilicon’s Ultra-Low Energy NPU Empowers On-Device LLM Inference with Over 40 TOPS for Mobile Applications
In a groundbreaking advancement for on-device artificial intelligence, VeriSilicon (688521.SH) has unveiled its latest ultra-low energy Neural Network Processing Unit (NPU) IP, designed to deliver exceptional AI computing performance while maintaining remarkable energy efficiency. This cutting-edge NPU architecture now supports the on-device inference of large language models (LLMs), achieving AI computing performance that scales beyond 40 trillion operations per second (TOPS). With this innovation, VeriSilicon is addressing the growing demand for generative AI capabilities in mobile platforms, including AI-enabled smartphones, AI PCs, and other edge devices.
Meeting the Demands of Generative AI on Mobile Platforms
As mobile devices evolve into personal AI servers, the need for powerful yet energy-efficient AI processing has become paramount. The rapid advancements in artificial intelligence-generated content (AIGC) and multi-modal large language models (LLMs) have driven exponential growth in AI computing requirements. These technologies are transforming mobile devices into versatile tools capable of handling complex tasks like natural language processing, image enhancement, and real-time translation. However, these capabilities come with significant challenges, particularly in managing energy consumption and ensuring smooth, responsive performance.
VeriSilicon’s ultra-low energy NPU IP rises to these challenges by delivering a highly optimized solution tailored for mobile platforms. Its energy-efficient design ensures that devices can perform intensive AI computations without compromising battery life or user experience. This makes it an ideal choice for AI phones, AI PCs, and other end devices that require both high performance and stringent energy efficiency.
A Scalable Architecture for Diverse AI Applications
At the heart of VeriSilicon’s NPU IP is a highly configurable and scalable architecture that supports mixed-precision computation, advanced sparsity optimization, and parallel processing. These features enable the NPU to handle a wide range of AI algorithms and models with remarkable efficiency. For example, it supports hundreds of AI algorithms, including AI-based noise reduction (AI-NR) and super-resolution (AI-SR), as well as leading AI models such as Stable Diffusion and LLaMA-7B.
The NPU’s design incorporates efficient memory management and sparsity-aware acceleration, which significantly reduce computational overhead and latency. By minimizing redundant calculations and optimizing data flow, the NPU ensures smooth and responsive AI processing, even for resource-intensive tasks. This level of optimization is critical for enabling seamless user experiences in applications like real-time language translation, image generation, and video enhancement.
Additionally, VeriSilicon’s ultra-low energy NPU IP can be seamlessly integrated with the company’s other processing IPs, enabling heterogeneous computing solutions. This flexibility empowers system-on-chip (SoC) designers to create comprehensive AI solutions that meet the diverse needs of various applications, from consumer electronics to industrial IoT devices.
Supporting Popular AI Frameworks for Simplified Integration
To accelerate deployment and simplify integration, VeriSilicon’s NPU IP supports popular AI frameworks such as TensorFlow Lite, ONNX, and PyTorch. This compatibility ensures that developers can easily port their AI models and applications to the NPU, reducing time-to-market and enabling faster adoption across a wide range of use cases. Whether it’s enhancing photography capabilities on smartphones or enabling real-time language processing on AI PCs, VeriSilicon’s NPU provides a versatile foundation for innovation.
Driving the Future of AI-Enabled Devices
“Mobile devices, such as smartphones, are evolving into personal AI servers,” said Weijin Dai, Chief Strategy Officer, Executive Vice President, and General Manager of the IP Division at VeriSilicon. “With the rapid advancement of AIGC and multi-modal LLM technologies, the demand for AI computing is growing exponentially and becoming a key differentiator in mobile products.”
Dai emphasized the importance of addressing energy consumption challenges in supporting high AI computing workloads. “One of the most critical challenges in supporting such high AI computing workloads is energy consumption control. VeriSilicon has been continuously investing in ultra-low energy NPU development for AI phones and AI PCs. Through close collaboration with leading SoC partners, we are excited to see that our technology has been realized in silicon for next-generation AI phones and AI PCs.”
This commitment to innovation underscores VeriSilicon’s leadership in the semiconductor industry. By collaborating closely with top-tier SoC partners, the company is helping to bring next-generation AI-enabled devices to life. These devices will not only deliver unprecedented levels of performance but also maintain the energy efficiency required for everyday use.
Enabling New Possibilities for On-Device AI
The ability to perform on-device inference of large language models with over 40 TOPS of computing power opens up exciting possibilities for mobile applications. For instance, users can enjoy real-time interactions with AI assistants, generate high-quality images or videos, and access personalized recommendations without relying on cloud connectivity. This not only enhances privacy and security but also reduces latency, providing a more responsive and reliable user experience.
Moreover, the scalability of VeriSilicon’s NPU architecture ensures that it can adapt to the evolving demands of AI applications. As AI models grow larger and more complex, the NPU’s flexible design allows it to scale accordingly, ensuring long-term relevance and performance.
About VeriSilicon
VeriSilicon is committed to providing customers with platform-based, all-around, one-stop custom silicon services and semiconductor IP licensing services leveraging its in-house semiconductor IP.
