Raspberry Pi has launched an AI Kit tailored for its single-board computer Raspberry Pi 5, centered around the Hailo-8L AI accelerator. This accessory, which comes with an M.2 Hat board and a preinstalled M.2 stick, marks Raspberry Pi's entry into dedicated AI hardware.
The Hailo-8L chip, a downsized version of the Hailo-8, is capable of 13 trillion operations per second (TOPS) at INT8 precision. This compares to the 26 TOPS of its predecessor. In the current market, others like Qualcomm, AMD, and Intel are putting out SoCs with NPUs delivering upwards of 45 to 50 TOPS. Despite this, the Hailo-8L distinguishes itself with its affordability, priced at $70. Together with the Raspberry Pi 5, which sells for about $80, it offers an economical alternative to more expensive NPU-equipped SoCs.
Energy Efficiency and Design Approach
The Hailo-8L is noted for its energy efficiency, consuming about one watt per three TOPS. For routine operations such as real-time 60 FPS video processing, it generally uses between one and two watts, with a peak usage around five watts under strenuous conditions. Hailo CTO Avi Baum has highlighted the chip's low power consumption.
Raspberry Pi CEO Eben Upton clarified the company's decision to use a detachable accelerator rather than building an NPU into the central processor. This choice aligns with Raspberry Pi's modular approach, wherein the CPU and GPU are on a 16nm chip, while I/O functions are on a more affordable 40nm chip. Incorporating an NPU into the 16nm chip would have raised costs, but the separate AI Kit allows users to add machine learning abilities without increasing the base price of the Raspberry Pi 5.
AI applications are not new to Raspberry Pi. Users have previously utilized AI for tasks like object detection, often depending on cloud processing to achieve this. Upton cited an example from Japan where a farm used a Raspberry Pi 3 to sort cucumbers with about 70% accuracy, though it required cloud-based AI. The new AI Kit shifts some of this processing to the device itself, reducing the dependence on internet connectivity.
Applications and Availability
The AI Kit integrates with Raspberry Pi's camera software, enabling the device to perform AI-based functions such as running chatbots. This move is part of a broader trend towards more compact, energy-efficient AI models and processors, making it easier to integrate AI into portable devices. The kits will shortly be available through Raspberry Pi-approved retailers globally.
There is a noticeable trend towards enhanced energy efficiency in AI processing. Microsoft's upcoming laptops will feature Copilot+ with integrated AI capabilities. AMD has introduced new Ryzen processors designed for generative AI tasks. Nvidia, famous for its H100 GPUs used in training large language models like GPT-4, is also including AI chips in its laptop lineup. According to Hailo CEO Orr Danon, the Hailo-8L's power consumption stays under 2W and it is passively cooled, emphasizing its suitability for portable AI contexts.
