Arduino will join forces with AI chip startup Axelera to bring more advanced machine learning to the masses. The collaboration will combine Axelera’s accelerator chips with Arduino’s families of system-on-modules (SOMs), giving them more than enough performance to run fully local large language models (LLMs) that are the bedrock of the AI boom.
The companies plan to debut the technology in a demo at CES 2025, formerly known as the Consumer Electronics Show.
>>Check out more of our CES 2025 coverage
The pact is more broadly about removing the obstacles to running high-performance AI on IoT devices. The companies touted the potential for the tie-up, which gives the more than 30 million developers working with Arduino easier access to Axelera’s solutions.
“By working with Axelera AI, we’re providing developers and businesses the means to integrate advanced AI capabilities into their projects,” stated Fabio Violante, CEO of Arduino, which is charting more of a commercial future for its open-source hardware.
Though it’s already widely used by engineers for rapid-prototyping IoT and other electronic devices, Arduino is also pushing high-performance hardware and software that’s all about moving into mass-produced products. The company’s Arduino Pro line includes the Portenta family of industrial-grade, gum-stick-shaped computer modules to the Nicla family of voice, vision, and other intelligent sensors. These are paired with its secure, scalable, and easy-to-access cloud plus a vast array of software libraries and other tools for developers.
Arduino said the Portenta family of chips can run relatively compact machine-learning models entirely on-device. But it tends to lack the performance to run many of the most advanced types of AI, including LLMs.
Metis: The AI Accelerator
Arduino is bringing in reinforcements with Axelera’s AI Processing Unit (AIPU), which is all about running AI inferencing very efficiently. The first in its family of accelerator chips, Metis, comes equipped with four AI cores that can each execute up to 53.5 trillion operations per second (TOPS). The silicon is supplemented by its software stack, the Voyager SDK, giving customers compilers and other tools to load machine-learning models into Metis.
Central to each core is Axelera’s digital in-memory computing architecture (D-IMC). It’s used to carry out the heavy-duty computations at the heart of AI—also called matrix multiplication—in its SRAM, which is widely utilized as cache memory in modern chips.
Feeding processors with data as fast as possible is one of the bottlenecks in machine learning. The most advanced AI chips in data centers are bordered by high rises of high-bandwidth memory (HBM) to reduce the power-hungry process of slinging data into the processor, which also adds latency.
By co-locating the logic and memory, Axelera said Metis can deliver higher power efficiency of up to 15 TOPS per watt. That’s ideal for battery-powered devices that can’t afford to use too much power or output too much heat.
Axelera said the cores can access a shared memory-hierarchy of more than 52 MB of on-chip memory, and the chip adds a memory controller that can connect it to additional memory over LPDDR4 so that it can run machine learning models that are too large to fit within the chip’s cache memory. Axelera said Metis uses PCIe to connect to the host processor at the heart of the system, which offloads the AI workloads to the SoC.
The Portenta X8 “Brain”
At CES, the companies plan to attach a carrier board based on Axelera’s Metis chip to the Arduino Portenta X8 module, which will act as the brain of the system. It can run through temperature, vibrations, humidity, and any other data from sensors strapped to the machinery on factory floors. Then it feeds the data to Metis, which runs Microsoft's Phi-3 model entirely on-device to identify potential issues in real-time before they can cause malfunctions or failures.
Phi is a fraction of the scale and complexity of the machine-learning models at the heart of OpenAI’s ChatGPT and other chatbots. The latter generally run in data centers due to their immense computational demands.
It is unclear whether Arduino plans to use Axelera’s chips in future Portenta modules. But Axelera CEO and co-founder Fabrizio Del Maffeo indicated that the tie-up is more broadly about delivering state-of-the-art AI with minimal power and cost.
He said, “This is critical for companies to realize true AI innovation.”
