The Benchmark Behind the Next Wave of Ultra-Low-Power AI

Machine learning (ML) is no longer confined to data centers and is transforming the world around us, adding more intelligence to our day-to-day lives. It now runs on doorbell cameras, hearing aids, factory sensors, and battery-powered wearables. These devices operate on a few milliwatts and must respond in real time. 

As that footprint expands, a hard question follows. How do you fairly measure their performance and efficiency when no two of these devices look alike? A single release can include anything from a 60 MHz microcontroller to a vector-enabled RISC-V core to a dedicated neural processing unit.

MLPerf® Tiny is the consortium-built answer. Developed by the MLCommons® Tiny Working Group with EEMBC, it provides an architecture-neutral way to compare ultra-low-power systems using the same workloads, models, and measurement methodology. 

This post covers what MLPerf Tiny measures, why energy is treated as a first-class metric, and what the v1.4 submission round reveals about the direction of edge AI.

The rise of TinyML and why it matters

TinyML refers to ML models that are small enough, typically under 2M weights, to run on microcontrollers and other constrained devices that draw sub-milliwatt to low-milliwatt power. That footprint matters because it opens up places that cloud or smartphone inference cannot go, unlocking new applications and capabilities. 

In 2026, that includes:

• Industrial vibration sensors flagging bearing wear before a machine fails
• Agricultural sensors lasting months on a single charge
• Hearing aids suppressing background noise on-device
• Always-on keyword spotters in earbuds
• Patch wearables monitoring ECG without uploading raw signals

The appeal is practical. Inference on the device keeps data where it’s created instead of sending it to the cloud. That means lower latency, lower cost, better privacy, and, for many battery-powered systems, much longer operating life.

The software ecosystem has matured just as quickly. TensorFlow Lite for Microcontrollers, ExecuTorch, Edge Impulse, STEdgeAI-Core, NXP eIQ, and vendor-specific compilers such as AndesAIRE and RUHMI have made it much easier to deploy models on embedded hardware. But they’ve also exposed the challenge of comparing results across substantially different platforms in a robust and trusted manner. Vendors benchmark different models across different datasets and under varying conditions, so performance numbers rarely tell the whole story.

The benchmarking gap no one had filled

Building a fair benchmark for ultra-low-power ML is hard in itself. The 2021 paper introducing MLPerf Tiny identified four challenges:

• Measuring energy fairly. Power consumption varies from one device to another. On top of that, vendors don’t always measure the same things. Some include peripherals, firmware startup, or I/O activity, while others measure only the inference itself. These differences can change the reported numbers.
• Fitting within tight memory budgets. TinyML devices typically work in kilobytes rather than gigabytes, roughly 6 orders of magnitude tighter than smartphone ML. Reference models, harness overhead, and multiple quantization paths all have to fit in that resource-constrained envelope, which lacks many of the conveniences of cloud systems.
• Software & Hardware heterogeneity. Devices range from general-purpose microcontrollers to neural processors, event-based architectures, and in-memory compute. Also, each vendor typically ships its own tightly coupled toolchain. That makes a portable benchmark non-trivial since that may require over-constraining the stack, which can erase the very performance the benchmark is meant to measure. 

Figure 1: Summary of the Tiny Machine Learning Stack. There is diversity at every level, which makes standardization for benchmarking challenging. 

For years, embedded developers had benchmarks, just not ones that answered the questions they actually cared about.

CoreMark became the standard for measuring Microcontroller Unit (MCU) performance, but it was never designed for ML workloads. MLMark moved closer by benchmarking ML inference, yet its reference models, including ResNet-50, MobileNet, and SSD-MobileNet, target edge AI processors rather than resource-constrained microcontrollers. It also stops short of measuring energy, which is often the limiting factor in embedded deployments.

That leaves a gap. Speed and energy efficiency aren’t the same thing. A model can run fast and still drain the battery. So in TinyML, both have to be measured. 

MLPerf Tiny was created to fill that gap. More than 50 organizations from industry and academia, including Harvard, Google, STMicroelectronics, Qualcomm, Syntiant, Renesas, Infineon, CERN, and Silicon Labs, spent roughly 18 months building the benchmark suite. It is based on compact reference models, representative workloads, and a standardized methodology that treats energy as a first-class metric. The first public results were released as version 0.5 in June 2021.

What MLPerf Tiny actually measures

MLPerf Tiny is designed to answer a simple question. If everyone runs the same ML task, which hardware executes it most efficiently?

For that, it evaluates three metrics, including accuracy, latency, and energy per inference. Each task carries a fixed quality target, so submissions are compared on speed and energy at equivalent accuracy rather than trading one against the other.

Energy is the differentiator. In TinyML, a device that runs an inference in 10 ms but at twice the joules of a competitor is often the wrong choice, because battery life is the binding constraint. MLPerf Tiny uses EEMBC’s EnergyRunner harness to perform calibrated power measurement on the device under test, making energy directly comparable across silicon.

MLPerf Tiny is built to be modular. Improvements can come from anywhere in the stack, whether that’s the chip, the compiler, the runtime, or the model itself. A fixed reference keeps every submission measured against the same bar. 

To make that comparison fair, each task includes a predefined model, dataset, and a minimum accuracy target. Vendors can’t swap in an easier model or lower the accuracy to improve their numbers. Instead, they focus on optimizing how the model runs on their hardware. Once the implementation is ready, MLPerf measures three things, including whether it meets the required accuracy, how long each inference takes, and how much energy each inference consumes.

Visual Wake Words shows how this works in practice. Every submission must determine whether a 96×96 image contains a person, achieving at least 80% accuracy on the official test set. Because every vendor solves the same problem under the same rules, the results are directly comparable. For example, ASYGN’s ColibriNPU completed each inference using just 22.2 microjoules. It’s low enough that a CR2032 coin-cell battery could perform one inference every second for more than three years.

That’s the key idea behind MLPerf Tiny. The workload, evaluation, and accuracy targets are all fixed. What changes is how efficiently each hardware and software stack delivers the result.

The five benchmarks and the real-world problems they represent

MLPerf Tiny started with four tasks in v0.5 (2021) and expanded to five in v1.3 (2025). Each task corresponds to a recurring TinyML deployment pattern:

• Keyword Spotting (KWS): A small DS-CNN trained on the Google Speech Commands dataset, representative of wake-word and voice-command detection in smart speakers, earbuds, and hearables.
• Visual Wake Words (VWW): A MobileNetV1 binary classifier on 96×96 images, mirroring person-presence detection in low-power vision sensors for doorbells, security cameras, and occupancy sensing.
• Image Classification (IC): A ResNet-style network on CIFAR-10, representative of general low-resolution vision workloads on edge devices.
• Anomaly Detection (AD): An autoencoder trained on industrial machine sounds (ToyADMOS), representative of predictive maintenance and condition monitoring.
• Streaming Wake Word: Added in v1.3, a 1D depthwise-separable CNN that detects a target word in a continuous audio stream. Unlike other benchmarks, this one also measures the device while it’s idle and listening, which is how wake-word systems actually run most of the time. These are the conditions under which most production wake-word systems actually run.

No benchmark can cover every embedded AI workload. These five capture the patterns that appear repeatedly in commercial TinyML products, making them a practical baseline for comparison. 

A benchmark built for everyone in the stack

MLPerf Tiny has two divisions because not everyone wants to measure the same thing.

1. The Closed Division fixes the reference model, dataset, and processing pipeline. That leaves hardware and low-level software optimizations as the only variables, making results directly comparable across platforms.
2. The Open Division is more flexible. Participants can use their own models, training methods, and optimizations, as long as they solve the same task and meet the required accuracy target. That makes it a place to evaluate new architectures, compiler techniques, and research ideas without losing the ability to compare against established benchmarks.

Figure 2: MLPerf Tiny’s modular design allows direct comparisons and shows improvement over the reference. Each reference implementation is swappable: green components can change in either division, orange only in the open division. 

MLPerf Tiny serves three different audiences:

• Hardware vendors use it to prove their silicon
• Software teams to evaluate their compilers and runtimes
• Researchers to validate novel architectures, and potential customers to evaluate candidate hardware and software platforms against a neutral standard.

The v1.4 round includes examples of all three. 

Audience	v1.4 example	What they showed
Hardware vendors	Andes Technology	Benchmarked five RISC-V configurations, ranging from the compact D23 core to the vector-enabled AX46MPV and an AX27 paired with the AnDLA I370 accelerator.
Software teams	DeepGate (first-time submitter)	Demonstrated its compiler across Arm Cortex-M microcontrollers and neural accelerators.
Researchers	University of Leeds (Open Division)	Evaluated a custom batch-normalization kernel running alongside an Xilinx DPU on a Versal Adaptive SoC.

What the results are already telling us

The v1.4 round closed with submissions from nine organizations, including Andes, ASYGN, DeepGate, Kai Jiang, Qualcomm, Renesas, STMicroelectronics, Syntiant, and the University of Leeds. Together, they submitted 25 system configurations across the Closed and Open Divisions. Five of the participating organizations were first-time submitters, more than doubling the participant count from v1.3.

These submissions highlight the real value of MLPerf Tiny. Every result is measured against the same task, model, accuracy target, and energy methodology, making improvements directly comparable instead of relying on vendor claims. 

A few signals stand out:

• Dedicated accelerators are moving into MCU-class parts. STMicroelectronics reports that enabling the hardware signal processor on the STM32U3 reduces image-classification inference time by up to 76.0% and 23.3% lower power on ic2 versus the same Cortex-M33 configuration. Additionally, the STM32H7P preview, with its Neural-ART NPU, improves inference time by up to 96.0% on the same workload compared to its CM7-only baseline.
• Sensing-hub architectures are advancing rapidly. Qualcomm’s submission on the Snapdragon 8 Elite Gen 5 Sensing Hub reports legacy workload latencies below 0.30 ms for keyword spotting, visual wake words, and image classification.
• Power efficiency is winning on streaming workloads. Syntiant’s NDP120 runs the streaming wake-word benchmark at a 3.3% duty cycle, meaning the device is idle ~97% of the time, with the remaining capacity available for concurrent tasks like noise cancellation or beamforming. 
• Open Division innovation is healthy. The University of Leeds reports a median throughput of 1,849.7 inferences per second for image classification on its Versal-based heterogeneous platform, with a custom on-fabric batch-normalization engine running at 313 MHz.

Together, the round shows that the field is maturing beyond CPU-only baselines. Hardware acceleration, streaming workloads, and heterogeneous compute have become the defining characteristics of competitive TinyML systems.

Conclusion

Standardization is what makes engineering progress measurable. By fixing the models, datasets, accuracy targets, and energy measurement methodology, MLPerf Tiny allows the entire stack, including silicon, compilers, runtimes, and models, to improve against a shared reference. That shared foundation is how responsible and efficient edge AI gets built. It depends not on the claims of any single vendor, but on a community-maintained benchmark that any submitter can reproduce and any practitioner can trust.

To explore the full v1.4 results, visit the MLPerf Tiny results page and read the supplemental containing details of all our submitters’ results. To contribute to future rounds, join the MLPerf Tiny Working Group.