MLPerf Tiny
Develop Tiny ML benchmarks to evaluate inference performance on ultra-low-power systems.
Purpose
ML inference on the edge is increasingly attractive to increase energy efficiency, privacy, responsiveness, and autonomy of edge devices. Recently there have been significant strides, in both academia and industry, towards expanding the scope of edge machine learning to a new class of ultra-low-power computational platforms. Tiny ML, or machine learning on extremely constrained devices, breaks the traditional paradigm of energy and compute hungry machine learning and allows for greater overall efficiency relative to a cloud-centric approach by eliminating networking overhead. This effort extends the accessibility and ubiquity of machine learning since its reach has traditionally been limited by the cost of larger computing platforms.
To enable the development and understanding of new, tiny machine learning devices, the MLPerf Tiny working group will extend the existing inference benchmark to include microcontrollers and other resource-constrained computing platforms.
Deliverables
- 3-4 benchmarks with defined datasets and reference models for the closed division
- Software framework to load inputs and measure latency
- Rules for benchmarking latency and energy
- Power and energy measurement with partners
Meeting Schedule
Monday July 15, 2024 Weekly – 09:05 – 10:00 Pacific Time
Results Publication
April 17, 2024 Wednesday
How to Join and Access MLPerf Tiny Resources
To sign up for the group mailing list, receive the meeting invite, and access shared documents and meeting minutes:
- Fill out our subscription form and indicate that you’d like to join the MLPerf Tiny Working Group.
- Associate a Google account with your organizational email address.
- Once your request to join the Tiny Working Group is approved, you’ll be able to access the Tiny folder in the Public Google Drive.
To engage in group discussions, join the group’s channels on the MLCommons Discord server.
To access the GitHub repositories (public):
- If you want to contribute code, please submit your GitHub ID to our subscription form.
- Visit the GitHub repository.
Tiny Working Group Chairs
To contact all Tiny working group chairs email [email protected].
Jeremy Holleman
Jeremy Holleman is the Chief Scientist at Syntiant Corp. and an Associate Professor of Electrical and Computer Engineering at the University of North Carolina, Charlotte. He has held positions at the University of Tennessee as well as Data I/O and National Semiconductor. He received his Ph.D. from the University of Washington where he studied micro-power integrated circuits for neural interfaces. His research interests span several disciplines including low-power circuit design, machine learning, and resource-constrained intelligent systems.
Peter Chang
Peter Chang is the business development manager, senior software engineer, and the co-founder of Skymizer Taiwan Inc. He earned his bachelor’s degree from the Department of Electrical Engineering, National Chiao Tung University, and his master’s degree from the Department of Computer Science, National Tsing Hua University in Taiwan. His research interests span areas in electrical control systems, operating systems, virtualization, performance engineering, and computer architecture. Currently, he focuses on topics in hardware/software co-design and benchmarking on Machine Learning. He is also devoted to participating in the MLPerf Tiny and the tinyML communities. He was also the maintainer of SkyPat, an open-source performance unit-test suite, and ARMvisor, one of the Kernel-based Virtual Machine solutions on Arm architecture.
