MLPerf Mobile Working Group
Purpose
The MLPerf Mobile working group aims to collaboratively develop a performance-accuracy benchmark suite for consumer mobile devices with different AI chips and software stacks. The working group draws from the expertise of mobile SoC vendors, ML framework providers, and model producers, and extends the MLPerf inference group’s efforts to a mobile context. We welcome new members that hope to raise the bar of ML performance for mobile devices.
Mission
Create a set of fair and representative inference benchmarks for mobile consumer devices such as smartphones, tablets, and notebooks that are representative of the end user experience.
Deliverables
- Mobile benchmark rules and definitions
- Mobile benchmark reference software
- Mobile benchmark submission rules
- Mobile benchmark roadmap
- Mobile benchmark app for Android and iOS (future version)
- Publish mobile benchmark results every ~6 months
Join
Meeting Schedule
Wednesday October 16, 2024
Weekly – 15:05 – 15:45 Pacific Time
Results Publication
October 16, 2024
Wednesday
Related Blog
-
MLPerf Mobile v4.0 application adds new benchmark, expands hardware support
The mobile benchmark suite adds a brand-new image classification model and supports neural acceleration on some of the latest mobile devices
-
MLPerf Inference Delivers Power Efficiency and Performance Gains
Record participation in MLCommons’ benchmark suite showcases improvements in efficiency and capabilities for deploying machine learning
-
MLPerf Results Show Advances in Machine Learning Inference Performance and Efficiency
MLCommons’ latest benchmarks illustrate focus on energy efficiency and up to 3.3X performance gains
How to Join and Access MLPerf Mobile Working Group Resources
The MLPerf Mobile working group is limited exclusively to MLCommons members and affiliates. If you are not already a member or affiliate or part of a member or affiliate company, you can learn more about MLCommons Membership here.
- To sign up for the group mailing list, receive the meeting invite, and access shared documents and meeting minutes:
- Fill out our subscription form and indicate that you’d like to join the MLPerf Mobile Working Group.
- Associate a Google account with your organizational email address.
- Once your request to join the Mobile Working Group is approved, you’ll be able to access the Mobile folder in the Members Google Drive.
- To engage in working group discussions join the group’s channels on the MLCommons Discord server.
- To access the GitHub repositories (public):
- If you want to contribute code, please submit your GitHub ID to our subscription form.
- Visit the GitHub repositories:
Mobile Working Group Chairs
To contact all Mobile working group chairs email [email protected].
Koan-Sin Tan
Koan-Sin Tan is a Senior Technical Manager with MediaTek Inc. He has worked on performance engineering for the past decade. He enjoys hacking together small utilities and contributing to some big open-source projects from time to time. He has a PhD in Information Management from Chiao-Tung University, Taiwan.
Mostafa El-Khamy
Mostafa El-Khamy received a B.S. and M.S. in electrical engineering from Alexandria University, Egypt, a M.S. and Ph.D. in electrical engineering from the California Institute of Technology (Caltech), USA, and a M.B.A. from the Edinburgh Business School, U.K. He is currently a Senior Principal Engineer with Samsung Device Solutions Research America, CA, USA.