MLCommons
Open Menu Close Menu

Inference Working Group

Mobile Working Group

Mission

Create a set of fair and representative inference benchmarks for mobile consumer devices such as smartphones, tablets, and notebooks that is representative of the end user experience.

Purpose

The MLPerf™ Mobile working group aims to collaboratively develop a performance-accuracy benchmark suite for consumer mobile devices with different AI chips and software stacks. The MLPerf Mobile working group draws from the expertise of mobile SoC vendors, ML framework providers, and model producers, and extends the MLPerf inference group’s efforts to a mobile context. We welcome new members that hope to raise the bar of ML performance for mobile devices.

Deliverables

  1. Mobile benchmark rules and definitions
  2. Mobile benchmark reference software
  3. Mobile benchmark submission rules
  4. Mobile benchmark roadmap
  5. Mobile benchmark app for Android and iOS (future version)
  6. Publish mobile benchmark results every ~6 months

Meeting Schedule

Weekly on Wednesday from 3:05-3:45PM Pacific.

How to Join

Use this link to request to join the group/mailing list, and receive the meeting invite:
Mobile Google Group.
Requests are manually reviewed, so please be patient.

Working Group Resources

Shared documents and meeting minutes:

  1. Associate a Google account with your e-mail address.
  2. Ask to join our Public Google Group.
  3. Ask to join our Members Google Group.
  4. Once approved, go to the Mobile folder in the Members Google Drive.

Working Group Chairs

Wei Zhao (weiz@mlcommons.org) - LinkedIn

Wei Zhao is the director of technical marketing at Zeku. He received his Ph.D. degree in electrical and computer engineering from University of Maryland College Park.

Mostafa El-Khamy (elkhamy@mlcommons.org) - LinkedIn - Google Scholar

Mostafa El-Khamy received the B.S. and M.S. degrees in electrical engineering from Alexandria University, Egypt, the M.S. and Ph.D. degrees in electrical engineering from the California Institute of Technology (Caltech), USA, and the M.B.A. degree from the Edinburgh Business School, U.K. He is currently a Senior Principal Engineer with Samsung Device Solutions Research America, CA, USA.