MLCommons

Inference Working Group

Mobile Working Group

Mission

Create a set of fair and representative inference benchmarks for mobile consumer devices such as smartphones, tablets, and notebooks that is representative of the end user experience.

Purpose

The MLPerf Mobile working group aims to collaboratively develop a performance-accuracy benchmark suite for consumer mobile devices with different AI chips and software stacks. The MLPerf Mobile working group draws from the expertise of mobile SoC vendors, ML framework providers, and model producers, and extends the MLPerf inference group’s efforts to a mobile context. We welcome new members that hope to raise the bar of ML performance for mobile devices.

Deliverables

  1. Mobile benchmark rules and definitions
  2. Mobile benchmark reference software
  3. Mobile benchmark submission rules
  4. Mobile benchmark roadmap
  5. Mobile benchmark app for Android and iOS (future version)
  6. Publish mobile benchmark results every ~6 months

Meeting Schedule

Weekly on Wednesday from 3:00-4:00PM Pacific.

Mailing List

mobile@mlcommons.org

Working Group Resources

Google Drive (Members only)

Working Group Chair Emails

William Chou (wchou@qti.qualcomm.com)

Ramesh Chukka (ramesh.n.chukka@intel.com)

Jungwook Hong (jwookie.hong@samsung.com)

Working Group Chair Bios

William Chou is a Product Manager at Qualcomm and he received his undergraduate and Master’s degree from the University of Toronto.

LinkedIn

Ramesh Chukka is a Deep Learning Manager at Intel with focus on performance analysis and benchmarking. He has 14+ years of experience leading benchmark development and working with industry benchmark consortiums. Ramesh received M.Tech from IIT Madras and B.E from Andhra University, India.

Jungwook (Wookie) Hong is an SoC product planner at Samsung Electronics’s S.LSI business and he received his undergraduate degree at the University of Virginia.

LinkedIn