MLCommons

Training Working Group

Mission

Define, develop, and conduct the MLPerf Training benchmarks.

Purpose

Benchmarking the performance of training ML models on a wide variety of use cases, software, and hardware drives AI performance across the tech industry. This working group draws on expertise in AI and the technology that powers AI from across the industry to design and create industry-standard benchmarks. Together, we create the reference implementations, rules, policies and procedures to benchmark a wide variety of AI workloads.

The training working group strives for a critical balance of perspectives to ensure fairness and accuracy in the benchmarks. This balance comes from our member's diverse experience in many different AI hardware and software spaces. We are always looking for new members to help us create the benchmarks that best capture innovation in AI.

Deliverables

  1. Training benchmark roadmap
  2. Training benchmark rules
  3. Training benchmark reference implementations
  4. Training benchmark results every ~6 months

Meeting Schedule

Weekly on Thursday from 8:30-10:00AM Pacific.

Mailing List

training@mlcommons.org

Working Group Resources

Google Drive (Members only)

Working Group Chair Emails

Victor Bittorf (vsb@fb.com)

John Tran (jotran@nvidia.com)

Working Group Chair Bios

Victor is an applied research scientist at Facebook driving PyTorch performance to new heights for both production and research. Previously, he was in Google Brain working on performance for Tensorflow and TPUs. Holding a Masters Degree in computer science from UW-Madison, his academic work focused on ML optimization algorithm design and it's efficient system implementation. Victor enjoys serving his local community through in-kind contributions as a photographer and videographer for not-for-profit organizations.

LinkedIn

John is a Director in the Deep Learning team at NVIDIA. He joined the company in 2005, contributing to almost every major GPU architecture released since then. He has worked on several efforts at NVIDIA, including the Texture unit, the SM team, graphics applications, compute applications, and NVIDIA Research. Now John leads several of NVIDIA’s Deep Learning architecture and software efforts. He completed his Master’s degree at the University of Virginia and undergraduate degree at Duke University.

LinkedIn