Research
Democratizing AI through an open community approach
Democratizing new technological capabilities and ensuring wide-spread adoption requires an open approach. MLCommons regularly publishes and presents at top conferences and industry events along with our broad community—allowing all researchers, scientists, and professionals in AI and ML to access and learn from our work.
Publications
MLCommons is a community-driven effort. We regularly co-author papers with community members to share our collective learnings with the broader community.
Introducing v0.5 of the AI Safety Benchmark from MLCommons
ArXiv 2024 | Paper
Dataperf: Benchmarks for data-centric ai development
NeurIPS 2023 | Poster
Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
arXiv 2023 | Paper
MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluation
Nature Machine Intelligence 2023 | Journal
Speech Wikimedia: A 77 Language Multilingual Speech Dataset
arXiv 2023 | Paper
The Dollar Street Dataset: Images Representing the Geographic and Socioeconomic Diversity of the World
NeurIPS 2022 | Paper
MLPerf Mobile Inference Benchmark
MLSys 2022 | Paper
MLPerf Tiny Benchmark
NeurIPS 2021 | Paper
Benchmarking tinyml systems: Challenges and direction
arXiv 2021 | Paper
The People’s Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage
NeurIPS 2021 | Paper
Multilingual Spoken Words Corpus
NeurIPS 2021 | Paper
MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems
arXiv 2021 | Paper
Software/hardware co-optimization on the IPU: An MLPerf™ case study
Hot Chips 33 2021 | Paper
Data Engineering for Everyone
arXiv 2021 | Paper
LSH methods for data deduplication in a Wikipedia artificial dataset
arXiv 2021 | Paper
MLPerf Training and Inference
Hot Chips 33 2021 | Tutorial
MLPerf Training Benchmark
MLSys 2020 | Paper
MLPerf Inference Benchmark
ISCA 2020 | Paper
MLPerf: A Benchmark Suite for Machine Learning from an Academic-Industry Cooperative
Hot Chips 31 2019 | Paper
Talks
Driving ML Forward in Automotive
David Kanter
CASPA Spring Symposium
What is MLCube
Victor Bittorf
ASPLOS 2021
MLPerf Automotive Overview
Tom St. John
ASPLOS 2021
Medical Imaging Benchmark using MLPerf
Alex Karargyris
ASPLOS 2021
MLPerf HPC: A Benchmark Suite for Large scale ML on HPC Systems
Murali Emani
ASPLOS 2021
Data-centric Speech for Machine Learning Systems
Greg Diamos
ASPLOS 2021
Mobile AI Performance Benchmarking & Analysis with the MLPerf App
Wookie Hong
ASPLOS 2021
MLPerf Inference Benchmark Suite
Christine Cheng
ASPLOS 2021
MLPerf Training Benchmark Suite
Peter Mattson
ASPLOS 2021
Multilingual Spoken Words Corpus
NeuRIPS 2021