About Us
An Open AI Engineering Consortium
MLCommons is an Artificial Intelligence engineering consortium, built on a philosophy of open collaboration to improve AI systems. Through our collective engineering efforts with industry and academia we continually measure and improve the accuracy, safety, speed, and efficiency of AI technologies–helping companies and universities around the world build better AI systems that will benefit society.
Our Five Key Principles
01
Grow AI markets and make the world a better place
02
Get everyone involved
- Be global, inclusive, and fair
- Bring together academia, small companies, large companies, non-profits, etc.
- Make it easy to get involved
- Be as open with our IP as possible while sustaining the community
03
Act through collaborative engineering
- Keep leadership mostly technical, with an emphasis on hands-on-involvement
- Favor data-driven decisions, design simplicity, and focus on real user value
04
Make fast but consensus-supported decisions
- Very low barrier for “experimental” working groups with well reviewed path to full endorsement
- Favor grudging consensus over 51/49 votes, especially for big decisions
- Make technical contributions easy
- Favor rapid development and iteration
05
Build a community that people want to be part of
- Be welcoming, informal, and friendly
- Encourage, recognize, and reward contributions
- Celebrate with cake
Our Mission
Our mission is to standardize AI measurement and enable informed decisions.
Measurement is fundamental to responsibly unlocking the potential of AI. AI systems are the ultimate black boxes: they use intricate patterns across billions of numbers to generate sophisticated behavior, and our explanations of how they work remain vague and unsatisfying even as they grow more complex. Training and operating them is expensive and power-hungry. Better measurements let us empirically characterize AI systems’ behavior and operational costs.
If a characteristic of AI hardware, software, or data matters to the value delivered – be it speed, power, reliability, or quality – we aim to measure it. In doing so, we support informed decisions about today’s AI systems, enable cost-benefit tradeoffs, provide a north star for innovation, and support risk management, including assurance and insurance. Better measurement de-risks and grows the AI market.
We develop and operate standardized measurements as industrial-quality benchmarks: detailed protocols combined with artifacts such as test harnesses, held-out test datasets, and evaluation models. Multiple implementations built to a paper standard cannot produce comparable measurements; many artifacts must be technically and legally protected to avoid overfitting, and measurements must evolve to keep pace with the technology, all of which requires a trusted, sustainably operating organization.
Our work touches everyone in the AI industry: academic researchers, hardware and software vendors, deployers, end users, and policymakers. We exist to meet these independent measurement needs and, in doing so, provide an essential service that supports the broader work of standards organizations such as ISO, CoSAI, and ADA.
Our Leadership
Our board of directors comprises a diverse set of representatives drawn from the MLCommons community. They provide corporate leadership and strategic guidance, accountability, and transparency to support the long-term future of MLCommons.
Our History
Community
Join our community
MLCommons is a community-driven and community-funded effort. We welcome all corporations, academic researchers, nonprofits, government organizations, and individuals on a non-discriminatory basis. Join us!
Contact Us
Address
MLCommons
8 The Green #20930
Dover, DE 19901