MLCommons AI Risk & Reliability
Creating a benchmark suite for safer AI
The MLCommons AI Risk & Reliability working group is composed of a global consortium of industry leaders, practitioners, researchers, and civil society experts committed to building a harmonized approach to safer AI.
The working group is creating a platform, tools, and tests for developing a standard AI Safety benchmark suite for different use cases to help guide responsible AI development.
Introducing the v0.5 AI Safety benchmark proof of concept
The v0.5 benchmark proof of concept (POC), announced April 15, 2024, focuses on measuring the safety of large language models (LLMs) by assessing the models’ responses to prompts across multiple hazard categories. The v0.5 POC includes:
- A benchmark that runs a series of tests for a taxonomy of hazards.
- A platform that defines the benchmark and generates a report.
- A new testing engine (inspired by the HELM framework from Stanford CRFM) to run the tests.
The v0.5 POC shows results from more than a dozen, anonymized systems-under-test (SUT). It is being shared with the community now for experimentation and feedback to inform improvements for a comprehensive v1.0 release later this year.
Join the working group to help shape the v1.0 benchmark suite and beyond.
AI Risk & Reliability related Blogs and News
-
MLCommons AI Safety Working Group’s Rapid Progress to a v1.0 Release
Building a comprehensive approach to measuring the safety of LLMs and beyond
-
MLCommons and AI Verify to collaborate on AI Safety Initiative
Agree to a memorandum of intent to collaborate on a set of AI safety benchmarks for LLMs
-
Creating a comprehensive Test Specification Schema for AI Safety
Helping to systematically document the creation, implementation, and execution of AI safety tests
AI Risk & Reliability paid opportunities for engagement
AI Safety Benchmark Prompt Generation – Expression of Interest
We are seeking expressions of interest (EOI) from qualified organizations to generate datasets of prompts for the upcoming v1.0 AI safety benchmark suite, launching later this fall.
AI Safety v0.5 benchmark POC supporting documentation
v0.5 POC Technical Glossary
April 2024 | v0.5 POC Documentation
v0.5 POC Test Specification Schema
April 2024 | v0.5 POC Documentation
v0.5 POC Taxonomy of Hazards
April 2024 | v0.5 POC Documentation
v0.5 POC AI Safety Benchmark Technical Whitepaper
ArXiv 2024 | Paper
AI Risk & Responsibility working group contributors
The MLCommons AI Risk & Responsibility working group is composed of a global group of industry leaders, practitioners, researchers, and civil society experts committed to building a harmonized approach to AI risk and reliability. The following organizations have contributed to the AI Risk & Responsibility working group.
- Accenture
- ActiveFence
- Anthropic
- Argonne National Laboratory
- Bain & Company
- Blue Yonder
- Bocconi University
- Broadcom
- cKnowledge, cTuning foundation
- Carnegie Mellon
- Center for Security and Emerging Technology
- Coactive AI
- Cohere
- Columbia University
- Common Crawl Foundation
- Commn Ground
- Context Fund
- Credo AI
- Deloitte
- Digital Safety Research Institute
- Dotphoton
- EleutherAI
- Ethriva
- Febus
- Futurewei Technologies
- Georgia Institute of Technology
- Hewlett Packard Enterprise
- Humanitas AI
- IIT Delhi
- Illinois Institute of Technology
- Inflection
- Intel
- Kaggle
- Lawrence Livermore National Laboratory
- Learn Prompting
- Lenovo
- MIT
- Meta FAIR
- Microsoft
- NASA
- Nebius
- NVIDIA Corporation
- NewsGuard
- Nutanix
- OpenAI
- Process Dynamics
- Protecto.ai
- Protiviti
- Qualcomm Technologies, Inc.
- RAND
- Reins AI
- SAP
- SaferAI
- Stanford
- Surescripts LLC
- Telecommunications Technology Association
- Toloka
- TU Eindhoven
- Turaco Strategy
- UC Irvine
- Univ. of British Columbia (UBC)
- Univ. of Birmingham
- Univ. of Cambridge
- Univ. of Chicago
- Univ. of Illinois at Urbana-Champaign
- Univ. of Southern California (USC)
- Univ. of Trento
Funding for the initial AI Risk & Responsibility working group effort was provided by Google, Intel, Meta, NVIDIA and Qualcomm Technologies, Inc. MLCommons is committed to supporting a long-term effort for this important work and welcomes additional funding contributors.