MLCommons AI Safety
Creating a benchmark suite for safer AI
The MLCommons AI Safety working group is composed of a global consortium of industry leaders, practitioners, researchers, and civil society experts committed to building a harmonized approach to AI safety.
The working group is creating a platform, tools, and tests for developing a standard AI Safety benchmark suite for different use cases to help guide responsible AI development.
![](https://mlcommons.org/wp-content/uploads/2024/04/MLC-AI-Safety-white-taller-1024x555.jpg)
![MLCommons AI Safety Working Group](https://mlcommons.org/wp-content/uploads/2024/04/AISafetyMeetingNov2023-2-2-1024x768.jpg)
Introducing the v0.5 AI Safety benchmark proof of concept
The v0.5 benchmark proof of concept (POC), announced April 15, 2024, focuses on measuring the safety of large language models (LLMs) by assessing the models’ responses to prompts across multiple hazard categories. The v0.5 POC includes:
- A benchmark that runs a series of tests for a taxonomy of hazards.
- A platform that defines the benchmark and generates a report.
- A new testing engine (inspired by the HELM framework from Stanford CRFM) to run the tests.
The v0.5 POC shows results from more than a dozen, anonymized systems-under-test (SUT). It is being shared with the community now for experimentation and feedback to inform improvements for a comprehensive v1.0 release later this year.
Join the working group to help shape the v1.0 benchmark suite and beyond.
AI Safety related Blogs and News
-
MLCommons and AI Verify to collaborate on AI Safety Initiative
Agree to a memorandum of intent to collaborate on a set of AI safety benchmarks for LLMs
-
Creating a comprehensive Test Specification Schema for AI Safety
Helping to systematically document the creation, implementation, and execution of AI safety tests
-
Announcing MLCommons AI Safety v0.5 Proof of Concept
Achieving a major milestone towards standard benchmarks for evaluating AI Safety
AI Safety paid opportunities for engagement
AI Safety Prompt Generation – Expression of Interest
We are seeking expressions of interest (EOI) from qualified organizations to generate datasets of prompts for the upcoming v1.0 AI safety benchmark suite, launching later this fall.
AI Safety v0.5 benchmark POC supporting documentation
v0.5 POC Technical Glossary
April 2024 | v0.5 POC Documentation
v0.5 POC Test Specification Schema
April 2024 | v0.5 POC Documentation
v0.5 POC Taxonomy of Hazards
April 2024 | v0.5 POC Documentation
Introducing v0.5 of the AI Safety Benchmark from MLCommons
ArXiv 2024 | Paper
AI Safety working group contributors
The MLCommons AI Safety working group is composed of a global group of industry leaders, practitioners, researchers, and civil society experts committed to building a harmonized approach to AI safety. The following organizations have contributed to the AI Safety working group.
- Accenture
- ActiveFence
- Anthropic
- Argonne National Laboratory
- Bain & Company
- Blue Yonder
- Bocconi University
- Broadcom
- cKnowledge, cTuning foundation
- Carnegie Mellon
- Center for Security and Emerging Technology
- Coactive AI
- Cohere
- Columbia University
- Common Crawl Foundation
- Commn Ground
- Context Fund
- Credo AI
- Deloitte
- Digital Safety Research Institute
- Dotphoton
- EleutherAI
- Ethriva
- Febus
- Futurewei Technologies
- Georgia Institute of Technology
- Hewlett Packard Enterprise
- Humanitas AI
- IIT Delhi
- Illinois Institute of Technology
- Inflection
- Intel
- Kaggle
- Lawrence Livermore National Laboratory
- Learn Prompting
- Lenovo
- MIT
- Meta FAIR
- Microsoft
- NASA
- Nebius
- NVIDIA Corporation
- NewsGuard
- Nutanix
- OpenAI
- Process Dynamics
- Protecto.ai
- Protiviti
- Qualcomm Technologies, Inc.
- RAND
- Reins AI
- SAP
- SaferAI
- Stanford
- Surescripts LLC
- Telecommunications Technology Association
- Toloka
- TU Eindhoven
- Turaco Strategy
- UC Irvine
- Univ. of British Columbia (UBC)
- Univ. of Birmingham
- Univ. of Cambridge
- Univ. of Chicago
- Univ. of Illinois at Urbana-Champaign
- Univ. of Southern California (USC)
- Univ. of Trento
Funding for the initial AI Safety working group effort was provided by Google, Intel, Meta, NVIDIA and Qualcomm Technologies, Inc. MLCommons is committed to supporting a long-term effort for this important work and welcomes additional funding contributors.