MLCommons AI Safety
Creating a benchmark suite for safer AI
The MLCommons AI Safety working group is composed of a global consortium of industry leaders, practitioners, researchers, and civil society experts committed to building a harmonized approach to AI safety.
The working group is creating a platform, tools, and tests for developing a standard AI Safety benchmark suite for different use cases to help guide responsible AI development.
Introducing the v0.5 AI Safety benchmark proof of concept
The v0.5 benchmark proof of concept (POC), announced April 15, 2024, focuses on measuring the safety of large language models (LLMs) by assessing the models’ responses to prompts across multiple hazard categories. The v0.5 POC includes:
- A benchmark that runs a series of tests for a taxonomy of hazards.
- A platform that defines the benchmark and generates a report.
- A new testing engine (inspired by the HELM framework from Stanford CRFM) to run the tests.
The v0.5 POC shows results from more than a dozen, anonymized systems-under-test (SUT). It is being shared with the community now for experimentation and feedback to inform improvements for a comprehensive v1.0 release later this year.
Join the working group to help shape the v1.0 benchmark suite and beyond.
AI Safety related Blogs and News
-
Announcing MLCommons AI Safety v0.5 Proof of Concept
Achieving a major milestone towards standard benchmarks for evaluating AI Safety
-
The AI Safety Ecosystem Needs Standard Benchmarks
IEEE Spectrum contributed blog excerpt, authored by the MLCommons AI Safety working group
-
Our comments to the NTIA on Open Foundation models
Open Foundations models play an important role in developing AI safety benchmarks.
AI Safety v0.5 benchmark POC supporting documentation
v0.5 POC Technical Glossary
April 2024 | v0.5 POC Documentation
v0.5 POC Test Specification Schema
April 2024 | v0.5 POC Documentation
v0.5 POC Taxonomy of Hazards
April 2024 | v0.5 POC Documentation
Whitepaper – Introducing v0.5 of the AI Safety Benchmark from MLCommons
April 2024 | v0.5 POC Documentation
AI Safety working group contributors
The MLCommons AI Safety working group is composed of a global group of industry leaders, practitioners, researchers, and civil society experts committed to building a harmonized approach to AI safety. The following organizations have contributed to the AI Safety working group.
- Accenture
- ActiveFence
- Anthropic
- Argonne National Laboratory
- Bain & Company
- Blue Yonder
- Bocconi University
- Broadcom
- cKnowledge, cTuning foundation
- Carnegie Mellon
- Center for Security and Emerging Technology
- Coactive AI
- Cohere
- Columbia University
- Common Crawl Foundation
- Commn Ground
- Context Fund
- Credo AI
- Deloitte
- Digital Safety Research Institute
- Dotphoton
- EleutherAI
- Ethriva
- Febus
- Futurewei Technologies
- Georgia Institute of Technology
- Hewlett Packard Enterprise
- Humanitas AI
- IIT Delhi
- Illinois Institute of Technology
- Inflection
- Intel
- Kaggle
- Lawrence Livermore National Laboratory
- Learn Prompting
- Lenovo
- MIT
- Meta FAIR
- Microsoft
- NASA
- Nebius
- NVIDIA Corporation
- NewsGuard
- Nutanix
- OpenAI
- Process Dynamics
- Protecto.ai
- Protiviti
- Qualcomm Technologies, Inc.
- RAND
- Reins AI
- SAP
- SaferAI
- Sony AI
- Stanford
- Surescripts LLC
- Telecommunications Technology Association
- Toloka
- TU Eindhoven
- Turaco Strategy
- UC Irvine
- Univ. of British Columbia (UBC)
- Univ. of Birmingham
- Univ. of Cambridge
- Univ. of Chicago
- Univ. of Illinois at Urbana-Champaign
- Univ. of Southern California (USC)
- Univ. of Trento
Funding for the initial AI Safety working group effort was provided by Google, Intel, Meta, NVIDIA and Qualcomm Technologies, Inc. MLCommons is committed to supporting a long-term effort for this important work and welcomes additional funding contributors.