AI Safety
Prompt Generation Expression of Interest
Now open for submissions!
MLCommons is seeking expressions of interest (EOI) from qualified organizations to generate datasets of prompts for the upcoming v1.0 AI safety benchmark suite, launching later this fall.
This is a paid opportunity. The deadline to submit is July 19, 2024. Early submissions are encouraged.
Deliverable options
Organizations can express interest in one or both of the following deliverable options. While MLCommons can adjust the submission dates of individual deliverables, the project must be completed by October 1, 2024.
All submissions must demonstrate that you can fulfill one or both deliverable options:
OPTION 1: Pilot Project
- Generation and delivery of ~200 prompts
- ~80% of prompts should be simple malicious / vulnerable
- ~20% of prompts should be generic adversarial
- Option to cover a target MLCommons language (English, French, Simplified Chinese, Hindi) or propose another language
- Option to cover one of 13 MLCommons hazards or propose another
- Smaller budget
OPTION 2: Full Coverage Project
- Generation and delivery of ~50,000 prompts in each of the target languages
- ~80% of the prompts should be simple malicious / vulnerable use
- ~20% of the prompts should be “generic adversarial” prompts
- Must cover English, French, Simplified Chinese and Hindi
- Must cover all MLCommons hazards and personas
- Bigger budget
Please read the EOI description for information on who should express interest, disqualifications, the budget, and the expected deliverable timeline. Clarifying questions and answers can be found here. Please email Mala at [email protected] with questions.