By Geoffrey Fox, Jeyan Thiyagalingam, Tony Hey, Juri Papay, Gregor von Laszewski and members of the MLCommons science working group
The MLCommons® Science working group is pleased to announce the availability of MLCommons Science GitHub, a series of 4 open source benchmarks. As a new addition to the MLCommons suite of benchmarks, these tools are aimed at domain scientists, machine learning experts, and students. The goal of MLCommons Science GitHub is to uncover and gather novel Machine Learning (ML) solutions to improve scientific discovery by improving accuracy on critical problems. Each benchmark includes open datasets, and reference models with algorithms and software. These rigorously documented benchmarks will also be used as tutorials and courses and run on new systems.
Join Us
We are calling on the international community to join us for an open call for participation to help increase model accuracy and/or extend uses with other datasets or scientific fields. The call for participation will have rolling submissions described in the overall policy and submissions documents. Participants should share their findings on the MLCommons Science Results GitHub. Ideal submissions will include a trained network that exceeds the target accuracy of one of the reference models for one of the benchmarks, together with a description of the improvements made.
Our Work
The MLCommons Science working group is part of the MLCommons research initiative which spans work in Algorithms, DataPerf, Dynabench, Medical, Science, and Storage. It collaborates with industry and research, with a shared mission to accelerate machine learning innovation to benefit society. The Science benchmarks differ from the flagship MLPerf benchmarks of MLCommons in terms of an emphasis on education and the impact on scientific discovery rather than system performance. New benchmarks are currently being considered. If you wish to join the open collaboration to discuss this work, contribute new challenges, or have any questions, please contact [email protected] or visit the science working group page. The working group meets every 2 weeks.
About MLCommons
MLCommons is an open engineering consortium with a mission to benefit society by accelerating innovation in machine learning. The foundation for MLCommons began with the MLPerf benchmark in 2018, which rapidly scaled as a set of industry metrics to measure machine learning performance and promote transparency of machine learning techniques. In collaboration with its 50+ founding partners – global technology providers, academics and researchers, MLCommons is focused on collaborative engineering work that builds tools for the entire machine learning industry through benchmarks and metrics, public datasets and best practices.