MLCommons

cm4mlops

A collection of portable, reusable and cross-platform automation recipes (CM scripts) with a human-friendly interface and minimal dependencies to make it easier to build, run, benchmark and optimize AI, ML and other applications and systems across diverse and continuously changing models, data sets, software and hardware (cloud/edge)

croissant

Croissant is a high-level format for machine learning datasets that brings together four rich layers.

logging

MLPerf™ logging library

GaNDLF

A generalizable application framework for segmentation, regression, and classification using PyTorch

modelbench

Run safety benchmarks against AI models and view detailed reports showing how well they performed.

submissions_algorithms

dynabench

ck

Collective Mind (CM) is a small, modular, cross-platform and decentralized workflow automation framework with a human-friendly interface and reusable automation recipes to make it easier to build, run, benchmark and optimize AI, ML and other applications and systems across diverse and continuously changing models, data, software and hardware

cm4mlperf-inference

mobile_app_open

Mobile App Open

hn7st6jxx77dbthjf9co

medperf

An open benchmarking platform for medical artificial intelligence using Federated Evaluation.

training_policies

Issues related to MLPerf™ training policies, including rules and suggested changes

cm4abtf

CM interface and automation recipes for ABTF

policies

General policies for MLPerf™ including submission rules, coding standards, etc.

inference

Reference implementations of MLPerf™ inference benchmarks

tiny

MLPerf™ Tiny is an ML benchmark suite for extremely low-power systems such as microcontrollers

modelgauge

Make it easy to automatically and uniformly measure the behavior of many AI Systems.

mlcube

MLCube® is a project that reduces friction for machine learning by ensuring that models are easily portable and reproducible.

chakra

Repository for MLCommons Chakra schema and tools

algorithmic-efficiency

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

inference_results_visualization_template

inference_results_v4.1

This repository contains the results and code for the MLPerf™ Inference v4.1 benchmark.

abtf-ssd-pytorch

training

Reference implementations of MLPerf™ training benchmarks

storage

MLPerf™ Storage Benchmark Suite

mobile_models

MLPerf™ Mobile models

algorithms_results_v0.5

This repository contains the results and code for the AlgoPerf v0.5 benchmark.

template

inference_results_v4.0

This repository contains the results and code for the MLPerf™ Inference v4.0 benchmark.