This is the project page for Model Assertions for Monitoring and Improving Machine Learning.
Please read the paper for full technical details.
pip install model_assertions
We provide two examples.
The first example (Tabular.ipynb) shows an example of predicting house prices from features.
This example trains a lienar model to predict the house price.
We define a model assertion that asserts the predicted house price should be positive. While seemingly simple, the predictions violate this assertion!
The second example (Consistency.ipynb) shows an example of predcting people in a TV show and several attributes of the person (gender and hair color).
In this example, we assume that the predictions are already provided.
This example shows how to use the attribute- and time-consistency APIs. It asserts that the same person in the same scene should have the same gender and hair color. It also asserts that a person in the same scene should appear across consecutive frames without gaps.
If you find this project useful, please cite us at
@article{kang2020model,
title={Model assertions for monitoring and improving ML model},
author={Kang, Daniel and Raghavan, Deepti and Bailis, Peter and Zaharia, Matei},
journal={MLSys},
year={2020}
}
and contact us if you deploy model assertions!