FINRAOS / model-validation-toolkit

Model Validation Toolkit is a collection of tools to assist with validating machine learning models prior to deploying them to production and monitoring them after deployment to production.

Model Validation Tookit

Installation

Run pip install mvtk.

Windows users: Until Jaxlib is supported on windows natively you will need to either use this library from a Linux subsystem or within a Docker container. Alternatively, you can build jaxlib from source.

Developers

Check out this repository and cd into the directory.

Run pip install -e ".[doc]".

The [doc] is used to install dependencies for building documentation. You will need pandoc installed.

Submodules

You can import:

• mvtk.credibility for assessing credibility from sample size.
• mvtk.interprenet for building interpretable neural nets.
• mvtk.thresholding for adaptive thresholding.
• mvtk.sobol for Sobol sensitivity analysis
• mvtk.supervisor for divergence analysis
• mvtk.metrics for specialised metrics
• mvtk.bias_variance for bias-variance decomposition

Documentation

You can run make -C docs html on a Mac or make.bat -C docs html on a PC to just rebuild the docs. In this case, point your browser to docs/_build/html/index.html to view the homepage. If your browser was already pointing to documentation that you changed, you can refresh the page to see the changes.