Giters

data-corentinv / awesome-reliable-ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

(WIP) Reliable IA

Table of Contents

Contributing

Please feel free to send me pull requests or email (vasseur.corentin@gmail.com) to add links.

Introduction : why

Notes :

Scientific Themes

  • Monitoring

Model observability

Transparency

  • Methodology
  • Explainability
  • Interpretable IA

Questions:

  • How does each features contribute to a model's prediction?
  • How does a prediction change dependent on feature inputs?
  • What features are or not are significant for a given outcome?
  • What features would you change to obtain a different prediction?
  • How robust is the model?

Tools:

Shap: Patrick Hall

A Selection of Medium articles :

  • Shap (Shapley Additive exPlanations) : Shap is a model agnostic and works by breaking down the contribution of each feature and attributing a score to each feature.

  • LIME (local Interpretable Model-agnostic Explanations) : LIME is another model agnostic method that works by approximinating the behavior of the model locally around a specific prediction.

  • Eli5 : library for debugging and explaining classifiers. It provides feature importance scores, as well as "reason codes" for scikit-learn, Keras, Xgboost, LightGBM, CatBoost

  • Shapash : python library which aims to make machine learning interpretable and understandable to everyone. Shapash provides several types of visualization with explicit labels.

  • Anchors : method for generating humain-interpretable rules that can be used to explain the predictions of a machine learning model.

  • XAI (eXplainable AI): XAI is a library for explaining and visualizeing the predictions of machine learning models including feature importance scores, decision trees, and rule-based explanations.

  • BreakDown : tool that can be used to explain the predictions of linear models. It works by decomposing the model's output into the contribution of each input feature.

  • Interpret-text: interpret-text is a library for explaining the predictions of natural language processing models.

  • iml (Interpretable MAchine Learning): iml currently contains the interface and IO code from the Shap project and it will potentially also do the same for the Lime project.

  • aix360 (AI Explainibility 360): aix360 includes a comprehensive set of algorithms that cover different dimensions.

  • OmnniXAI (short for Omni eXplainable AI), adresses several problems with interpreting judgements produced by machine learning models in practice.

  • Seldon (alibi explain / detect):

  • Source: https://www.linkedin.com/posts/maryammiradi_ai-explainability-beyond-the-surface-uncommon-activity-7212004104263380993-qmcn/?utm_source=share&utm_medium=member_ios

Ethics

Fairness

Robutness

  • Notes
  • Ex.
    • Confiance interval
    • Perturbations : graines aléatoire
    • MAPIE

Environment

  • Impact environment : carbonml
  • Solutions : small data ?

Reproducibility

Notes:

  • Tracking
  • Model registry
  • Train Dataset

Tools:

  • mlflow

Platform :

Stategic Themes

Human-Centered: "Contrôle Humain"

Responsability

Privacy

Quelques sources:

Books:

  • ML in High-Risk Applications: https://learning.oreilly.com/library/view/machine-learning-for/9781098102425/
    • Chp1: Contemporary Model Governance: "Going fast and breaking thinkgs. It can mean that a small group of data scientists and engineers causes real harm at scale to many people." -> Cas d'application sur la voiture autonome chez Uber (gestion incidents, risk management, documentation).
    • Chap2: Debugging ML Systems: "Tests data area under the curve (AUC) tells us almost nothing aboout harms or security vulnerabilities. Yet these problems are often whu AI systems fail once deployed." -> Cas d'application octroi de crédit (détection de dérives, stress-tests).
    • Chap3: Security for ML: "The worst ennemy of security is complexity. Unduly complex AI systems are innately insecure." -> Censure anti-terroriste de FB (Attaques, vol de données / modèles, sécurité IT)

About

Languages

Language:Jupyter Notebook 98.7%Language:Python 1.3%