This repository contains the experiment code required to generate the results presented in our submission to UAI2023:
L. Wimmer, Y. Sale, P. Hofman, B. Bischl & E. Hüllermeier:
"Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning:
Are Conditional Entropy and Mutual Information Appropriate Measures?"
Link to arXiv: TBD
The package requirements are listed in environment.yml.
A full Python environment can be created from this file using, e.g., conda
(conda env create -f environment.yml).
The experiments in both train.py (computer vision examples) and
train_tabular.py (tabular classification task) can be run from the command
line interface (CLI).
For convenience, CLI options can be specified in the respective bash files
run_experiment.sh and run_experiment_tabular.sh.
Analogously, the evaluation in eval.py, which will produce .csv files with
experiment results, can be triggered from the CLI via
run_eval.sh.