zohdit / unboxer

Heatmap Clustering to Understand the Misbehaviours Exposed by Automatically Generated Test Inputs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

πŸ₯‘ How to run the unboxer πŸ₯‘

First, you should install the environment and set the configurations based on the case study (MNIST or IMDB) you want to run:

πŸ“² Install πŸ“²

βš™οΈ Configure βš™οΈ

Generate inputs

You should run the following command to generate the inputs for corresponding case study.

python -m utls.generate_inputs

πŸ₯΅ Generate the heatmaps πŸ₯΅

You should run the following command to generate the heatmaps.

python -m steps.process_heatmaps

The tool will experiment with the different explainers, find the best configuration for the dimensionality reduction, and export the data collected during the experiment.

πŸ—Ί Generate the featuremaps πŸ—Ί

You can run the following command to generate the featuremaps.

python -m steps.process_featuremaps

The tool will generate the featuremaps, and export the data collected during the experiment.

πŸ“Š Export the insights πŸ“Š

You can run the following command to generate the insights about the data.

python -m steps.insights.insights

!!! IMPORTANT !!!
Remember to generate the heatmaps and the featuremaps before running this command.

The tool with prompt a menu with a set of options, and will guide you through the process.

πŸ€” Export the data for the human evaluation πŸ€”

You can run the following command to export the data for the human evaluation.

python -m steps.human_evaluation.export_samples

!!! IMPORTANT !!!
Remember to generate the heatmaps and the featuremaps before running this command.

The tool will generate samples for human study in out/human_evaluation.

About

Heatmap Clustering to Understand the Misbehaviours Exposed by Automatically Generated Test Inputs


Languages

Language:HTML 98.5%Language:Python 1.3%Language:Cython 0.1%Language:Makefile 0.0%Language:Batchfile 0.0%Language:Dockerfile 0.0%