WWW: A Unified Framework for Explaining What, Where and Why of Neural Networks by Interpretation of Neuron Concepts
This is the official source code for [WWW: A Unified Framework for Explaining What, Where and Why of Neural Networks by Interpretation of Neuron Concepts] IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024
It is tested under Ubuntu Linux 20.04 and Python 3.8 environment, and requries some packages to be installed:
Please download ImageNet-1k and place the training data and validation data in
./datasets/ILSVRC-2012/train and ./datasets/ILSVRC-2012/val, respectively.
For ImageNet, the model we used in the paper is the pre-trained ResNet-50 and vit is provided by Pytorch and timm. The download process
will start upon running.
For places365, please download http://places2.csail.mit.edu/models_places365/resnet18_places365.pth.tar and place in the ./utils folder.
WWW need precomputing for calculate Shapley value approximation.
Run ./extract_shap.py.
For ImageNet with ResNet-50 experiment we placed calcualted Class-wise Shapley value in
./utils/RN50_ImageNet_class_shap.pkl.
Run ./example_selection.py.
Run ./concept_matching.py.
Run ./image_heatmap.py.