Deep Learning models involving Convolutional Neural Networks enjoy stellar success in solving the classic problem of image classification. Despite being highly accurate in predicting class labels, many models fail to explain the underlying decisions taken to reach the final prediction. Most methods based on saliency maps explain only the final decision made by the model. In this work, we analyse the intermediate decisions involved in the task of classification by using saliency maps. This analysis work is based on the prior work of NBDT. We report our analysis on two publicly available basic datasets of CIFAR-10 and TinyImageNet and use RISE for saliency map generation.
Siddhant Agarwal, Aadarsh Sahoo, Adarsh Patnaik, Rajat Kumar Jenamani
Indian Institute of Technology Kharagpur, West Bengal, India