gchoi / class-activation-mapping

tensorflow implementation of cnn localization project by CSAIL@MIT (CVPR'16)

Class Activation Mapping

Tensorflow implementation of Learning Deep Features for Discriminative Localization by Zhou et al presented in CVPR'16.

Caffe version by the author is here

Prerequisites

• Python 2.7+
• NumPy
• SciPy
• tqdm
• Tensorflow r1.0+
• Caffe (only for extracting parameters from pre-trained model)
• matplotlib (for plotting the result)

Data

• CALTECH256

Preparation

1. Clone this repo, create log/ and caffe_pretrained/ folder:

   git clone https://github.com/markdtw/class-activation-mapping.git
   cd class-activation-mapping
   mkdir caffe_pretrained
   mkdir log

2. To train on CALTECH256 dataset, download the original VGG16 graph (.prototxt) and model (.caffemodel) from here, save them in caffe_pretrained/ folder.

3. To test directly from the pretrained ImageNet model, download the vgg16CAM graph and model from the author's repo, save them in caffe_pretrained/ folder as well.

4. If you went through both 2 and 3, your caffe_pretrained/ folder should contain these:

   • vgg16CAM_train_iter_90000.caffemodel
   • vgg16CAM_deploy.prototxt
   • VGG_ILSVRC_16_layers.caffemodel
   • VGG_ILSVRC_16_layers_deploy.prototxt

   We need these only to convert them into .npy format.

5. Run extract function in utils.py with proper input arguments, this will convert .caffemodel to .npy. Now your caffe_pretrained/ folder should have these two extra files:

   • vgg16CAM_train_iter_90000.npy
   • VGG_ILSVRC_16_layers.npy

   Let me know if you don't want to install caffe but still need them.

Train

Train (fine-tune) CALTECH256 from VGG_ILSVRC_16_layers with default settings:

python main.py --train

Train (fine-tune) CALTECH256 from previous checkpoint:

python main.py --train --modelpath=log/vgg16CAM_calt256-X

Check out tunable arguments:

python main.py

Test

Test the model provided by the authors (trained on ImageNet)

python main.py --test --imgpath=/path/to/img.jpg

Test the model trained on CALTECH256 by you given epoch X

python main.py --test --imgpath=/path/to/img.jpg --modelpath=log/vgg16CAM_calt256-X

This will save a result figure in this directory.

Some results

Others

• First time training will generate calt256_224_224.tfrecords file to your CALTECH256/ folder to load data in queue.
• Unfortunately training on CALTECH256 has not yet been successful/completed (super low accuracy). Please let me know if you can train the model with good result.
• Testing from the ImageNet model works perfectly with the same architecture.
• Issues are more than welcome!

Resources

• The project website
• The paper
• Author's repo
• vgg implementation highly based on this repo
• how to load data in queue with tensorflow here(sorry for self-promote)