ADL location in resnet & pytorch implementation

[Yassine]

First of all, great paper, and thank you for providing the tf implementation,

I was wondering about the correct locations to add the ADL module in a Resnet, in the command you only gave VGG example (end of block 3 & 4 and layer 53), but for resnet I see that you only apply ADL at the beginning of each Resnet block, can you please specify the best location to add ADL in this case.

Also, I am curious if the pytroch implementation will be available soon or it'll take some time (maybe I can start implementing it in this case).

[Junsuk Choe]

Thanks for your interest in our work.

I am sorry for the delay of releasing more detailed instruction and pytorch implementation. I think that it will take about a month to release them because I am totally occupied by other projects now.

For the ResNet, we apply ADL blocks to first residual blocks of group 3&4 and right before the global average pooling layer.

You can use this:

python CAM-resnet.py --gpu 0 --data /notebooks/dataset/ILSVRC2012/ --imagenet --base-lr 0.1 --logdir ResNet_ImageNet --load ResNet --batch 128 --depth 50 --mode se --attdrop 31 41 5 --threshold 0.90 --keep_prob 0.25

[Yassine]

Thank you.

[Yassine]

Hi, sorry for asking the same question again, but can you please provide the value of the argument --ADL-position for Resnet in the Pytorch version, I tired using "31 41 5" but its seems the function make_cfg a list of lists.

Thanks

[Junsuk Choe]

Hi, we are sorry for your inconvenience. Currently, the location numbers in pytorch version is not consistent with those of tensorflow version (In pytorch version, we use zero-based numbering).

You can use this:

gpu=0,1
name=resnet
epoch=200
decay=60
model=resnet50_ADL
server=tcp://127.0.0.1:12345
batch=128
wd=1e-4
lr=0.01
ADL_pos="30 40 42"
data_root="../CUB_200_2011"

CUDA_VISIBLE_DEVICES=${gpu} python train.py -a ${model} --dist-url ${server} \
    --multiprocessing-distributed --world-size 1 --pretrained \
    --data ${data_root} --dataset CUB \
    --train-list datalist/CUB/train.txt \
    --test-list datalist/CUB/test.txt \
    --data-list datalist/CUB/ \
    --task wsol \
    --batch-size ${batch} --epochs ${epoch} --LR-decay ${decay} \
    --wd ${wd} --lr ${lr} --nest --name ${name} \
    --use-ADL --ADL-thr 0.90 --ADL-rate 0.75 --ADL-pos ${ADL_pos}

[Yassine]

Thank you very much.