How to draw a heatmap for decoder?
ddwhzh opened this issue · comments
Sorry to interrupt, but I'm very interested in how to get a heatmap about the decoder like fig.3 mentioned in paper, because the input and output of decoder are both a pair of query(100, N, 256), and the multihead_attn function is torch wrapped. So, I would appreciate any advice.
plot_ours_feature.py.zip
Put this script under 'CDN' and run it with this command:
python plot_ours_feature.py --pretrained pretrained/hico_cdn_b.pth --dataset_file hico --hoi_path data/hico_20160224_det --num_obj_classes 80 --num_verb_classes 117 --backbone resnet50 --num_queries 64 --dec_layers_hopd 6 --dec_layers_interaction 6
Then, the feature maps of the decoded queries with top3 scores for ho-pd and interaction decoder are generated under 'outputs_hico'.
THX