Potential repeated activation function

[Qi Cai]

Hi, if "activation_mm" is set, it will first be applied here https://github.com/Cadene/vqa.pytorch/blob/master/vqa/models/fusion.py#L117 and then applied here https://github.com/Cadene/vqa.pytorch/blob/master/vqa/models/att.py#L74 again. I think it may cause unexpected results? Thanks.

[Remi]

Hello,

You are talking about MutanAtt, which inherits from AbstractAtt.
MutanAtt has two kind of fusion layers:

• fusion_classif which will be used to merge the visual and the question vectors with the MutanFusion module.
• fusion_att which will be used to merge the visual tensor from the ConvNet (14x14x2048) and the question vectors with the MutanFusion2d module in a "convolutional" way.

Thus, this activation function att.py#L74 will be computed after the call of the MutanFusion2d att.py#L72 module (and not the MutanFusion module which will be called at the end att.py#L140.

Also, notice that we do not use any activation_mm in our default MutanAtt model, but we use this option in our default MLBAtt model.

Are my explanations clear enough?

Greetings,
Remi

EDIT: but for sure, you have to be careful to not apply the same activation function twice (if you modify the default options).

[Qi Cai]

Thank you for your reply!