Potential repeated activation function
caiqi opened this issue · comments
Hi, if "activation_mm" is set, it will first be applied here https://github.com/Cadene/vqa.pytorch/blob/master/vqa/models/fusion.py#L117 and then applied here https://github.com/Cadene/vqa.pytorch/blob/master/vqa/models/att.py#L74 again. I think it may cause unexpected results? Thanks.
Hello,
You are talking about MutanAtt, which inherits from AbstractAtt.
MutanAtt has two kind of fusion layers:
- fusion_classif which will be used to merge the visual and the question vectors with the MutanFusion module.
- fusion_att which will be used to merge the visual tensor from the ConvNet (14x14x2048) and the question vectors with the MutanFusion2d module in a "convolutional" way.
Thus, this activation function att.py#L74 will be computed after the call of the MutanFusion2d att.py#L72 module (and not the MutanFusion module which will be called at the end att.py#L140.
Also, notice that we do not use any activation_mm
in our default MutanAtt model, but we use this option in our default MLBAtt model.
Are my explanations clear enough?
Greetings,
Remi
EDIT: but for sure, you have to be careful to not apply the same activation function twice (if you modify the default options).
Thank you for your reply!