mask_logits in AttentionModel is set to False by default
caiqi opened this issue · comments
Thanks for the awesome work and for sharing the code! The mask_logits is set to False by default and thus the leaf logits are normalized without removing the masked leaf nodes. Is this intended or a typo?
Thank you so much for your feedback, leaf logits are normalized without removing the masked leaf nodes directly since we find the policy training is not stable when all leaf nodes are not valid and set to '-inf'. Instead, leaf nodes are removed when calculating action probabilities (lines 139 - 144).