Modifying Eq 12/13 to deal with negative loss
MohitLamba94 opened this issue · comments
Hi,
Thanks for this great work. In Eq. 12/13 you have a log(variance)
that might go negative for variance<1. Assuming, variance to be always +ve can we modify the Eq. 12/13 to log(1+variance)
to prevent negative loss and allowing better optimization?
Will this modification hamper training or net output during inference?
Hi,
I never tried, because the negative loss never gave issues in my experiments.
You can use sigmoids over the outputs, but I think your solution should work as well.