small issue
ajtao opened this issue · comments
Hi, i thought i'd let you know, I had to change VideoFolder.py to call close() and not _close() to close video files, here: https://github.com/Atcold/pytorch-CortexNet/blob/master/data/VideoFolder.py#L146,L158
FWIW, with default training settings, i'm seeing the CE loss blow up and goes NAN after a few iterations. I am trying to lower LR but perhaps there a better set of default params to use?
Yes, you can lower the learning rate relative to the CE.
The values I wrote on the paper did work for me.
Are you training a MatchNet or a TempoNet?
You can send a PR, with your edit. I'll be more than welcome to accept it. Right now I'm transitioning from my PhD to my PostDoc, so I'm not much on the computer.
I'm training MatchNet. If i use the values in the paper, the CE loss appears to continue to explode.
These are my settings for the published MatchNet (λ is what I later renamed τ).
CLI arguments: --size 3 32 64 128 256 --lambda 0.001 --big-t 10 --log-interval 10 --cuda --save last/model.pth.tar --epochs 10 --model model_02 --lr-decay 3.16 3
Current commit hash: 1a162fad6ee96b55886ddcacf8cd735af0e343e6
So, try to use the same settings, but instead of --lambda use --tau.
The results must be the same, or a bug has been introduced somewhere.
At worst, checkout the commit hash.
The point of using quasi-exploding loss functions is that they maximise the learning speed.
Always keep your system loss on the verge of exploding, and you'll have the best learning experience.
Edit: oh, yes, there is a typo in the paper. It should be 0.001 and not 0.01. Thanks! I'll update it in the next version.
Edit2: my best network settings are the following. Once more, replace --lambda with --tau.
Validation predicting loss of 2 mMSE.
CLI arguments: --size 3 32 64 128 256 --lambda 0 --big-t 10 --log-interval 10 --cuda --save last/model.pth.tar --view 2 --show-x_hat --epochs 30 --model model_02 --lr-decay 10 10
Current commit hash: 8cbd3347472c57ae0749248d3a95d75bc6f73e25