Bugs of AssertionError?

[Shengkun Tang]

Thanks for your contribution. When I train the net for serveral iterations or epochs, some error occured. It's uncertain when this will happen, but it will always happen.
Traceback (most recent call last):
File "train_eval.py", line 236, in
start_epoch=cfg.TRAIN.START_EPOCH)
File "train_eval.py", line 110, in train_eval_model
loss = criterion(s_pred, perm_mat, n1_gt, n2_gt)
File "/home/tsk/anaconda3/envs/PCA_GM/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/home/tsk/PCA_GM/utils/permutation_loss.py", line 18, in forward
assert torch.all((pred_perm >= 0) * (pred_perm <= 1))
AssertionError

[Runzhong Wang]

Hi Shengku, thank you for your interest. Could you please go through these two issues #4 #10 and see whether your problem can be resolved?

[Shengkun Tang]

Hi Shengku, thank you for your interest. Could you please go through these two issues #4 #10 and see whether your problem can be resolved?

Thanks for your immediate reply. So, it will happen and there is no solution yet? This is maybe because of the problem of gradient?

[Runzhong Wang]

I am afraid there is no solution yet. And if the model provides a moderately good result before it crashes, I think it will be fine.

Gradient might be the issue because there are numerous iterations encountered in the Sinkhorn (Bi-Stochastic) layer.

[Shengkun Tang]

I am afraid there is no solution yet. And if the model provides a moderately good result before it crashes, I think it will be fine.

Gradient might be the issue because there are numerous iterations encountered in the Sinkhorn (Bi-Stochastic) layer.

Thanks!