DAGMM_LSTMAutoEncoder_withWindow: RuntimeError: cuda runtime error (77) : an illegal memory access was encountered at /pytorch/aten/src/THC/generic/THCTensorCopy.c:20
WGierke opened this issue · comments
2018-06-21 07:18:44 [ERROR] src.evaluation.evaluator: An exception occurred while training DAGMM_LSTMAutoEncoder_withWindow on Synthetic Variance Outliers: cuda runtime error (77) : an illegal memory access was encountered at /pytorch/aten/src/THC/generic/THCTensorCopy.c:20
2018-06-21 07:18:44 [ERROR] src.evaluation.evaluator: Traceback (most recent call last):
File "/repo/src/evaluation/evaluator.py", line 71, in evaluate
det.fit(X_train, y_train)
File "/repo/src/algorithms/dagmm.py", line 192, in fit
self.dagmm.cuda()
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 249, in cuda
return self._apply(lambda t: t.cuda(device))
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 176, in _apply
module._apply(fn)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 176, in _apply
module._apply(fn)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py", line 111, in _apply
ret = super(RNNBase, self)._apply(fn)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 182, in _apply
param.data = fn(param.data)
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 249, in <lambda>
return self._apply(lambda t: t.cuda(device))
RuntimeError: cuda runtime error (77) : an illegal memory access was encountered at /pytorch/aten/src/THC/generic/THCTensorCopy.c:20Closed via d5cced5