Cant print Grad tensor from CTCLoss function
adhiraj69 opened this issue · comments
when ever i try to print the gradient tensor obtained from gpu_ctc
i am using pytorch
and i am editing the warpctc_pytorch init.py
in the forward function i tried printing the grads tensor but i encounter the following error
i encounter this error
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCCachingHostAllocator.cpp line=278 error=700 : an illegal memory access was encountered
Traceback (most recent call last):
File "CTC_loss_2.py", line 765, in
loss1 = ctcloss(x1, labels, inp1,targs1)
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/warpctc_pytorch-0.1-py3.8-linux-x86_64.egg/warpctc_pytorch/init.py", line 83, in forward
return self.ctc(acts, labels, act_lens, label_lens, self.size_average,
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/warpctc_pytorch-0.1-py3.8-linux-x86_64.egg/warpctc_pytorch/init.py", line 46, in forward
print(grads)
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/tensor.py", line 153, in repr
return torch._tensor_str._str(self)
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 371, in _str
return _str_intern(self)
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 351, in _str_intern
tensor_str = _tensor_str(self, indent)
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 241, in _tensor_str
formatter = _Formatter(get_summarized_data(self) if summarize else self)
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 273, in get_summarized_data
return torch.stack([get_summarized_data(x) for x in (start + end)])
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 273, in
return torch.stack([get_summarized_data(x) for x in (start + end)])
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 275, in get_summarized_data
return torch.stack([get_summarized_data(x) for x in self])
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 275, in
return torch.stack([get_summarized_data(x) for x in self])
File "/home/dzilla69/anaconda3/lib/python3.8/site-packages/torch/_tensor_str.py", line 266, in get_summarized_data
return torch.cat((self[:PRINT_OPTS.edgeitems], self[-PRINT_OPTS.edgeitems:]))
RuntimeError: cuda runtime error (700) : an illegal memory access was encountered at /pytorch/aten/src/THC/THCCachingHostAllocator.cpp:278