error in correlation_forward_cuda_kernel: no kernel image is available for execution on the devic

[Neeraj Kumar]

I am getting below error:

Traceback (most recent call last):
File "train.py", line 148, in
train()
File "train.py", line 60, in train
flow_ref, conf_ref = flowNet(real_B, real_B_prev) # reference flows and confidences
File "/home/neerajku/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/neerajku/neerajAvatar/vid2vid/models/flownet.py", line 38, in forward
flow, conf = self.compute_flow_and_conf(input_A, input_B)
File "/home/neerajku/neerajAvatar/vid2vid/models/flownet.py", line 54, in compute_flow_and_conf
flow1 = self.flowNet(data1)
File "/home/neerajku/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/neerajku/neerajAvatar/vid2vid/models/flownet2_pytorch/models.py", line 105, in forward
flownetc_flow2 = self.flownetc(x)[0]
File "/home/neerajku/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/neerajku/neerajAvatar/vid2vid/models/flownet2_pytorch/networks/FlowNetC.py", line 89, in forward
out_corr = self.corr(out_conv3a, out_conv3b) # False
File "/home/neerajku/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, kwargs)
File "/home/neerajku/neerajAvatar/vid2vid/models/flownet2_pytorch/networks/correlation_package/correlation.py", line 59, in forward
result = CorrelationFunction(self.pad_size, self.kernel_size, self.max_displacement,self.stride1, self.stride2, self.corr_multiply)(input1, input2)
File "/home/neerajku/neerajAvatar/vid2vid/models/flownet2_pytorch/networks/correlation_package/correlation.py", line 27, in forward
self.pad_size, self.kernel_size, self.max_displacement,self.stride1, self.stride2, self.corr_multiply)
RuntimeError: CUDA call failed (correlation_forward_cuda at correlation_cuda.cc:82)
frame #0: std::function<std::string ()>::operator()() const + 0x11 (0x7fe3f3b21021 in /home/neerajku/anaconda3/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x2a (0x7fe3f3b208ea in /home/neerajku/anaconda3/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #2: correlation_forward_cuda(at::Tensor&, at::Tensor&, at::Tensor&, at::Tensor&, at::Tensor&, int, int, int, int, int, int) + 0x6e7 (0x7fe3e0099cb7 in /home/neerajku/.local/lib/python3.7/site-packages/correlation_cuda-0.0.0-py3.7-linux-x86_64.egg/correlation_cuda.cpython-37m-x86_64-linux-gnu.so)
frame #3: + 0x17e54 (0x7fe3e00a5e54 in /home/neerajku/.local/lib/python3.7/site-packages/correlation_cuda-0.0.0-py3.7-linux-x86_64.egg/correlation_cuda.cpython-37m-x86_64-linux-gnu.so)
frame #4: + 0x1808e (0x7fe3e00a608e in /home/neerajku/.local/lib/python3.7/site-packages/correlation_cuda-0.0.0-py3.7-linux-x86_64.egg/correlation_cuda.cpython-37m-x86_64-linux-gnu.so)
frame #5: + 0x14c25 (0x7fe3e00a2c25 in /home/neerajku/.local/lib/python3.7/site-packages/correlation_cuda-0.0.0-py3.7-linux-x86_64.egg/correlation_cuda.cpython-37m-x86_64-linux-gnu.so)

frame #12: THPFunction_do_forward(THPFunction, _object) + 0x15c (0x7fe42e0610ac in /home/neerajku/anaconda3/lib/python3.7/site-packages/torch/lib/libtorch_python.so)

configuration :

cuda - 10.2
pytorch- 1.3
python -3.7
V100 server

configuration in three files:
nvcc_args = [
'-gencode', 'arch=compute_30,code=sm_30',
'-gencode', 'arch=compute_35,code=sm_35',
'-gencode', 'arch=compute_37,code=sm_37',
'-gencode', 'arch=compute_52,code=sm_52',
'-gencode', 'arch=compute_60,code=sm_60',
'-gencode', 'arch=compute_61,code=sm_61',
'-gencode', 'arch=compute_70,code=sm_70',
' -gencode','arch=compute_75,code=sm_75'
'-gencode', 'arch=compute_70,code=compute_70'
]