No kernel image is available for execution on the device

[NenoSun]

Hi, everyone.

I kept having this error. I've spent two days googling this problem but still can't solve it.

(test) [root@localhost Faster-RCNN_TF]# python tools/demo.py --model ./output/faster_rcnn_end2end/voc_2007_trainval/VGGnet_fast_rcnn_iter_10000.ckpt
2019-01-21 : I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-01-21 : I tensorflow/core/common_runtime/gpu/gpu_device.cc:1105] Found device 0 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:08:00.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
2019-01-21 : I tensorflow/core/common_runtime/gpu/gpu_device.cc:1105] Found device 1 with properties:
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:09:00.0
totalMemory: 11.17GiB freeMemory: 11.10GiB
2019-01-21 : I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Device peer to peer matrix
2019-01-21 : I tensorflow/core/common_runtime/gpu/gpu_device.cc:1126] DMA: 0 1
2019-01-21 : I tensorflow/core/common_runtime/gpu/gpu_device.cc:1136] 0: Y Y
2019-01-21 : I tensorflow/core/common_runtime/gpu/gpu_device.cc:1136] 1: Y Y
2019-01-21 : I tensorflow/core/common_runtime/gpu/gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: Tesla K80, pci bus id: 0000:08:00.0, compute capability: 3.7)
2019-01-21 : I tensorflow/core/common_runtime/gpu/gpu_device.cc:1195] Creating TensorFlow device (/device:GPU:1) -> (device: 1, name: Tesla K80, pci bus id: 0000:09:00.0, compute capability: 3.7)
VGGnet_test
Loaded network ./output/faster_rcnn_end2end/voc_2007_trainval/VGGnet_fast_rcnn_iter_10000.ckpt
cudaCheckError() failed : no kernel image is available for execution on the device

Anyone met this error before? I am using tensorflow 1.5.0 and my CUDA version is 9.0

[NenoSun]

I just solved it. Days ago, I modified the make.sh file in the ./lib folder. It sets the nvcc compiler to compile with sm_53 architecture which is incompatible with my Telsa K80. Telsa K80 has 3.7 compute ability and so I changed the nvcc architecture from sm_53 to sm_35 and now everything works fine.