check failed::error == cudasuccess(2 vs 0) out of memory
lcj820 opened this issue · comments
according to baidu or google :to set batchsize low,but the batch size of faster rcnn is 1,which cannot change, and if i chnange the size of input image,this question will happen:
valueerror:attemot to get argmax of an empty sequence.
do you have any advice to solve it?
thanks
I encountered the same problem. I found that the last info before error begin is always
I0130 09:06:13.363144 13129 net.cpp:98] Creating Layer conv5_1/dw
I0130 09:06:13.363148 13129 net.cpp:443] conv5_1/dw <- conv4_2/sep
I0130 09:06:13.363157 13129 net.cpp:398] conv5_1/dw -> conv5_1/dw
I0130 09:06:13.363165 13129 net.cpp:133] Setting up conv5_1/dw
The main reason is that it cost memory about 20G with batch size 256. The other way to fix it is to optimize memory in pw layer. Maybe you could use Sensenet instead of Caffe.