theano gpu not working

[Alireza]

Hi,
I've followed instructions of how to run theano gpu using the deepo; unfortuantely i'm not able to run the theano code with gpu. it uses the cpu instead

steps that I took
docker run --gpus all -it ufoym/deepo:theano bash

and I'm running the following test code (from theano documentations)

# Code to test if theano is using GPU or CPU
# Reference: https://stackoverflow.com/questions/34328467/how-can-i-use-my-gpu-on-ipython-notebook

import os
os.environ["MKL_THREADING_LAYER"] = "GNU"
from theano import function, config, shared, sandbox
import theano.tensor as T
import numpy
import time
vlen = 10 * 30 * 768  # 10 x #cores x # threads per core
iters = 1000
rng = numpy.random.RandomState(22)
x = shared(numpy.asarray(rng.rand(vlen), config.floatX))
f = function([], T.exp(x))
print(f.maker.fgraph.toposort())
t0 = time.time()
# if you're using Python3, rename `xrange` to `range` in the following line
for i in range(iters):
    r = f()
t1 = time.time()
print("Looping %d times took %f seconds" % (iters, t1 - t0))
print("Result is %s" % (r,))
if numpy.any([isinstance(x.op, T.Elemwise) for x in f.maker.fgraph.toposort()]):
    print('Used the cpu')
else:
    print('Used the gpu')

I'm getting the following output which says it have used cpu instead of gpu:

/usr/local/lib/python3.6/dist-packages/theano/gpuarray/dnn.py:184: UserWarning: Your cuDNN version is more recent than Theano. If you encounter problems, try updating Theano or downgrading cuDNN to a version >= v5 and <= v7.
  warnings.warn("Your cuDNN version is more recent than "
Using cuDNN version 7605 on context None
Mapped name None to device cuda0: TITAN RTX (0000:65:00.0)
[GpuElemwise{exp,no_inplace}(<GpuArrayType<None>(float32, vector)>), HostFromGpu(gpuarray)(GpuElemwise{exp,no_inplace}.0)]
Looping 1000 times took 0.180456 seconds
Result is [1.2317803 1.6187935 1.5227807 ... 2.2077181 2.2996776 1.623233 ]
Used the cpu

here is my nvidia-smi inside my docker

Tue May  5 19:41:17 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.82       Driver Version: 440.82       CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  TITAN X (Pascal)    Off  | 00000000:17:00.0 Off |                  N/A |
| 23%   28C    P8     8W / 250W |      2MiB / 12196MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  TITAN RTX           Off  | 00000000:65:00.0  On |                  N/A |
| 41%   33C    P8     1W / 280W |    611MiB / 24217MiB |      4%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

any help is appreciated

[Ming]

Sorry for the late reply.

By default, there exists a file /root/.theanorc inside a deepo image with theano. And it should be OK to use GPU in this case, though theano is deprecated now. If you simply import theano in the latest deepo image, there should be a warning like this:

/usr/local/lib/python3.6/dist-packages/theano/gpuarray/dnn.py:184: UserWarning: Your cuDNN version is more recent than Theano. If you encounter problems, try updating Theano or downgrading cuDNN to a version >= v5 and <= v7.
  warnings.warn("Your cuDNN version is more recent than "
Using cuDNN version 8005 on context None
Mapped name None to device cuda0: NVIDIA GeForce RTX 2070 (0000:01:00.0)

We think the problem in your case is probably the lack of /root/.theanorc, especially when a directory from your host is mounted into the /root and therefore /root/.theanorc does not exist any more.

You can, of course, create a /root/.theanorc again with the following settings:

[global]
device = cuda
floatX=float32

or use theano in this way

THEANO_FLAGS='device=cuda,floatX=float32' python -c "import theano"