GPU usage?
smeylan opened this issue · comments
The documentation mentions deepspeech-gpu in the installation process but deepspeech-server appears not to use the GPU, which I need for higher-speed inference. I know it isn't a more general problem (misconfiguration of Nvidia drivers, Tensorflow, etc.) because the binary with arch = gpu works; rather it seems that the Model Python class doesn't use the GPU (this also seems the case with the native_client/python/client.py code, which uses that same class).
I did not have the opportunity to test on a GPU yet. I will have a GPU system within one or two weeks to test this. However I though that using the deepspeech package with GPU support would be enough. When using deepspeech-gpu, the bindings should be routed to the GPU enabled implementation.
It works for me with this build of deepspeech (tf1.5 with cuda9):
https://index.taskcluster.net/v1/task/project.deepspeech.deepspeech.native_client.tf-master.gpu/artifacts/public/deepspeech_gpu-0.1.1-cp36-cp36m-manylinux1_x86_64.whl
I see the GPU being initialized when tensorflow is starting, and inference is much faster. I will close this issue unless you have more information.