System.AccessViolationException when trying to modify the interpreter to run on GPU with delegate
minhduc66532 opened this issue · comments
Any ideas ?
In the Emgu TF Lite 2.4.0 release, the GPUDelegateV2 is only implemented on Android. If you are running on platforms other than Android the native pointer of TfLiteInvoke.DefaultGpuDelegateV2 points to IntPtr.Zero, Calling ModifiedGraphWithDelegate with a Zero pointer will results in AccessViolationException.
We will update the code to make it more clear, and throw exception message reflecting that.
FYI, Tensorflow Lite GPUDelegate support is discussed here:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/delegates/gpu
TFLite GPU backend uses OpenGL ES 3.1 compute shaders or OpenCL.
It is not intended to be used on Desktop (e.g. Windows or Ubuntu). Desktop systems use full OpenGL implementation while mobile system such as Android and iOS use OpenGL ES:
https://stackoverflow.com/questions/4519264/opengl-es-versus-opengl
Oof so basically it's impossible to run tflite model using CUDA GPU (nvidia jetson nano in my case, running ubuntu 18.04) ? TBH, there ahs to be a way right. Tflite is design to run on low power device like the jetson nano, so can't be able to run it on the nano sound pretty dumb. Anyway, thanks for your answer
EDIT: Good new, after a bit of googling i found out that linux package for the nano do have OpenGL ES
So i think that i'm gonna try to run the model on the nano and report the result. If everything goes smoothly then i'm gonna close the issues, thank you
Neither the the Emgu TF Lite build script, nor the Tensorflow Lite build script enable GPUDelegateV2 (OpenGL-ES) on platform other than Android. You may need to work through the tflite Bazel build script to enable GPUDelegateV2 on Linux build. You are quite likely needed to add the OpenGL-ES library into tflite Bazel linux build tree. It is not supported by Tensorflow lite or Emgu TF Lite.
If you have a Jetson nano platform. Using the full version of Tensorflow with CUDA should give you more performance than using Tensorflow lite through GPUDelegateV2 (if you manage to get it to work).
Oof that suck, but thank for your answer anyway