remicres / otbtf

Hello Remi,

In the docker list from here, it was mentioned that mdl4eo/otbtf3.1:gpu is built with no optimization and has dev files so I tried to use that. I tried to use the latest otbtf3.1 GPU on my WSL2 Docker. It gives me an error as:

The TensorFlow library was compiled to use AVX512F instructions, but these aren't available on your machine.

I also tried mdl4eo/otbtf3.1:gpu-dev it also gives the same error but I assume it has been built with optimization parameters. The only docker image that worked was mdl4eo/otbtf3.1:gpu-basic but sadly it doesn't have dev files 😢

So I did try to build otbtf 3.1 gpu docker image with dev files without optimization, it works the only drawback is it took a lot of time (~6 hrs) 🥲

Is it possible that we can have a GPU docker image with dev files with no optimization?

Hi @Pratyush1991 ,

Thanks.

Oops... it looks like there is a typo in the doc. Yes indeed, the gpu and gpu-dev are built with CPU optimizations flags. You are right, a gpu-basic-dev image could be useful, and we will add it soon on dockerhub.

Soon we will also have a better CI (the docker images should be published for each new release... for now it is done manually hence the multiple errors 😢 )

Hi @remicres , the problem I see here, we already talked about it : a lot of CPUs are SSE / AVX2, but not AVX512.
Actually, the choice right now is either no opt at all, or the AVX512 one which is not compatible with a lot of processors.
It would be nice to provide optimizations that are compatible with most CPUs, and eventually AVX512 could be yet another image ?

otbtf/tools/docker/build-env-tf.sh

Line 5 in ac52b65

    
           #export CC_OPT_FLAGS="-Wno-sign-compare --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2"

Hi Remi,

I have tested the mdl4eo/otbtf3.1:gpu-basic-dev docker image and this works correctly on my WSL2 Cuda docker. Thanks a lot.

OTBTF GPU Dev without optimization (AVX512F) for WSL2 Cuda