libomp/libiomp5 conflict macOS
FrederickGeek8 opened this issue · comments
Hello! I am currently getting the following abort in my Python code:
OMP: Error #15: Initializing libomp.a, but found libiomp5.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous,
since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP
runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe,
unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow
the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information,
please see http://openmp.llvm.org/
[1] 24366 abort python train.py
I believe this is probably caused by a conflict between torch and faiss-cpu since my code relies on both (as well as numpy but numpy is used by faiss). I'm trying to reproduce the issue outside of my code so I can post it here. I tried to compile a version myself by copying the build parameters used in the repository, however, I was unable to get the same performance so I gave up.
I am using faiss-cpu version 1.7.0 on macOS 11.2.3
I guess this seems to be a known issue with pytorch. Any suggestion from your end on how to get around this? pytorch/pytorch#44282
Yep, this seems a pytorch issue. Check if the official conda package avoids this issue.
Unfortunately conda isn't an option for me. Maybe in the future I'll change my replicated Python stack. Thank you for your help though, I'll close this issue now.