Import Error on M1 Mac when using Docker Image
nimski opened this issue · comments
First, thank you for all your work and support on binder.
I'm trying to run binder on a C++ project on an M1 Mac. To make building easier, I'm using the provided Dockerfile (based on Ubuntu 20.04) to quickly build binder. I then run it on a mounted host volume to generate the bindings from C++ files on the host. This all works well. However, when trying to import the generated module I get the following error:
dlopen(xxx.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__Z11bind_cpp_ttRNSt3__18functionIFRN8pybind117module_ERKNS_12basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEEEEE'
This seems like an STL error. I'm wondering if this is because the binder built in the Docker image was built in an Ubuntu 20.04 environment (I'm assuming using libstdc++), and now I'm building its resulting bindings on an M1 Mac with apple clang (which I think uses libc++).
I've tried building the bindings on Mac with libstdc++ but it creates all sorts of errors and I believe is unsupported by Apple clang.
Is what I'm doing an unsupported use case? Building binder natively on Mac is not easy (especially as part of a CI build environment) so I'd love to be able to use the Docker image to do it instead.
Thanks!
@nimski how exactly are you trying to build Binder on M1 Mac? Particularly have you tried using https://github.com/RosettaCommons/binder/blob/master/build.py script? (this is what i do when i run Binder on M1). One known issue with build on M1 Mac's is to use LLVM-13 or older (build.py script does that). - Could you please try it and see if it works for you?
re error above: in general recommended approach is to generate bindings and build them on the target platforms to avoid std lib incompatibility issues.
Let me know how it goes,
Thanks @lyskov! I tried building with build.py and this is what I'm getting. It looks like a configuration failure. Is build.py supported on M1 mac with Apple clang?
Thank you for detailed log @nimski - i will try to replicate this locally. To answer your question: - yes, build on M1 Mac's with Apple Clang is supported. In meanwhile: could you please double check that you are getting the same error on clean binder clone? Thanks,
ok, my local build on clean Binder clone on M1 seems to work. @nimski could you please try building on clean clone and let me know if you are still seeing the error above? Thanks,
Thank you for the quick responses @lyskov! Building from source (on a clean clone) indeed solved the problem. Do you have a BuyMeACoffee account by any chance? I certainly owe you one!
It looks like this was not the end of the saga. Although building using build.py
works well, I'm getting errors about missing headers:
fatal error: 'functional' file not found
[cmake] #include <functional>
[cmake] ^~~~~~~~~~~~
[cmake] 1 error generated.
On the docs I see that the remedy for this is to point binder to the appropriate clang includes, but having tried a few different options I'm not sure what the correct include directory is. For example, if I try I/Library/Developer/CommandLineTools/usr/include/c++/v1
then I get a different error message:
fatal error: 'stdlib.h' file not found
[cmake] #include_next <stdlib.h>
Any pointers on how to fix this would be greatly appreciated!
I routinely compile on Mac so it is certainly possible but might require a few tries to make include work. I would recommend to try adding this to your binder
command line (at the end): binder <YOUR PROJECT ARGS> -- -x c++ -std=c++11 -isysroot `xcrun --show-sdk-path`
and see if it helps.
Thanks so much! That fixed the problem.
Great, glad to hear that!