Apple M1 building x86 wheels but labelling them as arm
thoward27 opened this issue · comments
Hey there!
I have been trying to put my finger on why building wheels on my machine sometimes produces x86 shared object files when my system is arm64. I think there may be an issue in this library, but I am having a hard time proving it.
If anyone can help guide me here it would be greatly appreciated.
The journey started here: python-poetry/poetry#8786. After debugging poetry for a while I realized that the issue was actually that the build step was producing x86 wheels on my system, but calling them arm wheels. From there I was able to discover that in certain situations, I get this output when building python packages
ld: warning: ignoring file /opt/homebrew/lib/libcrypto.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /opt/homebrew/lib/postgresql@14/libpq.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
ld: warning: ignoring file /opt/homebrew/lib/libssl.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
When this happens, the final bundle command from clang looks like this
clang -bundle -undefined dynamic_lookup -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/thomas/.pyenv/versions/3.11.3/lib -Wl,-rpath,/Users/thomas/.pyenv/versions/3.11.3/lib -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/thomas/.pyenv/versions/3.11.3/lib -Wl,-rpath,/Users/thomas/.pyenv/versions/3.11.3/lib -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_asis.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_binary.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_datetime.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_list.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_pboolean.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_pdecimal.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_pfloat.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_pint.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_qstring.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/aix_support.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/bytes_format.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/column_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/connection_int.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/connection_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/conninfo_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/cursor_int.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/cursor_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/diagnostics_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/error_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/green.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/libpq_support.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/lobject_int.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/lobject_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/microprotocols.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/microprotocols_proto.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/notify_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/pqpath.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/psycopgmodule.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/replication_connection_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/replication_cursor_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/replication_message_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/solaris_support.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/typecast.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/utils.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/win32_support.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/xid_type.o -L/Users/thomas/.pyenv/versions/3.11.3/lib -L/opt/homebrew/lib/postgresql@14 -L/opt/homebrew/opt/openssl@3/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/Cellar/lz4/1.9.4/lib -lpq -lssl -lcrypto -o build/lib.macosx-12.4-arm64-cpython-311/psycopg2/_psycopg.cpython-311-darwin.so
However, in other situations, I get
clang -bundle -undefined dynamic_lookup -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/thomas/.pyenv/versions/3.11.3/lib -Wl,-rpath,/Users/thomas/.pyenv/versions/3.11.3/lib -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/thomas/.pyenv/versions/3.11.3/lib -Wl,-rpath,/Users/thomas/.pyenv/versions/3.11.3/lib -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_asis.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_binary.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_datetime.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_list.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_pboolean.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_pdecimal.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_pfloat.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_pint.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/adapter_qstring.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/aix_support.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/bytes_format.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/column_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/connection_int.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/connection_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/conninfo_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/cursor_int.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/cursor_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/diagnostics_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/error_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/green.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/libpq_support.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/lobject_int.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/lobject_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/microprotocols.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/microprotocols_proto.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/notify_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/pqpath.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/psycopgmodule.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/replication_connection_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/replication_cursor_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/replication_message_type.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/solaris_support.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/typecast.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/utils.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/win32_support.o build/temp.macosx-12.4-arm64-cpython-311/psycopg/xid_type.o -L/Users/thomas/.pyenv/versions/3.11.3/lib -L/opt/homebrew/lib/postgresql@14 -L/opt/homebrew/opt/openssl@3/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/Cellar/lz4/1.9.4/lib -lpq -lssl -lcrypto -o build/lib.macosx-12.4-arm64-cpython-311/psycopg2/_psycopg.cpython-311-darwin.so
installing to build/bdist.macosx-12.4-arm64/wheel
I don't have much experience compiling python packages, so I am hoping someone here a bit more knowledgeable than myself may be able to help me pin this issue down.
Edit 1:
Upon further investigation, I found that clang is targeting the wrong architecture, but I've no idea why there is a difference:
$ clang --version
Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: arm64-apple-darwin21.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
...
$ clang --version
Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: x86_64-apple-darwin21.5.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Okay, I think I found the root cause for my issue, it appears that inspec is running in i386 mode on my machine for some reason. Going to create an issue with that team and hopefully get this sorted out.