Perfetto build fails on ARM-based (M1) Macs
cphlipot1 opened this issue · comments
Perfetto build is currently failing on M1 based macs likely due to an incompatibility between the protoc used and Rosetta2
Reproducible on M1-pro based MBP running Mac OS 12.2.1 with the following commands:
$ git clone https://github.com/google/perfetto.git
$ cd perfetto
$ tools/install-build-deps
$ tools/build_all_configs
$ tools/ninja -C out/mac_debug
Build fails with the following:
ACTION //protos/perfetto/con...or(//gn/standalone/toolchain:gcc_like)
FAILED: gen/protos/perfetto/config/config.descriptor
python3 ../../gn/standalone/protoc.py ./protoc --proto_path ../../ --include_imports --descriptor_set_out gen/protos/perfetto/config/config.descriptor --dependency_out gen/protos/perfetto/config/config.descriptor.d ../../protos/perfetto/config/trace_config.proto
Traceback (most recent call last):
File "../../gn/standalone/protoc.py", line 60, in <module>
sys.exit(main())
File "../../gn/standalone/protoc.py", line 45, in main
subprocess.check_call(cmd)
File "/opt/homebrew/Cellar/python37/3.7.5_3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['./protoc', '--descriptor_set_out', 'gen/protos/perfetto/config/config.descriptor', '--dependency_out', '/var/folders/vy/cnh_8mzx0g935l4fwl2qhzvh0000gn/T/fa24eb39-0282-4310-8568-67dddf9cada5', '--proto_path', '../../', '--include_imports', '../../protos/perfetto/config/trace_config.proto']' died with <Signals.SIGILL: 4>.
if you attempt to run protoc under lldb you can see it faults on an AVX instruction (vmovss):
lldb out/mac_debug/protoc
(lldb) run
(lldb) disassemble
...
-> 0x100091264 <+84>: vmovss 0x560e54(%rip), %xmm0 ; xmm0 = mem[0],zero,zero,zero
...
AVX instructions are reported as unsupported by rosetta2 so i believe this is likely the cause of the failure: https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment
Rosetta translates all x86_64 instructions, but it doesn’t support the execution of some newer instruction sets and processor features, such as AVX, AVX2, and AVX512 vector instructions.
Which version did you try? I thought we fixed these issues with r.android.com/1972743
@LalitMaganti @chromy who have a M1
AVX instructions are reported as unsupported by rosetta2 so i believe this is likely the cause of the failure: https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment
So there are two problems there:
- We were using AVX and the detection code that prints a clear message had a bug. THis is not the key.
- We should be using the arm64 binaries on arm64, not using intel ones. We build (deliberately) the intel ones with -mavx, which I am aware are not compatible with rosetta.
Which version did you try?
The commands i posted above currently reproduce the issue for me. Those commands involve a fresh clone, which currently puts me on the master branch with the following commit as HEAD:
504b40994 (HEAD -> master, origin/master, origin/HEAD) Merge "Release contents of aosp/2010697 to stable"
Does this reproduce on the release build? (i.e. tools/ninja -C out/mac_release?)
Looked into this a bit further. I think the issue may be caused if install_build_deps uses an x86_64 version of python on an ARM-based mac to build.
I've noticed i'm getting a x86_64 gn binary in buildtools/mac/gn, and it looks like the script decides what to download based on the result from the platform.machine() Python API call.
% file `which python3`
/usr/local/bin/python3: Mach-O 64-bit executable x86_64
% python3
Python 3.7.5 (default, Oct 22 2019, 10:35:10)
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from platform import machine
>>> machine()
'x86_64'
I believe this is related to https://bugs.python.org/issue42704
Does this reproduce on the release build? (i.e. tools/ninja -C out/mac_release?)
Yes, if i run "tools/ninja -C out/mac_release" instead of "tools/ninja -C out/mac_debug" it also fails with the same error.
Ah the Python issue is definitely the problem. I don't think we can do anything on the Perfetto side to deal with this.
I don't think we can do anything on the Perfetto side to deal with this.
@LalitMaganti I think there are a few options here that i can think of that can be done on the Perfetto side:
- Explicitly require Python 3.9.1 (based on suggestion from https://bugs.python.org/issue42704), or at least call out other versions as unsupported in the build instructions: https://perfetto.dev/docs/contributing/build-instructions
- Add an argument to tools/install-build-deps to allow specifying of architecture
- Use A different method to determine the host architecture. eg. parsing platform version string:
% python3
Python 3.7.5 (default, Oct 22 2019, 10:35:10)
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.version()
'Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000'
Sent https://android-review.googlesource.com/c/platform/external/perfetto/+/2024525 to add this note to our docs.
TBH this feels like an edge case given that 3.9.1 was released <1 month after M1 was announced so unless you intentionally install an old version of Python, you will almost certainly be on a more recent version than 3.9.1.