Failed to compile on aarch64
hectorj2f opened this issue · comments
Hector Fernandez commented
Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
2.16.1
Custom code
No
OS platform and distribution
aarch64
Mobile device
No response
Python version
3.10
Bazel version
bazelisk using the default version requested by tensorflow
GCC/compiler version
clang 17 (I also tried with clang16).
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
When compiling the tensorflow source code on aarch64, I always get the following error:
WARN ERROR: /home/build/tensorflow/BUILD:1263:20: Linking tensorflow/libtensorflow.so.2.16.1 failed: (Exit 1): clang-17 failed: error executing command (from target //tensorflow:libtensorflow.so.2.16.1) /usr/bin/clang-17 @bazel-out/aarch64-opt/bin/tensorflow/libtensorflow.so.2.16.1-2.params
However the compilation works on x86_64.
export PYTHON_BIN_PATH=/usr/bin/python
export TF_PYTHON_VERSION=3.10
export USE_DEFAULT_PYTHON_LIB_PATH=1
export TF_NEED_JEMALLOC=1
export TF_NEED_KAFKA=1
export TF_NEED_OPENCL_SYCL=0
export TF_NEED_AWS=1
export TF_NEED_GCP=1
export TF_NEED_HDFS=1
export TF_NEED_S3=1
export TF_ENABLE_XLA=1
export TF_NEED_GDR=0
export TF_NEED_VERBS=0
export TF_NEED_OPENCL=0
export TF_NEED_MPI=0
export TF_NEED_TENSORRT=0
export TF_NEED_NGRAPH=0
export TF_NEED_IGNITE=0
export TF_NEED_ROCM=0
export TF_SYSTEM_LIBS="boringssl,curl,gif,icu,libjpeg_turbo,nasm,png,zlib"
export TF_SET_ANDROID_WORKSPACE=0
./configure
bazel --bazelrc=.tf_configure.bazelrc build \
--config=opt \
--config=mkl_threadpool \
//tensorflow:libtensorflow.so \
//tensorflow:libtensorflow_cc.so \
//tensorflow:install_headers \
//tensorflow/tools/pip_package:build_pip_package
Standalone code to reproduce the issue
export PYTHON_BIN_PATH=/usr/bin/python
export TF_PYTHON_VERSION=3.10
export USE_DEFAULT_PYTHON_LIB_PATH=1
export TF_NEED_JEMALLOC=1
export TF_NEED_KAFKA=1
export TF_NEED_OPENCL_SYCL=0
export TF_NEED_AWS=1
export TF_NEED_GCP=1
export TF_NEED_HDFS=1
export TF_NEED_S3=1
export TF_ENABLE_XLA=1
export TF_NEED_GDR=0
export TF_NEED_VERBS=0
export TF_NEED_OPENCL=0
export TF_NEED_MPI=0
export TF_NEED_TENSORRT=0
export TF_NEED_NGRAPH=0
export TF_NEED_IGNITE=0
export TF_NEED_ROCM=0
export TF_SYSTEM_LIBS="boringssl,curl,gif,icu,libjpeg_turbo,nasm,png,zlib"
export TF_SET_ANDROID_WORKSPACE=0
./configure
bazel --bazelrc=.tf_configure.bazelrc build \
--config=opt \
--config=mkl_threadpool \
//tensorflow:libtensorflow.so \
//tensorflow:libtensorflow_cc.so \
//tensorflow:install_headers \
//tensorflow/tools/pip_package:build_pip_package
Relevant log output
2024/05/09 13:03:51 WARN [14,447 / 14,578] Compiling tensorflow/compiler/jit/xla_platform_info.cc; 20s local ... (16 actions running)
2024/05/09 13:03:52 WARN [14,448 / 14,578] Compiling tensorflow/compiler/jit/xla_platform_info.cc; 21s local ... (16 actions running)
2024/05/09 13:03:55 WARN [14,449 / 14,578] Compiling tensorflow/compiler/jit/xla_platform_info.cc; 24s local ... (16 actions, 15 running)
2024/05/09 13:03:57 WARN [14,450 / 14,578] Compiling tensorflow/compiler/jit/xla_platform_info.cc; 26s local ... (16 actions running)
2024/05/09 13:03:58 WARN [14,452 / 14,578] Compiling tensorflow/compiler/jit/get_compiler_ir.cc; 26s local ... (16 actions running)
2024/05/09 13:04:00 WARN [14,454 / 14,578] Compiling tensorflow/compiler/jit/kernels/xla_ops.cc; 27s local ... (16 actions running)
2024/05/09 13:04:01 WARN [14,457 / 14,578] Compiling tensorflow/compiler/jit/xla_cpu_device.cc; 22s local ... (16 actions running)
2024/05/09 13:04:03 WARN [14,459 / 14,578] Compiling tensorflow/compiler/jit/xla_cpu_device.cc; 24s local ... (16 actions running)
2024/05/09 13:04:04 WARN [14,460 / 14,578] Compiling tensorflow/compiler/jit/xla_cpu_device.cc; 25s local ... (16 actions, 15 running)
2024/05/09 13:04:06 WARN [14,461 / 14,578] Compiling tensorflow/compiler/jit/xla_cpu_device.cc; 27s local ... (16 actions running)
2024/05/09 13:04:07 WARN [14,464 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 22s local ... (16 actions running)
2024/05/09 13:04:11 WARN [14,465 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 26s local ... (16 actions, 15 running)
2024/05/09 13:04:12 WARN [14,467 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 27s local ... (16 actions, 15 running)
2024/05/09 13:04:13 WARN [14,468 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 28s local ... (16 actions running)
2024/05/09 13:04:14 WARN [14,470 / 14,578] Compiling tensorflow/compiler/tf2xla/mlir_tf2xla.cc; 29s local ... (16 actions, 15 running)
2024/05/09 13:04:16 WARN [14,473 / 14,578] Compiling tensorflow/compiler/aot/codegen.cc; 19s local ... (16 actions, 15 running)
2024/05/09 13:04:17 WARN [14,476 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 16s local ... (16 actions running)
2024/05/09 13:04:18 WARN [14,478 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 17s local ... (16 actions, 15 running)
2024/05/09 13:04:20 WARN [14,479 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 19s local ... (16 actions, 15 running)
2024/05/09 13:04:21 WARN [14,481 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 20s local ... (16 actions, 14 running)
2024/05/09 13:04:22 WARN [14,482 / 14,578] Compiling tensorflow/core/common_runtime/pluggable_device/pluggable_device_plugin_init.cc; 21s local ... (16 actions, 15 running)
2024/05/09 13:04:23 WARN [14,486 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 21s local ... (16 actions running)
2024/05/09 13:04:25 WARN [14,487 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 23s local ... (16 actions running)
2024/05/09 13:04:26 WARN [14,489 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 24s local ... (16 actions running)
2024/05/09 13:04:28 WARN [14,491 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 25s local ... (16 actions, 15 running)
2024/05/09 13:04:29 WARN [14,492 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 27s local ... (16 actions, 15 running)
2024/05/09 13:04:30 WARN [14,494 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 28s local ... (16 actions, 15 running)
2024/05/09 13:04:31 WARN [14,497 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 29s local ... (14 actions running)
2024/05/09 13:04:32 WARN [14,498 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 30s local ... (13 actions running)
2024/05/09 13:04:34 WARN [14,500 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 32s local ... (11 actions running)
2024/05/09 13:04:36 WARN [14,505 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 34s local ... (6 actions running)
2024/05/09 13:04:38 WARN [14,506 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 36s local ... (5 actions running)
2024/05/09 13:04:40 WARN [14,508 / 14,578] Compiling tensorflow/compiler/mlir/python/mlir.cc; 38s local ... (3 actions running)
2024/05/09 13:04:40 WARN ERROR: /home/build/tensorflow/BUILD:1263:20: Linking tensorflow/libtensorflow.so.2.16.1 failed: (Exit 1): clang-17 failed: error executing command (from target //tensorflow:libtensorflow.so.2.16.1) /usr/bin/clang-17 @bazel-out/aarch64-opt/bin/tensorflow/libtensorflow.so.2.16.1-2.params
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN /usr/bin/../lib64/gcc/aarch64-unknown-linux-gnu/13.2.0/../../../../aarch64-unknown-linux-gnu/bin/ld.gold: error: Stub is too far away, try a smaller value for '--stub-group-size'. The current value is 0x7ffbffc.
2024/05/09 13:04:40 WARN clang-17: error: linker command failed with exit code 1 (use -v to see invocation)
2024/05/09 13:04:41 WARN INFO: Elapsed time: 4130.234s, Critical Path: 216.57s
2024/05/09 13:04:41 WARN INFO: 14511 processes: 1423 internal, 13088 local.
2024/05/09 13:04:41 WARN FAILED: Build did NOT complete successfully
tilakrayal commented
@hectorj2f,
Thanks for reporting the issue. Looks like this is a known issue while building the TensorFlow v2.16 with aarch64.
Could you please try to refer to the issue for the updates on the same.
Thank you!