Warp Clang/LLVM build error when python build_lib.py
Leon-LXA opened this issue · comments
I am building warp.so by "python build_lib.py --cuda_path="/usr/local/cuda-12" ", however it has problem:
Namespace(build_llvm=False, cuda_path='/usr/local/cuda-12/', debug_llvm=False, fast_math=False, mode='release', msvc_path=None, quick=False, sdk_path=None, standalone=True, verbose=True, verify_fp=False)
Finished writing /home/leon/origin/warp/warp/native/exports.h
Building /home/leon/origin/warp/warp/bin/warp.so
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/warp.cpp" -o "/home/leon/origin/warp/warp/native/warp.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/crt.cpp" -o "/home/leon/origin/warp/warp/native/crt.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/cuda_util.cpp" -o "/home/leon/origin/warp/warp/native/cuda_util.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/mesh.cpp" -o "/home/leon/origin/warp/warp/native/mesh.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/hashgrid.cpp" -o "/home/leon/origin/warp/warp/native/hashgrid.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/reduce.cpp" -o "/home/leon/origin/warp/warp/native/reduce.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/runlength_encode.cpp" -o "/home/leon/origin/warp/warp/native/runlength_encode.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/sort.cpp" -o "/home/leon/origin/warp/warp/native/sort.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/sparse.cpp" -o "/home/leon/origin/warp/warp/native/sparse.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/volume.cpp" -o "/home/leon/origin/warp/warp/native/volume.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/marching.cpp" -o "/home/leon/origin/warp/warp/native/marching.cpp.o"
g++ -O3 -DNDEBUG -DWP_ENABLE_DEBUG=0 -DWP_ENABLE_CUDA=1 -DWP_ENABLE_CUTLASS=1 -DWP_ENABLE_CUDA_COMPATIBILITY=1 -fPIC -fvisibility=hidden --std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -I"/home/leon/origin/warp/warp/native" -I"/home/leon/origin/warp/external/llvm-project/out/install/release-x86_64/include" -I"/home/leon/origin/warp/_build/host-deps/llvm-project/release-x86_64/include" -I"/usr/local/cuda-12//include" -c "/home/leon/origin/warp/warp/native/cutlass_gemm.cpp" -o "/home/leon/origin/warp/warp/native/cutlass_gemm.cpp.o"
build took 10073.37 ms
"/usr/local/cuda-12//bin/nvcc" -O3 --compiler-options -fPIC,-fvisibility=hidden -gencode=arch=compute_52,code=sm_52 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_80,code=sm_80 -gencode=arch=compute_86,code=sm_86 -gencode=arch=compute_89,code=sm_89 -gencode=arch=compute_90,code=sm_90 -gencode=arch=compute_90,code=compute_90 -t0 --extended-lambda -DNDEBUG -DWP_ENABLE_CUDA=1 -I"/home/leon/origin/warp/warp/native" -DWP_ENABLE_CUTLASS=1 -I"warp/native/cutlass/include" -I"warp/native/cutlass/tools/util/include" -o "/home/leon/origin/warp/warp/native/warp.cu.o" -c "/home/leon/origin/warp/warp/native/warp.cu"
/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used
const void* src_grad =
^
Remark: The warnings can be suppressed with "-diag-suppress "
/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used
void* dst_grad =
^
/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used
const void* src_grad =
^
Remark: The warnings can be suppressed with "-diag-suppress "
/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used
void* dst_grad =
^
/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used
const void* src_grad =
^
Remark: The warnings can be suppressed with "-diag-suppress "
/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used
void* dst_grad =
^
/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used
const void* src_grad =
^
Remark: The warnings can be suppressed with "-diag-suppress "
/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used
void* dst_grad =
^
/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used
const void* src_grad =
^
Remark: The warnings can be suppressed with "-diag-suppress "
/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used
void* dst_grad =
^
/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used
const void* src_grad =
^
Remark: The warnings can be suppressed with "-diag-suppress "
/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used
void* dst_grad =
^
/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used
const void* src_grad =
^
Remark: The warnings can be suppressed with "-diag-suppress "
/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used
void* dst_grad =
^
/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used
const void* src_grad =
^
Remark: The warnings can be suppressed with "-diag-suppress "
/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used
void* dst_grad =
^
/home/leon/origin/warp/warp/native/warp.cu(612): warning #550-D: variable "src_grad" was set but never used
const void* src_grad =
^
Remark: The warnings can be suppressed with "-diag-suppress "
/home/leon/origin/warp/warp/native/warp.cu(614): warning #550-D: variable "dst_grad" was set but never used
void* dst_grad =
^
build_cuda took 92215.20 ms
g++ -shared -Wl,-rpath,'$ORIGIN' -Wl,--no-undefined -Wl,--exclude-libs,ALL -o '/home/leon/origin/warp/warp/bin/warp.so' "/home/leon/origin/warp/warp/native/warp.cpp.o" "/home/leon/origin/warp/warp/native/crt.cpp.o" "/home/leon/origin/warp/warp/native/cuda_util.cpp.o" "/home/leon/origin/warp/warp/native/mesh.cpp.o" "/home/leon/origin/warp/warp/native/hashgrid.cpp.o" "/home/leon/origin/warp/warp/native/reduce.cpp.o" "/home/leon/origin/warp/warp/native/runlength_encode.cpp.o" "/home/leon/origin/warp/warp/native/sort.cpp.o" "/home/leon/origin/warp/warp/native/sparse.cpp.o" "/home/leon/origin/warp/warp/native/volume.cpp.o" "/home/leon/origin/warp/warp/native/marching.cpp.o" "/home/leon/origin/warp/warp/native/cutlass_gemm.cpp.o" "/home/leon/origin/warp/warp/native/warp.cu.o" -L"/usr/local/cuda-12//lib64" -lcudart_static -lnvrtc_static -lnvrtc-builtins_static -lnvptxcompiler_static -lpthread -ldl -lrt
strip --strip-all --keep-symbol=__jit_debug_register_code --keep-symbol=__jit_debug_descriptor /home/leon/origin/warp/warp/bin/warp.so
link took 1623.12 ms
Fetching packman-common@7.5.zip from bootstrap.packman.nvidia.com ...
Unpacking ...
Warp Clang/LLVM build error: run build.bat / build.sh
Looking at build_llvm.py, it seems like that message got printed because _build/host-deps/llvm-project wasn't successfully created. What happens if you run
./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose
@c0d1f1ed : We should change that message since build.sh is no longer there, right?
Looking at
build_llvm.py, it seems like that message got printed because_build/host-deps/llvm-projectwasn't successfully created. What happens if you run./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose@c0d1f1ed : We should change that message since
build.shis no longer there, right?
still the same error. And I do not find any path as "_build/host-deps/llvm-project"
Looking at
build_llvm.py, it seems like that message got printed because_build/host-deps/llvm-projectwasn't successfully created. What happens if you run./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose@c0d1f1ed : We should change that message since
build.shis no longer there, right?still the same error. And I do not find any path as "_build/host-deps/llvm-project"
Surely you get a different error when you just run the packman command directly, no?
Looking at
build_llvm.py, it seems like that message got printed because_build/host-deps/llvm-projectwasn't successfully created. What happens if you run./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose@c0d1f1ed : We should change that message since
build.shis no longer there, right?still the same error. And I do not find any path as "_build/host-deps/llvm-project"
Surely you get a different error when you just run the
packmancommand directly, no?
./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose
Fetching packman-common@7.5.zip from bootstrap.packman.nvidia.com ...
Unpacking ...
Looking at
build_llvm.py, it seems like that message got printed because_build/host-deps/llvm-projectwasn't successfully created. What happens if you run./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 --verbose@c0d1f1ed : We should change that message since
build.shis no longer there, right?still the same error. And I do not find any path as "_build/host-deps/llvm-project"
Surely you get a different error when you just run the
packmancommand directly, no?
My two computers act differently. This one just unpacking and finished. And the error's the same when i python build_lib.py. The other one unpacking and does many things, then no error when python build_lib.py
Thanks! It's hard to say exactly what's wrong with your setup but here's some thoughts. Packman needs to install "itself" somewhere on your system, and the environment variable PM_PACKAGES_ROOT controls the location. By default, it picks /home/eshi/.cache/packman/ on my system, and in that I have the directories packman-common, python, and chk. You can try setting PM_PACKAGES_ROOT to a different directory that you can write to...
Alternatively, you can also try to manually run what the install process is trying to do... Download http://bootstrap.packman.nvidia.com/packman-common@7.5.zip onto your system, then run python3 tools/packman/bootstrap/install_package.py packman-common@7.5.zip $PM_PACKAGES_ROOT/packman-common/7.5. Then you can try running ./tools/packman/packman install -l ./_build/host-deps/llvm-project/release-x86_64 clang+llvm-warp 15.0.7-linux-x86_64-ptx-gcc7.5-cxx11abi0 again.
I'm going to close this for now, but please reopen it or file a new bug if this issue still reproduces. The code that used to potentially produce this error has changed a fair bit in recent months so we'd need updated information anyway.
If you do run into this still, it might be worth trying to clone the Warp repo into a new directory and build from there. Interrupted downloads or builds could cause issues we currently don't detect and report properly.