NVlabs / instant-ngp

Instant neural graphics primitives: lightning fast NeRF and more

Home Page:https://nvlabs.github.io/instant-ngp

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Compilation error, underfined reference to cublasSetWorkspace_v2 and cudaGraphExecUpdate

chriskeraly-rios opened this issue · comments

Awesome work, I'm really looking forward to demoing this.

However I am currently running into build errors that I have not been able to troubleshoot myself, I have followed the instructions on your README.md. I am running Ubuntu 20 on a laptop with an NVIDIA GeForce RTX 2070 card and cuda 11.3.

Here are the logs of the build commands

`user@computer:~/software/instant-ngp$ cmake . -B build -DCMAKE_CUDA_COMPILER=/usr/local/cuda-11.3/bin/nvcc
-- The CUDA compiler identification is NVIDIA 11.3.109
-- Check for working CUDA compiler: /usr/local/cuda-11.3/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda-11.3/bin/nvcc - works
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- No release type specified. Setting to 'Release'.
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Using X11 for window creation
-- Found X11: /usr/include
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so
-- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so - found
-- Looking for gethostbyname
-- Looking for gethostbyname - found
-- Looking for connect
-- Looking for connect - found
-- Looking for remove
-- Looking for remove - found
-- Looking for shmat
-- Looking for shmat - found
-- Looking for IceConnectionNumber in ICE
-- Looking for IceConnectionNumber in ICE - found
-- Found GLEW: /usr/include (found version "2.1.0")
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- !!! Warning OptiX_INSTALL_DIR not set in environment. using default
-- OptiX_INSTALL_DIR value: /usr/local/NVIDIA-OptiX-SDK-7.3.0-linux64-x86_64
-- OptiX headers (optix.h and friends) not found.
CMake Warning at CMakeLists.txt:169 (message):
OptiX was not found. Neural graphics primitives will still compile and run
correctly. However, SDF training in 'raystab' and 'pathescape' modes will
be significantly slower.

-- Found Python: /usr/bin/python3.8 (found suitable version "3.8.10", minimum required is "3.7") found components: Interpreter Development Development.Module Development.Embed
-- pybind11 v2.7.1
CMake Warning (dev) at /home/user/software/cmake-3.22.1-linux-x86_64/share/cmake-3.22/Modules/CMakeDependentOption.cmake:84 (message):
Policy CMP0127 is not set: cmake_dependent_option() supports full Condition
Syntax. Run "cmake --help-policy CMP0127" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
Call Stack (most recent call first):
dependencies/pybind11/CMakeLists.txt:98 (cmake_dependent_option)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Targeting GPU architectures: 75
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/software/instant-ngp/build
user@computer:~/software/instant-ngp$ cmake --build build --config RelWithDebInfo -j 16
[ 1%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/common.cu.o
[ 3%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/common_device.cu.o
[ 5%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/context.c.o
[ 9%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/cutlass_resnet.cu.o
[ 9%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/init.c.o
[ 12%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/monitor.c.o
[ 12%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/input.c.o
[ 14%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/cutlass_mlp.cu.o
[ 16%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/vulkan.c.o
[ 18%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/window.c.o
[ 20%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/encoding.cu.o
[ 21%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/x11_init.c.o
[ 23%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/loss.cu.o
[ 25%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/network.cu.o
[ 27%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/object.cu.o
[ 29%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/optimizer.cu.o
[ 30%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/x11_monitor.c.o
[ 32%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/reduce_sum.cu.o
[ 34%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/x11_window.c.o
[ 36%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/xkb_unicode.c.o
[ 38%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/posix_time.c.o
[ 40%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/posix_thread.c.o
[ 41%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/glx_context.c.o
[ 43%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/egl_context.c.o
[ 45%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/osmesa_context.c.o
[ 47%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/linux_joystick.c.o
[ 49%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/fully_fused_mlp.cu.o
[ 49%] Built target glfw_objects
[ 50%] Linking CUDA static library libtiny-cuda-nn.a
[ 50%] Built target tiny-cuda-nn
[ 52%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/ImGuizmo.cpp.o
[ 54%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui.cpp.o
[ 56%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_impl_glfw.cpp.o
[ 58%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_impl_opengl3.cpp.o
[ 60%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_draw.cpp.o
[ 61%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_tables.cpp.o
[ 63%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_widgets.cpp.o
[ 65%] Building CUDA object CMakeFiles/ngp.dir/src/camera_path.cu.o
[ 67%] Building CUDA object CMakeFiles/ngp.dir/src/common_device.cu.o
[ 69%] Building CUDA object CMakeFiles/ngp.dir/src/marching_cubes.cu.o
[ 70%] Building CUDA object CMakeFiles/ngp.dir/src/nerf_loader.cu.o
[ 72%] Building CUDA object CMakeFiles/ngp.dir/src/render_buffer.cu.o
[ 74%] Building CUDA object CMakeFiles/ngp.dir/src/testbed.cu.o
[ 76%] Building CUDA object CMakeFiles/ngp.dir/src/testbed_image.cu.o
[ 78%] Building CUDA object CMakeFiles/ngp.dir/src/testbed_nerf.cu.o
[ 80%] Building CUDA object CMakeFiles/ngp.dir/src/testbed_sdf.cu.o
[ 81%] Building CUDA object CMakeFiles/ngp.dir/src/testbed_volume.cu.o
[ 83%] Building CXX object CMakeFiles/ngp.dir/src/thread_pool.cpp.o
[ 85%] Building CUDA object CMakeFiles/ngp.dir/src/tinyexr_wrapper.cu.o
[ 87%] Building CXX object CMakeFiles/ngp.dir/src/tinyobj_loader_wrapper.cpp.o
[ 89%] Building CUDA object CMakeFiles/ngp.dir/src/triangle_bvh.cu.o
[ 90%] Linking CUDA device code CMakeFiles/ngp.dir/cmake_device_link.o
[ 92%] Linking CXX static library libngp.a
[ 92%] Built target ngp
[ 96%] Building CUDA object CMakeFiles/pyngp.dir/src/python_api.cu.o
[ 96%] Building CUDA object CMakeFiles/testbed.dir/src/main.cu.o
[ 98%] Linking CXX executable testbed
/usr/bin/ld: libngp.a(testbed_image.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::Trainer<float, __half, __half>::training_step(CUstream_st*, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const&, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const&, float*, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const*)::{lambda()#1}>(CUstream_st*, bool, tcnn::Trainer<float, __half, __half>::training_step(CUstream_st*, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const&, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const&, float*, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const*)::{lambda()#1})': tmpxft_00005ad8_00000000-6_testbed_image.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_7TrainerIf6__halfS3_E13training_stepEP11CUstream_stRKNS_9GPUMatrixIfLNS_12MatrixLayoutE1EEESB_PfPSA_EUlvE_EEvS6_bT_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_7TrainerIf6__halfS3_E13training_stepEP11CUstream_stRKNS_9GPUMatrixIfLNS_12MatrixLayoutE1EEESB_PfPSA_EUlvE_EEvS6_bT_]+0x406): undefined reference to cudaGraphExecUpdate'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function tcnn::ShampooOptimizer<float>::step(CUstream_st*, float, float*, float*, float const*)::{lambda()#1}::operator()() const': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv]+0xacf): undefined reference to cublasSetWorkspace_v2'
/usr/bin/ld: tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv]+0xebb): undefined reference to cublasSetWorkspace_v2' /usr/bin/ld: tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv]+0x12c1): undefined reference to cublasSetWorkspace_v2'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function tcnn::ShampooOptimizer<__half>::step(CUstream_st*, float, float*, __half*, __half const*)::{lambda()#1}::operator()() const': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv]+0xaec): undefined reference to cublasSetWorkspace_v2'
/usr/bin/ld: tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv]+0xed8): undefined reference to cublasSetWorkspace_v2' /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o):tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv]+0x12de): more undefined references to cublasSetWorkspace_v2' follow
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<float>::step(CUstream_st*, float, float*, float*, float const*)::{lambda()#1}>(CUstream_st*, bool, tcnn::ShampooOptimizer<float>::step(CUstream_st*, float, float*, float*, float const*)::{lambda()#1})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE4stepEP11CUstream_stfPfS6_PKfEUlvE_EEvS5_bT_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE4stepEP11CUstream_stfPfS6_PKfEUlvE_EEvS5_bT_]+0x406): undefined reference to cudaGraphExecUpdate'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<float>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#1}>(CUstream_st*, bool, tcnn::ShampooOptimizer<float>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#1})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE_EEvS6_bS9_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE_EEvS6_bS9_]+0x406): undefined reference to cudaGraphExecUpdate'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<float>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#2}>(CUstream_st*, bool, tcnn::ShampooOptimizer<float>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#2})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE0_EEvS6_bS9_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE0_EEvS6_bS9_]+0x406): undefined reference to cudaGraphExecUpdate'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<__half>::step(CUstream_st*, float, float*, __half*, __half const*)::{lambda()#1}>(CUstream_st*, bool, tcnn::ShampooOptimizer<__half>::step(CUstream_st*, float, float*, __half*, __half const*)::{lambda()#1})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS3_PKS3_EUlvE_EEvS6_bT_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS3_PKS3_EUlvE_EEvS6_bT_]+0x406): undefined reference to cudaGraphExecUpdate'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<__half>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#1}>(CUstream_st*, bool, tcnn::ShampooOptimizer<__half>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#1})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE_EEvS7_bSA_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE_EEvS7_bSA_]+0x406): undefined reference to cudaGraphExecUpdate'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o):tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text.ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE0_EEvS7_bSA[ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE0_EEvS7_bSA]+0x406): more undefined references to cudaGraphExecUpdate' follow collect2: error: ld returned 1 exit status make[2]: *** [CMakeFiles/testbed.dir/build.make:116: testbed] Error 1 make[1]: *** [CMakeFiles/Makefile2:200: CMakeFiles/testbed.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [100%] Linking CXX shared library pyngp.cpython-38-x86_64-linux-gnu.so [100%] Built target pyngp make: *** [Makefile:91: all] Error 2

Awesome work, I'm really looking forward to demoing this.

However I am currently running into build errors that I have not been able to troubleshoot myself, I have followed the instructions on your README.md. I am running Ubuntu 20 on a laptop with an NVIDIA GeForce RTX 2070 card and cuda 11.3.

Here are the logs of the build commands

`user@computer:~/software/instant-ngp$ cmake . -B build -DCMAKE_CUDA_COMPILER=/usr/local/cuda-11.3/bin/nvcc -- The CUDA compiler identification is NVIDIA 11.3.109 -- Check for working CUDA compiler: /usr/local/cuda-11.3/bin/nvcc -- Check for working CUDA compiler: /usr/local/cuda-11.3/bin/nvcc - works -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- No release type specified. Setting to 'Release'. -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Using X11 for window creation -- Found X11: /usr/include -- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so -- Looking for XOpenDisplay in /usr/lib/x86_64-linux-gnu/libX11.so;/usr/lib/x86_64-linux-gnu/libXext.so - found -- Looking for gethostbyname -- Looking for gethostbyname - found -- Looking for connect -- Looking for connect - found -- Looking for remove -- Looking for remove - found -- Looking for shmat -- Looking for shmat - found -- Looking for IceConnectionNumber in ICE -- Looking for IceConnectionNumber in ICE - found -- Found GLEW: /usr/include (found version "2.1.0") -- Found OpenMP_C: -fopenmp (found version "4.5") -- Found OpenMP_CXX: -fopenmp (found version "4.5") -- Found OpenMP: TRUE (found version "4.5") -- !!! Warning OptiX_INSTALL_DIR not set in environment. using default -- OptiX_INSTALL_DIR value: /usr/local/NVIDIA-OptiX-SDK-7.3.0-linux64-x86_64 -- OptiX headers (optix.h and friends) not found. CMake Warning at CMakeLists.txt:169 (message): OptiX was not found. Neural graphics primitives will still compile and run correctly. However, SDF training in 'raystab' and 'pathescape' modes will be significantly slower.

-- Found Python: /usr/bin/python3.8 (found suitable version "3.8.10", minimum required is "3.7") found components: Interpreter Development Development.Module Development.Embed -- pybind11 v2.7.1 CMake Warning (dev) at /home/user/software/cmake-3.22.1-linux-x86_64/share/cmake-3.22/Modules/CMakeDependentOption.cmake:84 (message): Policy CMP0127 is not set: cmake_dependent_option() supports full Condition Syntax. Run "cmake --help-policy CMP0127" for policy details. Use the cmake_policy command to set the policy and suppress this warning. Call Stack (most recent call first): dependencies/pybind11/CMakeLists.txt:98 (cmake_dependent_option) This warning is for project developers. Use -Wno-dev to suppress it.

-- Performing Test HAS_FLTO -- Performing Test HAS_FLTO - Success -- Targeting GPU architectures: 75 -- Configuring done -- Generating done -- Build files have been written to: /home/user/software/instant-ngp/build user@computer:~/software/instant-ngp$ cmake --build build --config RelWithDebInfo -j 16 [ 1%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/common.cu.o [ 3%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/common_device.cu.o [ 5%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/context.c.o [ 9%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/cutlass_resnet.cu.o [ 9%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/init.c.o [ 12%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/monitor.c.o [ 12%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/input.c.o [ 14%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/cutlass_mlp.cu.o [ 16%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/vulkan.c.o [ 18%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/window.c.o [ 20%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/encoding.cu.o [ 21%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/x11_init.c.o [ 23%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/loss.cu.o [ 25%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/network.cu.o [ 27%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/object.cu.o [ 29%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/optimizer.cu.o [ 30%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/x11_monitor.c.o [ 32%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/reduce_sum.cu.o [ 34%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/x11_window.c.o [ 36%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/xkb_unicode.c.o [ 38%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/posix_time.c.o [ 40%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/posix_thread.c.o [ 41%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/glx_context.c.o [ 43%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/egl_context.c.o [ 45%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/osmesa_context.c.o [ 47%] Building C object dependencies/glfw/src/CMakeFiles/glfw_objects.dir/linux_joystick.c.o [ 49%] Building CUDA object dependencies/tiny-cuda-nn/src/CMakeFiles/tiny-cuda-nn.dir/fully_fused_mlp.cu.o [ 49%] Built target glfw_objects [ 50%] Linking CUDA static library libtiny-cuda-nn.a [ 50%] Built target tiny-cuda-nn [ 52%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/ImGuizmo.cpp.o [ 54%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui.cpp.o [ 56%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_impl_glfw.cpp.o [ 58%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_impl_opengl3.cpp.o [ 60%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_draw.cpp.o [ 61%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_tables.cpp.o [ 63%] Building CXX object CMakeFiles/ngp.dir/dependencies/imgui/imgui_widgets.cpp.o [ 65%] Building CUDA object CMakeFiles/ngp.dir/src/camera_path.cu.o [ 67%] Building CUDA object CMakeFiles/ngp.dir/src/common_device.cu.o [ 69%] Building CUDA object CMakeFiles/ngp.dir/src/marching_cubes.cu.o [ 70%] Building CUDA object CMakeFiles/ngp.dir/src/nerf_loader.cu.o [ 72%] Building CUDA object CMakeFiles/ngp.dir/src/render_buffer.cu.o [ 74%] Building CUDA object CMakeFiles/ngp.dir/src/testbed.cu.o [ 76%] Building CUDA object CMakeFiles/ngp.dir/src/testbed_image.cu.o [ 78%] Building CUDA object CMakeFiles/ngp.dir/src/testbed_nerf.cu.o [ 80%] Building CUDA object CMakeFiles/ngp.dir/src/testbed_sdf.cu.o [ 81%] Building CUDA object CMakeFiles/ngp.dir/src/testbed_volume.cu.o [ 83%] Building CXX object CMakeFiles/ngp.dir/src/thread_pool.cpp.o [ 85%] Building CUDA object CMakeFiles/ngp.dir/src/tinyexr_wrapper.cu.o [ 87%] Building CXX object CMakeFiles/ngp.dir/src/tinyobj_loader_wrapper.cpp.o [ 89%] Building CUDA object CMakeFiles/ngp.dir/src/triangle_bvh.cu.o [ 90%] Linking CUDA device code CMakeFiles/ngp.dir/cmake_device_link.o [ 92%] Linking CXX static library libngp.a [ 92%] Built target ngp [ 96%] Building CUDA object CMakeFiles/pyngp.dir/src/python_api.cu.o [ 96%] Building CUDA object CMakeFiles/testbed.dir/src/main.cu.o [ 98%] Linking CXX executable testbed /usr/bin/ld: libngp.a(testbed_image.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::Trainer<float, __half, __half>::training_step(CUstream_st*, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const&, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const&, float*, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const*)::{lambda()#1}>(CUstream_st*, bool, tcnn::Trainer<float, __half, __half>::training_step(CUstream_st*, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const&, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const&, float*, tcnn::GPUMatrix<float, (tcnn::MatrixLayout)1> const*)::{lambda()#1})': tmpxft_00005ad8_00000000-6_testbed_image.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_7TrainerIf6__halfS3_E13training_stepEP11CUstream_stRKNS_9GPUMatrixIfLNS_12MatrixLayoutE1EEESB_PfPSA_EUlvE_EEvS6_bT_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_7TrainerIf6__halfS3_E13training_stepEP11CUstream_stRKNS_9GPUMatrixIfLNS_12MatrixLayoutE1EEESB_PfPSA_EUlvE_EEvS6_bT_]+0x406): undefined reference to cudaGraphExecUpdate' /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function tcnn::ShampooOptimizer<float>::step(CUstream_st*, float, float*, float*, float const*)::{lambda()#1}::operator()() const': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv]+0xacf): undefined reference to cublasSetWorkspace_v2' /usr/bin/ld: tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv]+0xebb): undefined reference to cublasSetWorkspace_v2' /usr/bin/ld: tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv]+0x12c1): undefined reference to cublasSetWorkspace_v2' /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function tcnn::ShampooOptimizer<__half>::step(CUstream_st*, float, float*, __half*, __half const*)::{lambda()#1}::operator()() const': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv]+0xaec): undefined reference to cublasSetWorkspace_v2' /usr/bin/ld: tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv]+0xed8): undefined reference to cublasSetWorkspace_v2' /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o):tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv]+0x12de): more undefined references to cublasSetWorkspace_v2' follow /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<float>::step(CUstream_st*, float, float*, float*, float const*)::{lambda()#1}>(CUstream_st*, bool, tcnn::ShampooOptimizer<float>::step(CUstream_st*, float, float*, float*, float const*)::{lambda()#1})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE4stepEP11CUstream_stfPfS6_PKfEUlvE_EEvS5_bT_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE4stepEP11CUstream_stfPfS6_PKfEUlvE_EEvS5_bT_]+0x406): undefined reference to cudaGraphExecUpdate' /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<float>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#1}>(CUstream_st*, bool, tcnn::ShampooOptimizer<float>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#1})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE_EEvS6_bS9_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE_EEvS6_bS9_]+0x406): undefined reference to cudaGraphExecUpdate' /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<float>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#2}>(CUstream_st*, bool, tcnn::ShampooOptimizer<float>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#2})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE0_EEvS6_bS9_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerIfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE0_EEvS6_bS9_]+0x406): undefined reference to cudaGraphExecUpdate' /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<__half>::step(CUstream_st*, float, float*, __half*, __half const*)::{lambda()#1}>(CUstream_st*, bool, tcnn::ShampooOptimizer<__half>::step(CUstream_st*, float, float*, __half*, __half const*)::{lambda()#1})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS3_PKS3_EUlvE_EEvS6_bT_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS3_PKS3_EUlvE_EEvS6_bT_]+0x406): undefined reference to cudaGraphExecUpdate' /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function void tcnn::CudaGraph::capture_and_execute<tcnn::ShampooOptimizer<__half>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#1}>(CUstream_st*, bool, tcnn::ShampooOptimizer<__half>::inverse_pth_root_batched<float>(CUstream_st*, unsigned int, float*, tcnn::GPUMemory<float>&, unsigned int, unsigned int)::{lambda()#1})': tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text._ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE_EEvS7_bSA_[_ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE_EEvS7_bSA_]+0x406): undefined reference to cudaGraphExecUpdate' /usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o):tmpxft_00005870_00000000-6_optimizer.cudafe1.cpp:(.text.ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE0_EEvS7_bSA[ZN4tcnn9CudaGraph19capture_and_executeIZNS_16ShampooOptimizerI6__halfE24inverse_pth_root_batchedIfEEvP11CUstream_stjPfRNS_9GPUMemoryIT_EEjjEUlvE0_EEvS7_bSA]+0x406): more undefined references to cudaGraphExecUpdate' follow collect2: error: ld returned 1 exit status make[2]: *** [CMakeFiles/testbed.dir/build.make:116: testbed] Error 1 make[1]: *** [CMakeFiles/Makefile2:200: CMakeFiles/testbed.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [100%] Linking CXX shared library pyngp.cpython-38-x86_64-linux-gnu.so [100%] Built target pyngp make: *** [Makefile:91: all] Error 2

how to solve this problem? THX

commented

hey how to sovle the problem thx!!!!! help orz

I'm getting this issue with CUDA 11.7, even though the README says it can be fixed by upgrading to 11.3 or higher.

$ cmake --build build --config RelWithDebInfo
[ 21%] Built target tiny-cuda-nn
[ 50%] Built target glfw_objects
[ 92%] Built target ngp
[ 94%] Linking CXX executable testbed
/usr/bin/ld: libngp.a(testbed_image.cu.o): in function `tcnn::CudaGraph::capture_and_execute(CUstream_st*, bool, std::function<void ()>)::{lambda()#1}::operator()() const':
tmpxft_00000ce1_00000000-6_testbed_image.cudafe1.cpp:(.text._ZZN4tcnn9CudaGraph19capture_and_executeEP11CUstream_stbSt8functionIFvvEEENKUlvE_clEv[_ZZN4tcnn9CudaGraph19capture_and_executeEP11CUstream_stbSt8functionIFvvEEENKUlvE_clEv]+0x3ad): undefined reference to `cudaGraphExecUpdate'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function `tcnn::ShampooOptimizer<float>::step(CUstream_st*, float, float*, float*, float const*)::{lambda()#1}::operator()() const':
tmpxft_00007c4d_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv]+0xacf): undefined reference to `cublasSetWorkspace_v2'
/usr/bin/ld: tmpxft_00007c4d_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv]+0xebb): undefined reference to `cublasSetWorkspace_v2'
/usr/bin/ld: tmpxft_00007c4d_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerIfE4stepEP11CUstream_stfPfS4_PKfENKUlvE_clEv]+0x12c1): undefined reference to `cublasSetWorkspace_v2'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o): in function `tcnn::ShampooOptimizer<__half>::step(CUstream_st*, float, float*, __half*, __half const*)::{lambda()#1}::operator()() const':
tmpxft_00007c4d_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv]+0xaec): undefined reference to `cublasSetWorkspace_v2'
/usr/bin/ld: tmpxft_00007c4d_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv]+0xed8): undefined reference to `cublasSetWorkspace_v2'
/usr/bin/ld: dependencies/tiny-cuda-nn/src/libtiny-cuda-nn.a(optimizer.cu.o):tmpxft_00007c4d_00000000-6_optimizer.cudafe1.cpp:(.text._ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv[_ZZN4tcnn16ShampooOptimizerI6__halfE4stepEP11CUstream_stfPfPS1_PKS1_ENKUlvE_clEv]+0x12de): more undefined references to `cublasSetWorkspace_v2' follow
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/testbed.dir/build.make:117: testbed] Error 1
make[1]: *** [CMakeFiles/Makefile2:200: CMakeFiles/testbed.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Tue_May__3_18:49:52_PDT_2022
Cuda compilation tools, release 11.7, V11.7.64
Build cuda_11.7.r11.7/compiler.31294372_0
commented

Solved by setting in my .bashrc:

export CUDA_HOME="/usr/local/cuda"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64"
export LIBRARY_PATH="$LIBRARY_PATH:/usr/local/cuda/lib64"