Cannot find cudnn libs/headers
Andreybest opened this issue · comments
Hello!
I was trying to run stable diffusion with candle, but I stuck with 3 issues:
Issue 1
LINK : fatal error LNK1181: cannot open input file 'cudnn.lib'
I've fixed it by adding println!("cargo:rustc-link-search=native=C:\\Program Files\\NVIDIA\\CUDNN\\v9.2\\lib\\12.5\\x64"); to my build.rs. I think it's issue on my side, but feel that my fix is not proper one (PATH points to this folder)
Issue 2
Unable to find `include/cudnn.h` under any of: ["C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.4", "C:\\Program Files\\NVIDIA\\CUDNN\\v9.2\\include\\12.5", "/usr", "/usr/local/cuda", "/opt/cuda", "/usr/lib/cuda", "C:/Program Files/NVIDIA GPU Computing Toolkit", "C:/CUDA"]. Set the `CUDNN_LIB` environment variable to `$CUDNN_LIB/include/cudnn.h` to override path.
I've set CUDNN_LIB env to C:\Program Files\NVIDIA\CUDNN\v9.2, but the problem is that the header file is in \include*12.5*\ not directly in \include. So I've created include folder in C:\Program Files\NVIDIA\CUDNN\v9.2\include\12.5, and copy-pasted .h files that and after that it stopped failing.
Issue 3
thread 'main' panicked at C:\Users\AndreyBest\.cargo\registry\src\index.crates.io-6f17d22bba15001f\cudarc-0.11.5\src\cudnn\sys\mod.rs:61:9:
Unable to find cudnn lib under the names ["cudnn", "cudnn64", "cudnn64_12", "cudnn64_124", "cudnn64_124_0", "cudnn64_120_4", "cudnn64_10"]. Please open GitHub issue.
After examining CUDNN folders (and learning about dynamic linking/loading), I found out that it was looking for .dll files that does not exist there.
After copy-pasting cudnn64_9.dll and renaming it to cudnn64_12.dll - it worked.
File tree
file /f command runed in C:\Program Files\NVIDIA\CUDNN\v9.2
C:.
│ LICENSE
│
├───bin
│ ├───11.8
│ │ cudnn64_9.dll
│ │ cudnn_adv64_9.dll
│ │ cudnn_cnn64_9.dll
│ │ cudnn_engines_precompiled64_9.dll
│ │ cudnn_engines_runtime_compiled64_9.dll
│ │ cudnn_graph64_9.dll
│ │ cudnn_heuristic64_9.dll
│ │ cudnn_ops64_9.dll
│ │
│ └───12.5
│ cudnn64_12.dll (my copy-paste)
│ cudnn64_9.dll
│ cudnn_adv64_9.dll
│ cudnn_cnn64_9.dll
│ cudnn_engines_precompiled64_9.dll
│ cudnn_engines_runtime_compiled64_9.dll
│ cudnn_graph64_9.dll
│ cudnn_heuristic64_9.dll
│ cudnn_ops64_9.dll
│
├───cudnn_samples
│ │ CMakeLists.txt
│ │ samples_common.mk
│ │
│ ├───cmake
│ │ FindcuDNN.cmake
│ │
│ ├───common
│ │ CMakeLists.txt
│ │ error_util.h
│ │ fp16_dev.cu
│ │ fp16_dev.h
│ │ fp16_emu.cpp
│ │ fp16_emu.h
│ │
│ ├───conv_sample
│ │ CMakeLists.txt
│ │ conv_sample.cpp
│ │ error_util.h
│ │ fp16_dev.cu
│ │ fp16_dev.h
│ │ fp16_emu.cpp
│ │ fp16_emu.h
│ │ Makefile
│ │ README.txt
│ │ run_conv_sample.sh
│ │
│ ├───mnistCUDNN
│ │ │ CMakeLists.txt
│ │ │ error_util.h
│ │ │ fp16_dev.cu
│ │ │ fp16_dev.h
│ │ │ fp16_emu.cpp
│ │ │ fp16_emu.h
│ │ │ gemv.h
│ │ │ Makefile
│ │ │ mnistCUDNN.cpp
│ │ │ README.txt
│ │ │
│ │ └───data
│ │ conv1.bias.bin
│ │ conv1.bin
│ │ conv2.bias.bin
│ │ conv2.bin
│ │ five_28x28.pgm
│ │ ip1.bias.bin
│ │ ip1.bin
│ │ ip2.bias.bin
│ │ ip2.bin
│ │ one_28x28.pgm
│ │ three_28x28.pgm
│ │
│ ├───multiHeadAttention
│ │ attn_ref.py
│ │ CMakeLists.txt
│ │ fp16_emu.h
│ │ Makefile
│ │ multiHeadAttention.cpp
│ │ multiHeadAttention.h
│ │ README.txt
│ │ run_ref.sh
│ │
│ └───RNN_v8.0
│ CMakeLists.txt
│ compare.py
│ fp16_emu.cpp
│ fp16_emu.h
│ golden_1.txt
│ golden_2.txt
│ golden_3.txt
│ golden_4.txt
│ Makefile
│ README.txt
│ RNN_example.cu
│ RNN_example.h
│
├───include
│ ├───11.8
│ │ cudnn.h
│ │ cudnn_adv.h
│ │ cudnn_backend.h
│ │ cudnn_cnn.h
│ │ cudnn_graph.h
│ │ cudnn_ops.h
│ │ cudnn_version.h
│ │
│ └───12.5
│ │ cudnn.h
│ │ cudnn_adv.h
│ │ cudnn_backend.h
│ │ cudnn_cnn.h
│ │ cudnn_graph.h
│ │ cudnn_ops.h
│ │ cudnn_version.h
│ │
│ └───include (my copy-paste)
│ cudnn.h
│ cudnn_adv.h
│ cudnn_backend.h
│ cudnn_cnn.h
│ cudnn_graph.h
│ cudnn_ops.h
│ cudnn_version.h
│
└───lib
├───11.8
│ └───x64
│ cudnn.lib
│ cudnn64_9.lib
│ cudnn_adv.lib
│ cudnn_adv64_9.lib
│ cudnn_cnn.lib
│ cudnn_cnn64_9.lib
│ cudnn_engines_precompiled.lib
│ cudnn_engines_precompiled64_9.lib
│ cudnn_engines_runtime_compiled.lib
│ cudnn_engines_runtime_compiled64_9.lib
│ cudnn_graph.lib
│ cudnn_graph64_9.lib
│ cudnn_heuristic.lib
│ cudnn_heuristic64_9.lib
│ cudnn_ops.lib
│ cudnn_ops64_9.lib
│
└───12.5
└───x64
cudnn.lib
cudnn64_9.lib
cudnn_adv.lib
cudnn_adv64_9.lib
cudnn_cnn.lib
cudnn_cnn64_9.lib
cudnn_engines_precompiled.lib
cudnn_engines_precompiled64_9.lib
cudnn_engines_runtime_compiled.lib
cudnn_engines_runtime_compiled64_9.lib
cudnn_graph.lib
cudnn_graph64_9.lib
cudnn_heuristic.lib
cudnn_heuristic64_9.lib
cudnn_ops.lib
cudnn_ops64_9.lib
I've installed CUDNN 9.2.0 as installer, and left all defaults in place, maybe something changed in last CUDNN?
Thanks for reporting this and including so much detail, its super helpful ❤️
CUDA on Windows has been continually confusing. Will get in fixes to these this week