Error nvcc
mix127 opened this issue · comments
On ubuntu 25, last relase is 12-9
make train_gpt2fp32cu
→ cuDNN is manually disabled by default, run make with USE_CUDNN=1 to try to enable
✓ OpenMP found
✓ NCCL found, OK to train with multiple GPUs
✗ MPI not found
✓ nvcc found, including GPU/CUDA support
/usr/local/cuda-12.5/bin/nvcc --threads=0 -t=0 --use_fast_math -std=c++17 -O3 -DMULTI_GPU train_gpt2_fp32.cu -lcublas -lcublasLt -lnvidia-ml -lnccl -o train_gpt2fp32cu
In file included from /usr/local/cuda-12.5/bin/../targets/x86_64-linux/include/cuda_runtime.h:82,
from :
/usr/local/cuda-12.5/bin/../targets/x86_64-linux/include/crt/host_config.h:143:2: error: #error -- unsupported GNU version! gcc versions later than 13 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
143 | #error -- unsupported GNU version! gcc versions later than 13 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
| ^~~~~
In file included from /usr/local/cuda-12.5/bin/../targets/x86_64-linux/include/cuda_runtime.h:82,
from :
/usr/local/cuda-12.5/bin/../targets/x86_64-linux/include/crt/host_config.h:143:2: error: #error -- unsupported GNU version! gcc versions later than 13 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
143 | #error -- unsupported GNU version! gcc versions later than 13 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.
| ^~~~~
make: *** [Makefile:277: train_gpt2fp32cu] Błąd 255
It should work if fix your PATH to point to where 12.9 is installed (probably /usr/local/cuda-12.9) rather than 12.5.
This is expected with NVCC 12.5 + GCC 14: https://stackoverflow.com/a/4638060