qwopqwop200 / GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

No CUDA_ENV / conda-froce cudatoolkit-dev freezes

nathanleclaire opened this issue · comments

I had an unfortunate time installing the cuda toolkit on WSL to use this

Unrelated to repo directly but, conda install -c conda-forge cudatoolkit-dev just hung forever and couldn't solve

nvidia-cuda-toolkit was out of date to 11.7 so I fixed it by following these steps:

wget https://developer.download.nvidia.com/compute/cuda/repos/wsl-ubuntu/x86_64/cuda-wsl-ubuntu.pin
sudo mv cuda-wsl-ubuntu.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda-repo-wsl-ubuntu-11-7-local_11.7.0-1_amd64.deb
sudo dpkg -i cuda-repo-wsl-ubuntu-11-7-local_11.7.0-1_amd64.deb
sudo cp /var/cuda-repo-wsl-ubuntu-11-7-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda

Just documenting here so it's hopefully helpful to someone down the line.