bitsandbytes import error in colab

[Sougata Sarkar]

System Info

colab with T4

Reproduction

Could not load bitsandbytes native library: libcusparse.so.11: cannot open shared object file: No such file or directory
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cextension.py", line 109, in
lib = get_native_library()
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/cextension.py", line 96, in get_native_library
dll = ct.cdll.LoadLibrary(str(binary_path))
File "/usr/lib/python3.10/ctypes/init.py", line 452, in LoadLibrary
return self._dlltype(name)
File "/usr/lib/python3.10/ctypes/init.py", line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: libcusparse.so.11: cannot open shared object file: No such file or directory

CUDA Setup failed despite CUDA being available. Please run the following command to get more information:

python -m bitsandbytes

Inspect the output of the command and see if you can locate CUDA libraries. You might need to add them
to your LD_LIBRARY_PATH. If you suspect a bug, please take the information from python -m bitsandbytes
and open an issue at: https://github.com/TimDettmers/bitsandbytes/issues

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++ BUG REPORT INFORMATION ++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
CUDA specs: CUDASpecs(highest_compute_capability=(7, 5), cuda_version_string='117', cuda_version_tuple=(11, 7))
PyTorch settings found: CUDA_VERSION=117, Highest Compute Capability: (7, 5).
To manually override the PyTorch CUDA version please see: https://github.com/TimDettmers/bitsandbytes/blob/main/docs/source/nonpytorchcuda.mdx
The directory listed in your path is found to be non-existent: /sys/fs/cgroup/memory.events /var/colab/cgroup/jupyter-children/memory.events
The directory listed in your path is found to be non-existent: //172.28.0.1
The directory listed in your path is found to be non-existent: 8013
The directory listed in your path is found to be non-existent: //colab.research.google.com/tun/m/cc48301118ce562b961b3c22d803539adc1e0c19/gpu-t4-hm-w6fogzooaaau --tunnel_background_save_delay=10s --tunnel_periodic_background_save_frequency=30m0s --enable_output_coalescing=true --output_coalescing_required=true
The directory listed in your path is found to be non-existent: /datalab/web/pyright/typeshed-fallback/stdlib,/usr/local/lib/python3.10/dist-packages
The directory listed in your path is found to be non-existent: /env/python
The directory listed in your path is found to be non-existent: //ipykernel.pylab.backend_inline
CUDA SETUP: WARNING! CUDA runtime files not found in any environmental path.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Checking that the library is importable and CUDA is callable...
Couldn't load the bitsandbytes library, likely due to missing binaries.
Please ensure bitsandbytes is properly installed.

For source installations, compile the binaries with cmake -DCOMPUTE_BACKEND=cuda -S ..
See the documentation for more details if needed.

Trying a simple check anyway, but this will likely fail...
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/diagnostics/main.py", line 66, in main
sanity_check()
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/diagnostics/main.py", line 40, in sanity_check
adam.step()
File "/usr/local/lib/python3.10/dist-packages/torch/optim/optimizer.py", line 280, in wrapper
out = func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/optim/optimizer.py", line 287, in step
self.update_step(group, p, gindex, pindex)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/optim/optimizer.py", line 496, in update_step
F.optimizer_update_32bit(
File "/usr/local/lib/python3.10/dist-packages/bitsandbytes/functional.py", line 1584, in optimizer_update_32bit
optim_func = str2optimizer32bit[optimizer_name][0]
NameError: name 'str2optimizer32bit' is not defined
Above we output some debug information.
Please provide this info when creating an issue via https://github.com/TimDettmers/bitsandbytes/issues/new/choose
WARNING: Please be sure to sanitize sensitive info from the output before posting it.

Expected behavior

Should work with no errpr

[slou]

I met the same problem but not in Colab.
You can try to install libcusparse manually: https://anaconda.org/nvidia/libcusparse
I also exported LD_LIBRARY_PATH to include the library path explicitly so that environment can find the so file.