With unsloth installed under cuda12.2, running the command line with llama-factory for Rola fine-tuning reported an error

[weibo021]

Environment is cuda12.2 pytorch 2.3.1
Install unsloth using ：
pip install "unsloth[cu121-ampere-torch230] @ git+https://github.com/unslothai/unsloth.git"
Installation of unsloth succeeded!

Traceback (most recent call last):
File "/root/anaconda3/bin/llamafactory-cli", line 8, in
sys.exit(main())
^^^^^^
File "/root/LLaMA-Factory/src/llamafactory/cli.py", line 110, in main
run_exp()
File "/root/LLaMA-Factory/src/llamafactory/train/tuner.py", line 47, in run_exp
run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
File "/root/LLaMA-Factory/src/llamafactory/train/sft/workflow.py", line 49, in run_sft
model = load_model(tokenizer, model_args, finetuning_args, training_args.do_train)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/LLaMA-Factory/src/llamafactory/model/loader.py", line 138, in load_model
model = load_unsloth_pretrained_model(config, model_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/LLaMA-Factory/src/llamafactory/model/model_utils/unsloth.py", line 53, in load_unsloth_pretrained_model
from unsloth import FastLanguageModel
File "/root/anaconda3/lib/python3.11/site-packages/unsloth/init.py", line 26, in
raise ImportError(f"Unsloth: Please import Unsloth before {module}.")
ImportError: Unsloth: Please import Unsloth before bitsandbytes.

[weibo021]

accelerate 0.31.0
bitsandbytes 0.43.1
llamafactory 0.8.2.dev0 /root/LLaMA-Factory
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-ml-py 12.555.43
nvidia-nccl-cu12 2.20.5
nvidia-nvjitlink-cu12 12.5.40
nvidia-nvtx-cu12 12.1.105
openai 1.34.0
torch 2.3.0
torchvision 0.18.1
transformers 4.41.2
triton 2.3.0
trl 0.8.6
unsloth 2024.6

[Daniel Han]

I think I fixed it - I removed the bitsandbytes import check. Please update unsloth via:

pip uninstall unsloth -y
pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git

[weibo021]

I think I fixed it - I removed the bitsandbytes import check. Please update unsloth via:

pip uninstall unsloth -y
pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git

Okay, it's working.