Support Transformers >=4.40.0

Question

Support Transformers >=4.40.0

saattrupdan opened this issue a month ago · comments

Dan Saattrup Nielsen commented a month ago

The Issue

I would like to be able to load GPTQ models with the newest transformers version. This requires optimum, but optimum is not compatible with the newest transformers version.

System Info

Linux, Python 3.11, Optimum 1.19.0

Who can help?

@SunMarc

Information

The official example scripts
My own modified scripts

Tasks

An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)

pip install "transformers>=4.40.0" "optimum>=1.19.0"

Expected behavior

No dependency conflict.

Marc Sun · Answer 1 · Mon Apr 22 2024 21:21:51 GMT+0800 (China Standard Time)

Hi @saattrupdan, transformers version have been pinned in the latest version of optimum. See PR here. I guess the easier solution for now is installing optimum==1.18.0 in order to use gptq with transformers.

Dan Saattrup Nielsen · Answer 2 · Mon Apr 22 2024 21:43:21 GMT+0800 (China Standard Time)

Hi @saattrupdan, transformers version have been pinned in the latest version of optimum. See PR here. I guess the easier solution for now is installing optimum==1.18.0 in order to use gptq with transformers.

Thanks @SunMarc! I found that 1.18.0 still has the upper bound requirement, but 1.17.1 works. Thanks for that hotfix 🙂