python-jaxlib installation error
omega3 opened this issue · comments
I get errors while compiling on Manjaro https://pastebin.com/eCeAuS06
I want to run it with https://huggingface.co/microsoft/speecht5_tts
examples given in Transformers Usage, point 2 or 3.
but I get:
RuntimeError: This version of jaxlib was built using AVX instructions, which your CPU and/or operating system do not support. You may be able work around this issue by building jaxlib from source.
I also reported here
I am a linux beginner and I can only follow simple instructions. I want to have offline TTS on my cheap PC. I don't know what to do now.
Can this help? I don't know what to do with it.
Operating System: Manjaro Linux
KDE Plasma Version: 5.27.10
KDE Frameworks Version: 5.113.0
Qt Version: 5.15.11
Kernel Version: 6.6.8-2-MANJARO (64-bit)
Graphics Platform: X11
Processors: 4 × Intel® Pentium® CPU G4560 @ 3.50GHz
Memory: 7.7 GiB of RAM
Graphics Processor: Mesa Intel® HD Graphics 610
Manufacturer: Gigabyte Technology Co., Ltd.
Product Name: H110M-S2H
Hi, the problem is in the PKGBUILD --target_cpu_features is set to release. And now AVX is a target architecture by default in release configuration. If you are compiling the package yourself, then try setting --target_cpu_features to native. It will detect and fit your CPU instruction set locally.
The patch you mentioned in our cactus.yaml is used to change --target_cpu_features native to --target_cpu_features=release. So it is useless now since they already set --target_cpu_features=release in the PKGBUILD.
Thank you for this tip. It compiles but at 4767 out of 5900... my system hanged. Do you know it if it possible to compile it with less features, so that it may work only for TTS or resume compiling after system restart?
It compiles but at 4767 out of 5900.
Could you share your build log so we can see the actual error?
Do you know it if it possible to compile it with less features
I haven't tested it but I think this may work:
- Set
--target_cpu_featurestorelease. - In build/build.py, change
avx_posixtoposix. You can do it by adding a line likesed 's/avx_posix/posix/' -i build/build.pybefore the build command in PKGBUILD. - Build the package.
And you can also try just setting --target_cpu_features to default according to the help messages in build/build.py.
And you can also try just setting
--target_cpu_featurestodefaultaccording to the help messages inbuild/build.py.
@carlosal1015 Should we also apply this change in our repository to disable the AVX support?
tip
Could you share your build log so we can see the actual error?
I don't know where they are. I looked at /var/log/pacman.log but there is nothing about it there. System hanged and I restarted so maybe nothing was written.
I will follow instructions and try again later.
Edition:
Unfortunately it hanged again, even sooner. So, I think my pc is too weak for compiling this package and I give up.
Yes, if we disable AVX support, then more users will have jax working.
I am using python-jaxlib-cuda from arch4edu and its working fine.
Hi @omega3, could you list of packages that you need to have installed from AUR? Did you enable the arch4edu repository?
I see, Manjaro does not have full support, maybe can try run under docker.
Unfortunately it hanged again, even sooner. So, I think my pc is too weak for compiling this package and I give up.
You can try the prebuilt binary from arch4edu. You can just download it from https://github.com/arch4edu/cactus/actions/runs/7288311166.
It's built on Arch Linux so it's possible that it won't work on Manjaro. But it still worths a try.
I installed your prebuilt binary package but still get the same error. I guess it is because I installed system wide, while this TTS module I installed via pip along with torch and transformers and I run it as they advise in virtual environment, like this:
source .env/bin/activate
I installed your prebuilt binary package but still get the same error.
Does the same error mean RuntimeError: This version of jaxlib was built using AVX instructions? If so, then maybe there is another jax installation in your machine. It looks like you are using venv/conda so it's very likely that there is another jaxlib installed in the virtual environment. You can check it with pip list and pip show [package name].
If you are using conda, then you have to compile the package yourself with the toolchain in conda.
If you are using venv, then you can add the --system-site-packages option when creating the environment to also use the packages installed by pacman like:
$ virtualenv --system-site-packages .envI assume that you will then install the dependencies via requirements.txt. They always fixed the versions in requirements.txt so it's very likely that there are some conflicts. You have to resolve the conflicts yourself then. If lucky, you can just remove the versions in requirements.txt to use the latest ones. You may have to update the code to meet the api in the latest versions.
You can also try ignoring requirements.txt and manually install the dependencies when the code says it's needed.
Does the same error mean RuntimeError: This version of jaxlib was built using AVX instructions?
Yes. And I was using venv and had another jaxlib installed via pip.
I did:
python -m venv --system-site-packages .env
source .env/bin/activate --system-site-packages
and success.
Thank you very much for your support and your patience.
It allows me to use speecht5_tts now and play with other super interesting projects on Hugging Face.