jllllll / exllama

A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

how to install this module via requirement.txt file

Yusuf007R opened this issue · comments

I am attempting to utilize a service called Cerebrium, which requires the installation of Exllama through a requirement.txt file. However, I am encountering difficulties with the original repository because it does not have a pip module.
and with this repo I'm not sure how can I use python -m pip install git+https://github.com/jllllll/exllama in the requirement.txt

Direct links to the wheels can be used along with system filters to ensure proper installation:

https://github.com/jllllll/exllama/releases/download/0.0.6/exllama-0.0.6+cu117-cp310-cp310-win_amd64.whl; platform_system == "Windows"
https://github.com/jllllll/exllama/releases/download/0.0.6/exllama-0.0.6+cu117-cp310-cp310-linux_x86_64.whl; platform_system == "Linux" and platform_machine == "x86_64"

You can also directly use the git+ format:

git+https://github.com/jllllll/exllama

thanks you so much