Giters

kagisearch / pyllms

Minimal Python library to connect to LLMs (OpenAI, Anthropic, AI21, Cohere, Aleph Alpha, HuggingfaceHub, Google PaLM2, with a built-in model performance benchmark.

Home Page:https://kagi.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Heavy dependencies

denisolt-relyanceai opened this issue · comments

commented

Hey,
I have noticed that there is a heavy dependency accelerate included in the requirements.
accelerate has a dependency on torch, which results in a pretty big total size (I believe torch is about 600mb). I traced down the dependency to a PR that was adding support for local LLM runs.

Have you considered adding extras_require to setup.py so these local run dependencies are only installed if the user wants to run the LLM locally?

I am happy to open up a PR with a suggestion.

commented

thanks for bringing it up. afaik we do not have local LLM support yet so it must be something else (I vaguely remember huggingface requiring it). Can you link to the commit where accelerate was added?

commented

hey @vprelovac, commit where it was added: 6247eb0

commented

Ah you are correct, this is for local execution of HF models and that does require acelerate. Yes please go ahead with the PR to localize the use of this.