Giters

Atinoda / text-generation-webui-docker

Docker variants of oobabooga's text-generation-webui, including pre-built images.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Incorporate exllama?

uogbuji opened this issue · comments

commented

Any chance you could incorporate Ooba's new exllama support? It's more than just an upstream code update. Needs additional cloning of that repo, etc. https://github.com/oobabooga/text-generation-webui/blob/main/docs/ExLlama.md

commented

I have integrated ExLlama and tested with wizard-mega-13B-GPTQ - it runs quickly indeed! Struggled to load older models though. Docker images are being built and pushed now - let me know how it works for you!