Add support for Zephyr 7B
gvijqb opened this issue · comments
Zephyr 7B model is kind of the best in small size models and is able to outperform GPT 3.5 and Llama 70B models as well in some tasks. See benchmark:
https://huggingface.co/HuggingFaceH4/zephyr-7b-beta
It is also cost effective because of its low size.
Zephyr 7B API is available on MonsterAPI:
Free playground: https://monsterapi.ai/playground/zephyr-7b-beta
Zephyr API docs: https://developer.monsterapi.ai/reference/post_generate-zephyr-7b-beta
We can provide dedicated support for ensuring scalable and reliable integration.
you are responsible for your LLM
start your onw llm for example use oobabooga textgen webui with parameter "--listen --extension openai"
load your Zephyr Model in oobabooga
ChatDev to use your own model
write the commands in ChatDev console
export OPENAI_API_KEY="no key needed in oobabooga"
export OPENAI_API_BASE="http://YOUR-OOBABOOGA-IP/v1"
pythron run.py --tast "build a HTML one page with a clock" --name "my-oobabooga-zephyr-clock"
You could refer to #27 to get an insight on how to deploy local models into ChatDev.
ok so running it with LMstudio and openai version 1.3.3 you get an authentication error so i changed OPENAI_API_BASE to BASE_URL and it connected to LM studio but now i was getting the "messages" error referenced here. Switched it to llama2 and now it's working. cant wait to test this out.