GPT4generator broke down
mamscience opened this issue · comments
Hi guys
Don't know if anybody has noticed, but when you use Verba on a docker, this is the error you get.
verba_1 | INFO: 12.345.678.910:53140 - "POST /api/get_document HTTP/1.1" 200 OK
verba_1 | INFO: 127.0.0.1:55516 - "POST /chat/completions HTTP/1.1" 404 Not Found
verba_1 | ERROR: Exception in ASGI application
Seems that the call to OpenAI has broken down. Error is pinpointed to
verba_1 | File "/Verba/goldenverba/components/generation/GPT4Generator.py", line 116, in generate_stream
verba_1 | completion = await openai.ChatCompletion.acreate(
OpenAI has pushed a major release in november, but afaik Verba still uses the old version (0.27 I believe). Perhaps they are cleaning up the endpoints?
Error has been reported here too: https://forum.weaviate.io/t/error-in-docker-image-openai-endpoint-not-found/1180
PS love the project!
Found the bug!
After december's update (adding "OPENAI_BASE_URL" as env var), the SDK points to "https://0.0.0.0:8000/", which is of course not the OpenAI endpoint.
So unsetting this var or resetting it in the .env to the correct one works.
base_url = "https://api.openai.com/v1"
I also took a sneak peak if the newer version of the OpenAI Pip package should work. I have figured it out moslty, but parsing the stream of chunks is not my forté. Happy to share the code of what I have done if there's a branch for it.
Great catch! Thanks a lot
We'll look into updating to a newer openai version for the next update + we'll clean some things up
We added it to the documentation! Thanks for catching!