Default behaviors not respected?
rdegges opened this issue · comments
Hi there! First off, thanks for building this project! It's super cool and I'm using it extensively <3
I'm planning to roll this out to some colleagues at my organization and ran into an issue. In my case we have a few different environment variables provided with API credentials so that we have a number of models users can access.
I've set the environment variable NEXT_PUBLIC_DEFAULT_MODEL to the specified model I'd like to use as the default (gpt-4-32k), but the setting doesn't appear to be respected. The same is true for NEXT_PUBLIC_DEFAULT_TEMPERATURE when set to 1, which is the default value ChatGPT uses and what I'd like to default temperature to for my users.
Any ideas what I may be doing incorrectly here? Thanks again! <3
I am currently working on this branch: https://github.com/jorge-menjivar/unSAGED/tree/model-params-upgrade
Just keep in mind that this update will require a quick change to the database, in order to upgrade from an older version. See the README for more details.
This update will change some things:
- The names of the env variables will change, allowing for one default var per AI vendor (OpenAI, Google, Ollama, etc).
- It includes support for other parameters like top_p, seed, etc.
- Not setting a default var won't force a temperature like 0.7 anymore. It will simply leave them blank in the completion request.
The latest commit should also fix the issue with the default model not being respected.
Also if the default model env variable is not set, new conversations will default to the model of the last conversation that was used.
Oh thank you! This is awesome.