OpenAI seems like there is an API error
Jbkcrash2 opened this issue · comments
I am receiving this error when trying to use OpenAI. I believe I have the config.yaml correct, but based on the error, OpenAI doesn't like the structure. A fellow colleague and I started to trace it out, but we ran out of time on Friday, so I figured I would at least post the error to see if you see an easy fix.
2024-03-18 12:13:13,387 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 400 Bad Request"
2024-03-18 12:13:13,388 - ERROR - Error in Generation Step: Error code: 400 - {'error': {'message': "'$.stop' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}
Our .yaml for reference...
PATH:
INPUT: "./raw_txt_input"
OUTPUT: "./output"
DEFAULT_PROMPTS: "./prompts" # the baseline prompt folder that Augmentoolkit falls back to if it can't find a step in the PROMPTS path
PROMPTS: "./prompts" # Where Augmentoolkit first looks for prompts
API:
API_KEY: "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # Add the API key for your favorite provider here
BASE_URL: "https://api.openai.com/v1/" # add the base url for a provider, or local server, here. Some possible values: http://127.0.0.1:5000/v1/ # <- local models. # https://api.together.xyz # <- together.ai, which is real cheap, real flexible, and real high-quality, if a tad unreliable. # https://api.openai.com/v1/ # <- OpenAI. Will bankrupt you very fast. # anything else that accepts OAI-style requests, so basically any API out there (openrouter, fireworks, etc etc etc...)
LOGICAL_MODEL: "gpt-3.5-turbo-0125" # model used for everything except conversation generation at the very end
LARGE_LOGICAL_MODEL: "gpt-3.5-turbo-0125" # model used for conversation generation at the very end. A pretty tough task, if ASSISTANT_MODE isn't on.
SYSTEM:
USE_FILENAMES: False # give the AI context from the filenames provided to it. Useful if the filenames are meaningful, otherwise turn them off.
ASSISTANT_MODE: False # If off, the conversations generated are between a user and an AI assistant. If on, the generated convs are between fictional characters in historical or fictional settings, with randomized personalities (some are nsfw by default, because a lot of model creators make models for that purpose. Change this (or amplify it) in ./augmentoolkit/generation_functions/special_instructions.py, it only requires changes to some strings.)
DOUBLE_CHECK_COUNTER: 3 # How many times to check a question and answer pair during each validation step. Majority vote decides if it passes that step. There are three steps. So most questions are by default checked around 9 times (fewer if the first two checks for a step pass, obviously).
USE_SUBSET: True # Whether to take only the first 13 chunks from a text during the run. Useful for experimenting and iterating and seeing all the steps without costing too much money or time.
REARRANGEMENTS_TO_TAKE: 3 # How many times to rearrange the questions and answers for generating different conversations from the same group of questions and answers.
CONCURRENCY_LIMIT: 1 # Hard limit of how many calls can be run at the same time, useful for API mode (aphrodite automatically manages this and queues things, as far as I know)
COMPLETION_MODE: False # Change to false if you want to use chat (instruct) mode; this requires .json files in your chosen prompts directory, in the OpenAI API format. Not all APIs support completion mode.
MODE: "api" # can be one of "api"|"aphrodite"
GRAPH: False
Now this is a mysterious one. You're right: it looks like Augmentoolkit is erroring with gpt-3.5. Strangely enough it still works with open source APIs, which offer OpenAI compatible APIs. And the error message suggests that stop isn't a valid option for a request?
This is strange indeed. I'll give it a look later today when I'm off work, thanks for bringing this up!
That was the path we followed, i.e., looking at "stop."
It has an option for an array, though limited to 4 sequences according to API docs...
https://platform.openai.com/docs/api-reference/chat/create#chat-create-stop
Oh well that explains it then. Yeah I looked into it but couldn't figure it out since stop is valid, but their arbitrary restriction to 4 sequences would def be the cause then, yeah.
Not really sure what the best way to resolve this is though, since lots of those sequences are needed, especially with completion mode.
If you need a solution right now all the stop sequences are in control_flow_functions.py, you can find all for "stop": and truncate the lists. Maybe I'll add some custom logic that truncates the stop list if the API is OAI....
You could create a "stop" configuration item, note the restriction, and default to the current list. Since you already have a pretty comprehensive configuration, this would allow you to also recommend the setting per use case.
The interesting thing is that my colleague and I thought we turned it down and even tried to remove the stop parameter altogether from the API call. In hindsight, we must not have been successful, one plausible excuse, we were in a hurry. ;)
alright a recent push should fix this issue. However, unfortunately, since GPT-3.5's context window is so abysmally small, I don't think it can run the pipeline anyway (I can't get past qgen due to context window limits. Paragraph judging works though).
This means that you'd be limited to using GPT-4 Turbo if you had to use OpenAI... which would be... expensive.
Closing because I believe the issue's fixed, unless someone reports otherwise in which case I will reopen