Code: 1001, Text = openai embedding request error
elvin-du opened this issue · comments
when I run ./target/release/askbend -c askbend.toml --rebuild
cmd , I got the error as blow. how to fix this?
[2023-10-16T02:07:41Z INFO ] embedding 605/1936, time: 513.095121ms
Error: APIError: ResponseError with 1001: openai embedding request error:Internal. Code: 1001, Text = openai embedding request error: ApiError("{\"error\":{\"message\":\"This model's maximum context length is 8191 tokens, however you requested 8916 tokens (8916 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.\",\"type\":\"invalid_request_error\",\"param\":null,\"code\":null}}").
context length is 8191 tokens
please split your content to more small piece.
content
means the content of one file? I just put 390 files on the path which set in config file.
One of your file length is larger than 8191(prompts length + content length = 8916).
We should split this file into two parts, make sure not near 8191.