Unable to send BinaryPart to OpenAI Completions API
pilsnerbeer opened this issue · comments
Trying to directly send a image/png to OpenAI completions API.
(Model: Gpt-4o / Mini)
Snippet:
if imageData != nil {
historyWithBinary = append(historyWithBinary, llms.MessageContent{
Role: llms.ChatMessageTypeHuman,
Parts: []llms.ContentPart{
llms.BinaryPart("image/png", imageData),
},
})
}
choices, err := client.GenerateContent(context.Background(), historyWithBinary, llms.WithTools([]llms.Tool{FileWriteTool}))
Returns Error:
Error generating content: API returned unexpected status code: 400: Invalid value:
'binary'. Supported values are: 'text', 'image_ url', and 'audio_url'.
This seems to be only problematic with OpenAI. Gemini and Ollama worked fine when i tested it with the same snippet.
According to https://platform.openai.com/docs/guides/vision doc sending images directly to vision API shoudl be possible so the error is not clear to me