Add Gemini 1.5 Flash support
283142289 opened this issue · comments
@abi for gemini support, should the assemble_prompt functions be created for openai and gemini separately or in the same function using conditionals?
Ideally, we generate all prompts in OpenAI format and then, have unittested translation functions for other formats like Claude and Gemini.
Hi @abi, I am working on adding support for Gemini, have added the support but facing an issue if you could help please. Gemini requires the image to be in bytes format, which is creating problem in the write_logs function as bytes value is not JSON serializable while attempting to dump the assemble_prompt as JSON.
One solution I thought of was to convert it to base64 before dumping, either using a recursive function or directly accessing the keys. Is there any better alternative?
@naman1608 thanks for working on it. We pass images in base64 format to both OpenAI and Anthropic. Should be easy to convert an image to base64 with Python. We should generally store images in base64 for this project.