Can't pass genai.Content to model.GenerateContentStream
nullcache opened this issue · comments
I want to implement a api which need user pass all chat history every chat, but I can't pass genai.Content to model.GenerateContentStream. It just accept like genai.Text, which doesn't contain role information
If you look at the chat implementation, you'll see it's already doing that, and it takes care of the roles for you. Do you need something different?
When I check generative-ai-js, I find GenerateContentStream can pass Part or Content, which make it more flexible.
I mean, I can see the session implementation. I just want write an api with different input/output request params, I think every query should be stateless, and the history keep in the frontend. User pass all historys in every query. So I think the sdk should allow users to manage history by themselves or just can pass all historys to GenerateContentStream.
The only way I find to implement my needs I think is StartChat in every request (every request to me), and append old historys to ChatSession.History slice, then SendMessageStream. Does it meet expectations ?
model := g.Client.GenerativeModel(model_name)
model.SetTemperature(0.9)
model.SetTopP(0.2)
model.SetTopK(1)
model.SetMaxOutputTokens(4000)
model.SetCandidateCount(1)
chatSession := model.StartChat()
history := g.ChatCompletionMessageToGeminiHistory(messages[:len(messages)-1])
chatSession.History = history
stream := chatSession.SendMessageStream(ctx, genai.Text(messages[0].Content))
model := g.Client.GenerativeModel(model_name) model.SetTemperature(0.9) model.SetTopP(0.2) model.SetTopK(1) model.SetMaxOutputTokens(4000) model.SetCandidateCount(1) chatSession := model.StartChat() history := g.ChatCompletionMessageToGeminiHistory(messages[:len(messages)-1]) chatSession.History = history
stream := chatSession.SendMessageStream(ctx, genai.Text(messages[0].Content))
So as I have metioned, is it the only to resolve? Of course it is a way, but using StartChatin this way is maybe not reasonable semantically.
And I think it's better to provide a tool function in this library.
I just notice that js implementation is more flexible, so I think maybe toContent interface is better than toPart?
@nullcache it's not clear what you're asking for beyond what ChatSession (see example: https://pkg.go.dev/github.com/google/generative-ai-go/genai#example-ChatSession) already provides.
Can you please clarify?
Note that GenerateContent is designed for one-shot interactions, and you can still provide context by including it in the prompt. If you need a chat-like interaction with roles, that's what ChatSession is for; is there something you need that it doesn't provide?
The Chat API is designed to let you modify the history however you want before your next request. That is why the History field and all of the Content fields are exported.
It's true that there is no way using this API to provide more than one Content to a GenerateContent or GenerateContentStream call. We may consider adding this functionality later if there is enough demand for it.
Note that you can always use the lower-level client in cloud.google.com/go/ai/generativelanguage/apiv1 to get full access to the raw API.