[Feature Request] Increase the calculation of tokens
xuzeyu91 opened this issue · comments
zyxucp commented
Context / Scenario
When using Kernel Memory, whether it is Import or Ask, I hope to increase the consumption of returning tokens, so that I can analyze the cost of question and answer scenarios
The problem
Increase the return value of token consumption
Proposed solution
Increase the return value of token consumption
Importance
would be great to have
ksemenenko commented
I found this solution, try to set MaxTokenTotal
}).WithAzureOpenAITextGeneration(new AzureOpenAIConfig()
{
Endpoint = azureOpenAi.SummarizationModel.Endpoint,
APIKey = azureOpenAi.SummarizationModel.ApiKey,
Deployment = azureOpenAi.SummarizationModel.DeploymentOrModelId,
Auth = AzureOpenAIConfig.AuthTypes.APIKey,
// try this
MaxTokenTotal = 25000,
MaxRetries = 1230
}, httpClient: httpClient)
zyxucp commented
我找到了这个解决方案,尝试设置 MaxTokenTotal
}).WithAzureOpenAITextGeneration(new AzureOpenAIConfig() { Endpoint = azureOpenAi.SummarizationModel.Endpoint, APIKey = azureOpenAi.SummarizationModel.ApiKey, Deployment = azureOpenAi.SummarizationModel.DeploymentOrModelId, Auth = AzureOpenAIConfig.AuthTypes.APIKey, // try this MaxTokenTotal = 25000, MaxRetries = 1230 }, httpClient: httpClient)
What I hope to know is how many tokens are consumed per request
Example
"usage": {
"completion_tokens": 207,
"prompt_tokens": 15,
"total_tokens": 222
}
Devis Lucato commented
Please feel free to use the poll at #532 to vote for this feature