High API usage?
Helmasko opened this issue · comments
I noticed using unsaged.com does a lot of API requests even with very short conversations which shouldn't cost as much. I can see $0.10 increase after 10-30 mins of having used the tool (1 or 2 messages only).
What’s your OS/browser? I was analyzing streaming messages this past week trying to fix a streaming bug and did not notice anything out of the ordinary. I will still try to reproduce with your setup and see if it’s browser related (maybe sending double requests or something)
Windows 11, Microsoft Edge Stable. I just tried it again on a chat with some backlog (10 or so messages) and got a $0.03 increase.
- Check usage
- Send a single message totalling to 11 tokens (including the response I got)
- Wait 10-30 minutes (it doesn't happen instantly)
- Check usage and compare with before
So I did some tests and was not able to see any issues.
I sent two messages. The first request had 38 tokens and the second one had 59 tokens. The second message was a reply to the first message.
Here are the results:
The requests were not sent multiple times. They both also had the correct number of tokens.
See if you can test it as well. The newest commit turned on printing the tokenCount by default.
You could try to use https://developers.cloudflare.com/ai-gateway/ for your unsaged instance so you can check what requests are coming from that instance and what request could come from something else. I just added a short docs entry regarding that #82