Add session persistence
dkvdm opened this issue · comments
Upon redeployment the bot shouldn't lose track of its context.
I've yet to look into how the browser UI handles multiple sessions, but I assume that there is some sort of unique ID you can set. If that is the case, a potential solution would be to utilize the chat_id as a unique session ID.
There is currently no context or session comparable to the browser chat available for the API. Every request has to include the history (that's why the token usage grows).
Histories would have to be saved locally. However I have mixed feelings about storing the conversations of my users.
Anyway, if @n3d1117 is a fan of the idea I think the implementation can be very straightforward by adding it to the UsageTracker, as a field to the usage_logs .json files "current_conversation". Similarly #71 can be included in an additional field "saved_conversations". I am happy to do it, if @n3d1117 thinks it's worthwhile. But we have to discuss if it's feasable leaving it in JSONs and also in memory, as the usage numbers currently are. But that could be easily changed to I/O from the JSONs, but that might slow the bot down. Probably should be moved to a database if the amount of data we want to store is larger than the few usage numbers right now.
As an addition to this feature I want to suggest a command /opt_out that prevents saving the current conversations of the user.
Thanks for the suggestion @dkvdm. I like the idea of keeping track of the conversations between deployments.
@AlexHTW good thinking. Not a big fan of adding a whole database though. For what it's worth, I just noticed python-telegram-bot offers a few persistence solutions out of the box: https://github.com/python-telegram-bot/python-telegram-bot/wiki/Making-your-bot-persistent
Maybe the built-in PicklePersistence could be useful here, for storing data in binary format instead of plain json. Haven't tested it but I feel like it could be used to store usage data as well!
Pickling is great, and super simple as you can just save/load python data structures, I love it.
A full look under the hood on how the webapp deals with sessions: https://github.com/terminalcommandnewsletter/everything-chatgpt