long-term memory is expensive.
punksta opened this issue · comments
Hi, amazing work!
As I can tell, long term memory requires sending all memories to gpt-4 to fetch the more relevant memories, for each input.
If it stoes many memories, the cost of this feature will be very high.
Is there a way of optimizing this? I am thinking about using a cheaper model.
Hey 👋 thanks!
I'm looking at options for improving this but I haven't had a lot of time to work on it. There's definitely optimisations that can be made, but any pre-filtering done on the data (which ultimately is the only solution) means letting something that isn't GPT determine how relevant a memory is.
It's worth trying with GPT-3.5 for memory recall though (you can replace openai.GPT4 in the memory module code), other parts of GPTChat (commands, plugins) don't work particularly well but memory recall might be ok.
Hello there!
I saw your reddit post the other day where you mentioned this project. In the comments below I found the hint to use Vector Databases for Memory Storing.
I'm coding a kind of similiar project like you, but in Javascript.
I just wanted to share this video to you, because it helped me to integrate a vector db (I never worked with on before). Maybe it'll be useful to you: https://www.youtube.com/watch?v=tp0bQNDtLPc
And don't quit your project, it's a perfect way of learning because it keeps you motivated.