Unable to maintain chat history and continuous chat
Rkyzzy opened this issue · comments
Hi! Thanks very much about the great work.
Following the instruction provided, I successfully deploy llama7b chat and llama13b chat on my Jetson AGX Orin Device.
For single conversation, it works just fine. However, it seems that it does not have the ability to maintain chat history and do continuous chat. It will forget the previous context. Is it the problem with llama 2 model itself or with the engine. Any suggestion on how to solve this? @RaymondWang0 , Thanks in advance!
Hi @Rkyzzy, thank you for your interest in our work. The issue you noticed arose due to our previous implementation. But the good news is, we've addressed it. TinyChatEngine should be able to remember past contexts properly now. Please feel free to try it out!