Web GUI for longchat

Question

Web GUI for longchat

VVNMA opened this issue a year ago · comments

Hi, thanks for the great job!
But I'd like to know is there a web GUI for this longchat model, it seems that the model worker in Vicuna does not support this model directly, there should be a condense ratio process.

Dacheng Li · Answer 1 · Tue Jul 04 2023 11:48:33 GMT+0800 (China Standard Time)

@VVNMA Thanks for trying out! LongChat is compatible in FastChat that we have implemented the monkey patch in the load_model function. The model worker is using load_model so it should work. Let me know if it does not work and we will take time to fix it!

VVNMA · Answer 2 · Wed Jul 05 2023 14:49:52 GMT+0800 (China Standard Time)

@DachengLi1 Thanks. I saw the replace in the model adapter. But for the gradio server, I have to modify the input and max output token to fit the long chat.

Jalaj Thanaki · Answer 3 · Sat Jul 15 2023 16:59:30 GMT+0800 (China Standard Time)

How can I see an option on UI to increase the context length from 1024 to 16K?