Can i add some other models to fauxpilot except codegen series, like gpt-neo.
Inoria-cn opened this issue · comments
I think fauxpilot is a framework capable of deploying a LLM model locally. Can I also use other models with fauxpilot, similar to how I can download pre-trained models from Huggingface and load them into stable-diffusion-webui?
Hello there, thanks for opening your first issue. We welcome you to the FauxPilot community!
You should be able to use any of the models FasterTransfomer supports as well as (using the Python backend) anything supported in HuggingFace. But it isn't supported out of the box yet, and there are some assumptions in the proxy about the tokenizer used that would need to be fixed.
hello，in my understanding，fauxpilot uses the gptj inerface in triton ft backend due to codegen shares the same arch with gptj. If i want to add a model that has slightly different arch with those already included in triton ft backend，i should modify the src and rebuild. Is my understanding correct？