Giters

zetavg / LLaMA-LoRA-Tuner

UI tool for fine-tuning and testing your own LoRA models base on LLaMA, GPT-J and more. One-click run on Google Colab. + A Gradio ChatGPT-like Chat UI to demonstrate your language models.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Executing tuned Model locally

muni-wobo opened this issue · comments

commented

I was able to tune the model with our data on Google colab.
We would like to run locally, build a REST API for other applications to use the same. Is it possible to download the trained model , run locally and interact with the model through direct Python call, instead of using the UI?. Can you point me to any sample code?