Giters

zetavg / LLaMA-LoRA-Tuner

UI tool for fine-tuning and testing your own LoRA models base on LLaMA, GPT-J and more. One-click run on Google Colab. + A Gradio ChatGPT-like Chat UI to demonstrate your language models.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support smaller models

paplorinc opened this issue · comments

commented

How difficult would it be to support 3-4 bit models, e.g. https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml or https://huggingface.co/decapoda-research/llama-smallint-pt/blob/main/llama-7b-3bit.pt - or non-conversational ones that are smaller than the 7b?

commented

databricks/dolly-v2-3b is pretty small and seems to be working for prediction, but can't make it work for fine-tuning (not sure what LoRA Target Modules to set)

edit: maybe they're: https://github.com/huggingface/peft/blob/2822398fbe896f25d4dac5e468624dc5fd65a51b/src/peft/utils/other.py#L220

commented

forget it, the project name clearly states LLaMA :)