Giters

tloen / alpaca-lora

Instruct-tune LLaMA on consumer hardware

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Can anyone post already trained model?

HCBlackFox opened this issue · comments

commented

Hello, you can find this 13B one here: https://huggingface.co/samwit/alpaca13B-lora

Otherwise, there is the 7B one here: https://huggingface.co/tloen/alpaca-lora-7b

Please note these are LoRA models they need the base model to work.

And here is the base model for the 7B: https://huggingface.co/decapoda-research/llama-7b-hf

commented

Thank you

Hello, you can find this 13B one here: https://huggingface.co/samwit/alpaca13B-lora

Otherwise, there is the 7B one here: https://huggingface.co/tloen/alpaca-lora-7b

Please note these are LoRA models they need the base model to work.

And here is the base model for the 7B: https://huggingface.co/decapoda-research/llama-7b-hf

Thank you

commented

Is there a 30B-4bit lora out there? I think I read somewhere that finetuning in 4bit might not be supported?

commented

Hello, you can find this 13B one here: https://huggingface.co/samwit/alpaca13B-lora

Otherwise, there is the 7B one here: https://huggingface.co/tloen/alpaca-lora-7b

Please note these are LoRA models they need the base model to work.

And here is the base model for the 7B: https://huggingface.co/decapoda-research/llama-7b-hf

can the original LLaMA-7B weights (consolidated.00.pth) be used? or can I convert it to hf?

commented

Any links for models trained w/3-epochs on the new cleaned dataset?

commented

Any links for models trained w/3-epochs on the new cleaned dataset?

I just finished training this 13B one but haven't got it to work yet (I'm using multiple GPUs so maybe that's the issue) https://huggingface.co/mattreid/alpaca-lora-13b

commented

@collant can you help me understand how can I load the Lora model trained with the 52k dataset and use it to train on another data.json?

In finetune.py I can find the loading of the llama 7b model

model = LlamaForCausalLM.from_pretrained(
    "decapoda-research/llama-7b-hf",
    load_in_8bit=True,
    device_map=device_map,
)
tokenizer = LlamaTokenizer.from_pretrained(
    "decapoda-research/llama-7b-hf", add_eos_token=True
)

and after the lora config obj is being created

config = LoraConfig(
    r=LORA_R,
    lora_alpha=LORA_ALPHA,
    target_modules=TARGET_MODULES,
    lora_dropout=LORA_DROPOUT,
    bias="none",
    task_type="CAUSAL_LM",
)
model = get_peft_model(model, config)

does loading the Lora model from hf involves calling another function and loading that checkpoint? I can see that there is a save_pretrained function, maybe I need to load the Lora model via this? Sorry if this sounds confusing

edit: after a little bit more google I found this load_attn_procs function, maybe it's something around here

edit2: it seems that it was inside generate.py all along

    model = LlamaForCausalLM.from_pretrained(
        "decapoda-research/llama-7b-hf",
        load_in_8bit=True,
        torch_dtype=torch.float16,
        device_map="auto",
    )
    model = PeftModel.from_pretrained(
        model, "tloen/alpaca-lora-7b",
        torch_dtype=torch.float16
    )
commented

@collant can you help me understand how can I load the Lora model trained with the 52k dataset and use it to train on another data.json?

In finetune.py I can find the loading of the llama 7b model

model = LlamaForCausalLM.from_pretrained(
    "decapoda-research/llama-7b-hf",
    load_in_8bit=True,
    device_map=device_map,
)
tokenizer = LlamaTokenizer.from_pretrained(
    "decapoda-research/llama-7b-hf", add_eos_token=True
)

and after the lora config obj is being created

config = LoraConfig(
    r=LORA_R,
    lora_alpha=LORA_ALPHA,
    target_modules=TARGET_MODULES,
    lora_dropout=LORA_DROPOUT,
    bias="none",
    task_type="CAUSAL_LM",
)
model = get_peft_model(model, config)

does loading the Lora model from hf involves calling another function and loading that checkpoint? I can see that there is a save_pretrained function, maybe I need to load the Lora model via this? Sorry if this sounds confusing

edit: after a little bit more google I found this load_attn_procs function, maybe it's something around here

edit2: it seems that it was inside generate.py all along

    model = LlamaForCausalLM.from_pretrained(
        "decapoda-research/llama-7b-hf",
        load_in_8bit=True,
        torch_dtype=torch.float16,
        device_map="auto",
    )
    model = PeftModel.from_pretrained(
        model, "tloen/alpaca-lora-7b",
        torch_dtype=torch.float16
    )

Have you found solution？ #44 I found this may help? But I still confuse with what is

commented

Any links for models trained w/3-epochs on the new cleaned dataset?

+1

commented

Please, report @larasatistevany for spamming.

https://support.github.com/contact/report-abuse?category=report-abuse&report=larasatistevany

-> I want to report abusive content or behavior.
-> I want to report SPAM, a user that is disrupting me or my organization's experience on GitHub, or a user who is using my personal information without my permission
-> A user is disrupting me or my organization's experience and productivity by posting SPAM off-topic or other types of disruptive content in projects they do not own.

Put this in the form:

spamming in issue comments
https://github.com/tloen/alpaca-lora/issues/52#issuecomment-1570561693
https://github.com/tloen/alpaca-lora/issues/52#issuecomment-1571059071

Thanks!