Giters

horseee / LLM-Pruner

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support LLaMA, Llama-2, BLOOM, Vicuna, Baichuan, etc.

Home Page:https://arxiv.org/abs/2305.11627

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Question related to the model tuning

shawnricecake opened this issue · comments

commented

Hi,

Great work first!

I am confused with the model tuning part.

According to the code, it seemed that you used the lora method.
This, in my opinion, will destroy the sparsity you have made in the original model after merging the lora weights to the model weights.

could you explain this?

Thanks
Shawn

commented

Hi @shawnricecake, LLM-Pruner is a structural method and thus produces a dense model after pruning.

commented

Hi @shawnricecake, LLM-Pruner is a structural method and thus produces a dense model after pruning.

Hi, thanks for your reply, so, the model weights after merge the lora weights will be dense?

the main contribution of paper is the structure pruning?

Thanks
Shawn