horseee / LLM-Pruner

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support LLaMA, Llama-2, BLOOM, Vicuna, Baichuan, etc.

Home Page:https://arxiv.org/abs/2305.11627

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

I encountered the following error message when I assign iterative_steps = 2 during baichuan-7B pruning

yangd85 opened this issue · comments

Traceback (most recent call last):
File "/home/jovyan/honor/yangdong/LLM-Pruner-main/examples/baichuan.py", line 342, in
main(args)
File "/home/jovyan/honor/yangdong/LLM-Pruner-main/examples/baichuan.py", line 229, in main
pruner.step()
File "/home/jovyan/honor/yangdong/LLM-Pruner-main/LLMPruner/torch_pruning/pruner/algorithms/metapruner.py", line 186, in step
for group in self.prune_local():
File "/home/jovyan/honor/yangdong/LLM-Pruner-main/LLMPruner/torch_pruning/pruner/algorithms/metapruner.py", line 245, in prune_local
imp = self.estimate_importance(group, ch_groups=ch_groups, consecutive_groups=consecutive_groups)
File "/home/jovyan/honor/yangdong/LLM-Pruner-main/LLMPruner/torch_pruning/pruner/algorithms/metapruner.py", line 190, in estimate_importance
return self.importance(group, ch_groups=ch_groups, consecutive_groups=consecutive_groups)
File "/opt/miniconda3/envs/flash/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/jovyan/honor/yangdong/LLM-Pruner-main/LLMPruner/pruner/hf_baichuan_pruner.py", line 180, in call
local_norm = local_norm[idxs]
IndexError: index 3840 is out of bounds for dimension 0 with size 3840

Our code currently does not support iterative_steps > 1 for baichuan. Please try iterative_steps = 1.