Remove target_module LoRA mapping for Phi-2 model
arnavgarg1 opened this issue · comments
Arnav Garg commented
The current PEFT LoRA modules map to the wrong target modules for Phi-2 since the architecture was recently updated.
I created a PR on the PEFT repo to update this to new target modules: huggingface/peft#1375
For now, we will fall back to target_modules for Phi-2 that lead to good model performance.