finetune on 4096 context length
MohamedAliRashad opened this issue · comments
How to finetune Falcon-7B-Instruct on input or outputs of 4096 context length ?
how much VRAM i will need ?
Does it work to set --cutoff_len=4096?
Yes, I'm currently fine-tuning the 7B instruct model with 4096 and it's working fine.
Although I'm making the assumption that there's no truncation elsewhere in the code.