finetune on 4096 context length

[Mohamed Rashad]

How to finetune Falcon-7B-Instruct on input or outputs of 4096 context length ?
how much VRAM i will need ?

[yuhai-china]

Does it work to set --cutoff_len=4096?

[Richard Burleigh]

Yes, I'm currently fine-tuning the 7B instruct model with 4096 and it's working fine.

Although I'm making the assumption that there's no truncation elsewhere in the code.