Can I replace the base model with longformer? Is it a simple replacement or does the code also need to be changed? Thank you for your answer.
Struggle-lsl opened this issue · comments
Can I replace the base model with longformer? Is it a simple replacement or does the code also need to be changed? Thank you for your answer.
Hi,
I do believe some minor code change is needed, as it only supports bert or Roberta for now. But since everything is based on Huggingface, the change should be minimal. You can just follow the bert examples in the code and edit it to be long former accordingly.
Hi,
I do believe some minor code change is needed, as it only supports bert or Roberta for now. But since everything is based on Huggingface, the change should be minimal. You can just follow the bert examples in the code and edit it to be long former accordingly.
Hello, longformer is based on robert. My understanding is that you can directly replace the model file without changing the code.
If the two models share the same code but just different parameters, then yes; if there is additional modification, then you'll probably need to change the code too.
Stale issue message