Giters

deepset-ai / FARM

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

Home Page:https://farm.deepset.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Support for DeBERTa V2

ishabi opened this issue · comments

commented

Question
Will DeGERTa V2 be supported by FARM?

Additional context
I was experimenting DeBERTa language model to train NER model, but it always fails. Will it be supported in future?

Thanks

commented

Hey @ishabi there is currently no support for DeBERTa in FARM.

Would you like to contribute this?
It should not be too complex since you can use the underlying transformers implementation, you just have to create a wrapper in our farm.modelling.language_model script and write some test.

Any help would be appreciated.

commented

Hey @ishabi it seems adding DeBERTa v1 and v2 will be more difficult to implement because we currently only support fast tokenizers.

A fast tokenizer for DeBERTa v1 was merged 5 days ago (huggingface/transformers#11387). But there’s still no fast tokenizer for DeBERTa v2 (huggingface/transformers#11529). I think we might also have to wait until these are in the next release.

commented

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 21 days if no further activity occurs.