DoodleJZ / LIMIT-BERT

Source code for "LIMIT-BERT : Linguistics Informed Multi-Task BERT" published at Findings of EMNLP 2020

LIMIT-BERT

Contents

1. Requirements
2. Training

Requirements

• Python 3.6 or higher.
• Cython 0.25.2 or any compatible version.
• PyTorch 1.0.0+.
• EVALB. Before starting, run make inside the EVALB/ directory to compile an evalb executable. This will be called from Python for evaluation.
• pytorch-transformers PyTorch 1.0.0+ or any compatible version.

Pre-trained Models (PyTorch)

The following pre-trained models are available for download from Google Drive:

• LIMIT-BERT: PyTorch version, same setting with BERT-Large-WWM，loading model with pytorch-transformers.

Training

To train LIMIT-BERT, simply run:

sh run_limitbert.sh

Evaluation Instructions

To test after setting model path:

sh test_bert.sh