GeorgeLuImmortal / PUnifiedNER

PUnifiedNER:a Prompt-Based NER System Jointly Crossing Multiple NER Datasets (AAAI2023)

This repository is a replicate implementation based on paper PUnifiedNER:a Prompt-Based NER System Jointly Crossing Multiple NER Datasets (Accepted to the main conference of AAAI2023)

Datasets

People Daily (dev)

People Daily (test)

People Daily (train)

Other datasets can be found under ./ner_datasets/

Usage

Dependencies

Tested Python 3.6, and requiring the following packages, which are available via PIP:

• Required: numpy >= 1.19.5
• Required: scikit-learn >= 0.21.1
• Required: pandas >= 1.1.5
• Required: torch >= 1.9.0
• Required: transformers >= 4.8.2
• Required: datasets>=1.14.0
• Required: nltk>=3.6.5

Quick Start

1. Prefix language model adapatation: sh_lm_adaption_1e-4_multi_gpus.sh

2. Multi-dataset PUnifiedNER tuning: sh_train_all_multi_gpus.sh

3. Test set evaluation: sh_eval_{dataset_name}.sh

4. Quick demo using Gradio: PUnifiedNER_Demo_Gradio.ipynb

Demo video