jiachenwestlake / Cross-Domain_NER

Cross-domain NER using cross-domain language modeling, code for ACL 2019 paper

Cross-Domain_NER

Cross-domain NER using cross-domain language modeling, code for ACL 2019 paper.

Introduction

NER is a fundamental task in NLP. Due to the limitation of labeled resources, cross-domain NER has been a challenging task. Most previous work concentrates on the supervised scenario, making use of labeled data for both source and target domains. A disadvantage of such setting is that they can not train for domains which have no labeled data.

We address this issue, using cross-domain LM as a bridge cross-domains for NER domain adaptation. Performing cross-task and cross-domain transfer by designing a novel Parameter Generation Network.

Experiments on CBS SciTech News Dataset show that our model can effectively allow unsupervised domain adaptation, while also can deriving supervised domain adaption between domains with completely different entity types (i.e. news vs. biomedical).

The naive baseline of Single Task Model (STM in paper) mostly followed NCRF++.

For more details, please refer to our paper:

Cross-Domain NER using Cross-Domain Language Modeling
Chen Jia, Xiaobo Liang and Yue Zhang*
(* Corresponding Author)
ACL 2019

Requirements

Python 2 or 3 
PyTorch 0.3

The memory of one GPU should be no less than 8GB to fit the model.

Pretrained Embeddings

GloVe 100-dimension word vectors (Cite from Here).

DataSet

Source-domain:

CoNLL-2003 English NER data.

Target-domain

• Unsupervised: CBS SciTech News (test set) (In: `\unsupervised_domain_adaptation\data\news_tech\tech_test).
  
• Supervised: BioNLP13PC dataset and BioNLP13CG dataset.

LM raw data

Retures domain raw data is released together with the CoNLL-2003 dataset.
SciTech news domain raw data Download.
Reuters news domain raw data Download.

Usage

Command

\supervised_domain_adaptation, \unsupervised_domain_adaptationand \combined_SDA_and_UDA can use the following command to make it run.

python main.py --config train.NER.config

The file train.NER.config contains dataset path and model hyperparameters following NCRF++.

Input format

• We recommand using the IBOES label style for NER dataset.
• We recommand using an input style of one-sentence-per-line for raw data with word segmentation.

Cite:

If you use our data or code, please cite our paper as follows:

@inproceedings{jia2019cross,
  title={Cross-domain ner using cross-domain language modeling},
  author={Jia, Chen and Liang, Xiaobo and Zhang, Yue},
  booktitle={Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  pages={2464--2474},
  year={2019}
  organization={Association for Computational Linguistics}
}

Update

• V2. Combining supervised scenario and unsupervised scenario in \combined_SDA_and_UDA.
• V1. The previous supervised scenario in \supervised_domain_adaptation;
  The previous unsupervised scenario in \unsupervised_domain_adaptation;