Chinese clinical named entity recognition (CNER) using pre-trained BERT model
We pre-trained BERT model to improve the performance of Chinese CNER. Different layers such as Long Short-Term Memory (LSTM) and Conditional Random Field (CRF) were used to extract the text features and decode the predicted tags respectively. And we also proposed a new strategy to incorporate dictionary features into the model. Radical features of Chinese characters were also used to improve the model performance.
To replicate the result of CCKS-2018 dataset
python main.py \
--data_dir =data/ccks_2018 \
--bert_model =model/ \
--output_dir =./output \
--terminology_dicts_path =" {'medicine':'data/ccks_2018/drug_dict.txt','surgery':'data/ccks_2018/surgery_dict.txt'}" \
--radical_dict_path data/radical_dict.txt \
--constant =0 \
--add_radical_or_not =True \
--radical_one_hot =False \
--radical_emb_dim =20 \
--max_seq_length =480 \
--do_train =True \
--do_eval =True \
--train_batch_size =6 \
--eval_batch_size =4 \
--hidden_dim =64 \
--learning_rate =5 e-5 \
--num_train_epochs =5 \
--gpu_id =3 \
Method
P
R
F1
FT-BERT+BiLSTM+CRF
88.57
89.02
88.80
+dictionary
88.58
89.17
88.87
+radical(one-hot encoding)
88.51
89.39
88.95
+radical(random embedding)
89.24
89.11
89.17
+dictionary +radical
89.42
89.22
89.32
ensemble
89.59
89.54
89.56
Team Name
Method
F1
Yang and Huang (2018)
CRF(feature-rich + rule)
89.26
heiheihahei
LSTM-CRF(ensemble)
88.92
Luo et al.(2018)
LSTM-CRF(ensemble)
88.63
dous12
-
88.37
chengachengcheng
-
88.30
NUBT-IBDL
-
87.62
Our
FT-BERT+BiLSTM +CRF+Dictionary(ensemble)
89.56
Method
P
R
F1
FT-BERT+BiLSTM+CRF
91.64
90.98
91.31
+dictionary
91.49
90.97
91.23
+radical(one-hot encoding)
91.83
90.80
91.35
+radical(random embedding)
92.07
90.77
91.42
+dictionary+radical
91.76
90.88
91.32
ensemble
92.06
91.15
91.60
Team Name
Method
F1
Qiu et al. (2018b)
RD-CNN-CRF
91.32
Wang et al. (2019)
BiLSTM-CRF+Dictionary
91.24
Hu et al. (2017)
BiLSTM-FEA(ensemble)
91.03
Zhang et al. (2018)
BiLSTM-CRF(mt+att+ms)
90.52
Xia and Wang (2017)
BiLSTM-CRF(ensemble)
89.88
Ouyang et al. (2017)
BiRNN-CRF
88.85
Li et al. (2017)
BiLSTM-CRF(specialized +lexicons)
87.95
Our
FT-BERT+BiLSTM +CRF+Dictionary(ensemble)
91.60