smj0 / DC-Match

ACL-2022 paper: Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents.

DC-Match

Pytorch implementation of the ACL-2022 (findings) paper: Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents.

Environments

• Python 3.9.7

• pytorch 1.10.2

• transformers 4.17.0

• datasets 2.0.0

• RTX 3090 GPU & Titan RTX

• CUDA 11.4

Data

All the processed datastes used in our work are available at Google Drive or Baidu Pan (extract code: w2op), including QQP, MRPC, and Medical-SM.

Usage

• Download raw datasets from the above data links and put them into the directory raw_data like this:

   --- raw_data
     |
     |--- medical
     |
     |--- mrpc
     |
     |--- qqp
  

• We have tried various pre-trained models. The following models work fine with our code:

  • model names for QQP and MRPC:

    • roberta-base
    • roberta-large
    • bert-base-uncased
    • bert-large-uncased
    • albert-base-v2
    • albert-large-v2
    • microsoft/deberta-large
    • microsoft/deberta-base
    • funnel-transformer/medium

  • model names for Medical:

    • hfl/chinese-macbert-base
    • hfl/chinese-macbert-large
    • hfl/chinese-roberta-wwm-ext
    • hfl/chinese-roberta-wwm-ext-large

• Pre-process datasets.

  PYTHONPATH=. python ./src/preprocess.py -raw_path raw_data/mrpc
  

  PYTHONPATH=. python ./src/preprocess.py -raw_path raw_data/qqp
  

  PYTHONPATH=. python ./src/preprocess.py -raw_path raw_data/medical
  

• Training and Evaluation (Baseline)

  • MRPC

  PYTHONPATH=. python -u src/main.py \
      -baseline \
      -task mrpc \
      -model roberta-large \
      -num_labels 2 \
      -batch_size 16 \
      -accum_count 1 \
      -test_batch_size 128 \
      >> logs/mrpc.roberta_large.baseline.log
  

  • QQP

  PYTHONPATH=. python -u src/main.py \
      -baseline \
      -task qqp \
      -model roberta-large \
      -num_labels 2 \
      -batch_size 16 \
      -accum_count 4 \
      -test_batch_size 128 \
      >> logs/qqp.roberta_large.baseline.log
  

  • Medical

  PYTHONPATH=. python -u src/main.py \
      -baseline \
      -task medical \
      -model hfl/chinese-roberta-wwm-ext-large \
      -num_labels 3 \
      -batch_size 16 \
      -accum_count 4 \
      -test_batch_size 128 \
      >> logs/medical.roberta_large.baseline.log
  

• Training and Evaluation (DC-Match)

  • MRPC

  PYTHONPATH=. python -u src/main.py \
      -task mrpc \
      -model roberta-large \
      -num_labels 2 \
      -batch_size 16 \
      -accum_count 1 \
      -test_batch_size 128 \
      >> logs/mrpc.roberta_large.log
  

  • QQP

  PYTHONPATH=. python -u src/main.py \
      -task qqp \
      -model roberta-large \
      -num_labels 2 \
      -batch_size 16 \
      -accum_count 4 \
      -test_batch_size 128 \
      >> logs/qqp.roberta_large.log
  

  • Medical

  PYTHONPATH=. python -u src/main.py \
      -task medical \
      -model hfl/chinese-roberta-wwm-ext-large \
      -num_labels 3 \
      -batch_size 16 \
      -accum_count 4 \
      -test_batch_size 128 \
      >> logs/medical.roberta_large.log
  

Citation

@article{zou2022divide,
         title={Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents},
         author={Zou, Yicheng and Liu, Hongwei and Gui, Tao and Wang, Junzhe and Zhang, Qi and Tang, Meng and Li, Haixiang and Wang, Daniel},
         journal={arXiv preprint arXiv:2203.02898},
         year={2022}
}