kiranrohra / R2Gen

The official implementation of Generating Radiology Reports via Memory-driven Transformer at EMNLP-2020.

R2Gen

This is the implementation of Generating Radiology Reports via Memory-driven Transformer at EMNLP-2020.

Citations

If you use or extend our work, please cite our paper at EMNLP-2020.

@inproceedings{chen-emnlp-2020-r2gen,
    title = "Generating Radiology Reports via Memory-driven Transformer",
    author = "Chen, Zhihong and
      Song, Yan  and
      Chang, Tsung-Hui and
      Wan, Xiang",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2020",
}

Requirements

• torch==1.7.1
• torchvision==0.8.2
• opencv-python==4.4.0.42

Download R2Gen

You can download the models we trained for each dataset from here.

Datasets

We use two datasets (IU X-Ray and MIMIC-CXR) in our paper.

For IU X-Ray, you can download the dataset from here and then put the files in data/iu_xray.

For MIMIC-CXR, you can download the dataset from here and then put the files in data/mimic_cxr.

NOTE: The IU X-Ray dataset is of small size, and thus the variance of the results is large. There have been some works using MIMIC-CXR only and treating the whole IU X-Ray dataset as an extra test set.

Train

Run bash train_iu_xray.sh to train a model on the IU X-Ray data.

Run bash train_mimic_cxr.sh to train a model on the MIMIC-CXR data.

Test

Run bash test_iu_xray.sh to test a model on the IU X-Ray data.

Run bash test_mimic_cxr.sh to test a model on the MIMIC-CXR data.

Visualization

Run bash plot_mimic_cxr.sh to visualize the attention maps on the MIMIC-CXR data.