This repository contains the processing and training scripts for an object detector based on Faster R-CNN, specifically for the Chest Imagenome dataset.
The object detector is trained, evaluated, and tested on the Chest Imagenome dataset. The dataset comprises 29 anatomical regions extracted from the MIMIC-CXR image dataset for object detection.
To use this repository, ensure the following datasets are prepared:
- Chest Imagenome: Utilize the scene graph folder of the silver datasets, which contains JSON files detailing 29 anatomical regions with bbox coordinates and attributes.
- MIMIC-CXR JPG Dataset: Provides images for the Chest Imagenome.
- MIMIC-CXR Reports Dataset: Generates train, test, and validation datasets for subsequent vision-language model frameworks.
Scripts under object_detector/src/dataset
are used for processing and creating datasets:
constants.py
: Stores 29 regions defined by the Chest Imagenome dataset and images to be ignored due to detection failure.identify_faulty_data.py
: Detects faulty bounding boxes (bboxes) in the Chest Imagenome dataset. Outputs includefaulty_bbox_coordinates.csv
andfaulty_bbox_names.csv
for bboxes with incorrect dimensions or labels.
compute_stats_dataset.py
: Counts data in the Chest Imagenome dataset, like the proportion of abnormal bboxes, correspondence between bboxes and phrases, etc.create_dataset_new.py
: Generates CSV files for train, validation, and test datasets, excluding detected faulty bboxes. The train dataset CSV includes subject_id, study_id, image_id, path, bbox_coordinates, bbox_labels, bbox_phrases, bbox_phrases_exists, and bbox_is_abnormal. Validation and test datasets also include the correspondingreport
.
Scripts in object_detector/src/object_detector
are based on PyTorch's Faster R-CNN with modifications for the Chest Imagenome dataset:
image_list.py
: Adjusted to handle the consistent image size of the MIMIC-CXR JPG images.image_dataset.py
: Resize image size to 512×512 for customized anchors.rpn.py
androi_heads.py
: Modified to add loss calculation during evaluation.object_detector.py
: Tailored for the Chest Imagenome dataset.- Uses a ResNet50 model pre-trained on chest x-rays.
- Accepts grayscale images (1 channels instead of 3).
- The backbone ends before the final two layers (AdaptiveAvgPool2d and Linear) of the standard ResNet50, focusing only on feature extraction.
- Expects tensors of shape [batch_size, 1, H, W], where H and W are typically 512.
training_script_object_detector.py
: Script for training and evaluating the object detector. Outputs include a "runs" folder for saving checkpoints.
- Set up the paths for the dataset and training output checkpoints in
object_detector/src/path_datasets_and_weights.py
. - Follow the instructions in each script under the
object_detector/src/dataset
andobject_detector/src/object_detector
directories.