giddyyupp / coco-minitrain

a subset of coco dataset for faster experimentation

COCO minitrain

COCO minitrain is a curated mini training set (25K images ≈ 20% of train2017) for COCO. It is useful for hyperparameter tuning and reducing the cost of ablation experiments. minitrain's object instance statistics match those of train2017 (see the stats page). val2017 performance of a model trained on minitrain is strongly positively correlated with the performance of the same model trained on the full set, train2017 (see the Performance Correlation section below).

Reference

We introduced COCO minitrain in our ECCV'2020 paper. Please cite it, if you use COCO minitrain in your work:

N. Samet, S. Hicsonmez, E. Akbas, "HoughNet: Integrating near and long-range evidence for bottom-up object detection", ECCV 2020. arXiv 2007.02355.

Bibtex entry

@inproceedings{HoughNet, 
  author = {Nermin Samet and Samet Hicsonmez and Emre Akbas},
  title = {HoughNet: Integrating near and long-range evidence for bottom-up object detection},   
  booktitle = {European Conference on Computer Vision (ECCV)}, 
  year = {2020}, 
}

What's New

Feb 10, 2023

• We add a subset for Keypoint Detection Task. Our minitrain set for KP contains 15K images ≈ 20% of person_keypoints_train2017.

More information

COCO minitrain is a subset of the COCO train2017 dataset, and contains 25K images (about 20% of the train2017 set) and around 184K annotations across 80 object categories. We randomly sampled these images from the full set while preserving the following three quantities as much as possible:

• proportion of object instances from each class,
• overall ratios of small, medium and large objects,
• per class ratios of small, medium and large objects.

More information on minitrain statistics could be found in STATS.md.

Download

Object Detection & Segmentation

We share COCO style JSON file, and Pascal VOC style CSV file.

Json

CSV

Class Labels

Download the whole 25k dataset directly.

coco_minitrain_25k.zip [Huggingface] (maintained by @bryanbocao)

coco_minitrain_25k.zip [Kaggle] (maintained by @trungit)

Keypoint Detection

We share COCO style JSON file.

Json

Usage

If you want to sample according to your own needs (e.g. different number of images), run src/sample_coco.py with updated parameters.

Below script runs minicoco sampling to curated 25000 images and saves annotations (both bbox and segmentation) to instances_train2017_minicoco.json file.

cd src
python sample_coco.py --coco_path "path_to_your_coco_dataset" --save_file_name "instances_train2017_minicoco" --save_format "json" --sample_image_count 25000 --debug [--sample_kp]

Performance correlation of train2017 and minitrain

Object Detection Task

Object Detection performances. Models are trained on minitrain and evaluated on val2017:

Method	Backbone	Scale	AP	AP_50	AP_75	AP_S	AP_M	AP_L
Faster R-CNN	ResNet-50 w FPN	800	27.7	48.8	28.4	14.7	29.8	36.4
Mask R-CNN	ResNet-50 w FPN	800	28.5	49.5	29.4	14.7	30.7	37.6
RetinaNet	ResNet-50 w FPN	800	25.7	43.1	26.8	12.1	28.6	34.2
CornerNet	Hourglass-104	511	28.4	41.8	29.5	11.3	29.6	39.2
ExtremeNet	Hourglass-104	511	27.3	39.4	28.9	12.5	29.6	38.0

Object Detector performances trained on minitrain vs train2017. Models are evaluated on val2017.

Method	Backbone	Scale	minitrain AP	minitrain AP_50	minitrain AP_75	train2017 AP	train2017 AP_50	train2017 AP_75
Faster R-CNN	ResNet-50 w FPN	800	27.7	48.8	28.4	36.7	58.4	39.6
Mask R-CNN	ResNet-50 w FPN	800	28.5	49.5	29.4	37.7	59.2	40.9
RetinaNet	ResNet-50 w FPN	800	25.7	43.1	26.8	35.7	54.7	38.5
CornerNet	Hourglass-104	511	28.4	41.8	29.5	38.4	53.8	40.9
ExtremeNet	Hourglass-104	511	27.3	39.4	28.9	40.3	55.1	43.7
HoughNet	ResNet-101	512	23.4	40.1	23.6	34.3	53.6	36.6

Below figure compares object detection results on train2017 and minitrain. This figure also shows the positive correlation between train2017 and minitrain results. The Pearson correlation coefficients are 0.74 and 0.92 for COCO evaluation metrics AP and AP50 respectively. This figure is based on the object detection result table above. BaseModel corresponds HoughNet model with ResNet-101 backbone.

Keypoint Detection Task

Keypoint Detection model performances trained on minitrain vs train2017. Models are evaluated on val2017.

Method	minitrain AP	minitrain AP_50	minitrain AP_75	train2017 AP	train2017 AP_50	train2017 AP_75
VitPOSE	70.5	88.7	78.2	75.8	90.7	83.1
DEKR	60.0	82.3	65.8	68.0	86.8	74.5
HRNET	56.9	80.3	61.7	65.4	86.3	72.0
SimpleBaseline2D	66.6	87.6	74.3	71.7	89.8	79.3

Below figure compares keypoint detection results on train2017 and minitrain. This figure also shows the positive correlation between train2017 and minitrain results. The Pearson correlation coefficients are 0.99 and 0.99 for COCO evaluation metrics AP and AP50 respectively. This figure is based on the keypoint detection result table above.