aLRP Loss: A Ranking-based, Balanced Loss Function Unifying Classification and Localisation in Object Detection
Average Localisation-Recall-Precision (aLRP) Loss is a ranking-based loss function to train object detectors by unifying localisation and classification branches. We define aLRP Loss as the average Localisation Recall Precision [1] errors on positive examples. To tackle the nondifferentiable nature of ranking during backpropagation, we combine error-driven update of perceptron learning with backpropogation by generalizing the training approach of AP Loss [2] to ranking-based loss functions (see Section 4 in the paper for details).
With this formulation, aLRP Loss (i) enforces the predictions with large confidence scores to have better localisation, and correlates the classification and localisation tasks (see Figure below), (ii) has significantly less number of hyperparameters (i.e. only 1 hyperparameter) than the conventional loss formulation (i.e. the combination of classification and regression losses by a scaler weight), and (iii) guarantees balanced training (see Theorem 2 in the paper).
This repository provides the implementation of aLRP Loss based on official AP Loss repository. Within a short period of time, a more-efficient mmdetection-based implementation including different models (e.g. FoveaBox, Faster R-CNN) trained by aLRP Loss will be released.
How to Cite
Please cite the paper if you benefit from our paper or repository:
@inproceedings{aLRPLoss,
title = {A Ranking-based, Balanced Loss Function Unifying Classification and Localisation in Object Detection},
author = {Kemal Oksuz and Baris Can Cam and Emre Akbas and Sinan Kalkan},
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
year = {2020}
}
Results of the Base Models (with Scale 500)
Method | Backbone | oLRP (minival) | AP (minival) | Download |
---|---|---|---|---|
AP Loss | ResNet-50 | 71.0 | 35.5 | model |
aLRP Loss | ResNet-50 | 68.4 | 38.9 | model |
aLRP Loss + ATSS | ResNet-50 | 67.3 | 40.2 | model |
Table of Main Results with Pretrained Models
Backbone | Scale | oLRP (minival) | AP (minival) | AP (test-dev) | Download |
---|---|---|---|---|---|
ResNet-101 | 500 | 66.1 | 41.2 | 41.7 | model |
ResNext-101-32x8d | 500 | 64.8 | 42.8 | 43.4 | model |
ResNet-101 | 800 | 64.5 | 43.6 | 44.1 | model |
ResNext-101-32x8d | 800 | 62.7 | 45.4 | 45.9 | model |
Specification of Dependencies
- Python 3.7
- PyTorch 1.2+
- CUDA 10.0
- NumPy 1.16+
- mmcv
Preparation Instructions:
- Install required packages:
pip install "git+https://github.com/open-mmlab/cocoapi.git#subdirectory=PythonAPI"
pip install opencv-python
- Create necessary directories:
mkdir data models results
- Prepare Data: You can download the images and their annotations from the download page of COCO dataset. Please do not forget to change the name of the downloaded
test
folder name astest-dev2017
. If you have had the dataset, then just create a symbolic link to yours by:
ln -s $YOUR_PATH_TO_coco data/coco
Finally, the data directories should be arranged like:
├── data
│ ├── coco
│ │ ├── annotations
│ │ ├── train2017
│ │ ├── val2017
│ │ ├── test-dev2017
- Prepare the pre-trained backbone models under
models
directory by following the structure below:
├── models
│ ├── resnet50-pytorch.pth
| ├── resnet101-pytorch.pth
| ├── resnext101_32x8d-8ba56ff5.pth
We use ResNet-50 and ResNet-101 pre-trained models provided by the official AP-Loss repository. Accordingly, you can download them from this link. For ResNext-101, we use the model provided by Pytorch in this link to download.
Training Code
You can train a model by running the following code:
python train.py --cfg PATH_TO_CONFIG_FILE
The configuration files are in the config
folder. We provide all the configuration files used for comparison with SOTA methods in Table 4:
- aLRPLoss500_r50.py
- aLRPLoss500_r50_ATSS.py
- aLRPLoss500_r101.py
- aLRPLoss500_x101.py
- aLRPLoss800_r101.py
- aLRPLoss800_x101.py
Also using our code, the results of AP Loss can be reproduced with the following configuration files:
- APLoss500_r50.py
Test Code
You can test a model on "test-dev2017" or "val2017" splits of COCO dataset by running the following code:
python test.py --cfg PATH_TO_CONFIG_FILE
In addition to standard COCO-style performance metrics, our evaluation code reports APs for every IoU to compute COCO-Style AP, and also oLRP.
References
[1] Oksuz K, Cam BC, Akbas E, Kalkan S, Localization recall precision (LRP): A new performance metric for object detection, ECCV 2018.
[2] Chen K, Li J, Lin W, See J, Wang J, Duan L, Chen Z, He C, Zou J, Towards Accurate One-Stage Object Detection With AP-Loss, CVPR 2019 & TPAMI.