Detection mAP
A simple utility tool to evaluate Bounding box classification task following Pascal VOC paper.
To learn about this metric I recommend this excellent blog post by Sancho McCann before reading the paper : link
Note that the method is not compared with the original VOC implementation! (See Todo)
features
- Simple : numpy and matplotlib are the only dependencies
- Compute a running evaluation : input prediction/ground truth at each frames, no need to save in files
- Plot (matplotlib) per class pr-curves with interpolated average precision (default) or average precision
Method
Multiclass mAP
Handle every class as one against the others. (x against z)
- True positive (TP):
- Gt x predicted as x
- False positive (FP):
- Prediction x if Gt x has already a TP prediction
- Prediction x not overlapping any Gt x
- False negative (FN):
- Gt x not predicted as x
Example frame
Code
All you need is your predicted bounding boxes with class and confidence score and the ground truth bounding boxes with their classes.
main loop :
frames = [(pred_bb1, pred_cls1, pred_conf1, gt_bb1, gt_cls1),
(pred_bb2, pred_cls2, pred_conf2, gt_bb2, gt_cls2),
(pred_bb3, pred_cls3, pred_conf3, gt_bb3, gt_cls3)]
n_class = 7
mAP = DetectionMAP(n_class)
for frame in frames:
mAP.evaluate(*frame)
mAP.plot()
plt.show() # or plt.savefig(path)In this example a frame is a tuple containing:
- Predicted bounding boxes : numpy array [n, 4]
- Predicted classes: numpy array [n]
- Predicted confidences: numpy array [n]
- Ground truth bounding boxes:numpy array [m, 4]
- Ground truth classes: numpy array [m]
Note that the bounding boxes are represented as two corners points : [x1, y1, x2, y2]
TODO
Interpolated average precision- Test against VOC matlab implementation
Contribution
And of course any bugfixes/contribution are always welcome!