Generates a 3D object map for the ObChange dataset based on the paper
@article{oliveira2021spatio,
title={{Spatio-temporal Data Association for Object-augmented Mapping}},
author={de Oliveira, Felipe DB and da Silva, Marcondes R and Ara{\'u}jo, Aluizio FR},
journal={Journal of Intelligent \& Robotic Systems},
volume={103},
number={1},
pages={1--19},
year={2021}
}
and their code published on https://github.com/freicaneca/RTABMapMod.git
We used the same parameters as mentioned in the paper. Except that we generate object detections with Mask R-CNN (https://github.com/edith-langer/Mask_R-CNN_feature_vector). Additionally, we changed the spatial association distance (dfTh) to 0.3 m and the the distance threshold for BB centers (bbcTh) to 0.0001.
3d_obj_map Prerequisites:
- camera intrinsics
- camera pose for each frame. We use the .freiburg generated by ElasticFusion
- json file including following information for each object detection: id, label, score, bounding box
- for each frame and each detected object a file containing the extracted feature vector from Mask R-CNN
The result is stored in 3d_obj_map.txt. The file contains for each object in the 3D map the coordinates (e.g. [0.388456,-0.780053,0.515878];apple0). Additionally a file for each object is stored, which contains the paths to all the associated feature vectors (one feature vector for each image where the objects was detected).
scene_comparison Compares the generated 3d object map from each scene with each other. If an object is only in one scene, but not the other, it is counted as removed/novel. If there is exactly one object with a certain label in both scenes it is moved/static (depending on their spatial distance in the map). If several objects with the same label exist, the dot product between all feature vector from the two scenes is computed and the two objects with the highest value are categorized as moved/static. The categorized objects are stored in the respective files (static, moved, removed, novel) with coordinates and label. The associations of static and moved objects can be found in obj_associations.txt.
evaluation The ObChange dataset provides pointwise ground truth labeling of reconstructions. To evaluate the categorization, the centroid of each labeled object is compared against the coordinates of the objects in the 3D object map. If there is a match (within 10 cm) it counts as detected. Also the category of each object is evaluated. The assigned labels are not used because they are different between COCO and the ObChange dataset. A second evaluation is based on an adapted ground truth where only detected objects are used (an object formerly categorized as moved, is now categorized as novel if it was not detected in the reference scene).
The result files generated by scene_comparison together with the ground truth annotation of the ObChange dataset are the input.
The evaluation result contains, among other things,
- number of true positives and false positives for each category
- for each object in the ground truth how often it is visible, how often is was detected and how often it was correctly categorized
- the number of detected object, the number of correctly categorized objects and the number of wrongly categorized objects (including false positives like background objects not categorized as static)