JdeRobot / DetectionMetrics

Tool to evaluate deep-learning detection and segmentation models, and to create datasets

Home Page:https://jderobot.github.io/DetectionMetrics/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Detection Metrics

Publish Docker image

More info and documentation here.

Detection Metrics is a set of tools to evaluate object detection neural networks models over the common object detection datasets. The tools can be accessed using the GUI or the command line applications. In the picture below, the general architecture is displayed.

general_architecture

The tools provided are:

  • Viewer: view the dataset images with the annotations.
  • Detector: run a model over a dataset and get generate a new annotated dataset.
  • Evaluator: evaluate the ground truth dataset with another one and get the comparison metrics.
  • Deployer: run a model over different inputs like a video or webcam and generate a new annotated dataset.
  • Converter: convert a dataset into another dataset format.
  • Command line application (CLI): access Detection Metrics toolset through command line
  • Detection Metrics as ROS Node: use Detection Metrics as a ROS Node.
  • Labelling: add or modify labels in the datasets in runtime when running Deployer.

The idea is to offer a generic infrastructure to evaluate object detection models against a dataset and compute the common statistics:

  • mAP
  • mAR
  • Mean inference time.

What's supported in Detection Metrics.

Support Detail
Supported OS Multiplatform using Docker
Supported datasets
  • COCO
  • ImageNet
  • Pascal VOC
  • Jderobot recorder logs
  • Princeton RGB dataset
  • Spinello dataset
  • Open images dataset
Supported frameworks
  • TensorFlow
  • Keras
  • PyTorch
  • Yolo-OpenCV
  • Caffe
  • Background substraction
Supported inputs in Deployer
  • WebCamera/USB Camera
  • Videos
  • Streams from ROS
  • Streams from ICE
  • JdeRobot Recorder Logs

Installation

Install packaged image

To quickly get started with Detection Metrics, we provide a docker image.

  • Download docker image and run it
    docker run -dit --name detection-metrics -v [local_directory]:/root/volume/ -e DISPLAY=host.docker.internal:0 jderobot/detection-metrics:noetic

This will start the GUI, provide a configuration file (appConfig.yml can be used) and you are ready to go. Check out the web for more information

Installation from source (Linux only)

Check the installation guide here. This is also the recommended installation for contributors.

Starting with Detection Metrics

Check out the beginner's tutorial.

General Detection Metrics GUI

The top toolbar shows the different tools available.

Example of detection and console output in Detection Metrics

Two image views are displayed, one with the ground truth and the other with the detected annotations. In the console output, log info is shown.

detector

About

Tool to evaluate deep-learning detection and segmentation models, and to create datasets

https://jderobot.github.io/DetectionMetrics/

License:GNU General Public License v3.0


Languages

Language:C++ 77.9%Language:Python 12.5%Language:CMake 6.1%Language:C 2.3%Language:Slice 0.8%Language:Shell 0.3%Language:Dockerfile 0.2%Language:Ruby 0.0%