ROS1 nodes publishing low confidence detections

[Kostas Tsampazis]

It has been reported that some ROS nodes are publishing low confidence detections continuously even when no detection can be made.

How i tested the nodes

I tested all ROS1 nodes that i could from the develop branch by running them locally with a webcam and microphone. For the rest of the nodes that i could not run, i examined the code to determine, if i could, what they are publishing.

Expected behaviour

Expected behaviour would be for the nodes to output detection messages only when a detection is made. That means detections with some confidence threshold so as not to get false positives with low confidence (there are instances of detections with 1% confidence being continuously published). Logically, no detections should be published when no input is given, e.g. black image, silent audio, etc.

I think that the above reasoning is not applicable to the two segmentation nodes.

I also list some other minor issues that i discovered.

---

Comments on nodes that i could run

• Audiovisual emotion recognition

  • Getting detections with confidence ~0.3 to ~0.7 even when not talking and no face visible with various ids, 1, 2, 3
  • Input topic should be /audio and not /audio/audio?

• Speech command recognition

  • Getting detections of high confidence in various ids even when silent
  • Input topic should be /audio and not /audio/audio?

• Face detection retinaface

  • Getting empty detection messages when no face is found

• Object detection 2D centernet

  • Getting empty detection messages when no object is found
  • Re-downloads model

• Object Detection 2D detr

  • Getting empty detection messages when no object is found

• Object Detection 2D nanodet

  • Getting empty detection messages when no object is found
  • Getting initialization debugging messages
  • Getting several warnings messages about layer names
  • Re-downloads model
  • Log info says SSD node
  • Getting lots of rospy info messages about added connections when viewing results in rqt

• Object Detection 2D ssd

  • Getting empty detection messages when no object is found
  • Re-downloads model

• Object Detection 2D yolov3

  • Getting empty detection messages when no object is found

• Skeleton based action recognition

  • Continuously outputting detections with very low confidence ~0.03 even when screen is black
  • Continuously getting detections of very low confidence when pose is detected
  • Getting ~0.1, ~0.3 confidence detections that don't make sense when e.g. only head is detected - classes pointing something with finger, cross hands, etc.
  • Pose publishing is OK, no pose is published when no keypoints are detected

• Video activity recognition

  • Getting low confidence detections <~0.1 with regular input
  • Getting low confidence detections even when screen is black
  • Getting some good detections with high confidence and a lot of detections with low confidence that don't make sense

---

Comments on nodes that i could not run

• Heart anomaly detection

  • Seems that max of model's softmax output (class prediction) is returned from infer
  • No check in callback for what the learner.infer returns, feeds it straight into ros message and publishes
  • Probably publishes low confidence detections

• Landmark-based facial expression recognition

  • Seems that max of model's softmax output (class prediction) is returned from infer
  • No check in callback for what the learner.infer returns, feeds it straight into ros message and publishes
  • Probably publishes low confidence detections

• RGBD hand gesture recognition

  • Class prediction is returned from infer and fed into ROS message
  • Probably publishes low confidence detections

• Object Detection 2D GEM

  • Boxes returned from infer are fed into ROS message and published
  • Probably similar to other object detection 2D nodes, publishes empty detection messages when no object is found

• Object Detection 3D voxel

  • Boxes returned from infer are fed into ROS message and published
  • Probably publishes empty detection messages when no object is found

• Object Tracking 2D deep sort

  • It seems that there are no checks for contents of what is published, probably publishes empty detection messages

• Object Tracking 2D fair mot

  • It seems that there are no checks for contents of what is published, probably publishes empty detection messages

• Object Tracking 3D ab3dmot

  • It seems that there are no checks for contents of what is published, probably publishes empty detection messages