Combine visual descriptor and geometry distance in the data association part

[Hieu Pham Trung]

Hi, I want to ask about the data association part. If I understand correctly, the current implementation of matching detections with initialized tracked objects are consider the distance function and reid distance function separately. The distance matrix is constructed solely on one function. The distance function is used first to match initialized tracked objects with detections, and this function only considers the motion. I am using a reid function to utilize the visual characteristics of objects but this function only uses to match unmatched initialized tracked objects with not yet initialized tracked objects. I ask this question because I face a situation where 2 objects intersect and there is only 1 bounding box, this bounding box can be mistakenly matched with the object that is hidden behind if using only the motion feature. Is there an implementation that combines visual descriptors into the distance? Thank you.

[Agustín Castro]

Hello! Sorry for the late response. We don't currently have any out-of-the-box distance function that uses visual descriptors. However, you can always define a custom distance function taking a Detection and a TrackedObject instance that makes use of the attributes embedding (for the detection), and last_detection.embedding (for the tracked object). Remember that your tracked object also might have a list of several past detections in its attribute past_detections, so you can have the embeddings of those past detections at your disposal when defining your distance_function.

[Hieu Pham Trung]

Thank you very much for your reply!