It does not seem that the detector properly updates the tracker.

Question

It does not seem that the detector properly updates the tracker.

mjbedford2017 opened this issue 5 years ago · comments

mjbedford2017 commented 5 years ago

Love the project, works great for the most part. However, I have seen many times during my testing that two main things happen and I am not sure how/why:

The bounding box initially will be far too large or not size properly to the vehicle but will not update and correct itself on subsequent detections.
More importantly, I have seen times where cars will be tracking and leave the video frame but the bounding box gets left behind on the frame. It will stay there for a very long time, many times until another car comes by and then the bounding box that was left behind from the last car will snap to the new car.

Both of these issues lead me to believe that the detector is not updating the tracker all of the time. I have the detection interval set to 5 and the tracking failures set to 15. The tracking failures never seem to get to 15. But, if a detection happens every 5 frames and there is no car anymore, why does the tracker keep a bounding box hanging around for a very long time? It seems to me that if the detector does not detect a car anymore, the bounding box/tracker can be removed? I know we have to be careful here because the detector could normally miss a detection every so often and the goal of the tracker is to keep tracking when this happens but it seems that there should be a timeout or something where after several missed detections, we can then go ahead and remove?

These two issues are causing some cars to be missed when crossing the line.

Also, I have tried both KCF and CSRT trackers along with both SSD and YOLO detectors, in many different combinations and the issues seem to still happen which tells me that the issue isn't with the tracker or detector but rather, the code that ties the two together.

Thoughts? Thanks!

Nicholas Kajoh · Answer 1 · Tue Jun 04 2019 18:41:05 GMT+0800 (China Standard Time)

Hi @mjbedford2017! Thanks for trying out the VCS and for the feedback. And sorry for the semi-late reply. 🙃

This is a bug. The algorithm that matches new bounding boxes to existing blobs (vehicle objects) chooses the larger bounding box rather than the newer one. So if the vehicles are moving away from the camera, you'll get over-sized bounding boxes. I'll fix this ASAP. I have some refactors to do first though.
I've observed this as well. I think this happens because some trackers start tracking something else (most likely a portion of the road which barely changes) when they lose track of the vehicles that have left the frame. The reason the "hanging" boxes snap onto new vehicles when they come across them is because of an algorithm that removes duplicate trackers based on whether their bounding boxes "greatly" overlap or not. Since the trackers continue to report that they're tracking successfully and there's currently no mechanism to eliminate trackers that haven't been updated over several detection cycles, they just hang around until they snap or are snapped into other boxes by the "remove duplicates" function. I plan to fix this next but I'm not sure how soon I'll be able to get it out. I'm currently busy with some other work.

For now, I think you'll get better results if you make the MAX_CONSECUTIVE_TRACKING_FAILURES value less than DETECTION_INTERVAL. DETECTION_INTERVAL = 10 and MAX_CONSECUTIVE_TRACKING_FAILURES = 3 works pretty well with the videos I've tested the VCS on. Can you share your video with me (if you don't mind) so that I can look at things from your perspective? It's possible that there might be something else I haven't observed.

Cheers!

mjbedford2017 · Answer 2 · Thu Jun 06 2019 16:38:20 GMT+0800 (China Standard Time)

Hello @nicholaskajoh thank you for the detailed reply! Understood on the late reply, I am sure you are busy and have other things going on.

I understand on your comments. Thank you for offering to look at my videos, I don't mind sharing them at all! Attached is a zip file with two videos. Please let me know what you see. I am using the ssd detector and the csrt tracker and all other options set to default.

Let me know when you get bug number 1 fixed as well, that should clean up some of my issues.

Do you have a donate option? I would like to take care of you for the great project and your time looking at my videos.
Thanks!

Videos.zip

Nicholas Kajoh · Answer 3 · Fri Jun 07 2019 21:48:37 GMT+0800 (China Standard Time)

Great! I'll test with the videos and get back to you. Quick question: do you want to count vehicles going in all directions? If not, what position do you want the counting line?

I have a Buy Me A Coffee page. I'd appreciate a coffee or two. 😊

mjbedford2017 · Answer 4 · Mon Jun 10 2019 09:54:55 GMT+0800 (China Standard Time)

@nicholaskajoh No problem, I actually went to the site to buy you several coffees and I think the payment system is down, it kept giving me issues with many different cards. I will try again later. Shall we move this chat to email as well since we are moving beyond an issue to log in GitHub?

As for the actual issues. Please let me know when you have the bugs you mentioned fixed. As for tuning it to our video, I actually made some changes to your code to not use a line at all. I just track the centroid history in the blob and if it is moving down over a certain amount of pixels, I count it down. If it is moving up, I count it up. I can give you the changes I made so you can see them. Given my application and field of view, I just wanted to know if they are moving one way or another in general, without needing to cross a line.

Do you think a line is better? To answer your question, yes I do want to track both directions and have a count in or down and one for up or out.

For now though, if you want to at least get the detection and tracking working better, we can figure out the counting line part later.

Thank you so much!!

Mike

Nicholas Kajoh · Answer 5 · Tue Jun 18 2019 04:33:53 GMT+0800 (China Standard Time)

Hi @mjbedford2017! Thanks for trying out the VCS and for the feedback. And sorry for the semi-late reply.

This is a bug. The algorithm that matches new bounding boxes to existing blobs (vehicle objects) chooses the larger bounding box rather than the newer one. So if the vehicles are moving away from the camera, you'll get over-sized bounding boxes. I'll fix this ASAP. I have some refactors to do first though.

I've observed this as well. I think this happens because some trackers start tracking something else (most likely a portion of the road which barely changes) when they lose track of the vehicles that have left the frame. The reason the "hanging" boxes snap onto new vehicles when they come across them is because of an algorithm that removes duplicate trackers based on whether their bounding boxes "greatly" overlap or not. Since the trackers continue to report that they're tracking successfully and there's currently no mechanism to eliminate trackers that haven't been updated over several detection cycles, they just hang around until they snap or are snapped into other boxes by the "remove duplicates" function. I plan to fix this next but I'm not sure how soon I'll be able to get it out. I'm currently busy with some other work.

For now, I think you'll get better results if you make the MAX_CONSECUTIVE_TRACKING_FAILURES value less than DETECTION_INTERVAL. DETECTION_INTERVAL = 10 and MAX_CONSECUTIVE_TRACKING_FAILURES = 3 works pretty well with the videos I've tested the VCS on. Can you share your video with me (if you don't mind) so that I can look at things from your perspective? It's possible that there might be something else I haven't observed.

Cheers!

Fixed 1 and 2 in a5cda19.