What does 'mask' mean in event tracker in TS with polarity?
gyubeomim opened this issue · comments
Hello
Thank you for sharing such an great work!
I have a question while reading IROS2022 paper.
In "B. Event-corner Feature Detection and Tracking based on the Time Surface with Polarity." Section, Arc* is used for corner detection from the SAE first, then track the corner by using TS with polarity. (am I right?)
And the paper says "The detected event-corner would be further selected by setting the TS with polarity as the mask"
I'm curious what does mask mean in this case. You mean after detecting arc* corners from the SAE, then start tracking in TS with polarity? I couldn't figure out even though I watched your video (what is red, blue dot in the video btw?)
+) addtional question, your lab has plan to release the research code in the future? It would be really helpful for other researcher.
Thank you in advance!
In the video, red and blue dot seem like the polarity of the detected corner :-)
Thanks for your interest!
In our IROSS22 paper, we utilize the TS with polarity as a mask and for feature tracking, as depicted in Figure 1. This means that in addition to tracking features, the TS with polarity is also used to ensure an even distribution of event-corner features.
In our IROS2022 publication, we mentioned that the detected event-corners are further selected by employing the TS with polarity, denoted as Tp(x, y, t), as the mask. To achieve a uniform distribution, a minimum distance (10-20 pixels, depending on the resolution of the event camera) is set between neighboring event-corner features. We also retain the event-corners where the pixel value of Tp(x, y, t) is not equal to 128.0. This emphasizes the detected event-corner features located on strong edges, rather than including too many noisy features in low texture areas.
In our context, the term "mask" refers to the selection scheme used to ensure a uniform distribution of event-corner features.
Regarding the "red and blue dots" in our video, they represent the polarity of the events. We visualize the detected and tracked event-corner features in the raw event stream.
As for the code, unfortunately, it is currently not publicly available. It is on my to-do list, but cleaning it up will require a significant amount of time. In the meantime, one option is to evaluate the methods using our datasets. We also provide the raw results from our methods for performance comparison.
Thank you.
Thank you for the quick and kind reply!
Then, as far as I understand, TS with polarity is not only used for the corner tracker but also used for the even distribution of corners?
Detection:
- Arc* detector is used from SAE (same as original arc* paper).
- When detecting, TS with polarity is used for the even distribution of corners.
Tracking:
- Corners are being tracked in the sequence of TS with polarity by using LK optical flow.
Do I correctly understand?
Thank you in adavnce and I'm looking forward to seeing your next research!
Yes, you are right
Thank you!