Questions on Depth images

[Harvey Chang]

Thanks for providing this dataset~
I am currently testing fast-YCB dataset for my own algorithm. I found a problem on the provided depth images. The depth images seems to be pre-filtered. If you turned into 3D, you can find something like this.

From my understanding, this is because there is a filter/averaging happening here between the foreground scene and background scene. So I wonder if you can provide the raw depth image data? (Without filtering).

[Nicola Piga]

Hi @changhaonan,

thanks for showing interest in our dataset.

From my understanding, this is because there is a filter/averaging happening here between the foreground scene and background scene.

Actually, the depth map we provide is the exact output we got from the renderer we used, i.e. the xyz renderer of Mitsuba 2. Hence, no averaging between background and foreground is taking place.

Probably the behavior you are seeing depends on the inner mechanisms of the renderer which tends to introduce such "noise".
Anyways, let me add that such kind of noise resembles what you get as output from several stereo vision systems - although not all. Some other stereo systems just have holes between foreground and background and do not filter between the two.

So I wonder if you can provide the raw depth image data? (Without filtering).

Given the above, we could not provide any "raw" image as the raw ones are those already provided. Nonetheless, if you are working in the 3D domain directly, I would like to suggest using DBSCAN filtering to filter out the undesired noise.

Using DBSCAN filtering I was able to get this result:

the original being instead:

Of course, some tuning of the DBSCAN parameters might be required depending on your goals.

Let me know if this helps. I can provide the python code to reproduce the results above if you are interested in it.

[Harvey Chang]

OK. Thanks. I current solved this problem by cutting out the surfaces that are too close to camera view angle. I know DBSCAN is a clustering method. So it can also help to remove outliers? Sounds interesting.

[Nicola Piga]

I current solved this problem by cutting out the surfaces that are too close to camera view angle.

Glad to know that you have solved it.

I know DBSCAN is a clustering method. So it can also help to remove outliers?

Most DBSCAN libraries assign -1 as the label to a cluster including "noise", i.e. points not belonging to any valid cluster.

Feel free to close the issue if you think you are satisfied with your final solution. Thanks

[Harvey Chang]

Thanks.