TUI-NICR / ESANet

ESANet: Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Is it able to segment unseen object?

JANGSOONMYUN opened this issue · comments

Hi,
Is your work able to segment unseen objects, which are not labelled?
For example, if there is a big box in the center of room, and it is not labelled, then ESANet segment it without classification? Or just ignore it?

Every pixel in the image will be assigned a label, even if it does not fit the true category. Box is even one of the categories the model was trained on.

@mona0809 Thank you for your reply. Ok I will test it.

Even if each pixel is assigned to one of the known classes, the network might still be able to detect unknown objects with some height on the ground. During tests in real-world applications, we observed that the floor class is detected quite robustly and might help to find objects on the ground (see our video).

I am not sure whether this helps you. However, if your example above is your application scenario, give it a try.

@danielS91 Thank you.
I checked the NYUV2 dataset and there is 'void' in the classes. If there is unseen object detected, is it classified as 'void' ?

Void is ignored during training. You could try to train with void but there is no guarantee that every unkown object will be classified as void. As void is very versatile it is hard to learn a representation.

@mona0809 I see. Maybe I should train the model with additional dataset and see the result.
I appreciate.