liuyuan-pal / Gen6D

Dear author, thank you very much for proposing such an easy-to-use and high-performance 6D object pose estimation solution. Now I have some questions to ask:

First, taking the chair data of GenMOP as an example, during training or fine-tuning, only chair-ref participates in training, and chair-test does not participate, right?

Second, I would like to ask how the Mask of each object in the GenMOP data set is generated?

Looking forward to your reply! Thanks!

Hi, thank you for your interest in our work!

For training objects, we use both ref and test in training. We then evaluate on unseen objects, which are not included in the training set.
To generate the mask of the object, I use the reconstructed points of the object and project them onto images to get the object mask. You may also use other foreground mask predictor like carvekit.

Thank you very much for your prompt and detailed reply.
Regarding the problem of mask generation, I found that the frame-xx.jpg in the images folder in the GenMOP data does not correspond to the xx.png in the mask folder, such as frame0.jpg and 0.png in knife-ref:

Will this have an impact on training?

Yes, there is a map between the COLMAP id and the frame id. You may check

Gen6D/dataset/database.py

Line 173 in 50aa71b

if is_windows_colmap:

So you can retrieve the input image.

Thanks very much!
I have successfully generated the mask and carried out the training. Both the detector and selector have achieved good results, but the training results of the refiner are a bit less optimistic. Is there any tricks or things that need to be paid attention to?

before finetune

after finetune

Thanks very much! I have successfully generated the mask and carried out the training. Both the detector and selector have achieved good results, but the training results of the refiner are a bit less optimistic. Is there any tricks or things that need to be paid attention to?

before finetune

after finetune

Hi , I am interested on your fine-tuning result, is it possible to share the code with me, THANKS!

Data and masks of GenMOP involved in training