Giters

xxlong0 / SparseNeuS

SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse views

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

why is depth loss required to train sparse neus ?

Jiakui opened this issue · comments

commented

Dear author,

In the paper, I noticed that only color loss is mentioned, however, in your code it is required to provide the depth to train SparseNeus from scratch. I wonder, if I don not have depth information, can I train SparseNeus from stractch ? If so, how should I do it ?

Thanks so much ！

Jiakui

commented

No gt depth is needed for training. The depth loss in the code is just used to check whether the network predicts the correct geometry or not. You can find that the depth loss is not included in the total loss. If you don't have the depth, just modify the dataloader to avoid loading depth.

commented

thanks so much ! If I want to train my network for multiple scenes , how should I do it ?

commented

thanks so much ! If I want to train my network for multiple scenes , how should I do it ?

Could you make the question clear? You would like to handle the cases where the input images capture many objects?

commented

Dear author, I am familiar with NeuS. When I used NeuS, I ran per-scene optimization. For example, if I want to reconstuct person A, I just collect images for person A, and then run NeuS. If I want to reconstuct person B, I just collect images for person B, and then run NeuS. There is not any relationship between images of person A and person B.

In Sparse NeuS, since it is generalizable, so it should be meaningful to optimze for person A , and then keep the weight of neural network, and then optimize for person B , and so on. In this way, the neural network would be trained on images of different persons, so that it could generalize very well to kinds of images of person.

I just wonder, for the current code of Sparse NeuS, should I could do the above continous training without any tricks, or should I modify the code to acheive my goal ?

Thanks!

commented

Hello. To train the generic model of sparseneus, we train the model on DTU, which is made up of many objects. Then we run per-scene optimization on one specific object. So in your case, I think you should train sparseneus on a large dataset with many different people, and then run per-scene optimization on a specific person.