apple / ml-neuman

Official repository of NeuMan: Neural Human Radiance Field from a Single Video (ECCV 2022)

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

The loss is really small,but the result is weird

Net-Maker opened this issue · comments

Thanks for this excellent work!
I am using instant-ngp to accelerate the training speed.
And i think the loss is really small enough to get a nice image.
here is my loss in tensorboard:
图片
and here is my render output:
图片

Can you help me to figure out the reason why i get a bad result?
maybe the iters is not enough,or I should modify the render part?

instant-ngp in my work is just a accelerate tool and i did not break the framework,all things works fine(in my opinion...)

Will you be able to render the test images or training images, i.e. in observation space? are they looks good? what colors are rendered for the human part?

oh! I realized that the tensorboard result is rendered by only a few input images.
I got a better result after 4500 iters:
图片
And i will try to render another result in tensorboard that using more input images.
Thanks for your rapid reply!

instant-ngp 在我的工作中只是一个加速工具,我没有破坏框架,一切都很好(在我看来......)

How to use instant-ngp to speed up, I also want to train quickly.

instant-ngp 在我的工作中只是一个加速工具,我没有破坏框架,一切都很好(在我看来......)

How to use instant-ngp to speed up, I also want to train quickly.

先安装好tiny-cuda-nn:https://github.com/NVlabs/tiny-cuda-nn
这是instant-ngp的底层实施框架,提供了一些pytorch的接口,你可以去读一下项目的文档,也有一个demo可以参考
接下来你需要通读instant-NGP文章的appendix,了解用instant-ngp来实现nerf的结构和参数是怎么样的,最后先搭建一个你自己的nerf demo,最后就可以尝试去修改neuman的网络结构了。
我最后实现的效果并不是很好,我想可能存在一些原理层面的冲突(也可能有bug)。

instant-ngp 在我的工作中只是一个加速工具,我没有破坏框架,一切都很好(在我看来......)

How to use instant-ngp to speed up, I also want to train quickly.

先安装好tiny-cuda-nn:https://github.com/NVlabs/tiny-cuda-nn 这是instant-ngp的底层实施框架,提供了一些pytorch的接口,你可以去读一下项目的文档,也有一个demo可以参考 接下来你需要通读instant-NGP文章的appendix,了解用instant-ngp来实现nerf的结构和参数是怎么样的,最后先搭建一个你自己的nerf demo,最后就可以尝试去修改neuman的网络结构了。 我最后实现的效果并不是很好,我想可能存在一些原理层面的冲突(也可能有bug)。

instant-ngp 在我的工作中只是一个加速工具,我没有破坏框架,一切都很好(在我看来......)

How to use instant-ngp to speed up, I also want to train quickly.

先安装好tiny-cuda-nn:https://github.com/NVlabs/tiny-cuda-nn 这是instant-ngp的底层实施框架,提供了一些pytorch的接口,你可以去读一下项目的文档,也有一个demo可以参考 接下来你需要通读instant-NGP文章的appendix,了解用instant-ngp来实现nerf的结构和参数是怎么样的,最后先搭建一个你自己的nerf demo,最后就可以尝试去修改neuman的网络结构了。 我最后实现的效果并不是很好,我想可能存在一些原理层面的冲突(也可能有bug)。

OK! Thank you very much for your reply. Can I ask if the final effect you achieved is similar to the original text? How long is the training time? Could you please share your code, thank you very much!

@litchisea
actually there are still some bugs,and i will share it after I get a good result.
For some reasons I stoped this work,and I would work on it later.
So, I am sorry that I cannot help.

@litchisea actually there are still some bugs,and i will share it after I get a good result. For some reasons I stoped this work,and I would work on it later. So, I am sorry that I cannot help.

Are you still researching now? How is it going? Do you have any results? If so, can you share the code?