Giters

POSTECH-CVLab / SCNeRF

[ICCV21] Self-Calibrating Neural Radiance Fields

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Does `--dataset_type custom` work for 360 scenes with photos only ?

vishnukool opened this issue · comments

commented

Two questions, if you don't mind:

  1. Does the code in "custom" branch with --dataset_type custom work for 360 degree scenes like the "tanks_and_temples" images ? Or is it only for forward facing scenes like LLFF fern dataset?
  2. If it does work for 360 scenes, can you confirm that it doesn't need any COLMAP camera parameters, initialization, etc. ?
commented

Before the main description, I should mention that the "custom" branch is one of the examples of the custom dataset. I made this branch because of the issue #3 (comment). Thus this is not fully verified. I strongly recommend double-checking the code before using it.

First, the custom loader is designed to be similar to the "LLFF" dataset. Thus, you can feel free to use 360 scenes. However, the COLMAP process is required since we cannot get the camera pose without COLMAP.

In addition, if you ever want to use known camera parameters, instead of utilizing COLMAP, you should slightly modify the code.

commented

Ah thanks for the clarification @jeongyw12382
So basically for 360 scenes we still need COLMAP process? And there's no way around it ?

commented

Secondly is there a better starting template than main1_fern_ours.sh for training on 360 scenes ?
Since I assume that config is for forward facing scenes.

commented

@vishnukool
Hmm... my future extension attempts to enable 360 scene calibration.
However, I cannot ensure whether SCNeRF works perfectly fine for 360 scenes yet.

commented

@vishnukool
For non-forward-facing scenes, I strongly recommend removing the NDC setup since NDC is designed to better work on forward-facing scenes.
You should add the command "--no_ndc True".

commented

Thanks for the clarification @jeongyw12382
I see that the code example in main3_m60.sh already works for 360 scenes. However that tanks_and_temples dataset also has a lot of pose, intrinsics and other data, which I'm not sure how to generate.

  1. Would you recommend main3_m60.sh as a better starting point instead for 360 scenes?
  2. Also if I only have images, how to about generating the rest of the pose, intrinsics to make it work for 360 scenese?
commented

Oh. Sure I could add it.
I'm sorry for announcing it but I have an event that I should end up with until this week.
So, can I add the scripts at the end of this week?

commented

I'll close this issue after adding the utils about the "pose + intrinsic" calibration step using COLMAP.

commented

Thanks @jeongyw12382, yes that would be great!

commented

@vishnukool Sorry for being late. I have work that I should conclude until this Wednesday but it is still in progress. Let me know your email so that I could inform you whenever I update the code. I'll end up the additional code ASAP.

commented

I have just added colmap utils and updated the manual for usage. Could you check for it?

commented

Closing the issue after adding the COLMAP utils. Please let me know if you have any trouble using the COLMAP utils.