my own llff&colmap data set failed to build fine mesh at stage 1
tangyubbb opened this issue · comments
my own llff&colmap data set failed to build fine mesh at stage 1 by running like below:
stage 0: python main.py data/grape0/ --workspace trial_grape0 -O --data_format colmap --enable_cam_center --enable_cam_near_far --bound 2 --dt_gamma 0 --stage 0 --clean_min_f 16 --clean_min_d 10 --visibility_mask_dilation 50 --iters 10000 --decimate_target 3e5 --sdf
stage 1:python main.py data/grape0/ --workspace trial_grape0 -O --data_format colmap --bound 2 --enable_cam_center --enable_cam_near_far --dt_gamma 0 --stage 1 --iters 5000 --lambda_normal 1e-2 --refine_remesh_size 0.01 --sdf .
my data : 76 photos of reslution(480*270)
stage 0 is okay!
like:
mesh0& mesh1 in /home/ros/nerf2mesh1/trial_grape0/mesh_stage0
but stage 1 failed like this:
(nerf2mesh) ros@ros:~/nerf2mesh1$ python main.py data/grape0/ --workspace trial_grape0 -O --data_format colmap --bound 2 --enable_cam_center --enable_cam_near_far --dt_gamma 0 --stage 1 --iters 5000 --lambda_normal 1e-2 --refine_remesh_size 0.01 --sdf
Warning:
Unable to load the following plugins:
libio_e57.so: libio_e57.so does not seem to be a Qt Plugin.
Cannot load library /home/ros/anaconda3/envs/nerf2mesh/lib/python3.9/site-packages/pymeshlab/lib/plugins/libio_e57.so: (/lib/x86_64-linux-gnu/libp11-kit.so.0: undefined symbol: ffi_type_pointer, version LIBFFI_BASE_7.0)
[INFO] loaded cascade 0 mesh: (150590, 3), (299999, 3)
[INFO] loaded cascade 1 mesh: (239222, 3), (475456, 3)
[INFO] ColmapDataset: image H = 480, W = 270
[INFO] 68 image exists in all 68 colmap entries.
[INFO] ColmapDataset: load poses (68, 4, 4), points (5425, 3)
[INFO] ColmapDataset: auto-scale 0.3689
[WARN] ColmapDataset: estimated AABB [-23.008131078036023, -26.777362440560832, -8.252267374003914, 30.464630875674096, 14.855970668683902, 2.190616508554669] exceeds provided bound 2.0! Consider improving --bound to make scene included in trainable region.
[INFO] extracting sparse depth info...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 68/68 [00:00<00:00, 5380.66it/s]
[INFO] extracted 382.47 valid sparse depth on average per image
Loading train data: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 59/59 [00:00<00:00, 520.57it/s]
[INFO] max_epoch 85, eval every 17, save every 1.
[INFO] update_aabb: [-2.0, -2.0, -2.0, 2.0, 2.0, 2.0]
[INFO] Trainer: ngp_stage1 | 2023-07-14_15-32-16 | cuda | fp16 | trial_grape0
[INFO] #parameters: 20076965
Namespace(path='data/grape0/', O=True, workspace='trial_grape0', seed=0, stage=1, ckpt='latest', fp16=True, sdf=True, tcnn=False, progressive_level=False, test=False, test_no_video=False,
test_no_mesh=False, camera_traj='', data_format='colmap', train_split='train', preload=True, random_image_batch=True, downscale=1, bound=2.0, scale=-1, offset=[0, 0, 0], mesh='',
enable_cam_near_far=True, enable_cam_center=True, min_near=0.05, enable_sparse_depth=False, enable_dense_depth=False, iters=5000, lr=0.01, lr_vert=0.0001, pos_gradient_boost=1, cuda_ray=True,
max_steps=1024, update_extra_interval=16, max_ray_batch=4096, grid_size=128, mark_untrained=False, dt_gamma=0.0, density_thresh=0.001, diffuse_step=1000, diffuse_only=False, background='random',
enable_offset_nerf_grad=True, n_eval=5, n_ckpt=50, num_rays=4096, adaptive_num_rays=True, num_points=262144, lambda_density=0, lambda_entropy=0, lambda_tv=1e-08, lambda_depth=0.1, lambda_specular=1e-05,
lambda_eikonal=0.1, lambda_rgb=1, lambda_mask=0.1, wo_smooth=False, lambda_lpips=0, lambda_offsets=0.1, lambda_lap=0.001, lambda_normal=0.01, lambda_edgelen=0, contract=True, patch_size=1,
trainable_density_grid=False, color_space='srgb', ind_dim=0, ind_num=500, mcubes_reso=512, env_reso=256, decimate_target=300000.0, mesh_visibility_culling=True, visibility_mask_dilation=5, clean_min_f=8,
clean_min_d=5, ssaa=2, texture_size=4096, refine=True, refine_steps_ratio=[0.1, 0.2, 0.3, 0.4, 0.5, 0.7], refine_size=0, refine_decimate_ratio=0, refine_remesh_size=0.01, vis_pose=False, gui=False,
W=1000, H=1000, radius=5, fovy=50, max_spp=1, refine_steps=[500, 1000, 1500, 2000, 2500, 3500])
NeRFNetwork(
(encoder): GridEncoder: input_dim=3 num_levels=16 level_dim=1 resolution=16 -> 4096 per_level_scale=1.4473 params=(6299960, 1) gridtype=hash align_corners=True interpolation=linear
(sigma_net): MLP(
(net): ModuleList(
(0): Linear(in_features=19, out_features=32, bias=False)
(1): Linear(in_features=32, out_features=1, bias=False)
)
)
(encoder_color): GridEncoder: input_dim=3 num_levels=16 level_dim=2 resolution=16 -> 4096 per_level_scale=1.4473 params=(6299960, 2) gridtype=hash align_corners=True interpolation=linear
(color_net): MLP(
(net): ModuleList(
(0): Linear(in_features=35, out_features=64, bias=False)
(1): Linear(in_features=64, out_features=64, bias=False)
(2): Linear(in_features=64, out_features=6, bias=False)
)
)
(specular_net): MLP(
(net): ModuleList(
(0): Linear(in_features=6, out_features=32, bias=False)
(1): Linear(in_features=32, out_features=3, bias=False)
)
)
)
[INFO] Loading stage 0 model to init stage 1 ...
[INFO] loaded model.
[WARN] missing keys: ['vertices_offsets']
[INFO] Loading latest checkpoint ...
[WARN] No checkpoint found, abort loading latest model.
[INFO] ColmapDataset: image H = 480, W = 270
[INFO] 68 image exists in all 68 colmap entries.
[INFO] ColmapDataset: load poses (68, 4, 4), points (5425, 3)
[INFO] ColmapDataset: auto-scale 0.3689
[WARN] ColmapDataset: estimated AABB [-23.008131078036023, -26.777362440560832, -8.252267374003914, 30.464630875674096, 14.855970668683902, 2.190616508554669] exceeds provided bound 2.0! Consider improving --bound to make scene included in trainable region.
[INFO] extracting sparse depth info...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 68/68 [00:00<00:00, 7911.37it/s]
[INFO] extracted 382.47 valid sparse depth on average per image
Loading val data: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 512.44it/s]
==> Start Training Epoch 1, lr=0.000001 ...
0% 0/59 [00:00<?, ?it/s]/home/ros/anaconda3/envs/nerf2mesh/lib/python3.9/site-packages/torch/nn/functional.py:3631: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
warnings.warn(
段错误 (核心已转储)
and , i had tried vis_pose in stage 1
And that, I run those commands without masks.
AND, I tried to run the given original NeRF data format like data/nerf_synthetic/lego/.still the same ,stage 0 is ok, but stage 1 still goes like:
(nerf2mesh) ros@ros:~/nerf2mesh1$ python main.py data/nerf_synthetic/lego/ --workspace trial_syn_lego/ -O --bound 1 --scale 0.8 --dt_gamma 0 --stage 1
Warning:
Unable to load the following plugins:
libio_e57.so: libio_e57.so does not seem to be a Qt Plugin.
Cannot load library /home/ros/anaconda3/envs/nerf2mesh/lib/python3.9/site-packages/pymeshlab/lib/plugins/libio_e57.so: (/lib/x86_64-linux-gnu/libp11-kit.so.0: undefined symbol: ffi_type_pointer, version LIBFFI_BASE_7.0)
[INFO] loaded cascade 0 mesh: (150717, 3), (299999, 3)
Loading train data: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 117.40it/s]
[INFO] max_epoch 300, eval every 60, save every 6.
[INFO] Trainer: ngp_stage1 | 2023-07-14_15-47-39 | cuda | fp16 | trial_syn_lego/
[INFO] #parameters: 18754159
Namespace(path='data/nerf_synthetic/lego/', O=True, workspace='trial_syn_lego/', seed=0, stage=1, ckpt='latest', fp16=True, sdf=False, tcnn=False, progressive_level=False, test=False,
test_no_video=False, test_no_mesh=False, camera_traj='', data_format='nerf', train_split='train', preload=True, random_image_batch=True, downscale=1, bound=1.0, scale=0.8, offset=[0, 0, 0], mesh='',
enable_cam_near_far=False, enable_cam_center=False, min_near=0.05, enable_sparse_depth=False, enable_dense_depth=False, iters=30000, lr=0.01, lr_vert=0.0001, pos_gradient_boost=1, cuda_ray=True,
max_steps=1024, update_extra_interval=16, max_ray_batch=4096, grid_size=128, mark_untrained=True, dt_gamma=0.0, density_thresh=10, diffuse_step=1000, diffuse_only=False, background='random',
enable_offset_nerf_grad=False, n_eval=5, n_ckpt=50, num_rays=4096, adaptive_num_rays=True, num_points=262144, lambda_density=0, lambda_entropy=0, lambda_tv=1e-08, lambda_depth=0.1, lambda_specular=1e-05,
lambda_eikonal=0.1, lambda_rgb=1, lambda_mask=0.1, wo_smooth=False, lambda_lpips=0, lambda_offsets=0.1, lambda_lap=0.001, lambda_normal=0, lambda_edgelen=0, contract=False, patch_size=1,
trainable_density_grid=False, color_space='srgb', ind_dim=0, ind_num=500, mcubes_reso=512, env_reso=256, decimate_target=300000.0, mesh_visibility_culling=True, visibility_mask_dilation=5, clean_min_f=8,
clean_min_d=5, ssaa=2, texture_size=4096, refine=True, refine_steps_ratio=[0.1, 0.2, 0.3, 0.4, 0.5, 0.7], refine_size=0.01, refine_decimate_ratio=0.1, refine_remesh_size=0.02, vis_pose=False, gui=False,
W=1000, H=1000, radius=5, fovy=50, max_spp=1, refine_steps=[3000, 6000, 9000, 12000, 15000, 21000])
NeRFNetwork(
(encoder): GridEncoder: input_dim=3 num_levels=16 level_dim=1 resolution=16 -> 2048 per_level_scale=1.3819 params=(6098120, 1) gridtype=hash align_corners=True interpolation=linear
(sigma_net): MLP(
(net): ModuleList(
(0): Linear(in_features=19, out_features=32, bias=False)
(1): Linear(in_features=32, out_features=1, bias=False)
)
)
(encoder_color): GridEncoder: input_dim=3 num_levels=16 level_dim=2 resolution=16 -> 2048 per_level_scale=1.3819 params=(6098120, 2) gridtype=hash align_corners=True interpolation=linear
(color_net): MLP(
(net): ModuleList(
(0): Linear(in_features=35, out_features=64, bias=False)
(1): Linear(in_features=64, out_features=64, bias=False)
(2): Linear(in_features=64, out_features=6, bias=False)
)
)
(specular_net): MLP(
(net): ModuleList(
(0): Linear(in_features=6, out_features=32, bias=False)
(1): Linear(in_features=32, out_features=3, bias=False)
)
)
)
[INFO] Loading stage 0 model to init stage 1 ...
[INFO] loaded model.
[WARN] missing keys: ['vertices_offsets']
[INFO] Loading latest checkpoint ...
[WARN] No checkpoint found, abort loading latest model.
Loading val data: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:00<00:00, 122.73it/s]
[mark untrained grid] 0 from 2097152
==> Start Training Epoch 1, lr=0.000001 ...
0% 0/100 [00:00<?, ?it/s]/home/ros/anaconda3/envs/nerf2mesh/lib/python3.9/site-packages/torch/nn/functional.py:3631: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
warnings.warn(
段错误 (核心已转储)
My GPU is 3090 with cuda11.4 .