NerfAcc: Setting up CUDA (This may take a few minutes the first time)Killed
Flyingdog-Huang opened this issue · comments
2023-10-24 02:00:54.767 | INFO | main:main:25 - ==> Init dataloader ...
100%|█████████████████████████████████████████████████████████| 800/800 [00:00<00:00, 204837.51it/s]
2023-10-24 02:00:54.944 | INFO | dataset.ray_dataset:init:42 - ==> Find 4 cameras
100%|████████████████████████████████████████████████████████████| 200/200 [00:01<00:00, 111.36it/s]
100%|███████████████████████████████████████████████████████████████████| 200/200 [00:00<00:00, 550.07it/s]
100%|██████████████████████████████████████████████████████████████████| 200/200 [00:00<00:00, 2726.73it/s]
100%|██████████████████████████████████████████████████████████████████| 200/200 [00:00<00:00, 8469.83it/s]
100%|████████████████████████████████████████████████████████████████| 800/800 [00:00<00:00, 222524.25it/s]
2023-10-24 02:00:58.635 | INFO | dataset.ray_dataset:init:42 - ==> Find 4 cameras
100%|███████████████████████████████████████████████████████████████████| 200/200 [00:01<00:00, 111.41it/s]
100%|███████████████████████████████████████████████████████████████████| 200/200 [00:00<00:00, 486.54it/s]
100%|██████████████████████████████████████████████████████████████████| 200/200 [00:00<00:00, 5384.91it/s]
100%|██████████████████████████████████████████████████████████████████| 200/200 [00:00<00:00, 6841.64it/s]
2023-10-24 02:01:02.330 | INFO | main:main:49 - ==> Init model ...
2023-10-24 02:01:04.057 | INFO | main:main:51 - TriMipRFModel(
(field): TriMipRF(
(encoding): TriMipEncoding()
(direction_encoding): Encoding(n_input_dims=3, n_output_dims=16, seed=1337, dtype=torch.float16, hyperparams={'degree': 4, 'otype': 'SphericalHarmonics'})
(mlp_base): Network(n_input_dims=48, n_output_dims=16, seed=1337, dtype=torch.float16, hyperparams={'encoding': {'offset': 0.0, 'otype': 'Identity', 'scale': 1.0}, 'network': {'activation': 'ReLU', 'n_hidden_layers': 2, 'n_neurons': 128, 'otype': 'FullyFusedMLP', 'output_activation': 'None'}, 'otype': 'NetworkWithInputEncoding'})
(mlp_head): Network(n_input_dims=31, n_output_dims=3, seed=1337, dtype=torch.float16, hyperparams={'encoding': {'offset': 0.0, 'otype': 'Identity', 'scale': 1.0}, 'network': {'activation': 'ReLU', 'n_hidden_layers': 4, 'n_neurons': 128, 'otype': 'FullyFusedMLP', 'output_activation': 'Sigmoid'}, 'otype': 'NetworkWithInputEncoding'})
)
(ray_sampler): OccupancyGrid()
)
2023-10-24 02:01:04.058 | INFO | main:main:53 - ==> Init trainer ...
2023-10-24 02:01:04.098 | INFO | trainer.trainer:init:60 - # Parameters for trimipRF.get_optimizer:
==============================================================================
trimipRF.get_optimizer.feature_lr_scale = 10.0
trimipRF.get_optimizer.lr = 0.002
trimipRF.get_optimizer.weight_decay = 1e-05
Parameters for get_scheduler:
==============================================================================
get_scheduler.gamma = 0.6
Parameters for main:
==============================================================================
main.batch_size = 24
main.model_name = 'Tri-MipRF'
main.num_workers = 4
main.seed = 42
main.stages = 'train_eval'
main.train_split = 'trainval'
Parameters for RayDataset:
==============================================================================
RayDataset.base_path =
'/home/jovyan/vol-1/Tri-MipRF/data/nerf_synthetic_multiscale'
RayDataset.num_rays = 8192
RayDataset.render_bkgd = 'white'
RayDataset.scene = 'chair'
RayDataset.scene_type = 'nerf_synthetic_multiscale'
RayDataset.to_world = True
Parameters for Trainer:
==============================================================================
Trainer.base_exp_dir = '/home/jovyan/vol-1/Tri-MipRF/output'
Trainer.dynamic_batch_size = True
Trainer.eval_step = 25000
Trainer.exp_name = 'nerf_synthetic_multiscale/chair/Tri-MipRF/2023-10-24_02-00-54'
Trainer.log_step = 1000
Trainer.max_steps = 25001
Trainer.num_rays = 8192
Trainer.target_sample_batch_size = 65536
Trainer.test_chunk_size = 8192
Trainer.varied_eval_img = True
Parameters for TriMipRF:
==============================================================================
TriMipRF.feature_dim = 16
TriMipRF.geo_feat_dim = 15
TriMipRF.n_levels = 8
TriMipRF.net_depth_base = 2
TriMipRF.net_depth_color = 4
TriMipRF.net_width = 128
TriMipRF.plane_size = 512
Parameters for TriMipRFModel:
==============================================================================
TriMipRFModel.occ_grid_resolution = 128
TriMipRFModel.samples_per_ray = 1024
2023-10-24 02:01:04.101 | INFO | trainer.trainer:fit:106 - ==> Start training ...
(● ) NerfAcc: Setting up CUDA (This may take a few minutes the first time)Killed
It seems an issue that is related with nerfacc
- remove nerfacc cache using command "rm -rf ~/.cache/torch_extensions/py37_cu116/nerfacc_cuda"
- reboot your machine
These two method could work for you with only each one.
- remove nerfacc cache using command "rm -rf ~/.cache/torch_extensions/py37_cu116/nerfacc_cuda"
- reboot your machine
These two method could work for you with only each one.
thanks for your replys,
my mistake is error cuda version that not match nerfacc version,
when I set cuda-116, I solve my error,
thanks again!