Error when i do 'python run.py --ginc configs/nerf/blender.gin --scene chair'
jhbae1184 opened this issue · comments
Thank you for this repo,
I want to learn nerf model on blender dataset, but i meet this issue.
ㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡ
-
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
OSError: [Errno 22] Invalid argument -
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated
ㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡㅡ
Epoch 0: 0%| | 0/257813 [00:00<?, ?it/s]Traceback (most recent call last):
File "run.py", line 231, in
run(
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\gin\config.py", line 1605, in gin_wrapper
utils.augment_exception_message_and_reraise(e, err_str)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\gin\utils.py", line 41, in augment_exception_message_and_reraise
raise proxy.with_traceback(exception.traceback) from None
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\gin\config.py", line 1582, in gin_wrapper
return fn(*new_args, **new_kwargs)
File "run.py", line 175, in run
trainer.fit(model, data_module, ckpt_path=ckpt_path)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 608, in fit
call._call_and_handle_interrupt(
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\trainer\call.py", line 38, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 650, in _fit_impl
self._run(model, ckpt_path=self.ckpt_path)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1112, in _run
results = self._run_stage()
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1191, in _run_stage
self._run_train()
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1214, in _run_train
self.fit_loop.run()
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\loops\loop.py", line 199, in run
self.advance(*args, **kwargs)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\loops\fit_loop.py", line 267, in advance
self._outputs = self.epoch_loop.run(self._data_fetcher)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\loops\loop.py", line 194, in run
self.on_run_start(*args, **kwargs)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\loops\epoch\training_epoch_loop.py", line 160, in on_run_start
_ = iter(data_fetcher) # creates the iterator inside the fetcher
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\utilities\fetching.py", line 179, in iter
self._apply_patch()
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\utilities\fetching.py", line 120, in _apply_patch
apply_to_collections(self.loaders, self.loader_iters, (Iterator, DataLoader), _apply_patch_fn)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\utilities\fetching.py", line 156, in loader_iters
return self.dataloader_iter.loader_iters
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\trainer\supporters.py", line 556, in loader_iters
self._loader_iters = self.create_loader_iters(self.loaders)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\pytorch_lightning\trainer\supporters.py", line 596, in create_loader_iters
return apply_to_collection(loaders, Iterable, iter, wrong_dtype=(Sequence, Mapping))
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\lightning_utilities\core\apply_func.py", line 51, in apply_to_collection
return function(data, *args, **kwargs)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\torch\utils\data\dataloader.py", line 430, in iter
self._iterator = self._get_iterator()
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\torch\utils\data\dataloader.py", line 381, in _get_iterator
return _MultiProcessingDataLoaderIter(self)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\site-packages\torch\utils\data\dataloader.py", line 1034, in init
w.start()
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\multiprocessing\context.py", line 327, in _Popen
return Popen(process_obj)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\multiprocessing\popen_spawn_win32.py", line 93, in init
reduction.dump(process_obj, to_child)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
OSError: [Errno 22] Invalid argument
In call to configurable 'run' (<function run at 0x00000160C9D12E50>)
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\multiprocessing\spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File "C:\Users\JIHUNBAE\anaconda3\envs\nerf\lib\multiprocessing\spawn.py", line 126, in _main
self = reduction.pickle.load(from_parent)
_pickle.UnpicklingError: pickle data was truncated
I follow requirement.txt but because of my cuda version, i change the pytorch(=1.13.1), torch-scatter version.
Can you check and give me some solutions ?
Thank you very much.
One tricky solution is to remove DVGO relevant parts. If you are using DVGO models, then you should find for an appropriate version for PyTorch & torch-scatter.
We don't recommend using PyTorch 1.13.1 since it might lack the compatibility.
Thank you for answer :))