error in training
yejr0229 opened this issue · comments
Hello, my Monohuman made an error while training to 50000 iters. I started training again, but still encountered this error on the 50000 iter:
File "/home/yejr/miniconda3/envs/Monohuman/lib/python3.7/site-packages/torch/serialization.py", line 423, in save
_save(obj, opened_zipfile, pickle_module, pickle_protocol)
File "/home/yejr/miniconda3/envs/Monohuman/lib/python3.7/site-packages/torch/serialization.py", line 650, in _save
zip_file.write_record(name, storage.data_ptr(), num_bytes)
RuntimeError: [enforce fail at inline_container.cc:445] . PytorchStreamWriter failed writing file data/5: file write failed
can you tell me how to fix it?
Is it due to the SMPL model?
I realized I didn't remove the chumpy objects from the SMPL model.
Hello, my Monohuman made an error while training to 50000 iters. I started training again, but still encountered this error on the 50000 iter: File "/home/yejr/miniconda3/envs/Monohuman/lib/python3.7/site-packages/torch/serialization.py", line 423, in save _save(obj, opened_zipfile, pickle_module, pickle_protocol) File "/home/yejr/miniconda3/envs/Monohuman/lib/python3.7/site-packages/torch/serialization.py", line 650, in _save zip_file.write_record(name, storage.data_ptr(), num_bytes) RuntimeError: [enforce fail at inline_container.cc:445] . PytorchStreamWriter failed writing file data/5: file write failed
can you tell me how to fix it?
It seems like an I/0 problem. Please check if you have enough space or permission to write in your saving path.