copy_and_paste code error
so45jj45 opened this issue · comments
Thank you for your appreciation of our research.
When I read your paper, I understood that it uses the copy-and-paste technique on images to create synthetic videos. It appears that this code operates within the copy_and_paste(self, labeled_data, unlabeled_data) function of the train_loop.py.
The part keep = iou_matrix.max(1)[0] < 0.0 seems to always return false, leading me to believe that the current code does not copy instances from other images but rather copies the existing target frame for training.
When changing it to keep = iou_matrix.max(1)[0] < 0.5, it results in an error. I would appreciate any assistance in resolving this issue.
[02/16 08:27:16 mask2former_video.data_video.build]: Using training sampler TrainingSampler
[02/16 08:27:16 d2.data.common]: Serializing the dataset using: <class 'detectron2.data.common._TorchSerializedList'>
[02/16 08:27:16 d2.data.common]: Serializing 100 elements to byte tensors and concatenating them all ...
[02/16 08:27:16 d2.data.common]: Serialized dataset takes 0.69 MiB
[02/16 08:27:16 d2.data.build]: Making batched data loader with batch_size=2
INFO: use AMPTrainer.
copy_paste hyper-params: True 1.0 False
[02/16 08:27:36 d2.checkpoint.detection_checkpoint]: [DetectionCheckpointer] Loading from /workspace/CutLER/videocutler/pretrain/cutler_m2f_rn50.pth ...
[02/16 08:27:36 fvcore.common.checkpoint]: [Checkpointer] Loading from /workspace/CutLER/videocutler/pretrain/cutler_m2f_rn50.pth ...
[02/16 08:27:37 d2.engine.train_loop]: Starting training from iteration 0
ERROR [02/16 08:28:22 d2.engine.train_loop]: Exception during training:
Traceback (most recent call last):
File "/workspace/CutLER/detectron2/detectron2/engine/train_loop.py", line 155, in train
self.run_step()
File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/defaults.py", line 508, in run_step
self._trainer.run_step()
File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/train_loop.py", line 374, in run_step
data = self.copy_and_paste(copy.deepcopy(data[::-1]), data)
File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/train_loop.py", line 212, in copy_and_paste
copied_instances.gt_masks = copied_masks
File "/workspace/CutLER/detectron2/detectron2/structures/instances.py", line 62, in setattr
self.set(name, val)
File "/workspace/CutLER/detectron2/detectron2/structures/instances.py", line 78, in set
assert (
AssertionError: Adding a field of length 2 to a Instances of length 1
[02/16 08:28:22 d2.engine.hooks]: Total training time: 0:00:44 (0:00:00 on hooks)
[02/16 08:28:22 d2.utils.events]: iter: 0 lr: N/A max_mem: 171M
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/conda/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/main.py", line 39, in
cli.main()
File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
run()
File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
runpy.run_path(target, run_name="main")
File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
return _run_module_code(code, init_globals, run_name,
File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/root/.vscode-server/extensions/ms-python.python-2024.0.1/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
exec(code, run_globals)
File "/workspace/CutLER/CutLER/videocutler/train_net_video.py", line 314, in
launch(
File "/workspace/CutLER/detectron2/detectron2/engine/launch.py", line 84, in launch
main_func(*args)
File "/workspace/CutLER/CutLER/videocutler/train_net_video.py", line 305, in main
return trainer.train()
File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/defaults.py", line 498, in train
super().train(self.start_iter, self.max_iter)
File "/workspace/CutLER/detectron2/detectron2/engine/train_loop.py", line 155, in train
self.run_step()
File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/defaults.py", line 508, in run_step
self._trainer.run_step()
File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/train_loop.py", line 374, in run_step
data = self.copy_and_paste(copy.deepcopy(data[::-1]), data)
File "/workspace/CutLER/CutLER/videocutler/mask2former_video/engine/train_loop.py", line 212, in copy_and_paste
copied_instances.gt_masks = copied_masks
File "/workspace/CutLER/detectron2/detectron2/structures/instances.py", line 62, in setattr
self.set(name, val)
File "/workspace/CutLER/detectron2/detectron2/structures/instances.py", line 78, in set
assert (
AssertionError: Adding a field of length 2 to a Instances of length 1
Hi, as I've left Meta and no longer have the permissions to make direct changes to this repository, I've implemented the bug fix in my local fork available at: https://github.com/frank-xwang/CutLER. Please let me know if you encounter any additional issues. Thank you!
update: got the push permission. Have updated the codes. :-)