Unable to change maximum segment duration for BA-TFD
jeremy-swack opened this issue · comments
Hello, I am trying to change the max segment duration for the BA-TFD model (not plus) and I running into issues when doing so. I have changed the value in the config file, and everywhere in the codebase where the max_duration variable is manually set to 40, However, I am still getting the following error:
RuntimeError: The size of tensor a (100) must match the size of tensor b (40) at non-singleton dimension 0
Are there additional files that must be edited to change this value?
Hi, it is possible to use a different maximum numbers of the input temporal dimension. Please provide the full stack trace error message.
Here is the full error:
Traceback (most recent call last): File "/home/disinfolab/jswack/BA-TFD+/train.py", line 128, in <module> trainer.fit(model, dm) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 696, in fit self._call_and_handle_interrupt( File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _call_and_handle_interrupt return trainer_fn(*args, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 735, in _fit_impl results = self._run(model, ckpt_path=self.ckpt_path) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1166, in _run results = self._run_stage() File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1252, in _run_stage return self._run_train() File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1283, in _run_train self.fit_loop.run() File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 200, in run self.advance(*args, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/fit_loop.py", line 271, in advance self._outputs = self.epoch_loop.run(self._data_fetcher) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 200, in run self.advance(*args, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 203, in advance batch_output = self.batch_loop.run(kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 200, in run self.advance(*args, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 87, in advance outputs = self.optimizer_loop.run(optimizers, kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/loop.py", line 200, in run self.advance(*args, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 201, in advance result = self._run_optimization(kwargs, self._optimizers[self.optim_progress.optimizer_position]) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 248, in _run_optimization self._optimizer_step(optimizer, opt_idx, kwargs.get("batch_idx", 0), closure) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 358, in _optimizer_step self.trainer._call_lightning_module_hook( File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1550, in _call_lightning_module_hook output = fn(*args, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/core/module.py", line 1705, in optimizer_step optimizer.step(closure=optimizer_closure) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/core/optimizer.py", line 168, in step step_output = self._strategy.optimizer_step(self._optimizer, self._optimizer_idx, closure, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/strategies/ddp.py", line 289, in optimizer_step optimizer_output = super().optimizer_step(optimizer, opt_idx, closure, model, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/strategies/strategy.py", line 216, in optimizer_step return self.precision_plugin.optimizer_step(model, optimizer, opt_idx, closure, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/plugins/precision/native_amp.py", line 85, in optimizer_step closure_result = closure() File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 146, in __call__ self._result = self.closure(*args, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 132, in closure step_output = self._step_fn() File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 407, in _training_step training_step_output = self.trainer._call_strategy_hook("training_step", *kwargs.values()) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1704, in _call_strategy_hook output = fn(*args, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/strategies/ddp.py", line 352, in training_step return self.model(*args, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1040, in forward output = self._run_ddp_forward(*inputs, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1000, in _run_ddp_forward return module_to_run(*inputs[0], **kwargs[0]) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/pytorch_lightning/overrides/base.py", line 79, in forward output = self.module.training_step(*inputs, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/model/batfd.py", line 119, in training_step loss_dict = self.loss_fn(fusion_bm_map, v_bm_map, a_bm_map, v_frame_cla, a_frame_cla, label, n_frames, File "/home/disinfolab/jswack/BA-TFD+/model/batfd.py", line 92, in loss_fn fusion_bm_loss = self.bm_loss(fusion_bm_map, label, n_frames) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/loss.py", line 15, in forward loss.append(self.loss_fn(pred[i, :, :frame], true[i, :, :frame])) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/torch/nn/modules/loss.py", line 536, in forward return F.mse_loss(input, target, reduction=self.reduction) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/torch/nn/functional.py", line 3291, in mse_loss expanded_input, expanded_target = torch.broadcast_tensors(input, target) File "/home/disinfolab/jswack/BA-TFD+/env/lib/python3.8/site-packages/torch/functional.py", line 74, in broadcast_tensors return _VF.broadcast_tensors(tensors) # type: ignore[attr-defined] RuntimeError: The size of tensor a (100) must match the size of tensor b (40) at non-singleton dimension 0
From that output, these are the lines that cause errors from the LAVDF codebase:
File "/home/disinfolab/jswack/BA-TFD+/model/batfd.py", line 119, in training_step loss_dict = self.loss_fn(fusion_bm_map, v_bm_map, a_bm_map, v_frame_cla, a_frame_cla, label, n_frames, File "/home/disinfolab/jswack/BA-TFD+/model/batfd.py", line 92, in loss_fn fusion_bm_loss = self.bm_loss(fusion_bm_map, label, n_frames)
It is the error when calculating the loss function because the shape of predicted tensor and ground truth tensor are not matched. Please check the tensor inputs.
Could you elaborate a bit more on how to change the shape of the two tensors? Tensor 'a' seems to have changed from my code and config file changes, but I am not sure what else I can edit to change the shape of the other tensor. I've looked through loss.py and it doesn't seem clear to me what to change.
Solved in zoom meeting.