cannot backward tracking in online model with specific queries in middle frame

[Fang (Kevin) Li]

Hi,

I am using your online model for point tracking. and I want to process a video. I want to set up specific queries in one of the middle frame and try to use backward tracking. here is my code:

`video = torch.tensor(frames).permute(0, 3, 1, 2)[None].float().to(device)
cotracker = torch.hub.load("facebookresearch/co-tracker", "cotracker2_online").to(device)
cotracker(video_chunk = video, is_first_step = True, queries=maxima_coords[None], grid_query_frame=grid_query_frame, backward_tracking=True)
vis = Visualizer(save_dir="./co-tracker/result", pad_value=120, linewidth=2)
for ind in range(0, video.shape[1] - cotracker.step, cotracker.step):
pred_tracks, pred_visibility = cotracker(video_chunk=video[:, ind : ind + cotracker.step * 2], queries=maxima_coords[None], grid_query_frame=grid_query_frame, backward_tracking=True)

vis.visualize(video, pred_tracks, pred_visibility)`

and here is the bug i got:
---> 60 pred_tracks, pred_visibility = cotracker(video_chunk=video[:, ind : ind + cotracker.step * 2], queries=maxima_coords[None], grid_query_frame=grid_query_frame, backward_tracking=True)
61
62 vis.visualize(video, pred_tracks, pred_visibility)

~/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1128 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1129 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1130 return forward_call(*input, **kwargs)
1131 # Do not call functions when jit is used
1132 full_backward_hooks, non_full_backward_hooks = [], []

~/anaconda3/lib/python3.9/site-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
25 def decorate_context(*args, **kwargs):
26 with self.clone():
---> 27 return func(*args, **kwargs)
28 return cast(F, decorate_context)
29

TypeError: forward() got an unexpected keyword argument 'backward_tracking'

[Nikita Karaev]

Hi @fangli333, thank you for the question!
Backward tracking is only supported in the offline mode because you wouldn't need an online algorithm if you already have access to all the video frames.