facebookresearch / co-tracker

Great job, I have benefited a lot. I would like to train on the videos I have collected. What should I do if I want to train with my video dataset? Thank you.

I'd also like to know! Is there an "easy" way to take the tracked points and re-train/re-fine the model?

Hi @jdyjjj, @horsto, I think the easiest way to train on a custom dataset right now is to adapt this class to your use case:

co-tracker/cotracker/datasets/kubric_movif_dataset.py

Line 386 in 4f297a9

class KubricMovifDataset(CoTrackerDataset):

I'm not sure if this is considered an easy way, though :)

Hi @nikitakaraevv, I adapted the KubrikMovifDataset to a local dataset. In my case, I have 180 points to track per video. However, when sampling data with augmentations, I sometimes end up with fewer points. I these cases, the getitem_helper function return gotit=False. Is that a wished behaviour?

Thanks for your answers!

Hi @Anderstask1, some points don't satisfy the sampling criteria: being visible in the first frame or the middle frame of the sequence. That's why we sometimes end up with fewer points, especially if we don't have additional points to sample from. I think you can modify the sampling criteria to make it less strict:

co-tracker/cotracker/datasets/kubric_movif_dataset.py

Line 468 in 8d36403

visibile_pts_inds = torch.cat(

Hi again @nikitakaraevv, I modified the sampling criteria to make it less strict. However, during training, it seems like several points are sampled mid-sequence in the provided prediction videos (even though the ground truth video don't sample points mid-sequence). Do you know the reason for this?

How to prepare new dataset?