Data preprocessing and provided model

[tardigrade]

Hi and thanks for sharing the code. Makes a great school for me!

I have noticed one issue with regards to the currently committed code in conjunction with the committed pickle files for the train and validation data used.

Although everything runs smoothly when it comes to fetching and preprocessing data, as well as using the preprocessed data to train a new model from scratch, it is impossible for me to reproduce a model with performance, even remotely, close to the performance your provided pretrained model has. So in my effort to reproduce exactly what you do, I tried using your data pickle files, which you kindly provide in the datainfo subdir. Of course, the (absolute) path of the images needed some fixing to work for me but this was no issue. The issue is that the poses in your provided pickle files are lists of length 6, presumably angles and positions. But, the ImageSequenceDataset __getitem__ method needs a 15-long list (which, I have come to understand, is the 6-number part above, i.e. 3 angles and 3 positions, along with the flattened out rotational matrix.) Your ImageSequenceDataset class seems to use the rotational matrix to rotate all poses of a given sequence w.r.t. the first frame of that sequence.

What I am, effectively, saying is that the poses in the committed pickle files don't work with the committed code, at the moment of writing this. And when I use the committed code without your pickle files, I get seemingly some convergence in the training, when looking at train/valid losses, but the final model produces absolutely non-sensical results on the xy-plane (or is it xz-plane?).

Could you give a hint as to which hyper parameters and code commit you used to produce the provided model?