Some questions regarding to the related work
Jacobieee opened this issue · comments
Hello.
I noticed that you compared the Multiverse model with others like Social GAN and Next-prediction model, and the format of the dataset is compatible with those 2 models. Does that mean I can directly test pretrained Social GAN and Next-prediction models on the same dataset that this model does? I am quite confused of how to achieve the testing and compare results of different models. Could you give me some instructions?
And I found a very interesting idea in your Next-prediction model that you have used appearance features to predict future activities. I'm wondering if it's feasible to apply this method to the Multiverse model to facilitate the prediction.
I'll appreciate it so much for your response.
Hi.
Yes. This repo use similar preprocess protocol as the Next model. For Social GAN, you can use the "obs_traj" and "pred_traj" in the preprocessed files (see here) since Social GAN only needs trajectory inputs. For Next-prediction, it needs more features other than the scene semantic features. Follow the preprocessing protocol for Next-prediction here. If you keep the training/testing split the same, you can compare them.
Sure! you can try adding more features to Multiverse and see whether it helps. But it will take more GPU memory and harder to train.
@JunweiLiang Thanks for your reply!
Just to make sure if I understand it right. In preprocessing, the ActEv dataset and other features are processed and generated 3 .npz files in actev_preprocess directory. Since the Next model is trained in the same dataset, I only need to load the pre-trained model in the forking path dataset by running the multifuture inference here. Is that right?
And for Social GAN, I need to train it on ActEv dataset first, then test it on "next_x_v1_dataset_prepared_data/obs_data/traj_2.5fps" which only has testing dataset, to pass trajectory inputs into the model and start the evaluation. Does it also need to be run by multifuture inference?
To run Next-prediction on the Multiverse dataset, you need to follow its preprocessing protocol to get all needed features into the testing npz files. For Social GAN, yes, you only need to run on the traj_2.5fps/ files. They are compatible with the SocialGAN repo.
@JunweiLiang Hi, sorry for the delayed follow-up. I found an issue while testing sgan model, that it creates .pt files as checkpoints which is incompatible when running multifuture_inference.py. Could you show me how you evaluated that in your own evaluation?
And we don't need scene features like --scene_feat_path, and add --greedy instead, right?
Thanks a lot!
For SGAN experiments, you should use their script evaluate_model.py. I think you need to modify it a bit to add an dataset_path argument so you can run it like this:
python sgan/scripts/evaluate_model.py --dataset_path traj_2.5fps/test/ --model_path sgan-20V-20/checkpoint_with_model.pt