Phoenanabe / Interaction-aware-Trajectory-Prediction

A selection of state-of-the-art research materials on trajectory prediction

Interaction-aware Behavior and Trajectory Prediction

This is a checklist of state-of-the-art research materials (datasets, blogs, papers and public codes) related to trajectory prediction. Wish it could be helpful for both academia and industry. (Still updating)

Maintainer: Jiachen Li, Hengbo Ma, Jinning Li (UC Berkeley)

Emails: [e-mail] (Jiachen Li), [e-mail] (Hengbo Ma)

Please feel free to send emails to us for questions, discussion and collaborations.

Also welcome to check the current research in our MSC Lab at UC Berkeley.

A BAIR blog and a survey paper are coming soon!

Datasets

Vehicles and Traffic

• INTERACTION (newly released by UC Berkeley MSC Lab)
• KITTI
• HighD
• NGSIM
• Cyclists
• nuScenes
• BDD100k
• Apolloscapes
• Udacity
• Cityscapes

Pedestrians

• UCY
• ETH
• VIRAT
• KITTI
• ATC
• Daimler
• Central Station
• Town Center
• Edinburgh
• Cityscapes
• Argoverse
• Stanford Drone Dataset
• TrajNet

Sport Players

• Football

Literature and Codes

Survey Papers

1. Human Motion Trajectory Prediction: A Survey, 2019 [paper]
2. Survey on Vision-Based Path Prediction. [paper]
3. A survey on motion prediction and risk assessment for intelligent vehicles. [paper]
4. Autonomous vehicles that interact with pedestrians: A survey of theory and practice. [paper]
5. A literature review on the prediction of pedestrian behavior in urban scenarios, ITSC 2018. [paper]
6. Trajectory data mining: an overview. [paper]

Physics Systems with Interaction

1. Neural Relational Inference for Interacting Systems, ICML 2018. [paper] [code]
2. Factorised Neural Relational Inference for Multi-Interaction Systems, ICML workshop 2019. [paper] [code]
3. Relational inductive biases, deep learning, and graph networks, 2018. [paper]
4. Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions, ICLR 2018. [paper]
5. Graph networks as learnable physics engines for inference and control, ICML 2018. [paper]
6. Visual Interaction Networks, 2017. [paper]
7. Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks, UAI 2018. [paper]
8. Physics-as-Inverse-Graphics: Joint Unsupervised Learning of Objects and Physics from Video, 2019. [paper]
9. A Compositional Object-Based Approach to Learning Physical Dynamics, ICLR 2017. [paper]
10. Interaction Networks for Learning about Objects, Relations and Physics, 2016. [paper][code]
11. Flexible Neural Representation for Physics Prediction, 2018. [paper]
12. A simple neural network module for relational reasoning, 2017. [paper]
13. VAIN: Attentional Multi-agent Predictive Modeling, NIPS 2017. [paper]

Intelligent Vehicles & Traffic

1. Conditional generative neural system for probabilistic trajectory prediction, IROS 2019. [paper]
2. Interaction-aware multi-agent tracking and probabilistic behavior prediction via adversarial learning, ICRA 2019. [paper]
3. Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving, IEEE Trans. Intell. Transport. Systems, 2019. [paper]
4. Coordination and trajectory prediction for vehicle interactions via bayesian generative modeling, IV 2019. [paper]
5. Wasserstein generative learning with kinematic constraints for probabilistic interactive driving behavior prediction, IV 2019.
6. Generic probabilistic interactive situation recognition and prediction: From virtual to real, ITSC 2018. [paper]
7. Generic vehicle tracking framework capable of handling occlusions based on modified mixture particle filter, IV 2018. [paper]
8. Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs. [paper]
9. Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction. [paper]
10. Multi-Step Prediction of Occupancy Grid Maps with Recurrent Neural Networks, CVPR 2019. [paper]
11. Argoverse: 3D Tracking and Forecasting With Rich Maps, CVPR 2019 [paper]
12. Robust Aleatoric Modeling for Future Vehicle Localization, CVPR 2019. [paper]
13. Context-Aware Pedestrian Motion Prediction In Urban Intersections [paper]
14. Probabilistic long-term prediction for autonomous vehicles. [paper]
15. Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network, ITSC 2017. [paper]
16. Desire: Distant future prediction in dynamic scenes with interacting agents, CVPR 2017. [paper][code]
17. Sequence-to-sequence prediction of vehicle trajectory via lstm encoder-decoder architecture. [paper]
18. R2P2: A ReparameteRized Pushforward Policy for diverse, precise generative path forecasting, ECCV 2018. [paper]
19. Long-term planning by short-term prediction. [paper]
20. Predicting trajectories of vehicles using large-scale motion priors. [paper]
21. Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression. [paper]
22. Pedestrian occupancy prediction for autonomous vehicles, IRC 2019. [paper]
23. Vehicle trajectory prediction by integrating physics-and maneuver based approaches using interactive multiple models. [paper]
24. Mobile agent trajectory prediction using bayesian nonparametric reachability trees. [paper]
25. A game-theoretic approach to replanning-aware interactive scene prediction and planning. [paper]
26. Intention-aware online pomdp planning for autonomous driving in a crowd, ICRA 2015. [paper]
27. Long-term path prediction in urban scenarios using circular distributions. [paper]
28. Motion Prediction of Traffic Actors for Autonomous Driving using Deep Convolutional Networks. [paper]
29. Deep learning driven visual path prediction from a single image. [paper]
30. Context-based path prediction for targets with switching dynamics. [paper]
31. Imitating driver behavior with generative adversarial networks. [paper][code]
32. Understanding interactions between traffic participants based on learned behaviors. [paper]
33. Infogail: Interpretable imitation learning from visual demonstrations. [paper][code]
34. Deep Imitative Models for Flexible Inference, Planning, and Control. [paper]
35. Infer: Intermediate representations for future prediction. [paper][code]
36. Patch to the future: Unsupervised visual prediction, CVPR 2014. [paper]
37. Visual path prediction in complex scenes with crowded moving objects, CVPR 2016. [paper]
38. Generative multi-agent behavioral cloning. [paper]
39. Multi-agent tensor fusion for contextual trajectory prediction. [paper]
40. Deep Sequence Learning with Auxiliary Information for Traffic Prediction, KDD 2018. [paper], [code]

Mobile Robots

1. Bayesian intention inference for trajectory prediction with an unknown goal destination, IROS 2015. [paper]
2. Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning, ICRA 2017. [paper]
3. Augmented dictionary learning for motion prediction, ICRA 2016. [paper]
4. Learning to predict trajectories of cooperatively navigating agents, ICRA 2014. [paper]
5. Predicting future agent motions for dynamic environments, ICMLA 2016. [paper]
6. Multimodal probabilistic model-based planning for human-robot interaction, ICRA 2018. [paper][code]

Pedestrians

1. A data-driven model for interaction-aware pedestrian motion prediction in object cluttered environments, ICRA 2018. [paper]
2. Move, Attend and Predict: An attention-based neural model for people’s movement prediction, 2018 Pattern Recognition Letters [paper]
3. Situation-Aware Pedestrian Trajectory Prediction with Spatio-Temporal Attention Model, CVWW 2019. [paper]
4. GD-GAN: Generative Adversarial Networks for Trajectory Prediction and Group Detection in Crowds, ACCV 2018, [paper], [demo]
5. Path predictions using object attributes and semantic environment, VISIGRAPP 2019. [paper]
6. Probabilistic Path Planning using Obstacle Trajectory Prediction, CoDS-COMAD 2019. [paper]
7. Human Trajectory Prediction using Adversarial Loss, 2019. [paper]
8. Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANs, CVPR 2019. [Precognition Workshop], [paper], [code]
9. Peeking into the Future: Predicting Future Person Activities and Locations in Videos, CVPR 2019. [paper], [code]
10. Learning to Infer Relations for Future Trajectory Forecast, CVPR 2019. [paper]
11. TraPHic: Trajectory Prediction in Dense and Heterogeneous Traffic Using Weighted Interactions, CVPR 2019. [paper]
12. Which Way Are You Going? Imitative Decision Learning for Path Forecasting in Dynamic Scenes, CVPR 2019. [paper]
13. Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction, CVPR 2019. [paper]
14. Ss-lstm: a hierarchical lstm model for pedestrian trajectory prediction, WACV 2018. [paper]
15. Sophie: An attentive gan for predicting paths compliant to social and physical constraints, CVPR 2019. [paper][code]
16. Social Attention: Modeling Attention in Human Crowds, ICRA 2018. [paper][code]
17. Goal-directed pedestrian prediction, ICCV 2015. [paper]
18. Pedestrian prediction by planning using deep neural networks, ICRA 2018. [paper]
19. Joint long-term prediction of human motion using a planning-based social force approach, ICRA 2018. [paper]
20. Human motion prediction under social grouping constraints, IROS 2018. [paper]
21. Social LSTM: Human trajectory prediction in crowded spaces, CVPR 2016. [paper][code]
22. Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks, CVPR 2018. [paper][code]
23. Group LSTM: Group Trajectory Prediction in Crowded Scenarios, ECCV 2018. [paper]
24. Comparison and evaluation of pedestrian motion models for vehicle safety systems, ITSC 2016. [paper]
25. Learning intentions for improved human motion prediction. [paper]
26. Walking Ahead: The Headed Social Force Model. [paper]
27. Real-Time Predictive Modeling and Robust Avoidance of Pedestrians with Uncertain, Changing Intentions. [paper]
28. Behavior estimation for a complete framework for human motion prediction in crowded environments, ICRA 2014. [paper]
29. Pedestrian’s trajectory forecast in public traffic with artificial neural network, ICPR 2014. [paper]
30. Age and Group-driven Pedestrian Behaviour: from Observations to Simulations. [paper]
31. Mx-lstm: mixing tracklets and vislets to jointly forecast trajectories and head poses, CVPR 2018. [paper]
32. Real-time certified probabilistic pedestrian forecasting. [paper]
33. Structural-RNN: Deep learning on spatio-temporal graphs, CVPR 2016. [paper][code]
34. Intent-aware long-term prediction of pedestrian motion, ICRA 2016. [paper]
35. Will the pedestrian cross? A study on pedestrian path prediction. [paper]
36. BRVO: Predicting pedestrian trajectories using velocity-space reasoning. [paper]
37. Context-based pedestrian path prediction, ECCV 2014. [paper]
38. A multiple-predictor approach to human motion prediction, ICRA 2017. [paper]
39. Forecasting interactive dynamics of pedestrians with fictitious play, CVPR 2017. [paper]
40. Pedestrian path, pose, and intention prediction through gaussian process dynamical models and pedestrian activity recognition. [paper]
41. Trajectory analysis and prediction for improved pedestrian safety: Integrated framework and evaluations. [paper]
42. Predicting and recognizing human interactions in public spaces. [paper]
43. Multimodal Interaction-aware Motion Prediction for Autonomous Street Crossing. [paper]
44. Pedestrian path prediction using body language traits. [paper]
45. Intent prediction of pedestrians via motion trajectories using stacked recurrent neural networks. [paper]
46. Context-based detection of pedestrian crossing intention for autonomous driving in urban environments, IROS 2016. [paper]
47. The simpler the better: Constant velocity for pedestrian motion prediction. [paper]
48. Transferable pedestrian motion prediction models at intersections. [paper]
49. Pedestrian trajectory prediction in extremely crowded scenarios. [paper]
50. Forecast the plausible paths in crowd scenes, IJCAI 2017. [paper]
51. Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression. [paper]
52. Novel planning-based algorithms for human motion prediction, ICRA 2016. [paper]
53. Learning collective crowd behaviors with dynamic pedestrian-agents. [paper]
54. Srlstm: State refinement for lstm towards pedestrian trajectory prediction. [paper]
55. Location-velocity attention for pedestrian trajectory prediction, WACV 2019. [paper]
56. Bi-prediction: pedestrian trajectory prediction based on bidirectional lstm classification, DICTA 2017. [paper]
57. Modeling spatial-temporal dynamics of human movements for predicting future trajectories, AAAI 2015. [paper]
58. Probabilistic map-based pedestrian motion prediction taking traffic participants into consideration. [paper]
59. Unsupervised robot learning to predict person motion, ICRA 2015. [paper]
60. A controlled interactive multiple model filter for combined pedestrian intention recognition and path prediction, ITSC 2015. [paper]
61. Learning social etiquette: Human trajectory understanding in crowded scenes, ECCV 2016. [paper][code]
62. A Computationally Efficient Model for Pedestrian Motion Prediction, ECC 2018. [paper]
63. GLMP-realtime pedestrian path prediction using global and local movement patterns, ICRA 2016. [paper]
64. Aggressive, Tense or Shy? Identifying Personality Traits from Crowd Videos, IJCAI 2017. [paper]
65. Knowledge transfer for scene-specific motion prediction, ECCV 2016. [paper]
66. Context-aware trajectory prediction, ICPR 2018. [paper]
67. Set-based prediction of pedestrians in urban environments considering formalized traffic rules, ITSC 2018. [paper]
68. Natural vision based method for predicting pedestrian behaviour in urban environments, ITSC 2017. [paper]
69. Building prior knowledge: A markov based pedestrian prediction model using urban environmental data, ICARCV 2018. [paper]
70. Pedestrian Trajectory Prediction in Extremely Crowded Scenarios, Sensors, 2019. [paper]
71. Depth Information Guided Crowd Counting for Complex Crowd Scenes, 2018.
72. Tracking by Prediction: A Deep Generative Model for Mutli-Person Localisation and Tracking, WACV 2018.
73. “Seeing is Believing”: Pedestrian Trajectory Forecasting Using Visual Frustum of Attention, WACV 2018.
74. Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty, CVPR 2018. [paper], [code+data]
75. Encoding Crowd Interaction with Deep Neural Network for Pedestrian Trajectory Prediction, CVPR 2018. [paper], [code]
76. Human Trajectory Prediction using Spatially aware Deep Attention Models, 2017. [paper]
77. Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection, 2017. [paper]
78. Forecasting Interactive Dynamics of Pedestrians with Fictitious Play, CVPR 2017. [paper]
79. STF-RNN: Space Time Features-based Recurrent Neural Network for predicting People Next Location, SSCI 2016. [code]

Sport Players

1. Generating long-term trajectories using deep hierarchical networks. [paper]
2. Diverse Generation for Multi-Agent Sports Games, CVPR 2019. [paper]
3. Stochastic Prediction of Multi-Agent Interactions from Partial Observations, ICLR 2019. [paper]
4. Generative Multi-Agent Behavioral Cloning, ICML 2018. [paper]
5. Generating Multi-Agent Trajectories using Programmatic Weak Supervision, ICLR 2019. [paper]
6. Where Will They Go? Predicting Fine-Grained Adversarial Multi-Agent Motion using Conditional Variational Autoencoders, ECCV 2018. [paper]
7. Coordinated Multi-Agent Imitation Learning, ICML 2017. [paper]
8. Learning Fine-Grained Spatial Models for Dynamic Sports Play Prediction, ICDM 2014. [paper]

Benchmark and Evaluation Metrics

1. Towards a fatality-aware benchmark of probabilistic reaction prediction in highly interactive driving scenarios, ITSC 2018. [paper]
2. Trajnet: Towards a benchmark for human trajectory prediction. [website]
3. How good is my prediction? Finding a similarity measure for trajectory prediction evaluation, ITSC 2017. [paper]

Others

1. Using road topology to improve cyclist path prediction. [paper]
2. Cyclist trajectory prediction using bidirectional recurrent neural networks, AI 2018. [paper]
3. Trajectory prediction of cyclists using a physical model and an artificial neural network. [paper]
4. Road infrastructure indicators for trajectory prediction. [paper]