Giters

haoranD / World-Models-Autonomous-Driving-Latest-Survey

A curated list of world models for autonomous driving. Keep updated.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

World-Models-Autonomous-Driving-Latest-Survey

A curated list of world model for autonmous driving. Keep updated.

Papers

  • 2024-Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving arxiv Paper

  • 2024-ViDAR: Visual Point Cloud Forecasting enables Scalable Autonomous Driving CVPR 2024; Pre-training; from Shanghai AI Lab; NuScenes dataset Paper, Code

  • 2024-Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion ICLR 2024; Future Prediction; from Waabi; NuScenes, KITTI Odemetry, Argoverse2 Lidar datasets Paper

  • 2023-DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion model arxiv; Generative AI Paper, Code

  • 2023-MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representations arxiv; Pre-training; CARLA dataset Paper

  • 2023-Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving arxiv; Generative AI, Planning; NuScenes and Waymo datasets Paper

  • 2023-ADriver-I: A General World Model for Autonomous Driving arxiv; Generative AI; NuScenes & one private dataset Paper

  • 2023-OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving arxiv; Occupancy Future Prediction, Planning; Occ3D dataset for Occupancy Future Prediction, NuScenes for motion planning Paper, Code

  • 2023-GAIA-1: A Generative World Model for Autonomous Driving arxiv; Generative AI; Wayve's private data Paper

    Related papers & tutorials to understand this paper:

    FDM for video diffusion decoder: Paper, Code

    Denoising diffusion tutorials: CVPR 2022 tutorial, class from UC Berkeley, Video

  • 2023-DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving arxiv; Generative AI; NuScenes dataset Paper, Code (To be released soon)

  • 2023-Neural World Models for Computer Vision 'PhD Thesis'; from Wayve Paper

  • 2023-UniWorld: Autonomous Driving Pre-training via World Models arxiv; Pre-training; NuScenes dataset Paper

  • 2022-Separating the World and Ego Models for Self-Driving ICLR 2022 workshop on Generalizable Policy Learning in the Physical World; from Yann Lecun's Group Paper, Code

  • 2022-SEM2: Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model NeurIPS 2022 Deep Reinforcement Learning Workshop; RL; CARLA dataset Paper

  • 2022-MILE: Model-Based Imitation Learning for Urban Driving NeurIPS 2022; RL; from Wayve Paper, Code

  • 2022-Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models NeurIPS 2022 Paper, Code

  • 2021-FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras ICCV 2019; Future Prediction; from Wayve; NuScenes, Lyft datasets Paper, Code

  • 2021-Learning to drive from a world on rails CVPR 2021 Oral; RL Paper, Project Page, Code

  • 2019-Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic ICLR 2019; Future Prediction; from Yann Lecun's Group Paper, Code

Workshops/Challenges

  • 2024-CVPR Workshop, Foundation Models for Autonomous Systems, Challenges, Track 4: Predictive World Model Challenges Link

Tutorials/Talks/

  • 2023 from Wayve; Video
  • 2022-Neural World Models for Autonomous Driving Video

Surveys that Contain World Models for AD

  • 2024-Data-Centric Evolution in Autonomous Driving: A Comprehensive Survey of Big Data System, Data Mining, and Closed-Loop Technologies arxiv Paper
  • 2024-Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities arxiv Paper

Other General World Model Papers

  • 2024-Genie: Generative Interactive Environments Deepmind Paper, Website
  • 2024-Sora OpenAI, Generative AI Link, Technical Report
  • 2024-LWM: World Model on Million-Length Video And Language With RingAttention arxiv; Generative AI Paper, Code
  • 2024-WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens arxiv; Generative AI Paper
  • 2024-Video prediction models as rewards for reinforcement learning NeurIPS 2024 Paper, Code
  • 2024-V-JEPA: Revisiting Feature Prediction for Learning Visual Representations from Video from Yann Lecun's Group Paper, Code
  • 2023-I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture CVPR 2023; from Yann Lecun's Group Paper, Code
  • 2023-Temporally Consistent Transformers for Video Generation ICML 2023 Paper, Code
  • 2023-Learning to Model the World with Language arxiv Paper, Code
  • 2023-Transformers are sample-efficient world models ICLR 2023;RL Paper, Code
  • 2023-Gradient-based Planning with World Models arxiv; from Yann Lecun's Group; Planning; Paper
  • 2023-World Models via Policy-Guided Trajectory Diffusion arxiv; RL; Paper
  • 2023-DreamerV3: Mastering diverse domains through world models arxiv;RL; Paper, Code
  • 2022-Daydreamer: World models for physical robot learning CoRL 2022; Robotics Paper, Code
  • 2022-Masked World Models for Visual Control CoRL 2022; Robotics Paper, Code
  • 2022-A Path Towards Autonomous Machine Intelligence openreview; from Yann Lecun's Group; General Roadmap for World Models; Paper; Slides1, Slides2, Slides3; Videos
  • 2021-LEXA:Discovering and Achieving Goals via World Models NeurIPS 2021; Paper, Website & Code
  • 2021-DreamerV2: Mastering Atari with Discrete World Models ICLR 2021; RL; from Google & Deepmind Paper, Code
  • 2020-Dreamer: Dream to Control: Learning Behaviors by Latent Imagination ICLR 2020 Paper, Code
  • 2019-Learning Latent Dynamics for Planning from Pixels ICML 2019 Paper, Code
  • 2018-Model-Based Planning with Discrete and Continuous Actions arxiv; RL, Planning; from Yann Lecun's Group; Paper
  • 2018-Recurrent world models facilitate policy evolution NeurIPS 2018; Paper, Code

Other Related Papers

  • 2023-Occupancy Prediction-Guided Neural Planner for Autonomous Driving ITSC 2023; Planning, Neural Predicted-Guided Planning; Waymo Open Motion dataset Paper

References

About

A curated list of world models for autonomous driving. Keep updated.