Here is a collection of research papers for awesome knowledge-driven autonomous driving (AD). The repository will be continuously updated to track the frontier of knowledge-driven AD.
🌟 Welcome to star and contribute to (PR) this awesome knowledge-driven AD! 🌟
[2023.10.24] New: Our survey on the knowledge-driven AD is coming soon! [2023.10.24] New: We release the awesome knowledge-driven AD!
- Grounding human-to-vehicle advice for self-driving vehicles [
CVPR 2019] - ADAPT: Action-aware Driving Caption Transformer [
ICRA 2023, Github] - Talk2Car: Taking Control of Your Self-Driving Car [
EMNLP-IJNLP 2019, Project] - Textual explanations for self-driving vehicles [
ECCV 2018, Github] - Drive Like a Human: Rethinking Autonomous Driving with Large Language Models [
arxiv 2023, Github] - DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model] [
arxiv 2023, Project] - DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models [
arxiv 2023, Github] - GPT-Driver: Learning to Drive with GPT [
arxiv 2023, Github] - Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving [
arxiv 2023, Github] - LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving [
arxiv 2023, Project] - Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles [
arxiv 2023] - Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles [
arxiv 2023] - SurrealDriver: Designing Generative Driver Agent Simulation Framework in Urban Contexts based on Large Language Model [
arxiv 2023] - Language-Guided Traffic Simulation via Scene-Level Diffusion [
arxiv 2023] - Language Prompt for Autonomous Driving [
arxiv 2023, Github] - Talk2BEV: Language-Enhanced Bird's Eye View (BEV) Maps [
arxiv 2023, Project, Github] - BEVGPT: Generative Pre-trained Large Model for Autonomous Driving Prediction, Decision-Making, and Planning [
arxiv 2023] - DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving [
arxiv 2023] - MagicDrive: Street View Generation with Diverse 3D Geometry Control [
arxiv 2023] - GAIA-1: A Generative World Model for Autonomous Driving [
arxiv 2023] - HiLM-D: Towards High-Resolution Understanding in Multimodal Large Language Models for Autonomous Driving [
arxiv 2023] - Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving [
arxiv 2023] - OpenAnnotate3D: Open-Vocabulary Auto-Labeling System for Multi-modal 3D Data [
arxiv 2023, Github] - LangProp: A Code Optimization Framework Using Language Models Applied to Driving [
openreview 2023, Github] - Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion [
openreview 2023] - Planning with an Ensemble of World Models [
openreview 2023] - Large Language Models Can Design Game-Theoretic Objectives for Multi-Agent Planning [
openreview 2023] - TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction [
arxiv 2023] - BEV-CLIP: Multi-Modal BEV Retrieval Methodology for Complex Scene in Autonomous Driving [
arxiv 2023] - Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [
NeurIPS 2023, Github] - Large Language Models Can Design Game-theoretic Objectives for Multi-Agent Planning [
openreview 2023] - Semantic Anomaly Detection with Large Language Models [
arxiv 2023] - Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving [
arxiv 2023] - Drama: Joint risk localization and captioning in driving [
WACV 2023] - 3D Dense Captioning Beyond Nouns: A Middleware for Autonomous Driving [
openreview 2023] - SwapTransformer: Highway Overtaking Tactical Planner Model via Imitation Learning on OSHA Dataset [
openreview 2023] - NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenario [
arxiv 2023, Github] - Language Prompt for Autonomous Driving [
arxiv 2023, Github] - Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models [
arxiv 2023]
- [WACV2024 Workshop] MAPLM: A Large-Scale Vision-Language Dataset for Map and Traffic Scene Understanding
- [Blog] LINGO-1: Exploring Natural Language for Autonomous Driving
- [Blog] Introducing GAIA-1: A Cutting-Edge Generative AI Model for Autonomy
Awesome Knowledge-driven Autonomous Driving is released under the Apache 2.0 license.