papers.md

Paper Collection

We list key challenges from a wide span of candidate concerns, as well as trending methodologies.

• Survey
• Language / VLM for Driving
  • Review for VLM in Driving
  • Papers for VLM in Driving
• World Model & Model-based RL
• Multi-sensor Fusion
• Multi-task Learning
• Interpretability
  • Review for Interpretability
  • Attention Visualization
  • Interpretable Tasks
  • Cost Learning
  • Linguistic Explainability
  • Uncertainty Modeling
  • Counterfactual Explanations and Causal Inference
• Visual Abstraction / Representation Learning
• Policy Distillation
• Causal Confusion
• Robustness
  • Long-tailed Distribution
  • Covariate Shift
  • Domain Adaptation
• Affordance Learning
• BEV
• Transformer
• V2V Cooperative
• Distributed RL
• Data-driven Simulation
  • Parameter Initialization
  • Traffic Simulation
  • Sensor Simulation

Survey

• End-to-End Autonomous Driving: Challenges and Frontiers [TPAMI2024]

• Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey [TIV2023]

• Rethinking Integration of Prediction and Planning in Deep Learning-Based Automated Driving Systems: A Review [arXiv2023]

• End-to-end Autonomous Driving using Deep Learning: A Systematic Review [arXiv2023]

• Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives [TIV2023]

• Imitation Learning: Progress, Taxonomies and Challenges [TNNLS2022]

• A Review of End-to-End Autonomous Driving in Urban Environments [Access2022]

• A Survey on Imitation Learning Techniques for End-to-End Autonomous Vehicles [TITS2022]

• Deep Reinforcement Learning for Autonomous Driving: A Survey [TITS2021]

• A Survey of Deep RL and IL for Autonomous Driving Policy Learning [TITS2021]

• A Survey of End-to-End Driving: Architectures and Training Methods [TNNLS2020]

• Learning to Drive by Imitation: An Overview of Deep Behavior Cloning Methods [TIV2020]

• Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art [book]

(back to top)

Language / VLM for Driving

Review for VLM in Driving

• Vision Language Models in Autonomous Driving: A Survey and Outlook [TIV2024][Code]

• A Survey on Multimodal Large Language Models for Autonomous Driving [WACVWorkshop2024]

• Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities [arXiv2024][Code]

• LLM4Drive: A Survey of Large Language Models for Autonomous Driving [arXiv2023]

Papers for VLM in Driving

• DriveLM: Driving with Graph Visual Question Answering [ECCV2024][Code]

• Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving [ECCV2024][Code]

• Asynchronous Large Language Model Enhanced Planner for Autonomous Driving [ECCV2024][Code]

• LMDrive: Closed-Loop End-to-End Driving with Large Language Models [CVPR2024][Code]

• Driving Everywhere with Large Language Model Policy Adaptation [CVPR2024][Code]

• VLP: Vision Language Planning for Autonomous Driving [CVPR2024]

• A Language Agent for Autonomous Driving [COLM2024][Code]

• DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model [RAL2024]

• Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving [ICRA2024][Code]

• Prompting Multi-Modal Tokens to Enhance End-to-End Autonomous Driving Imitation Learning with LLMs [ICRA2024]

• DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiences [IROS2024]

• Pix2Planning: End-to-End Planning by Vision-language Model for Autonomous Driving on Carla Simulator [IV2024]

• LangProp: A code optimization framework using Large Language Models applied to driving [ICLRWorkshop2024][Code]

• SimpleLLM4AD: An End-to-End Vision-Language Model with Graph Visual Question Answering for Autonomous Driving [arXiv2024]

• An LLM-enhanced Multi-objective Evolutionary Search for Autonomous Driving Test Scenario Generation [arXiv2024]

• OmniDrive: A Holistic LLM-Agent Framework for Autonomous Driving with 3D Perception, Reasoning and Planning [arXiv2024][Code]

• Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Driving [arXiv2024][Code]

• Is a 3D-Tokenized LLM the Key to Reliable Autonomous Driving? [arXiv2024]

• DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving [arXiv2024]

• RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model [arXiv2024]

• DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models [arXiv2024]

• Hybrid Reasoning Based on Large Language Models for Autonomous Car Driving [arXiv2024]

• DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Driving [arXiv2024]

• LingoQA: Video Question Answering for Autonomous Driving [arXiv2023][Code]

• Dolphins: Multimodal Language Model for Driving [arXiv2023][Code]

• GPT-Driver: Learning to Drive with GPT [arXiv2023]

• Language Prompt for Autonomous Driving [arXiv2023][Code]

• DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents [EMNLP2022(Findings)][Code]

• LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action [CoRL2022]

• Ground then Navigate: Language-guided Navigation in Dynamic Scenes [arXiv2022]

• Generating Landmark Navigation Instructions from Maps as a Graph-to-Text Problem [ACL2021]

• Advisable Learning for Self-Driving Vehicles by Internalizing Observation-to-Action Rules [CVPR2020]

• Conditional Driving from Natural Language Instructions [CoRL2019]

• Grounding Human-to-Vehicle Advice for Self-driving Vehicles [CVPR2019][Dataset]

• Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars [IROS2019]

• Talk2Car: Taking Control of Your Self-Driving Car [EMNLP2019]

• TOUCHDOWN: Natural Language Navigation and Spatial Reasoning in Visual Street Environments [CVPR2019]

• Learning to Navigate in Cities Without a Map [NeurIPS2018][Code]

(back to top)

World Model & Model-based RL

• Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2) [ECCV2024]

• WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generation [ECCV2024][Code]

• OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving [ECCV2024][Code]

• Visual Point Cloud Forecasting enables Scalable Autonomous Driving [CVPR2024][Code]

• GenAD: Generalized Predictive Model for Autonomous Driving [CVPR2024][Code]

• DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving [CVPR2024]

• Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability [arXiv2024][Code]

• Enhancing End-to-End Autonomous Driving with Latent World Model [arXiv2024][Code]

• BEVWorld: A Multimodal World Model for Autonomous Driving via Unified BEV Latent Space [arXiv2024][Code]

• Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation [arXiv2024][Code]

• DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation [arXiv2024][Code]

• CarDreamer: Open-Source Learning Platform for World Model based Autonomous Driving [arXiv2024][Code]

• GAIA-1: A Generative World Model for Autonomous Driving [arXiv2023]

• ADriver-I: A General World Model for Autonomous Driving [arXiv2023]

• DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving [arXiv2023][Code]

• Uncertainty-Aware Model-Based Offline Reinforcement Learning for Automated Driving [RAL2023]

• Model-Based Imitation Learning for Urban Driving [NeurIPS2022)][Code]

• Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models [NeurIPS2022][Code]

• Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model [NeurIPSWorkshop2022]

• Addressing Optimism Bias in Sequence Modeling for Reinforcement Learning [ICML2022]

• Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning [TITS2022][Code]

• Learning To Drive From a World on Rails [ICCV2021][Code]

• Uncertainty-Aware Model-Based Reinforcement Learning: Methodology and Application in Autonomous Driving [IV2022]

• UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning [NeurIPSWorkshop2021]

• Deductive Reinforcement Learning for Visual Autonomous Urban Driving Navigation [TNNLS2021]

• Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic [ICLR2019]

(back to top)

Multi-sensor Fusion

• DualAT: Dual Attention Transformer for End-to-End Autonomous Driving [ICRA2024]

• DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mamba [arXiv2024][Code]

• MaskFuser: Masked Fusion of Joint Multi-Modal Tokenization for End-to-End Autonomous Driving [arXiv2024]

• M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Driving [arXiv2024]

• Utilizing Navigation Paths to Generate Target Points for Enhanced End-to-End Autonomous Driving Planning [arXiv2024]

• Hidden Biases of End-to-End Driving Models [ICCV2023][Code]

• Learning to Drive Anywhere [CoRL2023]

• Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving [CVPR2023][Code]

• ReasonNet: End-to-End Driving with Temporal and Global Reasoning [CVPR2023]

• Scaling Vision-Based End-to-End Autonomous Driving with Multi-View Attention Learning [IROS2023]

• FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Driving [arXiv2023]

• Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model [NeurIPSWorkshop2022]

• End-to-end Autonomous Driving with Semantic Depth Cloud Mapping and Multi-agent [IV2022]

• MMFN: Multi-Modal-Fusion-Net for End-to-End Driving [IROS2022][Code]

• Interpretable End-to-End Urban Autonomous Driving With Latent Deep Reinforcement Learning [TITS2022][Code]

• Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer [CoRL2022][Code]

• Learning from All Vehicles [CVPR2022][Code]

• TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving [TPAMI2022][Code]

• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [CVPR2021][Code]

• Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement Learning [arXiv2021]

• Multi-modal Sensor Fusion-based Deep Neural Network for End-to-end Autonomous Driving with Scene Understanding [IEEESJ2020]

• Probabilistic End-to-End Vehicle Navigation in Complex Dynamic Environments With Multimodal Sensor Fusion [RAL2020]

• Multimodal End-to-End Autonomous Driving [TITS2020]

• End-To-End Interpretable Neural Motion Planner [CVPR2019]

• Does Computer Vision Matter for Action? [ScienceRobotics2019]

• End-To-End Multi-Modal Sensors Fusion System For Urban Automated Driving [NeurIPSWorkshop2018]

• MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving [WACV2019]

• LiDAR-Video Driving Dataset: Learning Driving Policies Effectively [CVPR2018]

(back to top)

Multi-task Learning

• PARA-Drive: Parallelized Architecture for Real-time Autonomous Driving [CVPR2024]

• Planning-oriented Autonomous Driving [CVPR2023][Code]

• Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving [CVPR2023][Code]

• Coaching a Teachable Student [CVPR2023]

• ReasonNet: End-to-End Driving with Temporal and Global Reasoning [CVPR2023]

• Hidden Biases of End-to-End Driving Models [ICCV2023][Code]

• VAD: Vectorized Scene Representation for Efficient Autonomous Driving [ICCV2023][Code]

• TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving [TPAMI2022][Code]

• Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline [NeurIPS2022] [Code]

• Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer [CoRL2022][Code]

• Learning from All Vehicles [CVPR2022][Code]

• Multi-Task Learning With Attention for End-to-End Autonomous Driving [CVPRWorkshop2021]

• NEAT: Neural Attention Fields for End-to-End Autonomous Driving [ICCV2021][Code]

• SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning [CoRL2020][Code]

• Urban Driving with Conditional Imitation Learning [ICRA2020]

• Multi-modal Sensor Fusion-based Deep Neural Network for End-to-end Autonomous Driving with Scene Understanding [IEEESJ2020]

• Multi-task Learning with Future States for Vision-based Autonomous Driving [ACCV2020]

• Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks [AAAI2019][Code]

• MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving [WACV2019]

• Intentnet: Learning to Predict Intention from Raw Sensor Data [CoRL2018]

• Rethinking Self-driving: Multi-task Knowledge for Better Generalization and Accident Explanation Ability [arXiv2018][Code]

• Learning End-to-end Autonomous Driving using Guided Auxiliary Supervision [ICVGIP2018]

• End-to-end Learning of Driving Models from Large-scale Video Datasets [CVPR2017][Code]

(back to top)

Interpretability

Review for Interpretability

• Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review [arXiv2024]

• Explainability of Deep Vision-based Autonomous Driving Systems: Review and challenges [IJCV2022]

Attention Visualization

• Guiding Attention in End-to-End Driving Models [IV2024]

• Scaling Self-Supervised End-to-End Driving with Multi-View Attention Learning [arxiv2023]

• PlanT: Explainable Planning Transformers via Object-Level Representations [CoRL2022][Code]

• MMFN: Multi-Modal-Fusion-Net for End-to-End Driving [IROS2022][Code]

• TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving [TPAMI2022][Code]

• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [CVPR2021][Code]

• Multi-Task Learning With Attention for End-to-End Autonomous Driving [CVPRWorkshop2021]

• NEAT: Neural Attention Fields for End-to-End Autonomous Driving [ICCV2021][Code]

• Explaining Autonomous Driving by Learning End-to-End Visual Attention [CVPRWorkshop2020]

• Visual Explanation by Attention Branch Network for End-to-end Learning-based Self-driving [IV2019]

• Deep Object-Centric Policies for Autonomous Driving [ICRA2019]

• Textual Explanations for Self-Driving Vehicles [ECCV2018][Code]

• Learning End-to-end Autonomous Driving using Guided Auxiliary Supervision [ICVGIP2018]

• Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention [ICCV2017]

(back to top)

Interpretable Tasks

• Planning-oriented Autonomous Driving [CVPR2023][Code]

• Hidden Biases of End-to-End Driving Models [ICCV2023][Code]

• VAD: Vectorized Scene Representation for Efficient Autonomous Driving [ICCV2023][Code]

• Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer [CoRL2022][Code]

• TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving [TPAMI2022][Code]

• Learning from All Vehicles [CVPR2022][Code]

• Ground then Navigate: Language-guided Navigation in Dynamic Scenes [arXiv2022]

• NEAT: Neural Attention Fields for End-to-End Autonomous Driving [ICCV2021][Code]

• Multi-Task Learning With Attention for End-to-End Autonomous Driving [CVPRWorkshop2021]

• Urban Driving with Conditional Imitation Learning [ICRA2020]

• Using Eye Gaze to Enhance Generalization of Imitation Networks to Unseen Environments [TNNLS2020]

• Multi-modal Sensor Fusion-based Deep Neural Network for End-to-end Autonomous Driving with Scene Understanding [IEEESJ2020]

• Rethinking Self-driving: Multi-task Knowledge for Better Generalization and Accident Explanation Ability [arXiv2018][Code]

• Learning End-to-end Autonomous Driving using Guided Auxiliary Supervision [ICVGIP2018]

• End-to-end Learning of Driving Models from Large-scale Video Datasets [CVPR2017][Code]

(back to top)

Cost Learning

• QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Driving [ICRA2024]

• ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning [ECCV2022][Code]

• Differentiable Raycasting for Self-Supervised Occupancy Forecasting [ECCV2022][Code]

• MP3: A Unified Model To Map, Perceive, Predict and Plan [CVPR2021]

• Safe Local Motion Planning With Self-Supervised Freespace Forecasting [CVPR2021]

• LookOut: Diverse Multi-Future Prediction and Planning for Self-Driving [ICCV2021]

• DSDNet: Deep Structured Self-driving Network [ECCV2020]

• Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representations [ECCV2020]

• End-To-End Interpretable Neural Motion Planner [CVPR2019]

(back to top)

Linguistic Explainability

• ADAPT: Action-aware Driving Caption Transformer [ICRA2023][Code]

• Driving Behavior Explanation with Multi-level Fusion [PR2022][Code]

• Explainable Object-Induced Action Decision for Autonomous Vehicles [CVPR2020]

• Textual Explanations for Self-Driving Vehicles [ECCV2018][Code]

(back to top)

Uncertainty Modeling

• UAP-BEV: Uncertainty Aware Planning using Bird's Eye View generated from Surround Monocular Images [CASE2023][Code]

• Probabilistic End-to-End Vehicle Navigation in Complex Dynamic Environments With Multimodal Sensor Fusion [RAL2020]

• Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts? [ICML2020][Code]

• VTGNet: A Vision-Based Trajectory Generation Network for Autonomous Vehicles in Urban Environments [TIV2020][Code]

• Visual-based Autonomous Driving Deployment from a Stochastic and Uncertainty-aware Perspective [IROS2019]

• Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control [arXiv2018]

(back to top)

Counterfactual Explanations and Causal Inference

• OCTET: Object-aware Counterfactual Explanation [CVPR2023][Code]

• STEEX: Steering Counterfactual Explanations with Semantics [ECCV2022][Code]

• Who Make Drivers Stop? Towards Driver-centric Risk Assessment: Risk Object Identification via Causal Inference [IROS2020]

(back to top)

Visual Abstraction / Representation Learning

• Visual Point Cloud Forecasting enables Scalable Autonomous Driving [CVPR2024][Code]

• DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving [CVPR2024]

• End-to-End Autonomous Driving without Costly Modularization and 3D Manual Annotation [arXiv2024]

• Scene as Occupancy [ICCV2023][Code]

• DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving [ICCV2023][Code]

• Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modeling [ICLR2023][Code]

• An End-to-End Autonomous Driving Pre-trained Transformer Model for Multi-Behavior-Optimal Trajectory Generation [ITSC2023]

• Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning [NeurIPS2022]

• Task-Induced Representation Learning [ICLR2022][Code]

• Learning Generalizable Representations for Reinforcement Learning via Adaptive Meta-learner of Behavioral Similarities [ICLR2022][Code]

• Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretraining [ECCV2022][Code]

• Segmented Encoding for Sim2Real of RL-based End-to-End Autonomous Driving [IV2022]

• GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Driving [arXiv2021]

• Latent Attention Augmentation for Robust Autonomous Driving Policies [IROS2021]

• Multi-Task Long-Range Urban Driving Based on Hierarchical Planning and Reinforcement Learning [ITSC2021]

• Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement Learning [arXiv2021]

• A Versatile and Efficient Reinforcement Learning Framework for Autonomous Driving [arxiv2021]

• Deductive Reinforcement Learning for Visual Autonomous Urban Driving Navigation [TNNLS2021]

• End-to-End Model-Free Reinforcement Learning for Urban Driving Using Implicit Affordances [CVPR2020]

• Toward Deep Reinforcement Learning without a Simulator: An Autonomous Steering Example [AAAI2018]

(back to top)

Policy Distillation

• Feedback-Guided Autonomous Driving [CVPR2024]

• On the Road to Portability: Compressing End-to-End Motion Planner for Autonomous Driving [CVPR2024]

• Knowledge Distillation from Single-Task Teachers to Multi-Task Student for End-to-End Autonomous Driving [AAAI2024][Code]

• Multi-Task Adaptive Gating Network for Trajectory Distilled Control Prediction [RAL2024]

• DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving [ICCV2023][Code]

• Coaching a Teachable Student [CVPR2023]

• Detrive: Imitation Learning with Transformer Detection for End-to-End Autonomous Driving [DISA2023]

• Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline [NeurIPS2022][Code]

• Learning from All Vehicles [CVPR2022][Code]

• End-to-End Urban Driving by Imitating a Reinforcement Learning Coach [ICCV2021][Code]

• Learning To Drive From a World on Rails [ICCV2021][Code]

• Learning by Cheating [CoRL2020][Code]

• SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning [CoRL2020][Code]

• Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks [AAAI2019][Code]

(back to top)

Causal Confusion

• Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving? [CVPR2024][Code]

• Exploring the Causality of End-to-End Autonomous Driving [arXiv2024][Code]

• DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving [ICCV2023][Code]

• Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes [arxiv2023]

• Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving [arXiv2023]

• Fighting Fire with Fire: Avoiding DNN Shortcuts through Priming [ICML2022]

• Resolving Copycat Problems in Visual Imitation Learning via Residual Action Prediction [ECCV2022]

• Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning [NeurIPS2021][Code]

• Keyframe-Focused Visual Imitation Learning [ICML2021][Code]

• Fighting Copycat Agents in Behavioral Cloning from Observation Histories [NeurIPS2020]

• Shortcut Learning in Deep Neural Networks [NatureMachineIntelligence2020]

• Causal Confusion in Imitation Learning [NeurIPS2019]

• ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst [RSS2019]

• Exploring the Limitations of Behavior Cloning for Autonomous Driving [ICCV2019][Code]

• Off-Road Obstacle Avoidance through End-to-End Learning [NeurIPS2005]

(back to top)

Robustness

Long-tailed Distribution

• An LLM-enhanced Multi-objective Evolutionary Search for Autonomous Driving Test Scenario Generation [arXiv2024]

• Unleashing Generalization of End-to-End Autonomous Driving with Controllable Long Video Generation [arXiv2024][Code]

• CAT: Closed-loop Adversarial Training for Safe End-to-End Driving [CoRL2023]

• Adversarial Driving: Attacking End-to-End Autonomous Driving [IV2023][Code]

• KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients [ECCV2022][Code]

• AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles [CVPR2021]

• TrafficSim: Learning To Simulate Realistic Multi-Agent Behaviors [CVPR2021]

• Multimodal Safety-Critical Scenarios Generation for Decision-Making Algorithms Evaluation [RAL2021]

• Learning by Cheating [CoRL2020][Code]

• Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method [IROS2020]

• Enhanced Transfer Learning for Autonomous Driving with Systematic Accident Simulation [IROS2020]

• Improving the Generalization of End-to-End Driving through Procedural Generation [arXiv2020][Code]

• Generating Adversarial Driving Scenarios in High-Fidelity Simulators [ICRA2019]

• Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation [NeurIPS2018]

• Microscopic Traffic Simulation using SUMO [ITSC2018]

(back to top)

Covariate Shift

• Exploring Data Aggregation in Policy Learning for Vision-Based Urban Autonomous Driving [CVPR2020]

• Learning by Cheating [CoRL2020][Code]

• Agile Autonomous Driving using End-to-End Deep Imitation Learning [RSS2018]

• Query-Efficient Imitation Learning for End-to-End Simulated Driving [AAAI2017]

• Meta learning Framework for Automated Driving [arXiv2017]

• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning [AISTATS2011]

(back to top)

Domain Adaptation

• Uncertainty-Guided Never-Ending Learning to Drive [CVPR2024][Code]

• A Comparison of Imitation Learning Pipelines for Autonomous Driving on the Effect of Change in Ego-vehicle [IV2024]

• Balanced Training for the End-to-End Autonomous Driving Model Based on Kernel Density Estimation [IV2024]

• ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Driving [arXiv2024]

• DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving [ICCV2023][Code]

• Learning to Drive Anywhere [CoRL2023]

• SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation [CVPR2022][Code]

• Learning Interactive Driving Policies via Data-driven Simulation [ICRA2022]

• Segmented Encoding for Sim2Real of RL-based End-to-End Autonomous Driving [IV2022]

• Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation [AAAI2021][Code]

• A Versatile and Efficient Reinforcement Learning Framework for Autonomous Driving [arxiv2021]

• Enhanced Transfer Learning for Autonomous Driving with Systematic Accident Simulation [IROS2020]

• Simulation-Based Reinforcement Learning for Real-World Autonomous Driving [ICRA2020][Code]

• Learning to Drive from Simulation without Real World Labels [ICRA2019]

• Visual-based Autonomous Driving Deployment from a Stochastic and Uncertainty-aware Perspective [IROS2019]

• Virtual to Real Reinforcement Learning for Autonomous Driving [BMVC2017][Code]

(back to top)

Affordance Learning

• Enhance Planning with Physics-informed Safety Controller for End-to-end Autonomous Driving [arXiv2024]

• Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer [CoRL2022][Code]

• Multi-Task Learning With Attention for End-to-End Autonomous Driving [CVPRWorkshop2021]

• Driver Behavioral Cloning for Route Following in Autonomous Vehicles Using Task Knowledge Distillation [TIV2022]

• Policy-Based Reinforcement Learning for Training Autonomous Driving Agents in Urban Areas With Affordance Learning [TITS2021]

• Conditional Affordance Learning for Driving in Urban Environments [CoRL2018][Code]

(back to top)

BEV

• Visual Point Cloud Forecasting enables Scalable Autonomous Driving [CVPR2024][Code]

• DualAD: Disentangling the Dynamic and Static World for End-to-End Driving [CVPR2024]

• ParkingE2E: Camera-based End-to-end Parking Network, from Images to Planning [IROS2024][Code]

• E2E Parking: Autonomous Parking by the End-to-end Neural Network on the CARLA Simulator [IV2024][Code]

• BEVGPT: Generative Pre-trained Large Model for Autonomous Driving Prediction, Decision-Making, and Planning [AAAI2024]

• PolarPoint-BEV: Bird-eye-view Perception in Polar Points for Explainable End-to-end Autonomous Driving [TIV2024]

• Hybrid-Prediction Integrated Planning for Autonomous Driving [arXiv2024][Code]

• GraphAD: Interaction Scene Graph for End-to-end Autonomous Driving [arXiv2024][Code]

• DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving [ICCV2023][Code]

• Planning-oriented Autonomous Driving [CVPR2023][Code]

• Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving [CVPR2023][Code]

• Coaching a Teachable Student [CVPR2023]

• ReasonNet: End-to-End Driving with Temporal and Global Reasoning [CVPR2023]

• VAD: Vectorized Scene Representation for Efficient Autonomous Driving [ICCV2023][Code]

• FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Driving [arXiv2023]

• UAP-BEV: Uncertainty Aware Planning using Bird's Eye View generated from Surround Monocular Images [CASE2023][Code]

• Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model [NeurIPSWorkshop2022]

• Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer [CoRL2022][Code]

• Addressing Optimism Bias in Sequence Modeling for Reinforcement Learning [ICML2022]

• Learning Mixture of Domain-Specific Experts via Disentangled Factors for Autonomous Driving Authors [AAAI2022]

• ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning [ECCV2022][Code]

• TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving [TPAMI2022][Code]

• Learning from All Vehicles [CVPR2022][Code]

• Deep Federated Learning for Autonomous Driving [IV2022][Code]

• NEAT: Neural Attention Fields for End-to-End Autonomous Driving [ICCV2021][Code]

• ObserveNet Control: A Vision-Dynamics Learning Approach to Predictive Control in Autonomous Vehicles [RAL2021]

• Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D [ECCV2020][Code]

• Driving Through Ghosts: Behavioral Cloning with False Positives [IROS2020]

(back to top)

Transformer

• PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Driving [ECCV2024][Code]

• DualAD: Disentangling the Dynamic and Static World for End-to-End Driving [CVPR2024]

• Target-point Attention Transformer: A novel trajectory predict network for end-to-end autonomous driving [IV2024]

• Hybrid-Prediction Integrated Planning for Autonomous Driving [arXiv2024][Code]

• SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Driving [arXiv2024]

• VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning [arXiv2024]

• DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mamba [arXiv2024][Code]

• LeGo-Drive: Language-enhanced Goal-oriented Closed-Loop End-to-End Autonomous Driving [arXiv2024][Code]

• Planning-oriented Autonomous Driving [CVPR2023][Code]

• Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving [CVPR2023][Code]

• ReasonNet: End-to-End Driving with Temporal and Global Reasoning [CVPR2023]

• Hidden Biases of End-to-End Driving Models [ICCV2023][Code]

• VAD: Vectorized Scene Representation for Efficient Autonomous Driving [ICCV2023][Code]

• Detrive: Imitation Learning with Transformer Detection for End-to-End Autonomous Driving [DISA2023]

• Ground then Navigate: Language-guided Navigation in Dynamic Scenes [arXiv2022]

• Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer [CoRL2022][Code]

• MMFN: Multi-Modal-Fusion-Net for End-to-End Driving [IROS2022][Code]

• TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving [TPAMI2022][Code]

• Human-AI Shared Control via Policy Dissection [NeurIPS2022][Code]

• COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles [CVPR2022][Code]

• CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-Based Autonomous Urban Driving [AAAI2022][Code]

• Safe Driving via Expert Guided Policy Optimization [CoRL2022][Code]

• NEAT: Neural Attention Fields for End-to-End Autonomous Driving [ICCV2021][Code]

• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [CVPR2021][Code]

(back to top)

V2V Cooperative

• ICOP: Image-based Cooperative Perception for End-to-End Autonomous Driving [IV2024]

• Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End System [arXiv2024][Code]

• End-to-End Autonomous Driving through V2X Cooperation [arXiv2024][Code]

• CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-Based Autonomous Urban Driving [AAAI2022][Code]

• COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles [CVPR2022][Code]

(back to top)

Distributed RL

• Safe Driving via Expert Guided Policy Optimization [CoRL2022][Code]

• GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Driving [arXiv2021]

• End-to-End Model-Free Reinforcement Learning for Urban Driving Using Implicit Affordances [CVPR2020]

• Batch Policy Learning under Constraints [ICML2019][Code]

(back to top)

Data-driven Simulation

Parameter Initialization

• SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Traffic [ECCV2024][Code]

• NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking [arXiv2024][Code]

• TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios [ICRA2023][Code]

• KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients [ECCV2022][Code]

• AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles [CVPR2021]

• SceneGen: Learning To Generate Realistic Traffic Scenes [CVPR2021]

• HDMapGen: A Hierarchical Graph Generative Model of High Definition Maps [CVPR2021]

• SimNet: Learning Reactive Self-driving Simulations from Real-world Observations [ICRA2021]

• Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method [IROS2020]

(back to top)

Traffic Simulation

• Solving Motion Planning Tasks with a Scalable Generative Model [ECCV2024][Code]

• SMART: Scalable Multi-agent Real-time Simulation via Next-token Prediction [arXiv2024]

• Data-driven Traffic Simulation: A Comprehensive Review [arXiv2023]

• Scenario Diffusion: Controllable Driving Scenario Generation With Diffusion [NeurIPS2023]

• ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modeling [NeurIPSDataset2023][Code]

• MixSim: A Hierarchical Framework for Mixed Reality Traffic Simulation [CVPR2023]

• Learning Realistic Traffic Agents in Closed-loop [CoRL2023]

• TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction [arXiv2023]

• Language Conditioned Traffic Generation [arXiv2023][Code]

• TrafficGen: Learning to Generate Diverse and Realistic Traffic Scenarios [ICRA2023][Code]

• DriveSceneGen: Generating Diverse and Realistic Driving Scenarios from Scratch [arXiv2023]

• Guided Conditional Diffusion for Controllable Traffic Simulation [arXiv2022]

• BITS: Bi-level Imitation for Traffic Simulation [arXiv2022]

• TrafficSim: Learning To Simulate Realistic Multi-Agent Behaviors [CVPR2021]

• SimNet: Learning Reactive Self-driving Simulations from Real-world Observations [ICRA2021]

(back to top)

Sensor Simulation

• Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting [ECCV2024][Code]

• A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets [SIGGRAPH2024][Code]

• NeuRAD: Neural Rendering for Autonomous Driving [CVPR2024][Code]

• Multi-Level Neural Scene Graphs for Dynamic Urban Environments [CVPR2024][Code]

• Multiagent Multitraversal Multimodal Self-Driving: Open MARS Dataset [CVPR2024][Code]

• HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splatting [CVPR2024][Code]

• DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes [CVPR2024][Code]

• Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agents [CVPR2024][Code]

• LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes [CVPR2024]

• LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis [CVPR2024][Code]

• PaReNeRF: Toward Fast Large-scale Dynamic NeRF with Patch-based Reference [CVPR2024]

• Dynamic LiDAR Re-simulation using Compositional Neural Fields [CVPR2024][Code]

• Panacea: Panoramic and Controllable Video Generation for Autonomous Driving [CVPR2024][Code]

• EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision [ICLR2024][Code]

• UC-NeRF: Neural Radiance Field for Under-Calibrated Multi-view Cameras in Autonomous Driving [ICLR2024][Code]

• S3Gaussian: Self-Supervised Street Gaussians for Autonomous Driving [arXiv2024][Code]

• AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction [arXiv2024]

• Dynamic 3D Gaussian Fields for Urban Areas [arXiv2024][Code]

• MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenes [arXiv2024][Code]

• VDG: Vision-Only Dynamic Gaussian for Driving Simulation [arXiv2024][Code]

• HO-Gaussian: Hybrid Optimization of 3D Gaussian Splatting for Urban Scenes [arXiv2024]

• SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior [arXiv2024]

• LightSim: Neural Lighting Simulation for Urban Scenes [NeurIPS2023]

• Real-Time Neural Rasterization for Large Scenes [ICCV2023]

• UniSim: A Neural Closed-Loop Sensor Simulator [CVPR2023]

• Learning Compact Representations for LiDAR Completion and Generation [CVPR2023]

• Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation [CoRL2023]

• Reconstructing Objects in-the-wild for Realistic Sensor Simulation [ICRA2023]

• Enhancing Photorealism Enhancement [TPAMI2023][Code]

• UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fields [ICCV2023][Code]

• MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving [CICAI2023][Code]

• Mega-NERF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs [CVPR2022]

• Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation [CVPR2022]

• CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulation [CoRL2022]

• VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles [ICRA2022][Code]

• Learning Interactive Driving Policies via Data-driven Simulation [ICRA2022][Code]

• Learning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation [RAL2020]

(back to top)