-
Generative AI for Autonomous Driving: A Review
Authors:
Katharina Winter,
Abhishek Vivekanandan,
Rupert Polley,
Yinzhe Shen,
Christian Schlauch,
Mohamed-Khalil Bouzidi,
Bojan Derajic,
Natalie Grabowsky,
Annajoyce Mariani,
Dennis Rochau,
Giovanni Lucente,
Harsh Yadav,
Firas Mualla,
Adam Molin,
Sebastian Bernhard,
Christian Wirth,
Ömer Şahin Taş,
Nadja Klein,
Fabian B. Flohr,
Hanno Gottschalk
Abstract:
Generative AI (GenAI) is rapidly advancing the field of Autonomous Driving (AD), extending beyond traditional applications in text, image, and video generation. We explore how generative models can enhance automotive tasks, such as static map creation, dynamic scenario generation, trajectory forecasting, and vehicle motion planning. By examining multiple generative approaches ranging from Variatio…
▽ More
Generative AI (GenAI) is rapidly advancing the field of Autonomous Driving (AD), extending beyond traditional applications in text, image, and video generation. We explore how generative models can enhance automotive tasks, such as static map creation, dynamic scenario generation, trajectory forecasting, and vehicle motion planning. By examining multiple generative approaches ranging from Variational Autoencoder (VAEs) over Generative Adversarial Networks (GANs) and Invertible Neural Networks (INNs) to Generative Transformers (GTs) and Diffusion Models (DMs), we highlight and compare their capabilities and limitations for AD-specific applications. Additionally, we discuss hybrid methods integrating conventional techniques with generative approaches, and emphasize their improved adaptability and robustness. We also identify relevant datasets and outline open research questions to guide future developments in GenAI. Finally, we discuss three core challenges: safety, interpretability, and realtime capabilities, and present recommendations for image generation, dynamic scenario generation, and planning.
△ Less
Submitted 21 May, 2025;
originally announced May 2025.
-
Reachability-Based Contingency Planning against Multi-Modal Predictions with Branch MPC
Authors:
Mohamed-Khalil Bouzidi,
Bojan Derajic,
Daniel Goehring,
Joerg Reichardt
Abstract:
This paper presents a novel contingency planning framework that integrates learning-based multi-modal predictions of traffic participants into Branch Model Predictive Control (MPC). Leveraging reachability analysis, we address the computational challenges associated with Branch MPC by organizing the multitude of predictions into driving corridors. Analyzing the overlap between these corridors, the…
▽ More
This paper presents a novel contingency planning framework that integrates learning-based multi-modal predictions of traffic participants into Branch Model Predictive Control (MPC). Leveraging reachability analysis, we address the computational challenges associated with Branch MPC by organizing the multitude of predictions into driving corridors. Analyzing the overlap between these corridors, their number can be reduced through pruning and clustering while ensuring safety since all prediction modes are preserved. These processed corridors directly correspond to the distinct branches of the scenario tree and provide an efficient constraint representation for the Branch MPC. We further utilize the reachability for determining maximum feasible decision postponing times, ensuring that branching decisions remain executable. Qualitative and quantitative evaluations demonstrate significantly reduced computational complexity and enhanced safety and comfort.
△ Less
Submitted 4 February, 2025;
originally announced February 2025.
-
Learning Maximal Safe Sets Using Hypernetworks for MPC-based Local Trajectory Planning in Unknown Environments
Authors:
Bojan Derajić,
Mohamed-Khalil Bouzidi,
Sebastian Bernhard,
Wolfgang Hönig
Abstract:
This paper presents a novel learning-based approach for online estimation of maximal safe sets for local trajectory planning in unknown static environments. The neural representation of a set is used as the terminal set constraint for a model predictive control (MPC) local planner, resulting in improved recursive feasibility and safety. To achieve real-time performance and desired generalization p…
▽ More
This paper presents a novel learning-based approach for online estimation of maximal safe sets for local trajectory planning in unknown static environments. The neural representation of a set is used as the terminal set constraint for a model predictive control (MPC) local planner, resulting in improved recursive feasibility and safety. To achieve real-time performance and desired generalization properties, we employ the idea of hypernetworks. We use the Hamilton-Jacobi (HJ) reachability analysis as the source of supervision during the training process, allowing us to consider general nonlinear dynamics and arbitrary constraints. The proposed method is extensively evaluated against relevant baselines in simulations for different environments and robot dynamics. The results show a success rate increase of up to 52 \% compared to the best baseline while maintaining comparable execution speed. Additionally, we deploy our proposed method, NTC-MPC, on a physical robot and demonstrate its ability to safely avoid obstacles in scenarios where the baselines fail.
△ Less
Submitted 4 March, 2025; v1 submitted 26 October, 2024;
originally announced October 2024.
-
Motion Planning under Uncertainty: Integrating Learning-Based Multi-Modal Predictors into Branch Model Predictive Control
Authors:
Mohamed-Khalil Bouzidi,
Bojan Derajic,
Daniel Goehring,
Joerg Reichardt
Abstract:
In complex traffic environments, autonomous vehicles face multi-modal uncertainty about other agents' future behavior. To address this, recent advancements in learningbased motion predictors output multi-modal predictions. We present our novel framework that leverages Branch Model Predictive Control(BMPC) to account for these predictions. The framework includes an online scenario-selection process…
▽ More
In complex traffic environments, autonomous vehicles face multi-modal uncertainty about other agents' future behavior. To address this, recent advancements in learningbased motion predictors output multi-modal predictions. We present our novel framework that leverages Branch Model Predictive Control(BMPC) to account for these predictions. The framework includes an online scenario-selection process guided by topology and collision risk criteria. This efficiently selects a minimal set of predictions, rendering the BMPC realtime capable. Additionally, we introduce an adaptive decision postponing strategy that delays the planner's commitment to a single scenario until the uncertainty is resolved. Our comprehensive evaluations in traffic intersection and random highway merging scenarios demonstrate enhanced comfort and safety through our method.
△ Less
Submitted 6 May, 2024;
originally announced May 2024.
