-
Dual Control for Interactive Autonomous Merging with Model Predictive Diffusion
Authors:
Jacob Knaup,
Jovin D'sa,
Behdad Chalaki,
Hossein Nourkhiz Mahjoub,
Ehsan Moradi-Pari,
Panagiotis Tsiotras
Abstract:
Interactive decision-making is essential in applications such as autonomous driving, where the agent must infer the behavior of nearby human drivers while planning in real-time. Traditional predict-then-act frameworks are often insufficient or inefficient because accurate inference of human behavior requires a continuous interaction rather than isolated prediction. To address this, we propose an a…
▽ More
Interactive decision-making is essential in applications such as autonomous driving, where the agent must infer the behavior of nearby human drivers while planning in real-time. Traditional predict-then-act frameworks are often insufficient or inefficient because accurate inference of human behavior requires a continuous interaction rather than isolated prediction. To address this, we propose an active learning framework in which we rigorously derive predicted belief distributions. Additionally, we introduce a novel model-based diffusion solver tailored for online receding horizon control problems, demonstrated through a complex, non-convex highway merging scenario. Our approach extends previous high-fidelity dual control simulations to hardware experiments, which may be viewed at https://youtu.be/Q_JdZuopGL4, and verifies behavior inference in human-driven traffic scenarios, moving beyond idealized models. The results show improvements in adaptive planning under uncertainty, advancing the field of interactive decision-making for real-world applications.
△ Less
Submitted 14 February, 2025;
originally announced February 2025.
-
Stochastic Time-Optimal Trajectory Planning for Connected and Automated Vehicles in Mixed-Traffic Merging Scenarios
Authors:
Viet-Anh Le,
Behdad Chalaki,
Filippos N. Tzortzoglou,
Andreas A. Malikopoulos
Abstract:
Addressing safe and efficient interaction between connected and automated vehicles (CAVs) and human-driven vehicles in a mixed-traffic environment has attracted considerable attention. In this paper, we develop a framework for stochastic time-optimal trajectory planning for coordinating multiple CAVs in mixed-traffic merging scenarios. We present a data-driven model, combining Newell's car-followi…
▽ More
Addressing safe and efficient interaction between connected and automated vehicles (CAVs) and human-driven vehicles in a mixed-traffic environment has attracted considerable attention. In this paper, we develop a framework for stochastic time-optimal trajectory planning for coordinating multiple CAVs in mixed-traffic merging scenarios. We present a data-driven model, combining Newell's car-following model with Bayesian linear regression, for efficiently learning the driving behavior of human drivers online. Using the prediction model and uncertainty quantification, a stochastic time-optimal control problem is formulated to find robust trajectories for CAVs. We also integrate a replanning mechanism that determines when deriving new trajectories for CAVs is needed based on the accuracy of the Bayesian linear regression predictions. Finally, we demonstrate the performance of our proposed framework using a realistic simulation environment.
△ Less
Submitted 31 October, 2023;
originally announced November 2023.
-
MR-IDM -- Merge Reactive Intelligent Driver Model: Towards Enhancing Laterally Aware Car-following Models
Authors:
Dustin Holley,
Jovin D'sa,
Hossein Nourkhiz Mahjoub,
Gibran Ali,
Behdad Chalaki,
Ehsan Moradi-Pari
Abstract:
This paper discusses the limitations of existing microscopic traffic models in accounting for the potential impacts of on-ramp vehicles on the car-following behavior of main-lane vehicles on highways. We first surveyed U.S. on-ramps to choose a representative set of on-ramps and then collected real-world observational data from the merging vehicle's perspective in various traffic conditions rangin…
▽ More
This paper discusses the limitations of existing microscopic traffic models in accounting for the potential impacts of on-ramp vehicles on the car-following behavior of main-lane vehicles on highways. We first surveyed U.S. on-ramps to choose a representative set of on-ramps and then collected real-world observational data from the merging vehicle's perspective in various traffic conditions ranging from free-flowing to rush-hour traffic jams. Next, as our core contribution, we introduce a novel car-following model, called MR-IDM, for highway driving that reacts to merging vehicles in a realistic way. This proposed driving model can either be used in traffic simulators to generate realistic highway driving behavior or integrated into a prediction module for autonomous vehicles attempting to merge onto the highway. We quantitatively evaluated the effectiveness of our model and compared it against several other methods. We show that MR-IDM has the least error in mimicking the real-world data, while having features such as smoothness, stability, and lateral awareness.
△ Less
Submitted 19 May, 2023;
originally announced May 2023.
-
Minimally Disruptive Cooperative Lane-change Maneuvers
Authors:
Behdad Chalaki,
Vaishnav Tadiparthi,
Hossein Nourkhiz Mahjoub,
Jovin D'sa,
Ehsan Moradi-Pari,
Andres S. Chavez Armijos,
Anni Li,
Christos G. Cassandras
Abstract:
A lane-change maneuver on a congested highway could be severely disruptive or even infeasible without the cooperation of neighboring cars. However, cooperation with other vehicles does not guarantee that the performed maneuver will not have a negative impact on traffic flow unless it is explicitly considered in the cooperative controller design. In this letter, we present a socially compliant fram…
▽ More
A lane-change maneuver on a congested highway could be severely disruptive or even infeasible without the cooperation of neighboring cars. However, cooperation with other vehicles does not guarantee that the performed maneuver will not have a negative impact on traffic flow unless it is explicitly considered in the cooperative controller design. In this letter, we present a socially compliant framework for cooperative lane-change maneuvers for an arbitrary number of CAVs on highways that aims to interrupt traffic flow as minimally as possible. Moreover, we explicitly impose feasibility constraints in the optimization formulation by using reachability set theory, leading to a unified design that removes the need for an iterative procedure used in prior work. We quantitatively evaluate the effectiveness of our framework and compare it against previously offered approaches in terms of maneuver time and incurred throughput disruption.
△ Less
Submitted 10 March, 2023;
originally announced March 2023.
-
Cooperative Energy and Time-Optimal Lane Change Maneuvers with Minimal Highway Traffic Disruption
Authors:
Andres S. Chavez Armijos,
Anni Li,
Christos G. Cassandras,
Yasir K. Al-Nadawi,
Hidekazu Araki,
Behdad Chalaki,
Ehsan Moradi-Pari,
Hossein Nourkhiz Mahjoub,
Vaishnav Tadiparthi
Abstract:
We derive optimal control policies for a Connected Automated Vehicle (CAV) and cooperating neighboring CAVs to carry out a lane change maneuver consisting of a longitudinal phase where the CAV properly positions itself relative to the cooperating neighbors and a lateral phase where it safely changes lanes. In contrast to prior work on this problem, where the CAV "selfishly" only seeks to minimize…
▽ More
We derive optimal control policies for a Connected Automated Vehicle (CAV) and cooperating neighboring CAVs to carry out a lane change maneuver consisting of a longitudinal phase where the CAV properly positions itself relative to the cooperating neighbors and a lateral phase where it safely changes lanes. In contrast to prior work on this problem, where the CAV "selfishly" only seeks to minimize its maneuver time, we seek to ensure that the fast-lane traffic flow is minimally disrupted (through a properly defined metric). Additionally, when performing lane-changing maneuvers, we optimally select the cooperating vehicles from a set of feasible neighboring vehicles and experimentally show that the highway throughput is improved compared to the baseline case of human-driven vehicles changing lanes with no cooperation. When feasible solutions do not exist for a given maximal allowable disruption, we include a time relaxation method trading off a longer maneuver time with reduced disruption. Our analysis is also extended to multiple sequential maneuvers. Simulation results show the effectiveness of our controllers in terms of safety guarantees and up to 16% and 90% average throughput and maneuver time improvement respectively when compared to maneuvers with no cooperation.
△ Less
Submitted 15 November, 2022;
originally announced November 2022.
-
A Barrier-Certified Optimal Coordination Framework for Connected and Automated Vehicles
Authors:
Behdad Chalaki,
Andreas A. Malikopoulos
Abstract:
In this paper, we extend a framework that we developed earlier for coordination of connected and automated vehicles (CAVs) at a signal-free intersection by integrating a safety layer using control barrier functions. First, in our motion planning module, each CAV computes the optimal control trajectory using simple vehicle dynamics. The trajectory does not make any of the state, control, and safety…
▽ More
In this paper, we extend a framework that we developed earlier for coordination of connected and automated vehicles (CAVs) at a signal-free intersection by integrating a safety layer using control barrier functions. First, in our motion planning module, each CAV computes the optimal control trajectory using simple vehicle dynamics. The trajectory does not make any of the state, control, and safety constraints active. A vehicle-level tracking controller employs a combined feedforward-feedback control law to track the resulting optimal trajectory from the motion planning module. Then, a barrier-certificate module, acting as a middle layer between the vehicle-level tracking controller and physical vehicle, receives the control law from the vehicle-level tracking controller and using realistic vehicle dynamics ensures that none of the state, control, and safety constraints becomes active. The latter is achieved through a quadratic program, which can be solved efficiently in real time. We demonstrate the effectiveness of our extended framework through a numerical simulation.
△ Less
Submitted 30 March, 2022;
originally announced March 2022.
-
Combined Optimal Routing and Coordination of Connected and Automated Vehicles
Authors:
Heeseung Bang,
Behdad Chalaki,
Andreas A. Malikopoulos
Abstract:
In this letter, we consider a transportation network with a 100\% penetration rate of connected and automated vehicles (CAVs) and present an optimal routing approach that takes into account the efficiency achieved in the network by coordinating the CAVs at specific traffic scenarios, e.g., intersections, merging roadways, and roundabouts. To derive the optimal route of a travel request, we use the…
▽ More
In this letter, we consider a transportation network with a 100\% penetration rate of connected and automated vehicles (CAVs) and present an optimal routing approach that takes into account the efficiency achieved in the network by coordinating the CAVs at specific traffic scenarios, e.g., intersections, merging roadways, and roundabouts. To derive the optimal route of a travel request, we use the information of the CAVs that have already received a routing solution. This enables each CAV to consider the traffic conditions on the roads. The solution of any new travel request determines the optimal travel time at each traffic scenario while satisfying all state, control, and safety constraints. We validate the performance of our framework through numerical simulations. To the best of our knowledge, this is the first attempt to consider the coordination of CAVs in a routing problem.
△ Less
Submitted 17 May, 2022; v1 submitted 21 March, 2022;
originally announced March 2022.
-
A Multi-Agent Deep Reinforcement Learning Coordination Framework for Connected and Automated Vehicles at Merging Roadways
Authors:
Sai Krishna Sumanth Nakka,
Behdad Chalaki,
Andreas Malikopoulos
Abstract:
The steady increase in the number of vehicles operating on the highways continues to exacerbate congestion, accidents, energy consumption, and greenhouse gas emissions. Emerging mobility systems, e.g., connected and automated vehicles (CAVs), have the potential to directly address these issues and improve transportation network efficiency and safety. In this paper, we consider a highway merging sc…
▽ More
The steady increase in the number of vehicles operating on the highways continues to exacerbate congestion, accidents, energy consumption, and greenhouse gas emissions. Emerging mobility systems, e.g., connected and automated vehicles (CAVs), have the potential to directly address these issues and improve transportation network efficiency and safety. In this paper, we consider a highway merging scenario and propose a framework for coordinating CAVs such that stop-and-go driving is eliminated. We use a decentralized form of the actor-critic approach to deep reinforcement learning$-$multi-agent deep deterministic policy gradient. We demonstrate the coordination of CAVs through numerical simulations and show that a smooth traffic flow is achieved by eliminating stop-and-go driving. Videos and plots of the simulation results can be found at this supplemental $\href{https://sites.google.com/view/ud-ids-lab/MADRL}{\text{site}}$.
△ Less
Submitted 13 March, 2022; v1 submitted 23 September, 2021;
originally announced September 2021.
-
A Priority-Aware Replanning and Resequencing Framework for Coordination of Connected and Automated Vehicles
Authors:
Behdad Chalaki,
Andreas A. Malikopoulos
Abstract:
Deriving optimal control strategies for coordination of connected and automated vehicles (CAVs) often requires re-evaluating the strategies in order to respond to unexpected changes in the presence of disturbances and uncertainties. In this paper, we first extend a decentralized framework that we developed earlier for coordination of CAVs at a signal-free intersection to incorporate replanning. Th…
▽ More
Deriving optimal control strategies for coordination of connected and automated vehicles (CAVs) often requires re-evaluating the strategies in order to respond to unexpected changes in the presence of disturbances and uncertainties. In this paper, we first extend a decentralized framework that we developed earlier for coordination of CAVs at a signal-free intersection to incorporate replanning. Then, we further enhance the framework by introducing a priority-aware resequencing mechanism which designates the order of decision making of CAVs based on theory from the job-shop scheduling problem. Our enhanced framework relaxes the first-come-first-serve decision order which has been used extensively in these problems. We illustrate the effectiveness of our proposed approach through numerical simulations.
△ Less
Submitted 9 December, 2021; v1 submitted 12 September, 2021;
originally announced September 2021.
-
A Hysteretic Q-learning Coordination Framework for Emerging Mobility Systems in Smart Cities
Authors:
Behdad Chalaki,
Andreas A. Malikopoulos
Abstract:
Connected and automated vehicles (CAVs) can alleviate traffic congestion, air pollution, and improve safety. In this paper, we provide a decentralized coordination framework for CAVs at a signal-free intersection to minimize travel time and improve fuel efficiency. We employ a simple yet powerful reinforcement learning approach, an off-policy temporal difference learning called Q-learning, enhance…
▽ More
Connected and automated vehicles (CAVs) can alleviate traffic congestion, air pollution, and improve safety. In this paper, we provide a decentralized coordination framework for CAVs at a signal-free intersection to minimize travel time and improve fuel efficiency. We employ a simple yet powerful reinforcement learning approach, an off-policy temporal difference learning called Q-learning, enhanced with a coordination mechanism to address this problem. Then, we integrate a first-in-first-out queuing policy to improve the performance of our system. We demonstrate the efficacy of our proposed approach through simulation and comparison with the classical optimal control method based on Pontryagin's minimum principle.
△ Less
Submitted 5 November, 2020;
originally announced November 2020.
-
Optimal Control of Connected and Automated Vehicles at Multiple Adjacent Intersections
Authors:
Behdad Chalaki,
Andreas A. Malikopoulos
Abstract:
In this paper, we establish a decentralized optimal control framework for connected and automated vehicles (CAVs) crossing multiple adjacent, multi-lane signal-free intersections to minimize energy consumption and improve traffic throughput. Our framework consists of two layers of planning. In the upper-level planning, each CAV computes its optimal arrival time at each intersection recursively alo…
▽ More
In this paper, we establish a decentralized optimal control framework for connected and automated vehicles (CAVs) crossing multiple adjacent, multi-lane signal-free intersections to minimize energy consumption and improve traffic throughput. Our framework consists of two layers of planning. In the upper-level planning, each CAV computes its optimal arrival time at each intersection recursively along with the optimal lane to improve the traffic throughput. In the low-level planning, we formulate an energy-optimal control problem with interior-point constraints, the solution of which yields the optimal control input (acceleration/deceleration) of each CAV to cross the intersections at the time specified by the upper-level planning. Moreover, we extend the results of the proposed bi-level framework to include a bounded steady-state error in tracking the optimal position of the CAVs. Finally, we demonstrate the effectiveness of the proposed framework through simulation for symmetric and asymmetric intersections and comparison with traditional signalized intersections.
△ Less
Submitted 18 May, 2021; v1 submitted 5 August, 2020;
originally announced August 2020.
-
Experimental Validation of a Real-Time Optimal Controller for Coordination of CAVs in a Multi-Lane Roundabout
Authors:
Behdad Chalaki,
Logan E. Beaver,
Andreas A. Malikopoulos
Abstract:
Roundabouts in conjunction with other traffic scenarios, e.g., intersections, merging roadways, speed reduction zones, can induce congestion in a transportation network due to driver responses to various disturbances. Research efforts have shown that smoothing traffic flow and eliminating stop-and-go driving can both improve fuel efficiency of the vehicles and the throughput of a roundabout. In th…
▽ More
Roundabouts in conjunction with other traffic scenarios, e.g., intersections, merging roadways, speed reduction zones, can induce congestion in a transportation network due to driver responses to various disturbances. Research efforts have shown that smoothing traffic flow and eliminating stop-and-go driving can both improve fuel efficiency of the vehicles and the throughput of a roundabout. In this paper, we validate an optimal control framework developed earlier in a multi-lane roundabout scenario using the University of Delaware's scaled smart city (UDSSC). We first provide conditions where the solution is optimal. Then, we demonstrate the feasibility of the solution using experiments at UDSSC, and show that the optimal solution completely eliminates stop-and-go driving while preserving safety.
△ Less
Submitted 18 May, 2020; v1 submitted 29 January, 2020;
originally announced January 2020.
-
Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning
Authors:
Behdad Chalaki,
Logan E. Beaver,
Ben Remer,
Kathy Jang,
Eugene Vinitsky,
Alexandre M. Bayen,
Andreas A. Malikopoulos
Abstract:
In this article, we demonstrate a zero-shot transfer of an autonomous driving policy from simulation to University of Delaware's scaled smart city with adversarial multi-agent reinforcement learning, in which an adversary attempts to decrease the net reward by perturbing both the inputs and outputs of the autonomous vehicles during training. We train the autonomous vehicles to coordinate with each…
▽ More
In this article, we demonstrate a zero-shot transfer of an autonomous driving policy from simulation to University of Delaware's scaled smart city with adversarial multi-agent reinforcement learning, in which an adversary attempts to decrease the net reward by perturbing both the inputs and outputs of the autonomous vehicles during training. We train the autonomous vehicles to coordinate with each other while crossing a roundabout in the presence of an adversary in simulation. The adversarial policy successfully reproduces the simulated behavior and incidentally outperforms, in terms of travel time, both a human-driving baseline and adversary-free trained policies. Finally, we demonstrate that the addition of adversarial training considerably improves the performance \eat{stability and robustness} of the policies after transfer to the real world compared to Gaussian noise injection.
△ Less
Submitted 22 June, 2020; v1 submitted 12 March, 2019;
originally announced March 2019.
-
Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehicles
Authors:
Kathy Jang,
Eugene Vinitsky,
Behdad Chalaki,
Ben Remer,
Logan Beaver,
Andreas Malikopoulos,
Alexandre Bayen
Abstract:
Using deep reinforcement learning, we train control policies for autonomous vehicles leading a platoon of vehicles onto a roundabout. Using Flow, a library for deep reinforcement learning in micro-simulators, we train two policies, one policy with noise injected into the state and action space and one without any injected noise. In simulation, the autonomous vehicle learns an emergent metering beh…
▽ More
Using deep reinforcement learning, we train control policies for autonomous vehicles leading a platoon of vehicles onto a roundabout. Using Flow, a library for deep reinforcement learning in micro-simulators, we train two policies, one policy with noise injected into the state and action space and one without any injected noise. In simulation, the autonomous vehicle learns an emergent metering behavior for both policies in which it slows to allow for smoother merging. We then directly transfer this policy without any tuning to the University of Delaware Scaled Smart City (UDSSC), a 1:25 scale testbed for connected and automated vehicles. We characterize the performance of both policies on the scaled city. We show that the noise-free policy winds up crashing and only occasionally metering. However, the noise-injected policy consistently performs the metering behavior and remains collision-free, suggesting that the noise helps with the zero-shot policy transfer. Additionally, the transferred, noise-injected policy leads to a 5% reduction of average travel time and a reduction of 22% in maximum travel time in the UDSSC. Videos of the controllers can be found at https://sites.google.com/view/iccps-policy-transfer.
△ Less
Submitted 22 February, 2019; v1 submitted 14 December, 2018;
originally announced December 2018.
