Showing 1–22 of 22 results for author: Bujarbaruah, M

Output Feedback Stochastic MPC with Hard Input Constraints

Authors: Eunhyek Joa, Monimoy Bujarbaruah, Francesco Borrelli

Abstract: We present an output feedback stochastic model predictive controller (SMPC) for constrained linear time-invariant systems. The system is perturbed by additive Gaussian disturbances on state and additive Gaussian measurement noise on output. A Kalman filter is used for state estimation and an SMPC is designed to satisfy chance constraints on states and hard constraints on actuator inputs. The propo… ▽ More We present an output feedback stochastic model predictive controller (SMPC) for constrained linear time-invariant systems. The system is perturbed by additive Gaussian disturbances on state and additive Gaussian measurement noise on output. A Kalman filter is used for state estimation and an SMPC is designed to satisfy chance constraints on states and hard constraints on actuator inputs. The proposed SMPC constructs bounded sets for the state evolution and a tube-based constraint tightening strategy where the tightened constraints are time-invariant. We prove that the proposed SMPC can guarantee an infeasibility rate below a user-specified tolerance. We numerically compare our method with a classical output feedback SMPC with simulation results which highlight the efficacy of the proposed algorithm. △ Less

Submitted 27 November, 2023; v1 submitted 21 February, 2023; originally announced February 2023.

Comments: IEEE American Control Conference (ACC) 2023, May 31 - June 2, San Diego, CA, USA

Journal ref: 2023 American Control Conference (ACC) (pp. 2034-2039). IEEE

Stochastic MPC with Realization-Adaptive Constraint Tightening

Authors: Hotae Lee, Monimoy Bujarbaruah, Francesco Borrelli

Abstract: This paper presents a stochastic model predictive controller (SMPC) for linear time-invariant systems in the presence of additive disturbances. The distribution of the disturbance is unknown and is assumed to have a bounded support. A sample-based strategy is used to compute sets of disturbance sequences necessary for robustifying the state chance constraints. These sets are constructed offline us… ▽ More This paper presents a stochastic model predictive controller (SMPC) for linear time-invariant systems in the presence of additive disturbances. The distribution of the disturbance is unknown and is assumed to have a bounded support. A sample-based strategy is used to compute sets of disturbance sequences necessary for robustifying the state chance constraints. These sets are constructed offline using samples of the disturbance extracted from its support. For online MPC implementation, we propose a novel reformulation strategy of the chance constraints, where the constraint tightening is computed by adjusting the offline computed sets based on the previously realized disturbances along the trajectory. The proposed MPC is recursive feasible and can lower conservatism over existing SMPC approaches at the cost of higher offline computational time. Numerical simulations demonstrate the effectiveness of the proposed approach. △ Less

Submitted 29 September, 2022; v1 submitted 21 September, 2022; originally announced September 2022.

Comments: Submitted to ACC 2023

Safe Human-Robot Collaborative Transportation via Trust-Driven Role Adaptation

Authors: Tony Zheng, Monimoy Bujarbaruah, Yvonne R. Stürz, Francesco Borrelli

Abstract: We study a human-robot collaborative transportation task in presence of obstacles. The task for each agent is to carry a rigid object to a common target position, while safely avoiding obstacles and satisfying the compliance and actuation constraints of the other agent. Human and robot do not share the local view of the environment. The human policy either assists the robot when they deem the robo… ▽ More We study a human-robot collaborative transportation task in presence of obstacles. The task for each agent is to carry a rigid object to a common target position, while safely avoiding obstacles and satisfying the compliance and actuation constraints of the other agent. Human and robot do not share the local view of the environment. The human policy either assists the robot when they deem the robot actions safe based on their perception of the environment, or actively leads the task. Using estimated human inputs, the robot plans a trajectory for the transported object by solving a constrained finite time optimal control problem. Sensors on the robot measure the inputs applied by the human. The robot then appropriately applies a weighted combination of the human's applied and its own planned inputs, where the weights are chosen based on the robot's trust value on its estimates of the human's inputs. This allows for a dynamic leader-follower role adaptation of the robot throughout the task. Furthermore, under a low value of trust, if the robot approaches any obstacle potentially unknown to the human, it triggers a safe stopping policy, maintaining safety of the system and signaling a required change in the human's intent. With experimental results, we demonstrate the efficacy of the proposed approach. △ Less

Submitted 12 July, 2022; originally announced July 2022.

A Simple Robust MPC for Linear Systems with Parametric and Additive Uncertainty

Authors: Monimoy Bujarbaruah, Ugo Rosolia, Yvonne R. Stürz, Francesco Borrelli

Abstract: We propose a simple and computationally efficient approach for designing a robust Model Predictive Controller (MPC) for constrained uncertain linear systems. The uncertainty is modeled as an additive disturbance and an additive error on the system dynamics matrices. Set based bounds for each component of the model uncertainty are assumed to be known. We separate the constraint tightening strategy… ▽ More We propose a simple and computationally efficient approach for designing a robust Model Predictive Controller (MPC) for constrained uncertain linear systems. The uncertainty is modeled as an additive disturbance and an additive error on the system dynamics matrices. Set based bounds for each component of the model uncertainty are assumed to be known. We separate the constraint tightening strategy into two parts, depending on the length of the MPC horizon. For a horizon length of one, the robust MPC problem is solved exactly, whereas for other horizon lengths, the model uncertainty is over-approximated with a net-additive component. The resulting MPC controller guarantees robust satisfaction of state and input constraints in closed-loop with the uncertain system. With appropriately designed terminal components and an adaptive horizon strategy, we prove the controller's recursive feasibility and stability of the origin. With numerical simulations, we demonstrate that our proposed approach gains up to 15x online computation speedup over a tube MPC strategy, while stabilizing about 98$\%$ of the latter's region of attraction. △ Less

Submitted 23 March, 2021; originally announced March 2021.

Comments: Final version for IEEE American Control Conference (ACC), May 2021. arXiv admin note: text overlap with arXiv:2007.00930

Learning Environment Constraints in Collaborative Robotics: A Decentralized Leader-Follower Approach

Authors: Monimoy Bujarbaruah, Yvonne R. Stürz, Conrad Holda, Karl H. Johansson, Francesco Borrelli

Abstract: In this paper, we propose a leader-follower hierarchical strategy for two robots collaboratively transporting an object in a partially known environment with obstacles. Both robots sense the local surrounding environment and react to obstacles in their proximity. We consider no explicit communication, so the local environment information and the control actions are not shared between the robots. A… ▽ More In this paper, we propose a leader-follower hierarchical strategy for two robots collaboratively transporting an object in a partially known environment with obstacles. Both robots sense the local surrounding environment and react to obstacles in their proximity. We consider no explicit communication, so the local environment information and the control actions are not shared between the robots. At any given time step, the leader solves a model predictive control (MPC) problem with its known set of obstacles and plans a feasible trajectory to complete the task. The follower estimates the inputs of the leader and uses a policy to assist the leader while reacting to obstacles in its proximity. The leader infers obstacles in the follower's vicinity by using the difference between the predicted and the real-time estimated follower control action. A method to switch the leader-follower roles is used to improve the control performance in tight environments. The efficacy of our approach is demonstrated with detailed comparisons to two alternative strategies, where it achieves the highest success rate, while completing the task fastest. See the link www.dropbox.com/s/hexadigqkvspaeh/IROS_Video.mp4?dl=0 for a descriptive video of the algorithm. △ Less

Submitted 24 July, 2021; v1 submitted 7 March, 2021; originally announced March 2021.

Comments: Full version of our IROS 2021 paper

Learning How to Solve Bubble Ball

Authors: Hotae Lee, Monimoy Bujarbaruah, Francesco Borrelli

Abstract: "Bubble Ball" is a game built on a 2D physics engine, where a finite set of objects can modify the motion of a bubble-like ball. The objective is to choose the set and the initial configuration of the objects, in order to get the ball to reach a target flag. The presence of obstacles, friction, contact forces and combinatorial object choices make the game hard to solve. In this paper, we propose a… ▽ More "Bubble Ball" is a game built on a 2D physics engine, where a finite set of objects can modify the motion of a bubble-like ball. The objective is to choose the set and the initial configuration of the objects, in order to get the ball to reach a target flag. The presence of obstacles, friction, contact forces and combinatorial object choices make the game hard to solve. In this paper, we propose a hierarchical predictive framework which solves Bubble Ball. Geometric, kinematic and dynamic models are used at different levels of the hierarchy. At each level of the game, data collected during failed iterations are used to update models at all hierarchical level and converge to a feasible solution to the game. The proposed approach successfully solves a large set of Bubble Ball levels within reasonable number of trials. This proposed framework can also be used to solve other physics-based games, especially with limited training data from human demonstrations. △ Less

Submitted 28 April, 2021; v1 submitted 20 November, 2020; originally announced November 2020.

Comments: Accepted to L4DC 2021

Traction Adaptive Motion Planning at the Limits of Handling

Authors: Lars Svensson, Monimoy Bujarbaruah, Arpit Karsolia, Christian Berger, Martin Törngren

Abstract: In this paper, we address the problem of motion planning and control at the limits of handling, under locally varying traction conditions. We propose a novel solution method where traction variations over the prediction horizon are represented by time-varying tire force constraints, derived from a predictive friction estimate. A constrained finite time optimal control problem is solved in a recedi… ▽ More In this paper, we address the problem of motion planning and control at the limits of handling, under locally varying traction conditions. We propose a novel solution method where traction variations over the prediction horizon are represented by time-varying tire force constraints, derived from a predictive friction estimate. A constrained finite time optimal control problem is solved in a receding horizon fashion, imposing these time-varying constraints. Furthermore, our method features an integrated sampling augmentation procedure that addresses the problems of infeasibility and sensitivity to local minima that arise at abrupt constraint alterations, e.g., due to sudden friction changes. We validate the proposed algorithm on a Volvo FH16 heavy-duty vehicle, in a range of critical scenarios. Experimental results indicate that traction adaptive motion planning and control improves the vehicle's capacity to avoid accidents, both when adapting to low local traction, by ensuring dynamic feasibility of the planned motion, and when adapting to high local traction, by realizing high traction utilization. △ Less

Submitted 18 November, 2021; v1 submitted 9 September, 2020; originally announced September 2020.

Comments: 32 pages, 11 figures

Learning to Play Cup-and-Ball with Noisy Camera Observations

Authors: Monimoy Bujarbaruah, Tony Zheng, Akhil Shetty, Martin Sehr, Francesco Borrelli

Abstract: Playing the cup-and-ball game is an intriguing task for robotics research since it abstracts important problem characteristics including system nonlinearity, contact forces and precise positioning as terminal goal. In this paper, we present a learning model based control strategy for the cup-and-ball game, where a Universal Robots UR5e manipulator arm learns to catch a ball in one of the cups on a… ▽ More Playing the cup-and-ball game is an intriguing task for robotics research since it abstracts important problem characteristics including system nonlinearity, contact forces and precise positioning as terminal goal. In this paper, we present a learning model based control strategy for the cup-and-ball game, where a Universal Robots UR5e manipulator arm learns to catch a ball in one of the cups on a Kendama. Our control problem is divided into two sub-tasks, namely $(i)$ swinging the ball up in a constrained motion, and $(ii)$ catching the free-falling ball. The swing-up trajectory is computed offline, and applied in open-loop to the arm. Subsequently, a convex optimization problem is solved online during the ball's free-fall to control the manipulator and catch the ball. The controller utilizes noisy position feedback of the ball from an Intel RealSense D435 depth camera. We propose a novel iterative framework, where data is used to learn the support of the camera noise distribution iteratively in order to update the control policy. The probability of a catch with a fixed policy is computed empirically with a user specified number of roll-outs. Our design guarantees that probability of the catch increases in the limit, as the learned support nears the true support of the camera noise distribution. High-fidelity Mujoco simulations and preliminary experimental results support our theoretical analysis. △ Less

Submitted 18 July, 2020; originally announced July 2020.

Comments: Final paper for IEEE CASE, Hong Kong, August, 2020. First three authors contributed equally

Robust MPC for Linear Systems with Parametric and Additive Uncertainty: A Novel Constraint Tightening Approach

Authors: Monimoy Bujarbaruah, Ugo Rosolia, Yvonne R Stürz, Xiaojing Zhang, Francesco Borrelli

Abstract: We propose a novel approach to design a robust Model Predictive Controller (MPC) for constrained uncertain linear systems. The uncertain system is modeled as linear parameter varying with additive disturbance. Set bounds for the system matrices and the additive uncertainty are assumed to be known. We formulate a novel optimization-based constraint tightening strategy around a predicted nominal tra… ▽ More We propose a novel approach to design a robust Model Predictive Controller (MPC) for constrained uncertain linear systems. The uncertain system is modeled as linear parameter varying with additive disturbance. Set bounds for the system matrices and the additive uncertainty are assumed to be known. We formulate a novel optimization-based constraint tightening strategy around a predicted nominal trajectory which utilizes these bounds. With an appropriately designed terminal cost function and constraint set, we prove robust satisfaction of the imposed constraints by the resulting MPC in closed-loop with the uncertain system, and Input to State Stability of the origin. We highlight the efficacy of our proposed approach via a numerical example. △ Less

Submitted 10 August, 2022; v1 submitted 2 July, 2020; originally announced July 2020.

Comments: Dimension of matrices in the MPC problem fixed. Shortened and slightly altered version of this draft published as a full paper in Automatica

Learning to Satisfy Unknown Constraints in Iterative MPC

Authors: Monimoy Bujarbaruah, Charlott Vallon, Francesco Borrelli

Abstract: We propose a control design method for linear time-invariant systems that iteratively learns to satisfy unknown polyhedral state constraints. At each iteration of a repetitive task, the method constructs an estimate of the unknown environment constraints using collected closed-loop trajectory data. This estimated constraint set is improved iteratively upon collection of additional data. An MPC con… ▽ More We propose a control design method for linear time-invariant systems that iteratively learns to satisfy unknown polyhedral state constraints. At each iteration of a repetitive task, the method constructs an estimate of the unknown environment constraints using collected closed-loop trajectory data. This estimated constraint set is improved iteratively upon collection of additional data. An MPC controller is then designed to robustly satisfy the estimated constraint set. This paper presents the details of the proposed approach, and provides robust and probabilistic guarantees of constraint satisfaction as a function of the number of executed task iterations. We demonstrate the safety of the proposed framework and explore the safety vs. performance trade-off in a detailed numerical example. △ Less

Submitted 10 June, 2023; v1 submitted 9 June, 2020; originally announced June 2020.

Comments: Long version of the published paper for IEEE-CDC 2020. First two authors contributed equally. Added some very relevant citations that were missing

Near-Optimal Rapid MPC using Neural Networks: A Primal-Dual Policy Learning Framework

Authors: Xiaojing Zhang, Monimoy Bujarbaruah, Francesco Borrelli

Abstract: In this paper, we propose a novel framework for approximating the explicit MPC policy for linear parameter-varying systems using supervised learning. Our learning scheme guarantees feasibility and near-optimality of the approximated MPC policy with high probability. Furthermore, in contrast to most existing approaches that only learn the MPC policy, we also learn the "dual policy", which enables u… ▽ More In this paper, we propose a novel framework for approximating the explicit MPC policy for linear parameter-varying systems using supervised learning. Our learning scheme guarantees feasibility and near-optimality of the approximated MPC policy with high probability. Furthermore, in contrast to most existing approaches that only learn the MPC policy, we also learn the "dual policy", which enables us to keep a check on the approximated MPC's optimality online during the control process. If the check deems the control input from the approximated MPC policy safe and near-optimal, then it is applied to the plant, otherwise a backup controller is invoked, thus filtering out (severely) suboptimal control inputs. The backup controller is only invoked with a bounded (low) probability, where the exact probability level can be chosen by the user. Since our framework does not require solving any optimization problem during the control process, it enables the deployment of MPC on resource-constrained systems. Specifically, we illustrate the utility of the proposed framework on a vehicle dynamics control problem. Compared to online optimization methods, we demonstrate a speedup of up to 62x on a desktop computer and 10x on an automotive-grade electronic control unit, while maintaining a high control performance. △ Less

Submitted 8 December, 2019; originally announced December 2019.

Comments: First two authors contributed equally. arXiv admin note: text overlap with arXiv:1906.08257

Exploiting Model Sparsity in Adaptive MPC: A Compressed Sensing Viewpoint

Authors: Monimoy Bujarbaruah, Charlott Vallon

Abstract: This paper proposes an Adaptive Stochastic Model Predictive Control (MPC) strategy for stable linear time-invariant systems in the presence of bounded disturbances. We consider multi-input, multi-output systems that can be expressed by a Finite Impulse Response (FIR) model. The parameters of the FIR model corresponding to each output are unknown but assumed sparse. We estimate these parameters usi… ▽ More This paper proposes an Adaptive Stochastic Model Predictive Control (MPC) strategy for stable linear time-invariant systems in the presence of bounded disturbances. We consider multi-input, multi-output systems that can be expressed by a Finite Impulse Response (FIR) model. The parameters of the FIR model corresponding to each output are unknown but assumed sparse. We estimate these parameters using the Recursive Least Squares algorithm. The estimates are then improved using set-based bounds obtained by solving the Basis Pursuit Denoising [1] problem. Our approach is able to handle hard input constraints and probabilistic output constraints. Using tools from distributionally robust optimization, we reformulate the probabilistic output constraints as tractable convex second-order cone constraints, which enables us to pose our MPC design task as a convex optimization problem. The efficacy of the developed algorithm is highlighted with a thorough numerical example, where we demonstrate performance gain over the counterpart algorithm of [2], which does not utilize the sparsity information of the system impulse response parameters during control design. △ Less

Submitted 9 December, 2019; originally announced December 2019.

Comments: Both authors contributed equally. arXiv admin note: text overlap with arXiv:1804.09790

Learning Robustness with Bounded Failure: An Iterative MPC Approach

Authors: Monimoy Bujarbaruah, Akhil Shetty, Kameshwar Poolla, Francesco Borrelli

Abstract: We propose an approach to design a Model Predictive Controller (MPC) for constrained Linear Time Invariant systems performing an iterative task. The system is subject to an additive disturbance, and the goal is to learn to satisfy state and input constraints robustly. Using disturbance measurements after each iteration, we construct Confidence Support sets, which contain the true support of the di… ▽ More We propose an approach to design a Model Predictive Controller (MPC) for constrained Linear Time Invariant systems performing an iterative task. The system is subject to an additive disturbance, and the goal is to learn to satisfy state and input constraints robustly. Using disturbance measurements after each iteration, we construct Confidence Support sets, which contain the true support of the disturbance distribution with a given probability. As more data is collected, the Confidence Supports converge to the true support of the disturbance. This enables design of an MPC controller that avoids conservative estimate of the disturbance support, while simultaneously bounding the probability of constraint violation. The efficacy of the proposed approach is then demonstrated with a detailed numerical example. △ Less

Submitted 10 June, 2023; v1 submitted 22 November, 2019; originally announced November 2019.

Comments: Added a set of important references that were missing

A Semi-Definite Programming Approach to Robust Adaptive MPC under State Dependent Uncertainty

Authors: Monimoy Bujarbaruah, Siddharth H. Nair, Francesco Borrelli

Abstract: We propose an Adaptive MPC framework for uncertain linear systems to achieve robust satisfaction of state and input constraints. The uncertainty in the system is assumed additive, state dependent, and globally Lipschitz with a known Lipschitz constant. We use a non-parametric technique for online identification of the system uncertainty by approximating its graph via envelopes defined by quadratic… ▽ More We propose an Adaptive MPC framework for uncertain linear systems to achieve robust satisfaction of state and input constraints. The uncertainty in the system is assumed additive, state dependent, and globally Lipschitz with a known Lipschitz constant. We use a non-parametric technique for online identification of the system uncertainty by approximating its graph via envelopes defined by quadratic constraints. At any given time, by solving a set of convex optimization problems, the MPC controller guarantees robust constraint satisfaction for the closed loop system for all possible values of system uncertainty modeled by the envelope. The uncertainty envelope is refined with data using Set Membership Methods. We highlight the efficacy of the proposed framework via a detailed numerical example. △ Less

Submitted 15 February, 2020; v1 submitted 10 October, 2019; originally announced October 2019.

Comments: Accepted for European Control Conference (ECC), May 2020, Saint Petersburg, Russia

Modeling of Dynamical Systems via Successive Graph Approximations

Authors: Siddharth H. Nair, Monimoy Bujarbaruah, Francesco Borrelli

Abstract: In this work, we propose a non-parametric technique for online modeling of systems with unknown nonlinear Lipschitz dynamics. The key idea is to successively utilize measurements to approximate the graph of the state-update function using envelopes described by quadratic constraints. The proposed approach is then demonstrated on two control applications: (i) computation of tractable bounds for unm… ▽ More In this work, we propose a non-parametric technique for online modeling of systems with unknown nonlinear Lipschitz dynamics. The key idea is to successively utilize measurements to approximate the graph of the state-update function using envelopes described by quadratic constraints. The proposed approach is then demonstrated on two control applications: (i) computation of tractable bounds for unmodeled dynamics, and (ii) computation of positive invariant sets. We further highlight the efficacy of the proposed approach via a detailed numerical example. △ Less

Submitted 8 October, 2019; originally announced October 2019.

Optimization Based Planner Tracker Design for Safety Guarantees

Authors: He Yin, Monimoy Bujarbaruah, Murat Arcak, Andrew Packard

Abstract: We present a safe-by-design approach to path planning and control for nonlinear systems. The planner uses a low fidelity model of the plant to compute reference trajectories by solving an MPC problem, while the plant being controlled utilizes a feedback control law that tracks those trajectories with an upper-bound on the tracking error. Our main goal is to allow for maximum permissiveness (that i… ▽ More We present a safe-by-design approach to path planning and control for nonlinear systems. The planner uses a low fidelity model of the plant to compute reference trajectories by solving an MPC problem, while the plant being controlled utilizes a feedback control law that tracks those trajectories with an upper-bound on the tracking error. Our main goal is to allow for maximum permissiveness (that is, room for constraint feasibility) of the planner, while maintaining safety after accounting for the tracking error bound. We achieve this by parametrizing the state and input constraints imposed on the planner and deriving corresponding parametrized tracking control laws and tracking error bounds, which are computed offline through Sum-of-Squares programming. The parameters are then optimally chosen to maximize planner permissiveness, while guaranteeing safety. △ Less

Submitted 2 October, 2019; originally announced October 2019.

Comments: Submitted to IEEE American Control Conference (ACC), Denver, CO, USA, July 2020

Adaptive MPC under Time Varying Uncertainty: Robust and Stochastic

Authors: Monimoy Bujarbaruah, Xiaojing Zhang, Marko Tanaskovic, Francesco Borrelli

Abstract: This paper deals with the problem of formulating an adaptive Model Predictive Control strategy for constrained uncertain systems. We consider a linear system, in presence of bounded time varying additive uncertainty. The uncertainty is decoupled as the sum of a process noise with known bounds, and a time varying offset that we wish to identify. The time varying offset uncertainty is assumed unknow… ▽ More This paper deals with the problem of formulating an adaptive Model Predictive Control strategy for constrained uncertain systems. We consider a linear system, in presence of bounded time varying additive uncertainty. The uncertainty is decoupled as the sum of a process noise with known bounds, and a time varying offset that we wish to identify. The time varying offset uncertainty is assumed unknown point-wise in time. Its domain, called the Feasible Parameter Set, and its maximum rate of change are known to the control designer. As new data becomes available, we refine the Feasible Parameter Set with a Set Membership Method based approach, using the known bounds on process noise. We consider two separate cases of robust and probabilistic constraints on system states, with hard constraints on actuator inputs. In both cases, we robustly satisfy the imposed constraints for all possible values of the offset uncertainty in the Feasible Parameter Set. By imposing adequate terminal conditions, we prove recursive feasibility and stability of the proposed algorithms. The efficacy of the proposed robust and stochastic Adaptive MPC algorithms is illustrated with detailed numerical examples. △ Less

Submitted 11 April, 2021; v1 submitted 30 September, 2019; originally announced September 2019.

Comments: Corrected Figure 4, and added a missing compactness and containment of origin assumption for the constraint sets. The brief version of this will been published as a technical note in IEEE Transactions on Automatic Control, Vol 66, June 2021

Relaxed Actor-Critic with Convergence Guarantees for Continuous-Time Optimal Control of Nonlinear Systems

Authors: Jingliang Duan, Jie Li, Qiang Ge, Shengbo Eben Li, Monimoy Bujarbaruah, Fei Ma, Dezhao Zhang

Abstract: This paper presents the Relaxed Continuous-Time Actor-critic (RCTAC) algorithm, a method for finding the nearly optimal policy for nonlinear continuous-time (CT) systems with known dynamics and infinite horizon, such as the path-tracking control of vehicles. RCTAC has several advantages over existing adaptive dynamic programming algorithms for CT systems. It does not require the ``admissibility" o… ▽ More This paper presents the Relaxed Continuous-Time Actor-critic (RCTAC) algorithm, a method for finding the nearly optimal policy for nonlinear continuous-time (CT) systems with known dynamics and infinite horizon, such as the path-tracking control of vehicles. RCTAC has several advantages over existing adaptive dynamic programming algorithms for CT systems. It does not require the ``admissibility" of the initialized policy or the input-affine nature of controlled systems for convergence. Instead, given any initial policy, RCTAC can converge to an admissible, and subsequently nearly optimal policy for a general nonlinear system with a saturated controller. RCTAC consists of two phases: a warm-up phase and a generalized policy iteration phase. The warm-up phase minimizes the square of the Hamiltonian to achieve admissibility, while the generalized policy iteration phase relaxes the update termination conditions for faster convergence. The convergence and optimality of the algorithm are proven through Lyapunov analysis, and its effectiveness is demonstrated through simulations and real-world path-tracking tasks. △ Less

Submitted 30 March, 2023; v1 submitted 11 September, 2019; originally announced September 2019.

Journal ref: IEEE Transactions on Intelligent Vehicles, 2023 (Early Access)

Safe and Near-Optimal Policy Learning for Model Predictive Control using Primal-Dual Neural Networks

Authors: Xiaojing Zhang, Monimoy Bujarbaruah, Francesco Borrelli

Abstract: In this paper, we propose a novel framework for approximating the explicit MPC law for linear parameter-varying systems using supervised learning. In contrast to most existing approaches, we not only learn the control policy, but also a "certificate policy", that allows us to estimate the sub-optimality of the learned control policy online, during execution-time. We learn both these policies from… ▽ More In this paper, we propose a novel framework for approximating the explicit MPC law for linear parameter-varying systems using supervised learning. In contrast to most existing approaches, we not only learn the control policy, but also a "certificate policy", that allows us to estimate the sub-optimality of the learned control policy online, during execution-time. We learn both these policies from data using supervised learning techniques, and also provide a randomized method that allows us to guarantee the quality of each learned policy, measured in terms of feasibility and optimality. This in turn allows us to bound the probability of the learned control policy of being infeasible or suboptimal, where the check is performed by the certificate policy. Since our algorithm does not require the solution of an optimization problem during run-time, it can be deployed even on resource-constrained systems. We illustrate the efficacy of the proposed framework on a vehicle dynamics control problem where we demonstrate a speedup of up to two orders of magnitude compared to online optimization with minimal performance degradation. △ Less

Submitted 19 June, 2019; originally announced June 2019.

Comments: IEEE American Control Conference (ACC) 2019, July 9-12, Philadelphia, PA, USA

Adaptive MPC for Autonomous Lane Keeping

Authors: Monimoy Bujarbaruah, Xiaojing Zhang, H. Eric Tseng, Francesco Borrelli

Abstract: This paper proposes an Adaptive Robust Model Predictive Control strategy for lateral control in lane keeping problems, where we continuously learn an unknown, but constant steering angle offset present in the steering system. Longitudinal velocity is assumed constant. The goal is to minimize the outputs, which are distance from lane center line and the steady state heading angle error, while satis… ▽ More This paper proposes an Adaptive Robust Model Predictive Control strategy for lateral control in lane keeping problems, where we continuously learn an unknown, but constant steering angle offset present in the steering system. Longitudinal velocity is assumed constant. The goal is to minimize the outputs, which are distance from lane center line and the steady state heading angle error, while satisfying respective safety constraints. We do not assume perfect knowledge of the vehicle lateral dynamics model and estimate and adapt in real-time the maximum possible bound of the steering angle offset from data using a robust Set Membership Method based approach. Our approach is even well-suited for scenarios with sharp curvatures on high speed, where obtaining a precise model bias for constrained control is difficult, but learning from data can be helpful. We ensure persistent feasibility using a switching strategy during change of lane curvature. The proposed methodology is general and can be applied to more complex vehicle dynamics problems. △ Less

Submitted 30 November, 2018; v1 submitted 12 June, 2018; originally announced June 2018.

Comments: 14th International Symposium on Advanced Vehicle Control (AVEC), Beijing, China, July 2018

Adaptive MPC for Iterative Tasks

Authors: Monimoy Bujarbaruah, Xiaojing Zhang, Ugo Rosolia, Francesco Borrelli

Abstract: This paper proposes an Adaptive Learning Model Predictive Control strategy for uncertain constrained linear systems performing iterative tasks. The additive uncertainty is modeled as the sum of a bounded process noise and an unknown constant offset. As new data becomes available, the proposed algorithm iteratively adapts the believed domain of the unknown offset after each iteration. An MPC strate… ▽ More This paper proposes an Adaptive Learning Model Predictive Control strategy for uncertain constrained linear systems performing iterative tasks. The additive uncertainty is modeled as the sum of a bounded process noise and an unknown constant offset. As new data becomes available, the proposed algorithm iteratively adapts the believed domain of the unknown offset after each iteration. An MPC strategy robust to all feasible offsets is employed in order to guarantee recursive feasibility. We show that the adaptation of the feasible offset domain reduces conservatism of the proposed strategy, compared to classical robust MPC strategies. As a result, the controller performance improves. Performance is measured in terms of following trajectories with lower associated costs at each iteration. Numerical simulations highlight the main advantages of the proposed approach. △ Less

Submitted 25 April, 2018; originally announced April 2018.

Adaptive MPC with Chance Constraints for FIR Systems

Authors: Monimoy Bujarbaruah, Xiaojing Zhang, Francesco Borrelli

Abstract: This paper proposes an adaptive stochastic Model Predictive Control (MPC) strategy for stable linear time invariant systems in the presence of bounded disturbances. We consider multi-input multi-output systems that can be expressed by a finite impulse response model, whose parameters we estimate using a linear Recursive Least Squares algorithm. Building on the work of [1],[2], our approach is able… ▽ More This paper proposes an adaptive stochastic Model Predictive Control (MPC) strategy for stable linear time invariant systems in the presence of bounded disturbances. We consider multi-input multi-output systems that can be expressed by a finite impulse response model, whose parameters we estimate using a linear Recursive Least Squares algorithm. Building on the work of [1],[2], our approach is able to handle hard input constraints and probabilistic output constraints. By using tools from distributionally robust optimization, we formulate our MPC design task as a convex optimization problem that can be solved using existing tools. Furthermore, we show that our adaptive stochastic MPC algorithm is persistently feasible. The efficacy of the developed algorithm is demonstrated in a numerical example and the results are compared with the adaptive robust MPC algorithm of [2]. △ Less

Submitted 25 April, 2018; originally announced April 2018.

Journal ref: IEEE American Control Conference (ACC), June 2018, WI-USA