Showing 1–34 of 34 results for author: Tomlin, C J

Recursively Feasible Probabilistic Safe Online Learning with Control Barrier Functions

Authors: Fernando Castañeda, Jason J. Choi, Wonsuhk Jung, Bike Zhang, Claire J. Tomlin, Koushil Sreenath

Abstract: Learning-based control has recently shown great efficacy in performing complex tasks for various applications. However, to deploy it in real systems, it is of vital importance to guarantee the system will stay safe. Control Barrier Functions (CBFs) offer mathematical tools for designing safety-preserving controllers for systems with known dynamics. In this article, we first introduce a model-uncer… ▽ More Learning-based control has recently shown great efficacy in performing complex tasks for various applications. However, to deploy it in real systems, it is of vital importance to guarantee the system will stay safe. Control Barrier Functions (CBFs) offer mathematical tools for designing safety-preserving controllers for systems with known dynamics. In this article, we first introduce a model-uncertainty-aware reformulation of CBF-based safety-critical controllers using Gaussian Process (GP) regression to close the gap between an approximate mathematical model and the real system, which results in a second-order cone program (SOCP)-based control design. We then present the pointwise feasibility conditions of the resulting safety controller, highlighting the level of richness that the available system information must meet to ensure safety. We use these conditions to devise an event-triggered online data collection strategy that ensures the recursive feasibility of the learned safety controller. Our method works by constantly reasoning about whether the current information is sufficient to ensure safety or if new measurements under active safe exploration are required to reduce the uncertainty. As a result, our proposed framework can guarantee the forward invariance of the safe set defined by the CBF with high probability, even if it contains a priori unexplored regions. We validate the proposed framework in two numerical simulation experiments. △ Less

Submitted 3 September, 2024; v1 submitted 23 August, 2022; originally announced August 2022.

Comments: Journal article. Includes the results of the 2021 CDC paper titled "Pointwise feasibility of gaussian process-based safety-critical control under model uncertainty" and proposes a recursively feasible safe online learning algorithm as new contribution

On the Computational Consequences of Cost Function Design in Nonlinear Optimal Control

Authors: Tyler Westenbroek, Anand Siththaranjan, Mohsin Sarwari, Claire J. Tomlin, Shankar S. Sastry

Abstract: Optimal control is an essential tool for stabilizing complex nonlinear systems. However, despite the extensive impacts of methods such as receding horizon control, dynamic programming and reinforcement learning, the design of cost functions for a particular system often remains a heuristic-driven process of trial and error. In this paper we seek to gain insights into how the choice of cost functio… ▽ More Optimal control is an essential tool for stabilizing complex nonlinear systems. However, despite the extensive impacts of methods such as receding horizon control, dynamic programming and reinforcement learning, the design of cost functions for a particular system often remains a heuristic-driven process of trial and error. In this paper we seek to gain insights into how the choice of cost function interacts with the underlying structure of the control system and impacts the amount of computation required to obtain a stabilizing controller. We treat the cost design problem as a two-step process where the designer specifies outputs for the system that are to be penalized and then modulates the relative weighting of the inputs and the outputs in the cost. To characterize the computational burden associated to obtaining a stabilizing controller with a particular cost, we bound the prediction horizon required by receding horizon methods and the number of iterations required by dynamic programming methods to meet this requirement. Our theoretical results highlight a qualitative separation between what is possible, from a design perspective, when the chosen outputs induce either minimum-phase or non-minimum-phase behavior. Simulation studies indicate that this separation also holds for modern reinforcement learning methods. △ Less

Submitted 17 November, 2022; v1 submitted 5 April, 2022; originally announced April 2022.

Koopman-Based Neural Lyapunov Functions for General Attractors

Authors: Shankar A. Deka, Alonso M. Valle, Claire J. Tomlin

Abstract: Koopman spectral theory has grown in the past decade as a powerful tool for dynamical systems analysis and control. In this paper, we show how recent data-driven techniques for estimating Koopman-Invariant subspaces with neural networks can be leveraged to extract Lyapunov certificates for the underlying system. In our work, we specifically focus on systems with a limit-cycle, beyond just an isola… ▽ More Koopman spectral theory has grown in the past decade as a powerful tool for dynamical systems analysis and control. In this paper, we show how recent data-driven techniques for estimating Koopman-Invariant subspaces with neural networks can be leveraged to extract Lyapunov certificates for the underlying system. In our work, we specifically focus on systems with a limit-cycle, beyond just an isolated equilibrium point, and use Koopman eigenfunctions to efficiently parameterize candidate Lyapunov functions to construct forward-invariant sets under some (unknown) attractor dynamics. Additionally, when the dynamics are polynomial and when neural networks are replaced by polynomials as a choice of function approximators in our approach, one can further leverage Sum-of-Squares programs and/or nonlinear programs to yield provably correct Lyapunov certificates. In such a polynomial case, our Koopman-based approach for constructing Lyapunov functions uses significantly fewer decision variables compared to directly formulating and solving a Sum-of-Squares optimization problem. △ Less

Submitted 23 March, 2022; originally announced March 2022.

Comments: Submitted to CDC 2022

Infinite-Horizon Reach-Avoid Zero-Sum Games via Deep Reinforcement Learning

Authors: Jingqi Li, Donggun Lee, Somayeh Sojoudi, Claire J. Tomlin

Abstract: In this paper, we consider the infinite-horizon reach-avoid zero-sum game problem, where the goal is to find a set in the state space, referred to as the reach-avoid set, such that the system starting at a state therein could be controlled to reach a given target set without violating constraints under the worst-case disturbance. We address this problem by designing a new value function with a con… ▽ More In this paper, we consider the infinite-horizon reach-avoid zero-sum game problem, where the goal is to find a set in the state space, referred to as the reach-avoid set, such that the system starting at a state therein could be controlled to reach a given target set without violating constraints under the worst-case disturbance. We address this problem by designing a new value function with a contracting Bellman backup, where the super-zero level set, i.e., the set of states where the value function is evaluated to be non-negative, recovers the reach-avoid set. Building upon this, we prove that the proposed method can be adapted to compute the viability kernel, or the set of states which could be controlled to satisfy given constraints, and the backward reachable set, or the set of states that could be driven towards a given target set. Finally, we propose to alleviate the curse of dimensionality issue in high-dimensional problems by extending Conservative Q-Learning, a deep reinforcement learning technique, to learn a value function such that the super-zero level set of the learned value function serves as a (conservative) approximation to the reach-avoid set. Our theoretical and empirical results suggest that the proposed method could learn reliably the reach-avoid set and the optimal control policy even with neural network approximation. △ Less

Submitted 18 September, 2024; v1 submitted 18 March, 2022; originally announced March 2022.

Towards cyber-physical systems robust to communication delays: A differential game approach

Authors: Shankar A. Deka, Donggun Lee, Claire J. Tomlin

Abstract: Collaboration between interconnected cyber-physical systems is becoming increasingly pervasive. Time-delays in communication channels between such systems are known to induce catastrophic failure modes, like high frequency oscillations in robotic manipulators in bilateral teleoperation or string instability in platoons of autonomous vehicles. This paper considers nonlinear time-delay systems repre… ▽ More Collaboration between interconnected cyber-physical systems is becoming increasingly pervasive. Time-delays in communication channels between such systems are known to induce catastrophic failure modes, like high frequency oscillations in robotic manipulators in bilateral teleoperation or string instability in platoons of autonomous vehicles. This paper considers nonlinear time-delay systems representing coupled robotic agents, and proposes controllers that are robust to time-varying communication delays. We introduce approximations that allow the delays to be considered as implicit control inputs themselves, and formulate the problem as a zero-sum differential game between the stabilizing controllers and the delays acting adversarially. The ensuing optimal control law is finally compared to known results from Lyapunov-Krasovskii based approaches via numerical experiments. △ Less

Submitted 21 September, 2021; originally announced September 2021.

Comments: 7 pages, 5 figures, Submitted to IEEE Control Systems Letters

MSC Class: 34K35; 49L12; 93D21

Lax Formulae for Efficiently Solving Two Classes of State-Constrained Optimal Control Problems

Authors: Donggun Lee, Claire J. Tomlin

Abstract: This paper presents Lax formulae for solving the following optimal control problems: minimize the maximum (or the minimum) cost over a time horizon, while satisfying a state constraint. We present a viscosity theory, and by applying the theory to the Hamilton-Jacobi (HJ) equations, these Lax formulae are derived. A numerical algorithm for the Lax formulae is presented: under certain conditions, th… ▽ More This paper presents Lax formulae for solving the following optimal control problems: minimize the maximum (or the minimum) cost over a time horizon, while satisfying a state constraint. We present a viscosity theory, and by applying the theory to the Hamilton-Jacobi (HJ) equations, these Lax formulae are derived. A numerical algorithm for the Lax formulae is presented: under certain conditions, this algorithm's computational complexity is polynomial in the dimension of the state. For each class of optimal control problem, an example demonstrates the use and performance of the Lax formulae. △ Less

Submitted 31 August, 2021; originally announced September 2021.

Hamilton-Jacobi Equations for Two Classes of State-Constrained Zero-Sum Games

Authors: Donggun Lee, Claire J. Tomlin

Abstract: This paper presents Hamilton-Jacobi (HJ) formulations for two classes of two-player zero-sum games: one with a maximum cost value over time, and one with a minimum cost value over time. In the zero-sum game setting, player A minimizes the given cost while satisfying state constraints, and player B wants to prevent player A's success. For each class of problems, this paper presents two HJ equations… ▽ More This paper presents Hamilton-Jacobi (HJ) formulations for two classes of two-player zero-sum games: one with a maximum cost value over time, and one with a minimum cost value over time. In the zero-sum game setting, player A minimizes the given cost while satisfying state constraints, and player B wants to prevent player A's success. For each class of problems, this paper presents two HJ equations: one for time-varying dynamics, cost, and state constraint; the other for time-invariant dynamics, cost, and state constraint. Utilizing the HJ equations, the optimal control for each player is analyzed, and a numerical algorithm is presented to compute the solution to the HJ equations. A two-dimensional water system is introduced as an example to demonstrate the proposed HJ framework. △ Less

Submitted 28 June, 2021; originally announced June 2021.

A Computationally Efficient Hamilton-Jacobi-based Formula for State-Constrained Optimal Control Problems

Authors: Donggun Lee, Claire J. Tomlin

Abstract: This paper investigates a Hamilton-Jacobi (HJ) analysis to solve finite-horizon optimal control problems for high-dimensional systems. Although grid-based methods, such as the level-set method [1], numerically solve a general class of HJ partial differential equations, the computational complexity is exponential in the dimension of the continuous state. To manage this computational complexity, met… ▽ More This paper investigates a Hamilton-Jacobi (HJ) analysis to solve finite-horizon optimal control problems for high-dimensional systems. Although grid-based methods, such as the level-set method [1], numerically solve a general class of HJ partial differential equations, the computational complexity is exponential in the dimension of the continuous state. To manage this computational complexity, methods based on Lax-Hopf theory have been developed for the state-unconstrained optimal control problem under certain assumptions, such as affine dynamics and state-independent stage cost. Based on the Lax formula [2], this paper proposes an HJ formula for the state-constrained optimal control problem for nonlinear systems. We call this formula \textit{the generalized Lax formula} for the optimal control problem. The HJ formula provides both the optimal cost and an optimal control signal. We also provide an efficient computational method for a class of problems for which the dynamics is affine in the state, and for which the stage and terminal cost, as well as the state constraints, are convex in the state. This class of problems does not require affine dynamics and convex stage cost in the control. This paper also provides three practical examples. △ Less

Submitted 25 June, 2021; originally announced June 2021.

Pointwise Feasibility of Gaussian Process-based Safety-Critical Control under Model Uncertainty

Authors: Fernando Castañeda, Jason J. Choi, Bike Zhang, Claire J. Tomlin, Koushil Sreenath

Abstract: Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) are popular tools for enforcing safety and stability of a controlled system, respectively. They are commonly utilized to build constraints that can be incorporated in a min-norm quadratic program (CBF-CLF-QP) which solves for a safety-critical control input. However, since these constraints rely on a model of the system, when t… ▽ More Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) are popular tools for enforcing safety and stability of a controlled system, respectively. They are commonly utilized to build constraints that can be incorporated in a min-norm quadratic program (CBF-CLF-QP) which solves for a safety-critical control input. However, since these constraints rely on a model of the system, when this model is inaccurate the guarantees of safety and stability can be easily lost. In this paper, we present a Gaussian Process (GP)-based approach to tackle the problem of model uncertainty in safety-critical controllers that use CBFs and CLFs. The considered model uncertainty is affected by both state and control input. We derive probabilistic bounds on the effects that such model uncertainty has on the dynamics of the CBF and CLF. We then use these bounds to build safety and stability chance constraints that can be incorporated in a min-norm convex optimization-based controller, called GP-CBF-CLF-SOCP. As the main theoretical result of the paper, we present necessary and sufficient conditions for pointwise feasibility of the proposed optimization problem. We believe that these conditions could serve as a starting point towards understanding what are the minimal requirements on the distribution of data collected from the real system in order to guarantee safety. Finally, we validate the proposed framework with numerical simulations of an adaptive cruise controller for an automotive system. △ Less

Submitted 1 October, 2021; v1 submitted 13 June, 2021; originally announced June 2021.

Comments: The first two authors contributed equally. Accepted for publication in IEEE 60th Conference on Decision and Control (CDC 2021)

FaSTrack: a Modular Framework for Real-Time Motion Planning and Guaranteed Safe Tracking

Authors: Mo Chen, Sylvia L. Herbert, Haimin Hu, Ye Pu, Jaime F. Fisac, Somil Bansal, SooJean Han, Claire J. Tomlin

Abstract: Real-time, guaranteed safe trajectory planning is vital for navigation in unknown environments. However, real-time navigation algorithms typically sacrifice robustness for computation speed. Alternatively, provably safe trajectory planning tends to be too computationally intensive for real-time replanning. We propose FaSTrack, Fast and Safe Tracking, a framework that achieves both real-time replan… ▽ More Real-time, guaranteed safe trajectory planning is vital for navigation in unknown environments. However, real-time navigation algorithms typically sacrifice robustness for computation speed. Alternatively, provably safe trajectory planning tends to be too computationally intensive for real-time replanning. We propose FaSTrack, Fast and Safe Tracking, a framework that achieves both real-time replanning and guaranteed safety. In this framework, real-time computation is achieved by allowing any trajectory planner to use a simplified \textit{planning model} of the system. The plan is tracked by the system, represented by a more realistic, higher-dimensional \textit{tracking model}. We precompute the tracking error bound (TEB) due to mismatch between the two models and due to external disturbances. We also obtain the corresponding tracking controller used to stay within the TEB. The precomputation does not require prior knowledge of the environment. We demonstrate FaSTrack using Hamilton-Jacobi reachability for precomputation and three different real-time trajectory planners with three different tracking-planning model pairs. △ Less

Submitted 13 March, 2021; v1 submitted 13 February, 2021; originally announced February 2021.

Comments: Published in the IEEE Transactions on Automatic Control

Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics

Authors: Fernando Castañeda, Jason J. Choi, Bike Zhang, Claire J. Tomlin, Koushil Sreenath

Abstract: This paper presents a method to design a min-norm Control Lyapunov Function (CLF)-based stabilizing controller for a control-affine system with uncertain dynamics using Gaussian Process (GP) regression. In order to estimate both state and input-dependent model uncertainty, we propose a novel compound kernel that captures the control-affine nature of the problem. Furthermore, by the use of GP Upper… ▽ More This paper presents a method to design a min-norm Control Lyapunov Function (CLF)-based stabilizing controller for a control-affine system with uncertain dynamics using Gaussian Process (GP) regression. In order to estimate both state and input-dependent model uncertainty, we propose a novel compound kernel that captures the control-affine nature of the problem. Furthermore, by the use of GP Upper Confidence Bound analysis, we provide probabilistic bounds of the regression error, leading to the formulation of a CLF-based stability chance constraint which can be incorporated in a min-norm optimization problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP). The data-collection process and the training of the GP regression model are carried out in an episodic learning fashion. We validate the proposed algorithm and controller in numerical simulations of an inverted pendulum and a kinematic bicycle model, resulting in stable trajectories which are very similar to the ones obtained if we actually knew the true plant dynamics. △ Less

Submitted 23 March, 2021; v1 submitted 13 November, 2020; originally announced November 2020.

Comments: The first two authors contributed equally. To appear at the 2021 American Control Conference (ACC)

Dynamically Computing Adversarial Perturbations for Recurrent Neural Networks

Authors: Shankar A. Deka, Dušan M. Stipanović, Claire J. Tomlin

Abstract: Convolutional and recurrent neural networks have been widely employed to achieve state-of-the-art performance on classification tasks. However, it has also been noted that these networks can be manipulated adversarially with relative ease, by carefully crafted additive perturbations to the input. Though several experimentally established prior works exist on crafting and defending against attacks,… ▽ More Convolutional and recurrent neural networks have been widely employed to achieve state-of-the-art performance on classification tasks. However, it has also been noted that these networks can be manipulated adversarially with relative ease, by carefully crafted additive perturbations to the input. Though several experimentally established prior works exist on crafting and defending against attacks, it is also desirable to have theoretical guarantees on the existence of adversarial examples and robustness margins of the network to such examples. We provide both in this paper. We focus specifically on recurrent architectures and draw inspiration from dynamical systems theory to naturally cast this as a control problem, allowing us to dynamically compute adversarial perturbations at each timestep of the input sequence, thus resembling a feedback controller. Illustrative examples are provided to supplement the theoretical discussions. △ Less

Submitted 6 September, 2020; originally announced September 2020.

Comments: Submitted to IEEE Transactions on Neural Networks and Learning Systems

MSC Class: 68T07; 93B52; 93C10; 49N90 ACM Class: I.2.8

Technical Report: Adaptive Control for Linearizable Systems Using On-Policy Reinforcement Learning

Authors: Tyler Westenbroek, Eric Mazumdar, David Fridovich-Keil, Valmik Prabhu, Claire J. Tomlin, S. Shankar Sastry

Abstract: This paper proposes a framework for adaptively learning a feedback linearization-based tracking controller for an unknown system using discrete-time model-free policy-gradient parameter update rules. The primary advantage of the scheme over standard model-reference adaptive control techniques is that it does not require the learned inverse model to be invertible at all instances of time. This enab… ▽ More This paper proposes a framework for adaptively learning a feedback linearization-based tracking controller for an unknown system using discrete-time model-free policy-gradient parameter update rules. The primary advantage of the scheme over standard model-reference adaptive control techniques is that it does not require the learned inverse model to be invertible at all instances of time. This enables the use of general function approximators to approximate the linearizing controller for the system without having to worry about singularities. However, the discrete-time and stochastic nature of these algorithms precludes the direct application of standard machinery from the adaptive control literature to provide deterministic stability proofs for the system. Nevertheless, we leverage these techniques alongside tools from the stochastic approximation literature to demonstrate that with high probability the tracking and parameter errors concentrate near zero when a certain persistence of excitation condition is satisfied. A simulated example of a double pendulum demonstrates the utility of the proposed theory. 1 △ Less

Submitted 6 April, 2020; originally announced April 2020.

Feedback Linearization for Unknown Systems via Reinforcement Learning

Authors: Tyler Westenbroek, David Fridovich-Keil, Eric Mazumdar, Shreyas Arora, Valmik Prabhu, S. Shankar Sastry, Claire J. Tomlin

Abstract: We present a novel approach to control design for nonlinear systems which leverages model-free policy optimization techniques to learn a linearizing controller for a physical plant with unknown dynamics. Feedback linearization is a technique from nonlinear control which renders the input-output dynamics of a nonlinear plant \emph{linear} under application of an appropriate feedback controller. Onc… ▽ More We present a novel approach to control design for nonlinear systems which leverages model-free policy optimization techniques to learn a linearizing controller for a physical plant with unknown dynamics. Feedback linearization is a technique from nonlinear control which renders the input-output dynamics of a nonlinear plant \emph{linear} under application of an appropriate feedback controller. Once a linearizing controller has been constructed, desired output trajectories for the nonlinear plant can be tracked using a variety of linear control techniques. However, the calculation of a linearizing controller requires a precise dynamics model for the system. As a result, model-based approaches for learning exact linearizing controllers generally require a simple, highly structured model of the system with easily identifiable parameters. In contrast, the model-free approach presented in this paper is able to approximate the linearizing controller for the plant using general function approximation architectures. Specifically, we formulate a continuous-time optimization problem over the parameters of a learned linearizing controller whose optima are the set of parameters which best linearize the plant. We derive conditions under which the learning problem is (strongly) convex and provide guarantees which ensure the true linearizing controller for the plant is recovered. We then discuss how model-free policy optimization algorithms can be used to solve a discrete-time approximation to the problem using data collected from the real-world plant. The utility of the framework is demonstrated in simulation and on a real-world robotic platform. △ Less

Submitted 21 April, 2020; v1 submitted 29 October, 2019; originally announced October 2019.

A Minimum Discounted Reward Hamilton-Jacobi Formulation for Computing Reachable Sets

Authors: Anayo K. Akametalu, Shromona Ghosh, Jaime F. Fisac, Claire J. Tomlin

Abstract: We propose a novel formulation for approximating reachable sets through a minimum discounted reward optimal control problem. The formulation yields a continuous solution that can be obtained by solving a Hamilton-Jacobi equation. Furthermore, the numerical approximation to this solution can be obtained as the unique fixed-point to a contraction mapping. This allows for more efficient solution meth… ▽ More We propose a novel formulation for approximating reachable sets through a minimum discounted reward optimal control problem. The formulation yields a continuous solution that can be obtained by solving a Hamilton-Jacobi equation. Furthermore, the numerical approximation to this solution can be obtained as the unique fixed-point to a contraction mapping. This allows for more efficient solution methods that could not be applied under traditional formulations for solving reachable sets. In addition, this formulation provides a link between reinforcement learning and learning reachable sets for systems with unknown dynamics, allowing algorithms from the former to be applied to the latter. We use two benchmark examples, double integrator, and pursuit-evasion games, to show the correctness of the formulation as well as its strengths in comparison to previous work. △ Less

Submitted 3 September, 2018; originally announced September 2018.

Robust Tracking with Model Mismatch for Fast and Safe Planning: an SOS Optimization Approach

Authors: Sumeet Singh, Mo Chen, Sylvia L. Herbert, Claire J. Tomlin, Marco Pavone

Abstract: In the pursuit of real-time motion planning, a commonly adopted practice is to compute a trajectory by running a planning algorithm on a simplified, low-dimensional dynamical model, and then employ a feedback tracking controller that tracks such a trajectory by accounting for the full, high-dimensional system dynamics. While this strategy of planning with model mismatch generally yields fast compu… ▽ More In the pursuit of real-time motion planning, a commonly adopted practice is to compute a trajectory by running a planning algorithm on a simplified, low-dimensional dynamical model, and then employ a feedback tracking controller that tracks such a trajectory by accounting for the full, high-dimensional system dynamics. While this strategy of planning with model mismatch generally yields fast computation times, there are no guarantees of dynamic feasibility, which hampers application to safety-critical systems. Building upon recent work that addressed this problem through the lens of Hamilton-Jacobi (HJ) reachability, we devise an algorithmic framework whereby one computes, offline, for a pair of "planner" (i.e., low-dimensional) and "tracking" (i.e., high-dimensional) models, a feedback tracking controller and associated tracking bound. This bound is then used as a safety margin when generating motion plans via the low-dimensional model. Specifically, we harness the computational tool of sum-of-squares (SOS) programming to design a bilinear optimization algorithm for the computation of the feedback tracking controller and associated tracking bound. The algorithm is demonstrated via numerical experiments, with an emphasis on investigating the trade-off between the increased computational scalability afforded by SOS and its intrinsic conservativeness. Collectively, our results enable scaling the appealing strategy of planning with model mismatch to systems that are beyond the reach of HJ analysis, while maintaining safety guarantees. △ Less

Submitted 28 July, 2019; v1 submitted 1 August, 2018; originally announced August 2018.

Comments: Presented at WAFR 2018; final version v2 -- fixed typos

Safe and Resilient Multi-vehicle Trajectory Planning Under Adversarial Intruder

Authors: Somil Bansal, Mo Chen, Claire J. Tomlin

Abstract: Provably safe and scalable multi-vehicle trajectory planning is an important and urgent problem. Hamilton-Jacobi (HJ) reachability is an ideal tool for analyzing such safety-critical systems and has been successfully applied to several small-scale problems. However, a direct application of HJ reachability to multi-vehicle trajectory planning is often intractable due to the "curse of dimensionality… ▽ More Provably safe and scalable multi-vehicle trajectory planning is an important and urgent problem. Hamilton-Jacobi (HJ) reachability is an ideal tool for analyzing such safety-critical systems and has been successfully applied to several small-scale problems. However, a direct application of HJ reachability to multi-vehicle trajectory planning is often intractable due to the "curse of dimensionality." To overcome this problem, the sequential trajectory planning (STP) method, which assigns strict priorities to vehicles, was proposed, STP allows multi-vehicle trajectory planning to be done with a linearly-scaling computation complexity. However, if a vehicle not in the set of STP vehicles enters the system, or even worse, if this vehicle is an adversarial intruder, the previous formulation requires the entire system to perform replanning, an intractable task for large-scale systems. In this paper, we make STP more practical by providing a new algorithm where replanning is only needed only for a fixed number of vehicles, irrespective of the total number of STP vehicles. Moreover, this number is a design parameter, which can be chosen based on the computational resources available during run time. We demonstrate this algorithm in a representative simulation of an urban airspace environment. △ Less

Submitted 5 November, 2017; originally announced November 2017.

Comments: Submitted to IEEE Transactions on Automatic Control. arXiv admin note: text overlap with arXiv:1611.08364

Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances

Authors: Somil Bansal, Mo Chen, Sylvia Herbert, Claire J. Tomlin

Abstract: Hamilton-Jacobi (HJ) reachability analysis is an important formal verification method for guaranteeing performance and safety properties of dynamical systems; it has been applied to many small-scale systems in the past decade. Its advantages include compatibility with general nonlinear system dynamics, formal treatment of bounded disturbances, and the availability of well-developed numerical tools… ▽ More Hamilton-Jacobi (HJ) reachability analysis is an important formal verification method for guaranteeing performance and safety properties of dynamical systems; it has been applied to many small-scale systems in the past decade. Its advantages include compatibility with general nonlinear system dynamics, formal treatment of bounded disturbances, and the availability of well-developed numerical tools. The main challenge is addressing its exponential computational complexity with respect to the number of state variables. In this tutorial, we present an overview of basic HJ reachability theory and provide instructions for using the most recent numerical tools, including an efficient GPU-parallelized implementation of a Level Set Toolbox for computing reachable sets. In addition, we review some of the current work in high-dimensional HJ reachability to show how the dimensionality challenge can be alleviated via various general theoretical and application-specific insights. △ Less

Submitted 21 September, 2017; originally announced September 2017.

Comments: This paper summarizes the recent major research efforts in reachability. The paper will be presented in the proceedings of CDC 2017

A Hybrid Framework for Multi-Vehicle Collision Avoidance

Authors: Aparna Dhinakaran, Mo Chen, Glen Chou, Jennifer C. Shih, Claire J. Tomlin

Abstract: With the recent surge of interest in UAVs for civilian services, the importance of developing tractable multi-agent analysis techniques that provide safety and performance guarantees have drastically increased. Hamilton-Jacobi (HJ) reachability has successfully provided these guarantees to small-scale systems and is flexible in terms of system dynamics. However, the exponential complexity scaling… ▽ More With the recent surge of interest in UAVs for civilian services, the importance of developing tractable multi-agent analysis techniques that provide safety and performance guarantees have drastically increased. Hamilton-Jacobi (HJ) reachability has successfully provided these guarantees to small-scale systems and is flexible in terms of system dynamics. However, the exponential complexity scaling of HJ reachability with respect to system dimension prevents its direct application to larger-scale problems where the number of vehicles is greater than two. In this paper, we propose a collision avoidance algorithm using a hybrid framework for N+1 vehicles through higher-level control logic given any N-vehicle collision avoidance algorithm. Our algorithm conservatively approximates a guaranteed-safe region in the joint state space of the N+1 vehicles and produces a safety-preserving controller. In addition, our algorithm does not incur significant additional computation cost. We demonstrate our proposed method in simulation. △ Less

Submitted 21 March, 2017; originally announced March 2017.

Comments: Submitted to IEEE Conference on Decision and Control, 2017

Multiplayer Reach-Avoid Games via Pairwise Outcomes

Authors: Mo Chen, Zhengyuan Zhou, Claire J. Tomlin

Abstract: A multiplayer reach-avoid game is a differential game between an attacking team with NA attackers and a defending team with ND defenders playing on a compact domain with obstacles. The attacking team aims to send M of the NA attackers to some target location, while the defending team aims to prevent that by capturing attackers or indefinitely delaying attackers from reaching the target. Although t… ▽ More A multiplayer reach-avoid game is a differential game between an attacking team with NA attackers and a defending team with ND defenders playing on a compact domain with obstacles. The attacking team aims to send M of the NA attackers to some target location, while the defending team aims to prevent that by capturing attackers or indefinitely delaying attackers from reaching the target. Although the analysis of this game plays an important role in many applications, the optimal solution to this game is computationally intractable when NA>1 or ND>1. In this paper, we present two approaches for the NA=ND=1 case to determine pairwise outcomes, and a graph theoretic maximum matching approach to merge these pairwise outcomes for an NA,ND>1 solution that provides guarantees on the performance of the defending team. We will show that the four-dimensional Hamilton-Jacobi-Isaacs approach allows for real-time updates to the maximum matching, and that the two-dimensional "path defense" approach is considerably more scalable with the number of players while maintaining defender performance guarantees. △ Less

Submitted 1 November, 2016; originally announced November 2016.

Comments: to appear in IEEE Transactions on Automatic Control

Decomposition of Reachable Sets and Tubes for a Class of Nonlinear Systems

Authors: Mo Chen, Sylvia L. Herbert, Mahesh S. Vashishtha, Somil Bansal, Claire J. Tomlin

Abstract: Reachability analysis provides formal guarantees for performance and safety properties of nonlinear control systems. Here, one aims to compute the backward reachable set (BRS) or tube (BRT) -- the set of states from which the system can be driven into a target set at a particular time or within a time interval, respectively. The computational complexity of current approaches scales exponentially,… ▽ More Reachability analysis provides formal guarantees for performance and safety properties of nonlinear control systems. Here, one aims to compute the backward reachable set (BRS) or tube (BRT) -- the set of states from which the system can be driven into a target set at a particular time or within a time interval, respectively. The computational complexity of current approaches scales exponentially, making application to high-dimensional systems intractable. We propose a technique that decomposes the dynamics of a general class of nonlinear systems into subsystems which may be coupled through common states, controls, and disturbances. Despite this coupling, BRSs and BRTs can be computed efficiently using our technique without incurring additional approximation errors and without the need for linearizing dynamics or approximating sets as polytopes. Computations of BRSs and BRTs now become orders of magnitude faster, and for the first time BRSs and BRTs for many high-dimensional nonlinear control systems can be computed using the Hamilton-Jacobi (HJ) formulation. In situations involving bounded adversarial disturbances, our proposed method can obtain slightly conservative results. We demonstrate our theory by numerically computing BRSs and BRTs using the HJ formulation for several systems, including the 6D Acrobatic Quadrotor and the 10D Near-Hover Quadrotor. △ Less

Submitted 14 July, 2017; v1 submitted 1 November, 2016; originally announced November 2016.

Comments: to appear in IEEE Transactions on Automatic Control

Learning Quadrotor Dynamics Using Neural Network for Flight Control

Authors: Somil Bansal, Anayo K. Akametalu, Frank J. Jiang, Forrest Laine, Claire J. Tomlin

Abstract: Traditional learning approaches proposed for controlling quadrotors or helicopters have focused on improving performance for specific trajectories by iteratively improving upon a nominal controller, for example learning from demonstrations, iterative learning, and reinforcement learning. In these schemes, however, it is not clear how the information gathered from the training trajectories can be u… ▽ More Traditional learning approaches proposed for controlling quadrotors or helicopters have focused on improving performance for specific trajectories by iteratively improving upon a nominal controller, for example learning from demonstrations, iterative learning, and reinforcement learning. In these schemes, however, it is not clear how the information gathered from the training trajectories can be used to synthesize controllers for more general trajectories. Recently, the efficacy of deep learning in inferring helicopter dynamics has been shown. Motivated by the generalization capability of deep learning, this paper investigates whether a neural network based dynamics model can be employed to synthesize control for trajectories different than those used for training. To test this, we learn a quadrotor dynamics model using only translational and only rotational training trajectories, each of which can be controlled independently, and then use it to simultaneously control the yaw and position of a quadrotor, which is non-trivial because of nonlinear couplings between the two motions. We validate our approach in experiments on a quadrotor testbed. △ Less

Submitted 19 October, 2016; originally announced October 2016.

Exact and Efficient Hamilton-Jacobi-based Guaranteed Safety Analysis via System Decomposition

Authors: Mo Chen, Sylvia Herbert, Claire J. Tomlin

Abstract: Hamilton-Jacobi (HJ) reachability is a method that provides rigorous analyses of the safety properties of dynamical systems. This method has been successfully applied to many low-dimensional dynamical system models such as coarse models of aircraft and quadrotors in order to provide safety guarantees in potentially dangerous scenarios. These guarantees can be provided by the computation of a backw… ▽ More Hamilton-Jacobi (HJ) reachability is a method that provides rigorous analyses of the safety properties of dynamical systems. This method has been successfully applied to many low-dimensional dynamical system models such as coarse models of aircraft and quadrotors in order to provide safety guarantees in potentially dangerous scenarios. These guarantees can be provided by the computation of a backward reachable set (BRS), which represents the set of states from which the system may be driven into violating safety properties despite the system's best effort to remain safe. Unfortunately, HJ reachability is not practical for high-dimensional systems because the complexity of the BRS computation scales exponentially with the number of state dimensions. Although numerous approximation techniques are able to tractably provide conservative estimates of the BRS, they often require restrictive assumptions about system dynamics without providing an exact solution. In this paper we propose a general method for decomposing dynamical systems. Even when the resulting subsystems are coupled, relatively high-dimensional BRSs that were previously intractable or expensive to compute can now be quickly and exactly computed in lower-dimensional subspaces. As a result, the curse of dimensionality is alleviated to a large degree without sacrificing optimality. We demonstrate our theoretical results through two numerical examples: a 3D Dubins Car model and a 6D Acrobatic Quadrotor model. △ Less

Submitted 16 September, 2016; originally announced September 2016.

Comments: Submitted to ICRA 2017

On the Powerball Method for Optimization

Authors: Ye Yuan, Mu Li, Jun Liu, Claire J. Tomlin

Abstract: We propose a new method to accelerate the convergence of optimization algorithms. This method simply adds a power coefficient $γ\in[0,1)$ to the gradient during optimization. We call this the Powerball method and analyze the convergence rate for the Powerball method for strongly convex functions. While theoretically the Powerball method is guaranteed to have a linear convergence rate in the same o… ▽ More We propose a new method to accelerate the convergence of optimization algorithms. This method simply adds a power coefficient $γ\in[0,1)$ to the gradient during optimization. We call this the Powerball method and analyze the convergence rate for the Powerball method for strongly convex functions. While theoretically the Powerball method is guaranteed to have a linear convergence rate in the same order of the gradient method, we show that empirically it significantly outperforms the gradient descent and Newton's method, especially during the initial iterations. We demonstrate that the Powerball method provides a $10$-fold speedup of the convergence of both gradient descent and L-BFGS on multiple real datasets. △ Less

Submitted 1 September, 2017; v1 submitted 23 March, 2016; originally announced March 2016.

Fast Reachable Set Approximations via State Decoupling Disturbances

Authors: Mo Chen, Sylvia Herbert, Claire J. Tomlin

Abstract: With the recent surge of interest in using robotics and automation for civil purposes, providing safety and performance guarantees has become extremely important. In the past, differential games have been successfully used for the analysis of safety-critical systems. In particular, the Hamilton-Jacobi (HJ) formulation of differential games provides a flexible way to compute the reachable set, whic… ▽ More With the recent surge of interest in using robotics and automation for civil purposes, providing safety and performance guarantees has become extremely important. In the past, differential games have been successfully used for the analysis of safety-critical systems. In particular, the Hamilton-Jacobi (HJ) formulation of differential games provides a flexible way to compute the reachable set, which can characterize the set of states which lead to either desirable or undesirable configurations, depending on the application. While HJ reachability is applicable to many small practical systems, the curse of dimensionality prevents the direct application of HJ reachability to many larger systems. To address computation complexity issues, various efficient computation methods in the literature have been developed for approximating or exactly computing the solution to HJ partial differential equations, but only when the system dynamics are of specific forms. In this paper, we propose a flexible method to trade off optimality with computation complexity in HJ reachability analysis. To achieve this, we propose to simplify system dynamics by treating state variables as disturbances. We prove that the resulting approximation is conservative in the desired direction, and demonstrate our method using a four-dimensional plane model. △ Less

Submitted 20 April, 2017; v1 submitted 16 March, 2016; originally announced March 2016.

Comments: in Proceedings of the IEE Conference on Decision and Control, 2016

Exact and Efficient Hamilton-Jacobi Reachability for Decoupled Systems

Authors: Mo Chen, Claire J. Tomlin

Abstract: Reachability analysis is important for studying optimal control problems and differential games, which are powerful theoretical tools for analyzing and modeling many practical problems in robotics, aircraft control, among other application areas. In reachability analysis, one is interested in computing the reachable set, defined as the set of states from which there exists a control, despite the w… ▽ More Reachability analysis is important for studying optimal control problems and differential games, which are powerful theoretical tools for analyzing and modeling many practical problems in robotics, aircraft control, among other application areas. In reachability analysis, one is interested in computing the reachable set, defined as the set of states from which there exists a control, despite the worst disturbance, that can drive the system into a set of target states. The target states can be used to model either unsafe or desirable configurations, depending on the application. Many Hamilton-Jacobi formulations allow the computation of reachable sets; however, due to the exponential complexity scaling in computation time and space, problems involving approximately 5 dimensions become intractable. A number of methods that compute an approximate solution exist in the literature, but these methods trade off complexity for optimality. In this paper, we eliminate complexity-optimality trade-offs for time-invariant decoupled systems using a decoupled Hamilton-Jacobi formulation that enables the exact reconstruction of high dimensional solutions via low dimensional solutions of the decoupled subsystems. Our formulation is compatible with existing numerical tools, and we show the accuracy, computation benefits, and an application of our novel approach using two numerical examples. △ Less

Submitted 20 March, 2016; v1 submitted 19 March, 2015; originally announced March 2015.

Comments: 54th IEEE Conference on Decision and Control

Reach-Avoid Problems with Time-Varying Dynamics, Targets and Constraints

Authors: Jaime F. Fisac, Mo Chen, Claire J. Tomlin, S. Shankar Sastry

Abstract: We consider a reach-avoid differential game, in which one of the players aims to steer the system into a target set without violating a set of state constraints, while the other player tries to prevent the first from succeeding; the system dynamics, target set, and state constraints may all be time-varying. The analysis of this problem plays an important role in collision avoidance, motion plannin… ▽ More We consider a reach-avoid differential game, in which one of the players aims to steer the system into a target set without violating a set of state constraints, while the other player tries to prevent the first from succeeding; the system dynamics, target set, and state constraints may all be time-varying. The analysis of this problem plays an important role in collision avoidance, motion planning and aircraft control, among other applications. Previous methods for computing the guaranteed winning initial conditions and strategies for each player have either required augmenting the state vector to include time, or have been limited to problems with either no state constraints or entirely static targets, constraints and dynamics. To incorporate time-varying dynamics, targets and constraints without the need for state augmentation, we propose a modified Hamilton-Jacobi-Isaacs equation in the form of a double-obstacle variational inequality, and prove that the zero sublevel set of its viscosity solution characterizes the capture basin for the target under the state constraints. Through this formulation, our method can compute the capture basin and winning strategies for time-varying games at no additional computational cost with respect to the time-invariant case. We provide an implementation of this method based on well-known numerical schemes and show its convergence through a simple example; we include a second example in which our method substantially outperforms the state augmentation approach. △ Less

Submitted 28 October, 2014; v1 submitted 23 October, 2014; originally announced October 2014.

Comments: Submitted to 18th International Conference on Hybrid Systems: Computation and Control (HSCC 2015) v2: Corrected typos and introduced minor esthetic changes (equation 11 moved to the top of page 4); conclusion slightly shortened for conciseness

MSC Class: 49N75 (Primary); 49L25; 91A23 (Secondary)

Approximation Algorithms for Optimization of Combinatorial Dynamical Systems

Authors: Insoon Yang, Samuel A. Burden, Ram Rajagopal, S. Shankar Sastry, Claire J. Tomlin

Abstract: This paper considers an optimization problem for a dynamical system whose evolution depends on a collection of binary decision variables. We develop scalable approximation algorithms with provable suboptimality bounds to provide computationally tractable solution methods even when the dimension of the system and the number of the binary variables are large. The proposed method employs a linear app… ▽ More This paper considers an optimization problem for a dynamical system whose evolution depends on a collection of binary decision variables. We develop scalable approximation algorithms with provable suboptimality bounds to provide computationally tractable solution methods even when the dimension of the system and the number of the binary variables are large. The proposed method employs a linear approximation of the objective function such that the approximate problem is defined over the feasible space of the binary decision variables, which is a discrete set. To define such a linear approximation, we propose two different variation methods: one uses continuous relaxation of the discrete space and the other uses convex combinations of the vector field and running payoff. The approximate problem is a 0-1 linear program, which can be solved by existing polynomial-time exact or approximation algorithms, and does not require the solution of the dynamical system. Furthermore, we characterize a sufficient condition ensuring the approximate solution has a provable suboptimality bound. We show that this condition can be interpreted as the concavity of the objective function. The performance and utility of the proposed algorithms are demonstrated with the ON/OFF control problems of interdependent refrigeration systems. △ Less

Submitted 7 January, 2015; v1 submitted 27 September, 2014; originally announced September 2014.

Risk-Limiting Dynamic Contracts for Direct Load Control

Authors: Insoon Yang, Duncan S. Callaway, Claire J. Tomlin

Abstract: This paper proposes a novel continuous-time dynamic contract framework that has a risk-limiting capability. If a principal and an agent enter into such a contract, the principal can optimally manage its performance and risk with a guarantee that the agent's risk is less than or equal to a pre-specified level and that the agent's expected payoff is greater than or equal to another pre-specified thr… ▽ More This paper proposes a novel continuous-time dynamic contract framework that has a risk-limiting capability. If a principal and an agent enter into such a contract, the principal can optimally manage its performance and risk with a guarantee that the agent's risk is less than or equal to a pre-specified level and that the agent's expected payoff is greater than or equal to another pre-specified threshold. We achieve such risk-management capabilities by formulating the contract design problem as mean-variance constrained risk-sensitive control. A dynamic programming-based method is developed to solve the problem. The key idea of our proposed solution method is to reformulate the inequality constraints on the mean and the variance of the agent's payoff as dynamical system constraints by introducing new state and control variables. The reformulations use the martingale representation theorem. The proposed contract method enables us to develop a new direct load control method that provides the load-serving entity with financial risk management solutions in real-time electricity markets. We also propose an approximate decomposition of the optimal contract design problem for multiple customers into multiple low-dimensional contract problems for one customer. This allows the direct load control program to work with a large number of customers without any scalability issues. Furthermore, the contract design procedure can be completely parallelized. The performance and usefulness of the proposed contract method and its application to direct load control are demonstrated using data on the electric energy consumption of customers in Austin, Texas as well as the Electricity Reliability Council of Texas' locational marginal price data. △ Less

Submitted 8 October, 2014; v1 submitted 6 September, 2014; originally announced September 2014.

Path integral formulation of stochastic optimal control with generalized costs

Authors: Insoon Yang, Matthias Morzfeld, Claire J. Tomlin, Alexandre J. Chorin

Abstract: Path integral control solves a class of stochastic optimal control problems with a Monte Carlo (MC) method for an associated Hamilton-Jacobi-Bellman (HJB) equation. The MC approach avoids the need for a global grid of the domain of the HJB equation and, therefore, path integral control is in principle applicable to control problems of moderate to large dimension. The class of problems path integra… ▽ More Path integral control solves a class of stochastic optimal control problems with a Monte Carlo (MC) method for an associated Hamilton-Jacobi-Bellman (HJB) equation. The MC approach avoids the need for a global grid of the domain of the HJB equation and, therefore, path integral control is in principle applicable to control problems of moderate to large dimension. The class of problems path integral control can solve, however, is defined by requirements on the cost function, the noise covariance matrix and the control input matrix. We relax the requirements on the cost function by introducing a new state that represents an augmented running cost. In our new formulation the cost function can contain stochastic integral terms and linear control costs, which are important in applications in engineering, economics and finance. We find an efficient numerical implementation of our grid-free MC approach and demonstrate its performance and usefulness in examples from hierarchical electric load management. The dimension of one of our examples is large enough to make classical grid-based HJB solvers impractical. △ Less

Submitted 23 August, 2014; v1 submitted 30 June, 2014; originally announced June 2014.

A sampling-based approach to scalable constraint satisfaction in linear sampled-data systems---Part I: Computation

Authors: Shahab Kaynama, Jeremy H. Gillula, Claire J. Tomlin

Abstract: Sampled-data (SD) systems, which are composed of both discrete- and continuous-time components, are arguably one of the most common classes of cyberphysical systems in practice; most modern controllers are implemented on digital platforms while the plant dynamics that are being controlled evolve continuously in time. As with all cyberphysical systems, ensuring hard constraint satisfaction is key i… ▽ More Sampled-data (SD) systems, which are composed of both discrete- and continuous-time components, are arguably one of the most common classes of cyberphysical systems in practice; most modern controllers are implemented on digital platforms while the plant dynamics that are being controlled evolve continuously in time. As with all cyberphysical systems, ensuring hard constraint satisfaction is key in the safe operation of SD systems. A powerful analytical tool for guaranteeing such constraint satisfaction is the viability kernel: the set of all initial conditions for which a safety-preserving control law (that is, a control law that satisfies all input and state constraints) exists. In this paper we present a novel sampling-based algorithm that tightly approximates the viability kernel for high-dimensional sampled-data linear time-invariant (LTI) systems. Unlike prior work in this area, our algorithm formally handles both the discrete and continuous characteristics of SD systems. We prove the correctness and convergence of our approximation technique, provide discussions on heuristic methods to optimally bias the sampling process, and demonstrate the results on a twelve-dimensional flight envelope protection problem. △ Less

Submitted 9 May, 2014; originally announced May 2014.

Dynamic Contracts with Partial Observations: Application to Indirect Load Control

Authors: Insoon Yang, Duncan S. Callaway, Claire J. Tomlin

Abstract: This paper proposes a method to design an optimal dynamic contract between a principal and an agent, who has the authority to control both the principal's revenue and an engineered system. The key characteristic of our problem setting is that the principal has very limited information: the principal has no capability to monitor the agent's control or the state of the engineered system. The agent h… ▽ More This paper proposes a method to design an optimal dynamic contract between a principal and an agent, who has the authority to control both the principal's revenue and an engineered system. The key characteristic of our problem setting is that the principal has very limited information: the principal has no capability to monitor the agent's control or the state of the engineered system. The agent has perfect observations. With this asymmetry of information, we show that the principal can induce the agent to control both the revenue and the system processes in a way that maximizes the principal's utility, if the principal offers appropriate real-time and end-time compensation. We reformulate the dynamic contract design problem as a stochastic optimal control of both the engineered system and the agent's future expected payoff, which can be numerically solved using an associated Hamilton-Jacobi-Bellman equation. The performance and usefulness of the proposed contract are demonstrated with an indirect load control problem. △ Less

Submitted 20 March, 2014; originally announced March 2014.

Comments: 13 pages, 3 figures

On the Optimal Solutions of the Infinite-Horizon Linear Sensor Scheduling Problem

Authors: Lin Zhao, Wei Zhang, Jianghai Hu, Alessandro Abate, Claire J. Tomlin

Abstract: This paper studies the infinite-horizon sensor scheduling problem for linear Gaussian processes with linear measurement functions. Several important properties of the optimal infinite-horizon schedules are derived. In particular, it is proved that under some mild conditions, both the optimal infinite-horizon average-per-stage cost and the corresponding optimal sensor schedules are independent of t… ▽ More This paper studies the infinite-horizon sensor scheduling problem for linear Gaussian processes with linear measurement functions. Several important properties of the optimal infinite-horizon schedules are derived. In particular, it is proved that under some mild conditions, both the optimal infinite-horizon average-per-stage cost and the corresponding optimal sensor schedules are independent of the covariance matrix of the initial state. It is also proved that the optimal estimation cost can be approximated arbitrarily closely by a periodic schedule with a finite period. Moreover, it is shown that the sequence of the average-per-stage costs of the optimal schedule must converge. These theoretical results provide valuable insights into the design and analysis of various infinite-horizon sensor scheduling algorithms. △ Less

Submitted 20 March, 2014; v1 submitted 30 November, 2013; originally announced December 2013.

Comments: 6 pages, 1 figure

Identification of Parameters and Initial Values for Reaction-Diffusion Systems in Protein Networks (Extended Version)

Authors: Insoon Yang, Claire J. Tomlin

Abstract: Spatio-temporal biochemical signaling in a large class of protein-protein interaction networks is well modeled by a reaction-diffusion system. The global existence of the solution to the reaction-diffusion system is determined by the reaction kinetics model and the protein network topology. We propose a novel reaction kinetics model that guarantees that the reaction-diffusion system with this mode… ▽ More Spatio-temporal biochemical signaling in a large class of protein-protein interaction networks is well modeled by a reaction-diffusion system. The global existence of the solution to the reaction-diffusion system is determined by the reaction kinetics model and the protein network topology. We propose a novel reaction kinetics model that guarantees that the reaction-diffusion system with this model has a nonnegative invariant global classical solution for any network topology. We then present a computational method to identify the unknown parameters and initial values for a reaction-diffusion system with this reaction kinetics model. The identification approach solves an optimization problem that minimizes the cost function defined as the $L^2$-norm of the difference between the data and the solution of the reaction-diffusion system. We utilize an adjoint-based optimal control method to obtain the gradients of the cost function with respect to the parameters and initial values. The regularity of the global classical solutions of the reaction-diffusion system and its corresponding adjoint system avoids situations in which the gradients blow up, and therefore guarantees the success of the identification method for any network structure. Utilizing this gradient information, an efficient algorithm to solve the optimization problem is proposed and applied to estimate the mass diffusivities, rate constants and initial values of a reaction-diffusion system that models protein-protein interactions in a signaling network that regulates the actin cytoskeleton in a malignant breast cell. △ Less

Submitted 8 August, 2013; originally announced August 2013.