-
Solving Reach- and Stabilize-Avoid Problems Using Discounted Reachability
Authors:
Boyang Li,
Zheng Gong,
Sylvia Herbert
Abstract:
In this article, we consider the infinite-horizon reach-avoid (RA) and stabilize-avoid (SA) zero-sum game problems for general nonlinear continuous-time systems, where the goal is to find the set of states that can be controlled to reach or stabilize to a target set, without violating constraints even under the worst-case disturbance. Based on the Hamilton-Jacobi reachability method, we address th…
▽ More
In this article, we consider the infinite-horizon reach-avoid (RA) and stabilize-avoid (SA) zero-sum game problems for general nonlinear continuous-time systems, where the goal is to find the set of states that can be controlled to reach or stabilize to a target set, without violating constraints even under the worst-case disturbance. Based on the Hamilton-Jacobi reachability method, we address the RA problem by designing a new Lipschitz continuous RA value function, whose zero sublevel set exactly characterizes the RA set. We establish that the associated Bellman backup operator is contractive and that the RA value function is the unique viscosity solution of a Hamilton-Jacobi variational inequality. Finally, we develop a two-step framework for the SA problem by integrating our RA strategies with a recently proposed Robust Control Lyapunov-Value Function, thereby ensuring both target reachability and long-term stability. We numerically verify our RA and SA frameworks on a 3D Dubins car system to demonstrate the efficacy of the proposed approach.
△ Less
Submitted 13 May, 2025;
originally announced May 2025.
-
Linear Supervision for Nonlinear, High-Dimensional Neural Control and Differential Games
Authors:
William Sharpless,
Zeyuan Feng,
Somil Bansal,
Sylvia Herbert
Abstract:
As the dimension of a system increases, traditional methods for control and differential games rapidly become intractable, making the design of safe autonomous agents challenging in complex or team settings. Deep-learning approaches avoid discretization and yield numerous successes in robotics and autonomy, but at a higher dimensional limit, accuracy falls as sampling becomes less efficient. We pr…
▽ More
As the dimension of a system increases, traditional methods for control and differential games rapidly become intractable, making the design of safe autonomous agents challenging in complex or team settings. Deep-learning approaches avoid discretization and yield numerous successes in robotics and autonomy, but at a higher dimensional limit, accuracy falls as sampling becomes less efficient. We propose using rapidly generated linear solutions to the partial differential equation (PDE) arising in the problem to accelerate and improve learned value functions for guidance in high-dimensional, nonlinear problems. We define two programs that combine supervision of the linear solution with a standard PDE loss. We demonstrate that these programs offer improvements in speed and accuracy in both a 50-D differential game problem and a 10-D quadrotor control problem.
△ Less
Submitted 26 April, 2025; v1 submitted 2 December, 2024;
originally announced December 2024.
-
Sensor-Based Distributionally Robust Control for Safe Robot Navigation in Dynamic Environments
Authors:
Kehan Long,
Yinzhuang Yi,
Zhirui Dai,
Sylvia Herbert,
Jorge Cortés,
Nikolay Atanasov
Abstract:
We introduce a novel method for mobile robot navigation in dynamic, unknown environments, leveraging onboard sensing and distributionally robust optimization to impose probabilistic safety constraints. Our method introduces a distributionally robust control barrier function (DR-CBF) that directly integrates noisy sensor measurements and state estimates to define safety constraints. This approach i…
▽ More
We introduce a novel method for mobile robot navigation in dynamic, unknown environments, leveraging onboard sensing and distributionally robust optimization to impose probabilistic safety constraints. Our method introduces a distributionally robust control barrier function (DR-CBF) that directly integrates noisy sensor measurements and state estimates to define safety constraints. This approach is applicable to a wide range of control-affine dynamics, generalizable to robots with complex geometries, and capable of operating at real-time control frequencies. Coupled with a control Lyapunov function (CLF) for path following, the proposed CLF-DR-CBF control synthesis method achieves safe, robust, and efficient navigation in challenging environments. We demonstrate the effectiveness and robustness of our approach for safe autonomous navigation under uncertainty in simulations and real-world experiments with differential-drive robots.
△ Less
Submitted 5 May, 2025; v1 submitted 28 May, 2024;
originally announced May 2024.
-
Synthesizing Control Lyapunov-Value Functions for High-Dimensional Systems Using System Decomposition and Admissible Control Sets
Authors:
Zheng Gong,
Hyun Joe Jeong,
Sylvia Herbert
Abstract:
Control Lyapunov functions (CLFs) play a vital role in modern control applications, but finding them remains a problem. Recently, the control Lyapunov-value function (CLVF) and robust CLVF have been proposed as solutions for nonlinear time-invariant systems with bounded control and disturbance. However, the CLVF suffers from the ''curse of dimensionality,'' which hinders its application to practic…
▽ More
Control Lyapunov functions (CLFs) play a vital role in modern control applications, but finding them remains a problem. Recently, the control Lyapunov-value function (CLVF) and robust CLVF have been proposed as solutions for nonlinear time-invariant systems with bounded control and disturbance. However, the CLVF suffers from the ''curse of dimensionality,'' which hinders its application to practical high-dimensional systems. In this paper, we propose a method to decompose systems of a particular coupled nonlinear structure, in order to solve for the CLVF in each low-dimensional subsystem. We then reconstruct the full-dimensional CLVF and provide sufficient conditions for when this reconstruction is exact. Moreover, a point-wise optimal controller can be obtained using a quadratic program. We also show that when the exact reconstruction is impossible, the subsystems' CLVFs and their ``admissible control sets'' can be used to generate a Lipschitz continuous CLF. We provide several numerical examples to validate the theory and show computational efficiency.
△ Less
Submitted 2 April, 2024;
originally announced April 2024.
-
State-Augmented Linear Games with Antagonistic Error for High-Dimensional, Nonlinear Hamilton-Jacobi Reachability
Authors:
Will Sharpless,
Yat Tin Chow,
Sylvia Herbert
Abstract:
Hamilton-Jacobi Reachability (HJR) is a popular method for analyzing the liveness and safety of a dynamical system with bounded control and disturbance. The corresponding HJ value function offers a robust controller and characterizes the reachable sets, but is traditionally solved with Dynamic Programming (DP) and limited to systems of dimension less than six. Recently, the space-parallelizeable,…
▽ More
Hamilton-Jacobi Reachability (HJR) is a popular method for analyzing the liveness and safety of a dynamical system with bounded control and disturbance. The corresponding HJ value function offers a robust controller and characterizes the reachable sets, but is traditionally solved with Dynamic Programming (DP) and limited to systems of dimension less than six. Recently, the space-parallelizeable, generalized Hopf formula has been shown to also solve the HJ value with a nearly three-log increase in dimension limit, but is limited to linear systems. To extend this potential, we demonstrate how state-augmented (SA) spaces, which are well-known for their improved linearization accuracy, may be used to solve tighter, conservative approximations of the value function with any linear model in this SA space. Namely, we show that with a representation of the true dynamics in the SA space, a series of inequalities confirms that the value of a SA linear game with antagonistic error is a conservative envelope of the true value function. It follows that if the optimal controller for the HJ SA linear game with error may succeed, it will also succeed in the true system. Unlike previous methods, this result offers the ability to safely approximate reachable sets and their corresponding controllers with the Hopf formula in a non-convex manner. Finally, we demonstrate this in the slow manifold system for clarity, and in the controlled Van der Pol system with different lifting functions.
△ Less
Submitted 19 June, 2025; v1 submitted 25 March, 2024;
originally announced March 2024.
-
Conservative Linear Envelopes for Nonlinear, High-Dimensional, Hamilton-Jacobi Reachability
Authors:
Will Sharpless,
Yat Tin Chow,
Sylvia Herbert
Abstract:
Hamilton-Jacobi reachability (HJR) provides a value function that encodes the set of states from which a system with bounded control inputs can reach or avoid a target despite any bounded disturbance, and the corresponding robust, optimal control policy. Though powerful, traditional methods for HJR rely on dynamic programming (DP) and suffer from exponential computation growth with respect to stat…
▽ More
Hamilton-Jacobi reachability (HJR) provides a value function that encodes the set of states from which a system with bounded control inputs can reach or avoid a target despite any bounded disturbance, and the corresponding robust, optimal control policy. Though powerful, traditional methods for HJR rely on dynamic programming (DP) and suffer from exponential computation growth with respect to state dimension. The recently favored Hopf formula mitigates this ``curse of dimensionality'' by providing an efficient and space-parallelizable approach for solving the reachability problem. However, the Hopf formula can only be applied to linear time-varying systems. To overcome this limitation, we show that the error between a nonlinear system and a linear model can be transformed into an adversarial bounded artificial disturbance. One may then solve the dimension-robust generalized Hopf formula for a linear game with this ``antagonistic error" to perform guaranteed conservative reachability analysis and control synthesis of nonlinear systems; this can be done for problem formulations in which no other HJR method is both computationally feasible and guaranteed. In addition, we offer several technical methods for reducing conservativeness in the analysis. We demonstrate the effectiveness of our results through one illustrative example (the controlled Van der Pol system) that can be compared to standard DP, and one higher-dimensional 15D example (a 5-agent pursuit-evasion game with Dubins cars).
△ Less
Submitted 19 June, 2025; v1 submitted 21 March, 2024;
originally announced March 2024.
-
Robust Control Lyapunov-Value Functions for Nonlinear Disturbed Systems
Authors:
Zheng Gong,
Sylvia Herbert
Abstract:
Control Lyapunov Functions (CLFs) have been extensively used in the control community. A well-known drawback is the absence of a systematic way to construct CLFs for general nonlinear systems, and the problem can become more complex with input or state constraints. Our preliminary work on constructing Control Lyapunov Value Functions (CLVFs) using Hamilton-Jacobi (HJ) reachability analysis provide…
▽ More
Control Lyapunov Functions (CLFs) have been extensively used in the control community. A well-known drawback is the absence of a systematic way to construct CLFs for general nonlinear systems, and the problem can become more complex with input or state constraints. Our preliminary work on constructing Control Lyapunov Value Functions (CLVFs) using Hamilton-Jacobi (HJ) reachability analysis provides a method for finding a non-smooth CLF. In this paper, we extend our work on CLVFs to systems with bounded disturbance and define the Robust CLVF (R-CLVF). The R-CLVF naturally inherits all properties of the CLVF; i.e., it first identifies the "smallest robust control invariant set (SRCIS)" and stabilizes the system to it with a user-specified exponential rate. The region from which the exponential rate can be met is called the "region of exponential stabilizability (ROES)." We provide clearer definitions of the SRCIS and more rigorous proofs of several important theorems. Since the computation of the R-CLVF suffers from the "curse of dimensionality," we also provide two techniques (warmstart and system decomposition) that solve it, along with necessary proofs. Three numerical examples are provided, validating our definition of SRCIS, illustrating the trade-off between a faster decay rate and a smaller ROES, and demonstrating the efficiency of computation using warmstart and decomposition.
△ Less
Submitted 6 June, 2025; v1 submitted 5 March, 2024;
originally announced March 2024.
-
Koopman-Hopf Hamilton-Jacobi Reachability and Control
Authors:
Will Sharpless,
Nikhil Shinde,
Matthew Kim,
Yat Tin Chow,
Sylvia Herbert
Abstract:
The Hopf formula for Hamilton-Jacobi reachability (HJR) analysis has been proposed to solve high-dimensional differential games, producing the set of initial states and corresponding controller required to reach (or avoid) a target despite bounded disturbances. As a space-parallelizable method, the Hopf formula avoids the curse of dimensionality that afflicts standard dynamic-programming HJR, but…
▽ More
The Hopf formula for Hamilton-Jacobi reachability (HJR) analysis has been proposed to solve high-dimensional differential games, producing the set of initial states and corresponding controller required to reach (or avoid) a target despite bounded disturbances. As a space-parallelizable method, the Hopf formula avoids the curse of dimensionality that afflicts standard dynamic-programming HJR, but is restricted to linear time-varying systems. To compute reachable sets for high-dimensional nonlinear systems, we pair the Hopf solution with Koopman theory for global linearization. By first lifting a nonlinear system to a linear space and then solving the Hopf formula, approximate reachable sets can be efficiently computed that are much more accurate than local linearizations. Furthermore, we construct a Koopman-Hopf disturbance-rejecting controller, and test its ability to drive a 10-dimensional nonlinear glycolysis model. We find that it significantly out-competes expectation-minimizing and game-theoretic model predictive controllers with the same Koopman linearization in the presence of bounded stochastic disturbance. In summary, we demonstrate a dimension-robust method to approximately solve HJR, allowing novel application to analyze and control high-dimensional, nonlinear systems with disturbance. An open-source toolbox in Julia is introduced for both Hopf and Koopman-Hopf reachability and control.
△ Less
Submitted 19 June, 2025; v1 submitted 21 March, 2023;
originally announced March 2023.
-
FaSTrack: a Modular Framework for Real-Time Motion Planning and Guaranteed Safe Tracking
Authors:
Mo Chen,
Sylvia L. Herbert,
Haimin Hu,
Ye Pu,
Jaime F. Fisac,
Somil Bansal,
SooJean Han,
Claire J. Tomlin
Abstract:
Real-time, guaranteed safe trajectory planning is vital for navigation in unknown environments. However, real-time navigation algorithms typically sacrifice robustness for computation speed. Alternatively, provably safe trajectory planning tends to be too computationally intensive for real-time replanning. We propose FaSTrack, Fast and Safe Tracking, a framework that achieves both real-time replan…
▽ More
Real-time, guaranteed safe trajectory planning is vital for navigation in unknown environments. However, real-time navigation algorithms typically sacrifice robustness for computation speed. Alternatively, provably safe trajectory planning tends to be too computationally intensive for real-time replanning. We propose FaSTrack, Fast and Safe Tracking, a framework that achieves both real-time replanning and guaranteed safety. In this framework, real-time computation is achieved by allowing any trajectory planner to use a simplified \textit{planning model} of the system. The plan is tracked by the system, represented by a more realistic, higher-dimensional \textit{tracking model}. We precompute the tracking error bound (TEB) due to mismatch between the two models and due to external disturbances. We also obtain the corresponding tracking controller used to stay within the TEB. The precomputation does not require prior knowledge of the environment. We demonstrate FaSTrack using Hamilton-Jacobi reachability for precomputation and three different real-time trajectory planners with three different tracking-planning model pairs.
△ Less
Submitted 13 March, 2021; v1 submitted 13 February, 2021;
originally announced February 2021.
-
Robust Tracking with Model Mismatch for Fast and Safe Planning: an SOS Optimization Approach
Authors:
Sumeet Singh,
Mo Chen,
Sylvia L. Herbert,
Claire J. Tomlin,
Marco Pavone
Abstract:
In the pursuit of real-time motion planning, a commonly adopted practice is to compute a trajectory by running a planning algorithm on a simplified, low-dimensional dynamical model, and then employ a feedback tracking controller that tracks such a trajectory by accounting for the full, high-dimensional system dynamics. While this strategy of planning with model mismatch generally yields fast compu…
▽ More
In the pursuit of real-time motion planning, a commonly adopted practice is to compute a trajectory by running a planning algorithm on a simplified, low-dimensional dynamical model, and then employ a feedback tracking controller that tracks such a trajectory by accounting for the full, high-dimensional system dynamics. While this strategy of planning with model mismatch generally yields fast computation times, there are no guarantees of dynamic feasibility, which hampers application to safety-critical systems. Building upon recent work that addressed this problem through the lens of Hamilton-Jacobi (HJ) reachability, we devise an algorithmic framework whereby one computes, offline, for a pair of "planner" (i.e., low-dimensional) and "tracking" (i.e., high-dimensional) models, a feedback tracking controller and associated tracking bound. This bound is then used as a safety margin when generating motion plans via the low-dimensional model. Specifically, we harness the computational tool of sum-of-squares (SOS) programming to design a bilinear optimization algorithm for the computation of the feedback tracking controller and associated tracking bound. The algorithm is demonstrated via numerical experiments, with an emphasis on investigating the trade-off between the increased computational scalability afforded by SOS and its intrinsic conservativeness. Collectively, our results enable scaling the appealing strategy of planning with model mismatch to systems that are beyond the reach of HJ analysis, while maintaining safety guarantees.
△ Less
Submitted 28 July, 2019; v1 submitted 1 August, 2018;
originally announced August 2018.
-
Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances
Authors:
Somil Bansal,
Mo Chen,
Sylvia Herbert,
Claire J. Tomlin
Abstract:
Hamilton-Jacobi (HJ) reachability analysis is an important formal verification method for guaranteeing performance and safety properties of dynamical systems; it has been applied to many small-scale systems in the past decade. Its advantages include compatibility with general nonlinear system dynamics, formal treatment of bounded disturbances, and the availability of well-developed numerical tools…
▽ More
Hamilton-Jacobi (HJ) reachability analysis is an important formal verification method for guaranteeing performance and safety properties of dynamical systems; it has been applied to many small-scale systems in the past decade. Its advantages include compatibility with general nonlinear system dynamics, formal treatment of bounded disturbances, and the availability of well-developed numerical tools. The main challenge is addressing its exponential computational complexity with respect to the number of state variables. In this tutorial, we present an overview of basic HJ reachability theory and provide instructions for using the most recent numerical tools, including an efficient GPU-parallelized implementation of a Level Set Toolbox for computing reachable sets. In addition, we review some of the current work in high-dimensional HJ reachability to show how the dimensionality challenge can be alleviated via various general theoretical and application-specific insights.
△ Less
Submitted 21 September, 2017;
originally announced September 2017.
-
Decomposition of Reachable Sets and Tubes for a Class of Nonlinear Systems
Authors:
Mo Chen,
Sylvia L. Herbert,
Mahesh S. Vashishtha,
Somil Bansal,
Claire J. Tomlin
Abstract:
Reachability analysis provides formal guarantees for performance and safety properties of nonlinear control systems. Here, one aims to compute the backward reachable set (BRS) or tube (BRT) -- the set of states from which the system can be driven into a target set at a particular time or within a time interval, respectively. The computational complexity of current approaches scales exponentially,…
▽ More
Reachability analysis provides formal guarantees for performance and safety properties of nonlinear control systems. Here, one aims to compute the backward reachable set (BRS) or tube (BRT) -- the set of states from which the system can be driven into a target set at a particular time or within a time interval, respectively. The computational complexity of current approaches scales exponentially, making application to high-dimensional systems intractable. We propose a technique that decomposes the dynamics of a general class of nonlinear systems into subsystems which may be coupled through common states, controls, and disturbances. Despite this coupling, BRSs and BRTs can be computed efficiently using our technique without incurring additional approximation errors and without the need for linearizing dynamics or approximating sets as polytopes. Computations of BRSs and BRTs now become orders of magnitude faster, and for the first time BRSs and BRTs for many high-dimensional nonlinear control systems can be computed using the Hamilton-Jacobi (HJ) formulation. In situations involving bounded adversarial disturbances, our proposed method can obtain slightly conservative results. We demonstrate our theory by numerically computing BRSs and BRTs using the HJ formulation for several systems, including the 6D Acrobatic Quadrotor and the 10D Near-Hover Quadrotor.
△ Less
Submitted 14 July, 2017; v1 submitted 1 November, 2016;
originally announced November 2016.
-
Exact and Efficient Hamilton-Jacobi-based Guaranteed Safety Analysis via System Decomposition
Authors:
Mo Chen,
Sylvia Herbert,
Claire J. Tomlin
Abstract:
Hamilton-Jacobi (HJ) reachability is a method that provides rigorous analyses of the safety properties of dynamical systems. This method has been successfully applied to many low-dimensional dynamical system models such as coarse models of aircraft and quadrotors in order to provide safety guarantees in potentially dangerous scenarios. These guarantees can be provided by the computation of a backw…
▽ More
Hamilton-Jacobi (HJ) reachability is a method that provides rigorous analyses of the safety properties of dynamical systems. This method has been successfully applied to many low-dimensional dynamical system models such as coarse models of aircraft and quadrotors in order to provide safety guarantees in potentially dangerous scenarios. These guarantees can be provided by the computation of a backward reachable set (BRS), which represents the set of states from which the system may be driven into violating safety properties despite the system's best effort to remain safe. Unfortunately, HJ reachability is not practical for high-dimensional systems because the complexity of the BRS computation scales exponentially with the number of state dimensions. Although numerous approximation techniques are able to tractably provide conservative estimates of the BRS, they often require restrictive assumptions about system dynamics without providing an exact solution. In this paper we propose a general method for decomposing dynamical systems. Even when the resulting subsystems are coupled, relatively high-dimensional BRSs that were previously intractable or expensive to compute can now be quickly and exactly computed in lower-dimensional subspaces. As a result, the curse of dimensionality is alleviated to a large degree without sacrificing optimality. We demonstrate our theoretical results through two numerical examples: a 3D Dubins Car model and a 6D Acrobatic Quadrotor model.
△ Less
Submitted 16 September, 2016;
originally announced September 2016.
-
Fast Reachable Set Approximations via State Decoupling Disturbances
Authors:
Mo Chen,
Sylvia Herbert,
Claire J. Tomlin
Abstract:
With the recent surge of interest in using robotics and automation for civil purposes, providing safety and performance guarantees has become extremely important. In the past, differential games have been successfully used for the analysis of safety-critical systems. In particular, the Hamilton-Jacobi (HJ) formulation of differential games provides a flexible way to compute the reachable set, whic…
▽ More
With the recent surge of interest in using robotics and automation for civil purposes, providing safety and performance guarantees has become extremely important. In the past, differential games have been successfully used for the analysis of safety-critical systems. In particular, the Hamilton-Jacobi (HJ) formulation of differential games provides a flexible way to compute the reachable set, which can characterize the set of states which lead to either desirable or undesirable configurations, depending on the application. While HJ reachability is applicable to many small practical systems, the curse of dimensionality prevents the direct application of HJ reachability to many larger systems. To address computation complexity issues, various efficient computation methods in the literature have been developed for approximating or exactly computing the solution to HJ partial differential equations, but only when the system dynamics are of specific forms. In this paper, we propose a flexible method to trade off optimality with computation complexity in HJ reachability analysis. To achieve this, we propose to simplify system dynamics by treating state variables as disturbances. We prove that the resulting approximation is conservative in the desired direction, and demonstrate our method using a four-dimensional plane model.
△ Less
Submitted 20 April, 2017; v1 submitted 16 March, 2016;
originally announced March 2016.
