-
Upper and Lower Bounds for a Class of Constrained Linear Time-Varying Games
Authors:
Vincent Liu,
Chris Manzie,
Peter M. Dower
Abstract:
This paper develops an algorithm for upper- and lower-bounding the value function for a class of linear time-varying games subject to convex control sets. In particular, a two-player zero-sum differential game is considered where the respective players aim to minimise and maximise a convex terminal state cost. A collection of solutions of a single-player dynamical system subject to a trimmed contr…
▽ More
This paper develops an algorithm for upper- and lower-bounding the value function for a class of linear time-varying games subject to convex control sets. In particular, a two-player zero-sum differential game is considered where the respective players aim to minimise and maximise a convex terminal state cost. A collection of solutions of a single-player dynamical system subject to a trimmed control set is used to characterise a viscosity supersolution of a Hamilton-Jacobi (HJ) equation, which in turn yields an upper bound for the value function. Analogously, a collection of hyperplanes is used to characterise a viscosity subsolution of the HJ equation, which yields a lower bound. The computational complexity and memory requirement of the proposed algorithm scales with the number of solutions and hyperplanes that characterise the bounds, which is not explicitly tied to the number of system states. Thus, the algorithm is tractable for systems of moderately high dimension whilst preserving rigorous guarantees for optimal control and differential game applications.
△ Less
Submitted 11 March, 2025;
originally announced March 2025.
-
A Hamilton-Jacobi-Bellman Approach to Ellipsoidal Approximations of Reachable Sets for Linear Time-Varying Systems
Authors:
Vincent Liu,
Chris Manzie,
Peter M. Dower
Abstract:
Reachable sets for a dynamical system describe collections of system states that can be reached in finite time, subject to system dynamics. They can be used to guarantee goal satisfaction in controller design or to verify that unsafe regions will be avoided. However, general-purpose methods for computing these sets suffer from the curse of dimensionality, which typically prohibits their use for sy…
▽ More
Reachable sets for a dynamical system describe collections of system states that can be reached in finite time, subject to system dynamics. They can be used to guarantee goal satisfaction in controller design or to verify that unsafe regions will be avoided. However, general-purpose methods for computing these sets suffer from the curse of dimensionality, which typically prohibits their use for systems with more than a small number of states, even if they are linear. In this paper, we demonstrate that viscosity supersolutions and subsolutions of a Hamilton-Jacobi-Bellman equation can be used to generate, respectively, under-approximating and over-approximating reachable sets for time-varying nonlinear systems. Based on this observation, we derive dynamics for a union and intersection of ellipsoidal sets that, respectively, under-approximate and over-approximate the reachable set for linear time-varying systems subject to an ellipsoidal input constraint and an ellipsoidal terminal (or initial) set. We demonstrate that the dynamics for these ellipsoids can be selected to ensure that their boundaries coincide with the boundary of the exact reachable set along a solution of the system. The ellipsoidal sets can be generated with polynomial computational complexity in the number of states, making our approximation scheme computationally tractable for continuous-time linear time-varying systems of relatively high dimension.
△ Less
Submitted 21 February, 2025; v1 submitted 11 January, 2024;
originally announced January 2024.
-
A Tree Structure Approach to Reachability Analysis
Authors:
Alessandro Alla,
Peter M. Dower,
Vincent Liu
Abstract:
Reachability analysis is a powerful tool when it comes to capturing the behaviour, thus verifying the safety, of autonomous systems. However, general-purpose methods, such as Hamilton-Jacobi approaches, suffer from the curse of dimensionality. In this paper, we mitigate this problem for systems of moderate dimension and we propose a new algorithm based on a tree structure approach with geometric p…
▽ More
Reachability analysis is a powerful tool when it comes to capturing the behaviour, thus verifying the safety, of autonomous systems. However, general-purpose methods, such as Hamilton-Jacobi approaches, suffer from the curse of dimensionality. In this paper, we mitigate this problem for systems of moderate dimension and we propose a new algorithm based on a tree structure approach with geometric pruning. The numerical examples will include a comparison with a standard finite-difference method for linear and nonlinear problems
△ Less
Submitted 26 October, 2022; v1 submitted 25 October, 2022;
originally announced October 2022.
-
Neural network architectures using min-plus algebra for solving certain high dimensional optimal control problems and Hamilton-Jacobi PDEs
Authors:
Jérôme Darbon,
Peter M. Dower,
Tingwei Meng
Abstract:
Solving high dimensional optimal control problems and corresponding Hamilton-Jacobi PDEs are important but challenging problems in control engineering. In this paper, we propose two abstract neural network architectures which are respectively used to compute the value function and the optimal control for certain class of high dimensional optimal control problems. We provide the mathematical analys…
▽ More
Solving high dimensional optimal control problems and corresponding Hamilton-Jacobi PDEs are important but challenging problems in control engineering. In this paper, we propose two abstract neural network architectures which are respectively used to compute the value function and the optimal control for certain class of high dimensional optimal control problems. We provide the mathematical analysis for the two abstract architectures. We also show several numerical results computed using the deep neural network implementations of these abstract architectures. A preliminary implementation of our proposed neural network architecture on FPGAs shows promising speed up compared to CPUs. This work paves the way to leverage efficient dedicated hardware designed for neural networks to solve high dimensional optimal control problems and Hamilton-Jacobi PDEs.
△ Less
Submitted 29 March, 2023; v1 submitted 7 May, 2021;
originally announced May 2021.
-
Exploiting characteristics in stationary action problems
Authors:
V. Basco,
P. M. Dower,
W. M. McEneaney,
I. Yegorov
Abstract:
Connections between the principle of stationary action and optimal control, and between established notions of minimax and viscosity solutions, are combined to describe trajectories of energy conserving systems as solutions of corresponding Cauchy problems defined with respect to attendant systems of characteristics equations.
Connections between the principle of stationary action and optimal control, and between established notions of minimax and viscosity solutions, are combined to describe trajectories of energy conserving systems as solutions of corresponding Cauchy problems defined with respect to attendant systems of characteristics equations.
△ Less
Submitted 19 February, 2020;
originally announced February 2020.
-
A two-player game representation for a class of infinite horizon control problems under state constraints
Authors:
Vincenzo Basco,
Peter M. Dower
Abstract:
In this paper we give feedback laws for a class of parametrized infinite horizon control problems under state constraints. We provide a two-player representation assuming time dependent dynamics and Lagrangian and the set constraints merely compact. Using viability results recently investigated for infinite horizon settings in presence of state constraints, we extend some known results for the lin…
▽ More
In this paper we give feedback laws for a class of parametrized infinite horizon control problems under state constraints. We provide a two-player representation assuming time dependent dynamics and Lagrangian and the set constraints merely compact. Using viability results recently investigated for infinite horizon settings in presence of state constraints, we extend some known results for the linear-quadratic regulator problem to control problems with nonlinear dynamics in the state and affine in the control. Feedback laws are obtained under suitable controllability assumptions.
△ Less
Submitted 1 March, 2020; v1 submitted 18 December, 2019;
originally announced December 2019.
-
Verifying fundamental solution groups for lossless wave equations via stationary action and optimal control
Authors:
Peter M. Dower,
William M. McEneaney
Abstract:
A representation of a fundamental solution group for a class of wave equations is constructed by exploiting connections between stationary action and optimal control. By using a Yosida approximation of the associated generator, an approximation of the group of interest is represented for sufficiently short time horizons via an idempotent convolution kernel that describes all possible solutions of…
▽ More
A representation of a fundamental solution group for a class of wave equations is constructed by exploiting connections between stationary action and optimal control. By using a Yosida approximation of the associated generator, an approximation of the group of interest is represented for sufficiently short time horizons via an idempotent convolution kernel that describes all possible solutions of a corresponding short time horizon optimal control problem. It is shown that this representation of the approximate group can be extended to arbitrary longer horizons via a concatenation of such short horizon optimal control problems, provided that the associated initial and terminal conditions employed in concatenating trajectories are determined via a stationarity rather than an optimality based condition. The long horizon approximate group obtained is shown to converge strongly to the exact group of interest, under reasonable conditions. The construction is illustrated by its application to the solution of a two point boundary value problem.
△ Less
Submitted 9 June, 2019;
originally announced June 2019.
-
Synthesis of control Lyapunov functions and stabilizing feedback strategies using exit-time optimal control
Authors:
Ivan Yegorov,
Peter M. Dower,
Lars Grüne
Abstract:
This paper studies the problem of constructing control Lyapunov functions (CLFs) and feedback stabilization strategies for deterministic nonlinear control systems described by ordinary differential equations. Many numerical methods for solving the Hamilton-Jacobi-Bellman partial differential equations specifying CLFs typically require dense state space discretizations and consequently suffer from…
▽ More
This paper studies the problem of constructing control Lyapunov functions (CLFs) and feedback stabilization strategies for deterministic nonlinear control systems described by ordinary differential equations. Many numerical methods for solving the Hamilton-Jacobi-Bellman partial differential equations specifying CLFs typically require dense state space discretizations and consequently suffer from the curse of dimensionality. A relevant direction of attenuating the curse of dimensionality concerns reducing the computation of the values of CLFs and associated feedbacks at any selected states to finite-dimensional nonlinear programming problems. In this work, exit-time optimal control is used for that purpose. First, we state an exit-time optimal control problem with respect to a sublevel set of an appropriate local CLF and establish that, under a number of reasonable conditions, the concatenation of the corresponding value function and the local CLF is a global CLF in the whole domain of asymptotic null-controllability. This leads to a curse-of-dimensionality-free approach to feedback stabilization. We also investigate the formulated optimal control problem. A modification of these constructions for the case when one does not find a suitable local CLF is provided as well. Supporting numerical simulation results that illustrate our development are subsequently presented and discussed. Furthermore, it is pointed out that the curse of complexity may cause significant issues in practical implementation even if the curse of dimensionality is mitigated.
△ Less
Submitted 6 June, 2019;
originally announced June 2019.
-
Game representations for state constrained continuous time linear regulator problems
Authors:
Peter M. Dower,
William M. McEneaney,
Michael Cantoni
Abstract:
A supremum-of-quadratics representation for convex barrier-type constraints is developed and applied within the context of a class of continuous time state constrained linear regulator problems. Using this representation, it is shown that a linear regulator problem subjected to such a convex barrier-type constraint can be equivalently formulated as an unconstrained two-player linear quadratic game…
▽ More
A supremum-of-quadratics representation for convex barrier-type constraints is developed and applied within the context of a class of continuous time state constrained linear regulator problems. Using this representation, it is shown that a linear regulator problem subjected to such a convex barrier-type constraint can be equivalently formulated as an unconstrained two-player linear quadratic game. By demonstrating equivalence of the upper and lower values of this game, state feedback characterizations for the optimal policies of both players are developed. These characterizations are subsequently illustrated by example.
△ Less
Submitted 11 April, 2019;
originally announced April 2019.
-
Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton-Jacobi equations
Authors:
Ivan Yegorov,
Peter Dower
Abstract:
This paper extends the considerations of the works [1, 2] regarding curse-of-dimensionality-free numerical approaches to solve certain types of Hamilton-Jacobi equations arising in optimal control problems, differential games and elsewhere. A rigorous formulation and justification for the extended Hopf-Lax formula of [2] is provided together with novel theoretical and practical discussions includi…
▽ More
This paper extends the considerations of the works [1, 2] regarding curse-of-dimensionality-free numerical approaches to solve certain types of Hamilton-Jacobi equations arising in optimal control problems, differential games and elsewhere. A rigorous formulation and justification for the extended Hopf-Lax formula of [2] is provided together with novel theoretical and practical discussions including useful recommendations. By using the method of characteristics, the solutions of some problem classes under convexity/concavity conditions on Hamiltonians (in particular, the solutions of Hamilton-Jacobi-Bellman equations in optimal control problems) are evaluated separately at different initial positions. This allows for the avoidance of the curse of dimensionality, as well as for choosing arbitrary computational regions. The corresponding feedback control strategies are obtained at selected positions without approximating the partial derivatives of the solutions. The results of numerical simulations demonstrate the high potential of the proposed techniques. It is also pointed out that, despite the indicated advantages, the related approaches still have a limited range of applicability, and their extensions to Hamilton-Jacobi-Isaacs equations in zero-sum two-player differential games are currently developed only for sufficiently narrow classes of control systems. That is why further extensions are worth investigating.
△ Less
Submitted 9 November, 2017;
originally announced November 2017.
-
A new fundamental solution for a class of differential Riccati equations
Authors:
Peter M. Dower,
Huan Zhang
Abstract:
A class of differential Riccati equations (DREs) is considered whereby the evolution of any solution can be identified with the propagation of a value function of a corresponding optimal control problem arising in L2-gain analysis. By exploiting the semigroup properties inherited from the attendant dynamic programming principle, a max-plus primal space fundamental solution semigroup of max-plus li…
▽ More
A class of differential Riccati equations (DREs) is considered whereby the evolution of any solution can be identified with the propagation of a value function of a corresponding optimal control problem arising in L2-gain analysis. By exploiting the semigroup properties inherited from the attendant dynamic programming principle, a max-plus primal space fundamental solution semigroup of max-plus linear max-plus integral operators is developed that encapsulates all such value function propagations. Using this semigroup, a new one-parameter fundamental solution semigroup of matrices is developed for the aforementioned class of DREs. It is demonstrated that this new semigroup can be used to compute particular solutions of these DREs, and to characterize finite escape times (should they exist) in a relatively simple way compared with that provided by the standard symplectic fundamental solution semigroup.
△ Less
Submitted 1 June, 2015;
originally announced June 2015.
-
Solving two-point boundary value problems for a wave equation via the principle of stationary action and optimal control
Authors:
Peter M. Dower,
William M. McEneaney
Abstract:
A new approach to solving two-point boundary value problems for a wave equation is developed. This new approach exploits the principle of stationary action to reformulate and solve such problems in the framework of optimal control. In particular, an infinite dimensional optimal control problem is posed so that the wave equation dynamics and temporal boundary data of interest are captured via the c…
▽ More
A new approach to solving two-point boundary value problems for a wave equation is developed. This new approach exploits the principle of stationary action to reformulate and solve such problems in the framework of optimal control. In particular, an infinite dimensional optimal control problem is posed so that the wave equation dynamics and temporal boundary data of interest are captured via the characteristics of the associated Hamiltonian and choice of terminal payoff respectively. In order to solve this optimal control problem for any such terminal payoff, and hence solve any two-point boundary value problem corresponding to the boundary data encapsulated by that terminal payoff, a fundamental solution to the optimal control problem is constructed. Specifically, the optimal control problem corresponding to any given terminal payoff can be solved via a max-plus convolution of this fundamental solution with the specified terminal payoff. Crucially, the fundamental solution is shown to be a quadratic functional that is defined with respect to the unique solution of a set of operator differential equations, and computable using spectral methods. An example is presented in which this fundamental solution is computed and applied to solve a two-point boundary value problem for the wave equation of interest.
△ Less
Submitted 8 January, 2015;
originally announced January 2015.
-
Optimization Methods on Riemannian Manifolds via Extremum Seeking Algorithms
Authors:
Farzin Taringoo,
Peter M. Dower,
Dragan Nesic,
Ying Tan
Abstract:
This paper formulates the problem of Extremum Seeking for optimization of cost functions defined on Riemannian manifolds. We extend the conventional extremum seeking algorithms for optimization problems in Euclidean spaces to optimization of cost functions defined on smooth Riemannian manifolds. This problem falls within the category of online optimization methods. We introduce the notion of geode…
▽ More
This paper formulates the problem of Extremum Seeking for optimization of cost functions defined on Riemannian manifolds. We extend the conventional extremum seeking algorithms for optimization problems in Euclidean spaces to optimization of cost functions defined on smooth Riemannian manifolds. This problem falls within the category of online optimization methods. We introduce the notion of geodesic dithers which is a perturbation of the optimizing trajectory in the tangent bundle of the ambient state manifolds and obtain the extremum seeking closed loop as a perturbation of the averaged gradient system. The main results are obtained by applying closeness of solutions and averaging theory on Riemannian manifolds. The main results are further extended for optimization on Lie groups. Numerical examples on Riemannian manifolds (Lie groups) SO(3) and SE(3) are presented at the end of the paper.
△ Less
Submitted 8 December, 2014;
originally announced December 2014.
-
Input-to-State Stability, integral Input-to-State Stability, and $\mathcal{L}_2$-Gain Properties: Qualitative Equivalences and Interconnected Systems
Authors:
Christopher M. Kellett,
Peter M. Dower
Abstract:
Input-to-state stability (ISS) and $\mathcal{L}_2$-gain are well-known robust stability properties that continue to find wide application in the analysis and control of nonlinear dynamical systems and their interconnections. We investigate the relationship between ISS-type and $\mathcal{L}_2$-gain properties, demonstrating several qualitative equivalences between these two approaches. We subsequen…
▽ More
Input-to-state stability (ISS) and $\mathcal{L}_2$-gain are well-known robust stability properties that continue to find wide application in the analysis and control of nonlinear dynamical systems and their interconnections. We investigate the relationship between ISS-type and $\mathcal{L}_2$-gain properties, demonstrating several qualitative equivalences between these two approaches. We subsequently present several new sufficient conditions for the stability of interconnected systems derived by exploiting these qualitative equivalences.
△ Less
Submitted 1 May, 2014;
originally announced May 2014.
-
Max-plus fundamental solution semigroups for a class of difference Riccati equations
Authors:
Huan Zhang,
Peter M. Dower
Abstract:
Recently, a max-plus dual space fundamental solution semigroup for a class of difference Riccati equation (DRE) has been developed. This fundamental solution semigroup is represented in terms of the kernel of a specific max-plus linear operator that plays the role of the dynamic programming evolution operator in a max-plus dual space. In order to fully understand connections between this dual spac…
▽ More
Recently, a max-plus dual space fundamental solution semigroup for a class of difference Riccati equation (DRE) has been developed. This fundamental solution semigroup is represented in terms of the kernel of a specific max-plus linear operator that plays the role of the dynamic programming evolution operator in a max-plus dual space. In order to fully understand connections between this dual space fundamental solution semigroup and evolution of the value function of the underlying optimal control problem, a new max-plus primal space fundamental solution semigroup for the same class of difference Riccati equations is presented. Connections and commutation results between this new primal space fundamental solution semigroup and the recently developed dual space fundamental solution semigroup are established.
△ Less
Submitted 25 May, 2015; v1 submitted 30 April, 2014;
originally announced April 2014.
-
A max-plus dual space fundamental solution for a class of operator differential Riccati equations
Authors:
Peter M. Dower,
William M. McEneaney
Abstract:
A new fundamental solution semigroup for operator differential Riccati equations is developed. This fundamental solution semigroup is constructed via an auxiliary finite horizon optimal control problem whose value functional growth with respect to time horizon is determined by a particular solution of the operator differential Riccati equation of interest. By exploiting semiconvexity of this value…
▽ More
A new fundamental solution semigroup for operator differential Riccati equations is developed. This fundamental solution semigroup is constructed via an auxiliary finite horizon optimal control problem whose value functional growth with respect to time horizon is determined by a particular solution of the operator differential Riccati equation of interest. By exploiting semiconvexity of this value functional, and the attendant max-plus linearity and semigroup properties of the associated dynamic programming evolution operator, a semigroup of max-plus integral operators is constructed in a dual space defined via the Legendre-Fenchel transform. It is demonstrated that this semigroup of max-plus integral operators can be used to propagate all solutions of the operator differential Riccati equation that are initialized from a specified class of initial conditions. As this semigroup of max-plus integral operators can be identified with a semigroup of quadratic kernels, an explicit recipe for the aforementioned solution propagation is also rendered possible.
△ Less
Submitted 25 May, 2015; v1 submitted 28 April, 2014;
originally announced April 2014.
-
Averaging for nonlinear systems evolving on Riemannian manifolds
Authors:
Farzin Taringoo,
Dragan Nešić,
Ying Tan,
Peter M. Dower
Abstract:
This paper presents an averaging method for nonlinear systems defined on Riemannian manifolds. We extend closeness of solutions results for ordinary differential equations on $R^{n}$ to dynamical systems defined on Riemannian manifolds by employing differential geometry. A generalization of closeness of solutions for periodic dynamical systems on compact time intervals is derived for dynamical sys…
▽ More
This paper presents an averaging method for nonlinear systems defined on Riemannian manifolds. We extend closeness of solutions results for ordinary differential equations on $R^{n}$ to dynamical systems defined on Riemannian manifolds by employing differential geometry. A generalization of closeness of solutions for periodic dynamical systems on compact time intervals is derived for dynamical systems evolving on compact Riemannian manifolds. Under local asymptotic (exponential) stability of the average vector field, we further relax the compactness of the ambient Riemannian manifold and obtain the closeness of solutions on the infinite time interval by employing the notion of uniform normal neighborhoods of an equilibrium point of a vector field. These results are also presented for time-varying dynamical systems where their averaged systems are almost globally asymptotically or exponentially stable on compact manifolds. The main results of the paper are illustrated by several examples.
△ Less
Submitted 29 April, 2014; v1 submitted 11 November, 2013;
originally announced November 2013.
-
A Local Characterization of Lyapunov Functions and Robust Stability of Perturbed Systems on Riemannian Manifolds
Authors:
Farzin Taringoo,
Peter M. Dower,
Dragan Nešić,
Ying Tan
Abstract:
This paper proposes several Converse Lyapunov Theorems for nonlinear dynamical systems defined on smooth connected Riemannian manifolds and characterizes properties of corresponding Lyapunov functions in a normal neighborhood of an equilibrium. We extend the methods of constructing of Lyapunov functions for ordinary differential equations on $\mathds{R}^{n}$ to dynamical systems defined on Riemann…
▽ More
This paper proposes several Converse Lyapunov Theorems for nonlinear dynamical systems defined on smooth connected Riemannian manifolds and characterizes properties of corresponding Lyapunov functions in a normal neighborhood of an equilibrium. We extend the methods of constructing of Lyapunov functions for ordinary differential equations on $\mathds{R}^{n}$ to dynamical systems defined on Riemannian manifolds by employing the differential geometry. By employing the derived properties of Lyapunov functions, we obtained the stability of perturbed dynamical systems on Riemannian manifolds. The results are obtained by employing the notions of normal neighborhoods, the injectivity radius on Riemannian manifolds and existence of bump functions on manifolds.
△ Less
Submitted 24 June, 2014; v1 submitted 31 October, 2013;
originally announced November 2013.
-
A max-plus based fundamental solution for a class of discrete time linear regulator problems
Authors:
Huan Zhang,
Peter M. Dower
Abstract:
Efficient Riccati equation based techniques for the approximate solution of discrete time linear regulator problems are restricted in their application to problems with quadratic terminal payoffs. Where non-quadratic terminal payoffs are required, these techniques fail due to the attendant non-quadratic value functions involved. In order to compute these non-quadratic value functions, it is often…
▽ More
Efficient Riccati equation based techniques for the approximate solution of discrete time linear regulator problems are restricted in their application to problems with quadratic terminal payoffs. Where non-quadratic terminal payoffs are required, these techniques fail due to the attendant non-quadratic value functions involved. In order to compute these non-quadratic value functions, it is often necessary to appeal directly to dynamic programming in the form of grid- or element-based iterations for the value function. These iterations suffer from poor scalability with respect to problem dimension and time horizon. In this paper, a new max-plus based method is developed for the approximate solution of discrete time linear regulator problems with non-quadratic payoffs. This new method is underpinned by the development of new fundamental solutions to such linear regulator problems, via max-plus duality. In comparison with a typical grid-based approach, a substantial reduction in computational effort is observed in applying this new max-plus method. A number of simple examples are presented that illustrate this and other observations.
△ Less
Submitted 30 April, 2014; v1 submitted 21 June, 2013;
originally announced June 2013.
