Showing 1–50 of 74 results for author: Goulart, P

Synthesis of safety certificates for discrete-time uncertain systems via convex optimization

Authors: Marta Fochesato, Han Wang, Antonis Papachristodoulou, Paul Goulart

Abstract: We study the problem of co-designing control barrier functions and linear state feedback controllers for discrete-time linear systems affected by additive disturbances. For disturbances of bounded magnitude, we provide a semi-definite program whose feasibility implies the existence of a control law and a certificate ensuring safety in the infinite horizon with respect to the worst-case disturbance… ▽ More We study the problem of co-designing control barrier functions and linear state feedback controllers for discrete-time linear systems affected by additive disturbances. For disturbances of bounded magnitude, we provide a semi-definite program whose feasibility implies the existence of a control law and a certificate ensuring safety in the infinite horizon with respect to the worst-case disturbance realization in the uncertainty set. For disturbances with unbounded support, we rely on martingale theory to derive a second semi-definite program whose feasibility provides probabilistic safety guarantees holding joint-in-time over a finite time horizon. We examine several extensions, including (i) encoding of different types of input constraints, (ii) robustification against distributional ambiguity around the true distribution, (iii) design of safety filters, and (iv) extension to general safety specifications such as obstacle avoidance. △ Less

Submitted 13 May, 2025; originally announced May 2025.

Predictive Control Barrier Functions: Bridging model predictive control and control barrier functions

Authors: Jingyi Huang, Han Wang, Kostas Margellos, Paul Goulart

Abstract: In this paper, we establish a connection between model predictive control (MPC) techniques and Control Barrier Functions (CBFs). Recognizing the similarity between CBFs and Control Lyapunov Functions (CLFs), we propose a safe MPC formulation that ensures invariance and safety without relying on explicit stability conditions. The value function of our proposed safe MPC is a CBF, which we refer to a… ▽ More In this paper, we establish a connection between model predictive control (MPC) techniques and Control Barrier Functions (CBFs). Recognizing the similarity between CBFs and Control Lyapunov Functions (CLFs), we propose a safe MPC formulation that ensures invariance and safety without relying on explicit stability conditions. The value function of our proposed safe MPC is a CBF, which we refer to as the Predictive Control Barrier Function (PCBF), similar to traditional MPC formulations which encode stability by having value functions as CLFs. Our formulation is simpler than previous PCBF approaches and is based on weaker assumptions while proving a similar theorem that guarantees safety recovery. Notably, our safe MPC formulation does not require the value function to be strictly decreasing to ensure convergence to a safe invariant set. Numerical examples demonstrate the effectiveness of our approach in guaranteeing safety and constructing non-conservative CBFs. △ Less

Submitted 12 February, 2025; originally announced February 2025.

CuClarabel: GPU Acceleration for a Conic Optimization Solver

Authors: Yuwen Chen, Danny Tse, Parth Nobel, Paul Goulart, Stephen Boyd

Abstract: We present the GPU implementation of the general-purpose interior-point solver Clarabel for convex optimization problems with conic constraints. We introduce a mixed parallel computing strategy that processes linear constraints first, then handles other conic constraints in parallel. This mixed parallel computing strategy currently supports linear, second-order cone, exponential cone, and power co… ▽ More We present the GPU implementation of the general-purpose interior-point solver Clarabel for convex optimization problems with conic constraints. We introduce a mixed parallel computing strategy that processes linear constraints first, then handles other conic constraints in parallel. This mixed parallel computing strategy currently supports linear, second-order cone, exponential cone, and power cone constraints. We demonstrate that integrating a mixed parallel computing strategy with GPU-based direct linear system solvers enhances the performance of GPU-based conic solvers, surpassing their CPU-based counterparts across a wide range of conic optimization problems. We also show that employing mixed-precision linear system solvers can potentially achieve additional acceleration without compromising solution accuracy. △ Less

Submitted 30 December, 2024; v1 submitted 25 December, 2024; originally announced December 2024.

Can a Large Language Model Learn Matrix Functions In Context?

Authors: Paimon Goulart, Evangelos E. Papalexakis

Abstract: Large Language Models (LLMs) have demonstrated the ability to solve complex tasks through In-Context Learning (ICL), where models learn from a few input-output pairs without explicit fine-tuning. In this paper, we explore the capacity of LLMs to solve non-linear numerical computations, with specific emphasis on functions of the Singular Value Decomposition. Our experiments show that while LLMs per… ▽ More Large Language Models (LLMs) have demonstrated the ability to solve complex tasks through In-Context Learning (ICL), where models learn from a few input-output pairs without explicit fine-tuning. In this paper, we explore the capacity of LLMs to solve non-linear numerical computations, with specific emphasis on functions of the Singular Value Decomposition. Our experiments show that while LLMs perform comparably to traditional models such as Stochastic Gradient Descent (SGD) based Linear Regression and Neural Networks (NN) for simpler tasks, they outperform these models on more complex tasks, particularly in the case of top-k Singular Values. Furthermore, LLMs demonstrate strong scalability, maintaining high accuracy even as the matrix size increases. Additionally, we found that LLMs can achieve high accuracy with minimal prior examples, converging quickly and avoiding the overfitting seen in classical models. These results suggest that LLMs could provide an efficient alternative to classical methods for solving high-dimensional problems. Future work will focus on extending these findings to larger matrices and more complex matrix operations while exploring the effect of using different numerical representations in ICL. △ Less

Submitted 23 November, 2024; originally announced November 2024.

Clarabel: An interior-point solver for conic programs with quadratic objectives

Authors: Paul J. Goulart, Yuwen Chen

Abstract: We present a general-purpose interior-point solver for convex optimization problems with conic constraints. Our method is based on a homogeneous embedding method originally developed for general monotone complementarity problems and more recently applied to operator splitting methods, and here specialized to an interior-point method for problems with quadratic objectives. We allow for a variety of… ▽ More We present a general-purpose interior-point solver for convex optimization problems with conic constraints. Our method is based on a homogeneous embedding method originally developed for general monotone complementarity problems and more recently applied to operator splitting methods, and here specialized to an interior-point method for problems with quadratic objectives. We allow for a variety of standard symmetric and non-symmetric cones, and provide support for chordal decomposition methods in the case of semidefinite cones. We describe the implementation of this method in the open-source solver Clarabel, and provide a detailed numerical evaluation of its performance versus several state-of-the-art solvers on a wide range of standard benchmarks problems. Clarabel is faster and more robust than competing commercial and open-source solvers across a range of test sets, with a particularly large performance advantage for problems with quadratic objectives. Clarabel is currently distributed as a standard solver for the Python CVXPY optimization suite. △ Less

Submitted 21 May, 2024; originally announced May 2024.

A neural network-based approach to hybrid systems identification for control

Authors: Filippo Fabiani, Bartolomeo Stellato, Daniele Masti, Paul J. Goulart

Abstract: We consider the problem of designing a machine learning-based model of an unknown dynamical system from a finite number of (state-input)-successor state data points, such that the model obtained is also suitable for optimal control design. We adopt a neural network (NN) architecture that, once suitably trained, yields a hybrid system with continuous piecewise-affine (PWA) dynamics that is differen… ▽ More We consider the problem of designing a machine learning-based model of an unknown dynamical system from a finite number of (state-input)-successor state data points, such that the model obtained is also suitable for optimal control design. We adopt a neural network (NN) architecture that, once suitably trained, yields a hybrid system with continuous piecewise-affine (PWA) dynamics that is differentiable with respect to the network's parameters, thereby enabling the use of derivative-based training procedures. We show that a careful choice of our NN's weights produces a hybrid system model with structural properties that are highly favorable when used as part of a finite horizon optimal control problem (OCP). Specifically, we rely on available results to establish that optimal solutions with strong local optimality guarantees can be computed via nonlinear programming (NLP), in contrast to classical OCPs for general hybrid systems which typically require mixed-integer optimization. Besides being well-suited for optimal control design, numerical simulations illustrate that our NN-based technique enjoys very similar performance to state-of-the-art system identification methods for hybrid systems and it is competitive on nonlinear benchmarks. △ Less

Submitted 9 October, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Control of Cross-Directional Systems using the Generalised Singular Value Decomposition

Authors: Idris Kempf, Paul J. Goulart, Stephen Duncan

Abstract: Diamond Light Source produces synchrotron radiation by accelerating electrons to relativistic speeds. In order to maximise the intensity of the radiation, vibrations of the electron beam are attenuated by a multi-input multi-output (MIMO) control system actuating hundreds of magnets at kilohertz rates. For future accelerator configurations, in which two separate arrays of magnets with different ba… ▽ More Diamond Light Source produces synchrotron radiation by accelerating electrons to relativistic speeds. In order to maximise the intensity of the radiation, vibrations of the electron beam are attenuated by a multi-input multi-output (MIMO) control system actuating hundreds of magnets at kilohertz rates. For future accelerator configurations, in which two separate arrays of magnets with different bandwidths are used in combination, standard accelerator control design methods based on the singular value decomposition (SVD) of the system gain matrix are not suitable. We therefore propose to use the generalised singular value decomposition (GSVD) to decouple a two-array cross-directional (CD) system into sets of two-input single-output (TISO) and single-input single-output (SISO) systems. We demonstrate that the two-array decomposition is linked to a single-array system, which is used to accommodate ill-conditioned systems and compensate for the non-orthogonality of the GSVD. The GSVD-based design is implemented and validated through real-world experiments at Diamond. Our approach provides a natural extension of single-array methods and has potential application in other CD systems, including paper making, steel rolling or battery manufacturing processes. △ Less

Submitted 16 August, 2023; originally announced August 2023.

MSC Class: 93B11; 93B52

Control of Cross-Directional Systems with Approximate Symmetries

Authors: Idris Kempf, Paul Goulart, Stephen Duncan

Abstract: Structural symmetries of linear dynamical systems can be exploited for decoupling the dynamics and reducing the computational complexity of the controller implementation. However, in practical applications, inexact structural symmetries undermine the ability to decouple the system, resulting in the loss of any potential complexity reduction. To address this, we propose substituting an approximatio… ▽ More Structural symmetries of linear dynamical systems can be exploited for decoupling the dynamics and reducing the computational complexity of the controller implementation. However, in practical applications, inexact structural symmetries undermine the ability to decouple the system, resulting in the loss of any potential complexity reduction. To address this, we propose substituting an approximation with exact structural symmetries for the original system model, thereby introducing an approximation error. We focus on internal model controllers for cross-directional systems encountered in large-scale and high-speed control problems of synchrotrons or the process industry and characterise the stability, performance, and robustness properties of the resulting closed loop. While existing approaches replace the original system model with one that minimises the Frobenius norm of the approximation error, we show that this can lead to instability or poor performance. Instead, we propose approximations that are obtained from semidefinite programming problems. We show that our proposed approximations can yield stable systems even when the Frobenius norm approximation does not. The paper concludes with numerical examples and a case study of a synchrotron light source with inexact structural symmetries. Exploiting structural symmetries in large-scale and high-speed systems enables faster sampling times and the use of more advanced control techniques, even when the symmetries are approximate. △ Less

Submitted 30 June, 2023; originally announced June 2023.

MSC Class: 93B11; 93B51; 93B52 ACM Class: C.3; C.4; J.2; J.7

An Efficient IPM Implementation for A Class of Nonsymmetric Cones

Authors: Yuwen Chen, Paul Goulart

Abstract: We present an efficient implementation of interior point methods for a family of nonsymmetric cones, including generalized power cones, power mean cones and relative entropy cones, by exploiting underlying low-rank and sparse properties of Hessians of homogeneous self-concordant barrier functions. We prove that the augmented linear system in our interior point method is sparse and quasi-definite,… ▽ More We present an efficient implementation of interior point methods for a family of nonsymmetric cones, including generalized power cones, power mean cones and relative entropy cones, by exploiting underlying low-rank and sparse properties of Hessians of homogeneous self-concordant barrier functions. We prove that the augmented linear system in our interior point method is sparse and quasi-definite, enabling the use of sparse LDL factorization with a dual scaling strategy for nonsymmetric cones. Numerical results show that our proposed implementation for nonsymmetric cones performs much faster than the state-of-art solvers for spare problems and scales well for large problems. △ Less

Submitted 20 May, 2023; originally announced May 2023.

A Unified Early Termination Technique for Primal-dual Algorithms in Mixed Integer Conic Programming

Authors: Yuwen Chen, Catherine Ning, Paul Goulart

Abstract: We propose an early termination technique for mixed integer conic programming for use within branch-and-bound based solvers. Our approach generalizes previous early termination results for ADMM-based solvers to a broader class of primal-dual algorithms, including both operator splitting methods and interior point methods. The complexity for checking early termination is $O(n)$ for each termination… ▽ More We propose an early termination technique for mixed integer conic programming for use within branch-and-bound based solvers. Our approach generalizes previous early termination results for ADMM-based solvers to a broader class of primal-dual algorithms, including both operator splitting methods and interior point methods. The complexity for checking early termination is $O(n)$ for each termination check assuming a bounded problem domain. We show that this domain restriction can be relaxed for problems whose data satisfies a simple rank condition, in which case each check requires an $O(n^2)$ solve using a linear system that must be factored only once at the root node. We further show how this approach can be used in hybrid model predictive control as long as system inputs are bounded. Numerical results show that our method leads to a moderate reduction in the total iterations required for branch-and-bound conic solvers with interior-point based subsolvers. △ Less

Submitted 16 March, 2023; originally announced March 2023.

Burer-Monteiro ADMM for Large-scale SDPs

Authors: Yuwen Chen, Paul Goulart

Abstract: We propose a bilinear decomposition for the Burer-Monteiro method and combine it with the standard Alternating Direction Method of Multipliers algorithm for semidefinite programming. Bilinear decomposition reduces the degree of the augmented Lagrangian from four to two, which makes each of the subproblems a quadratic programming and hence computationally efficient. Our approach is able to solve a… ▽ More We propose a bilinear decomposition for the Burer-Monteiro method and combine it with the standard Alternating Direction Method of Multipliers algorithm for semidefinite programming. Bilinear decomposition reduces the degree of the augmented Lagrangian from four to two, which makes each of the subproblems a quadratic programming and hence computationally efficient. Our approach is able to solve a class of large-scale SDPs with diagonal constraints. We prove that our ADMM algorithm converges globally to a first-order stationary point, and show by exploiting the negative curvature that the algorithm converges to a point within $O(1-1/r)$ of the optimal objective value. Additionally, the proximal variant of the algorithm can solve block-diagonally constrained SDPs with global convergence to a first-order stationary point. Numerical results show that both our ADMM algorithm and the proximal variant outperform the state-of-art Riemannian manifold algorithms and can reach the global optimum empirically. △ Less

Submitted 8 February, 2023; originally announced February 2023.

The Past Does Matter: Correlation of Subsequent States in Trajectory Predictions of Gaussian Process Models

Authors: Steffen Ridderbusch, Sina Ober-Blöbaum, Paul Goulart

Abstract: Computing the distribution of trajectories from a Gaussian Process model of a dynamical system is an important challenge in utilizing such models. Motivated by the computational cost of sampling-based approaches, we consider approximations of the model's output and trajectory distribution. We show that previous work on uncertainty propagation, focussed on discrete state-space models, incorrectly i… ▽ More Computing the distribution of trajectories from a Gaussian Process model of a dynamical system is an important challenge in utilizing such models. Motivated by the computational cost of sampling-based approaches, we consider approximations of the model's output and trajectory distribution. We show that previous work on uncertainty propagation, focussed on discrete state-space models, incorrectly included an independence assumption between subsequent states of the predicted trajectories. Expanding these ideas to continuous ordinary differential equation models, we illustrate the implications of this assumption and propose a novel piecewise linear approximation of Gaussian Processes to mitigate them. △ Less

Submitted 13 May, 2023; v1 submitted 20 November, 2022; originally announced November 2022.

Comments: 9 Pages, 4 Figures, Accepted at UAI2023, not camera-ready

Safeguarded Anderson acceleration for parametric nonexpansive operators

Authors: Michael Garstka, Mark Cannon, Paul Goulart

Abstract: This paper describes the design of a safeguarding scheme for Anderson acceleration to improve its practical performance and stability when used for first-order optimisation methods. We show how the combination of a non-expansiveness condition, conditioning constraints, and memory restarts integrate well with solver algorithms that can be represented as fixed point operators with dynamically varyin… ▽ More This paper describes the design of a safeguarding scheme for Anderson acceleration to improve its practical performance and stability when used for first-order optimisation methods. We show how the combination of a non-expansiveness condition, conditioning constraints, and memory restarts integrate well with solver algorithms that can be represented as fixed point operators with dynamically varying parameters. The performance of the scheme is demonstrated on seven different QP and SDP problem types, including more than 500 problems. The safeguarded Anderson acceleration scheme proposed in this paper is implemented in the open-source ADMM-based conic solver COSMO. △ Less

Submitted 4 August, 2022; originally announced August 2022.

Robust stabilization of polytopic systems via fast and reliable neural network-based approximations

Authors: Filippo Fabiani, Paul J. Goulart

Abstract: We consider the design of fast and reliable neural network (NN)-based approximations of traditional stabilizing controllers for linear systems with polytopic uncertainty, including control laws with variable structure and those based on a (minimal) selection policy. Building upon recent approaches for the design of reliable control surrogates with guaranteed structural properties, we develop a sys… ▽ More We consider the design of fast and reliable neural network (NN)-based approximations of traditional stabilizing controllers for linear systems with polytopic uncertainty, including control laws with variable structure and those based on a (minimal) selection policy. Building upon recent approaches for the design of reliable control surrogates with guaranteed structural properties, we develop a systematic procedure to certify the closed-loop stability and performance of a linear uncertain system when a trained rectified linear unit (ReLU)-based approximation replaces such traditional controllers. First, we provide a sufficient condition, which involves the worst-case approximation error between ReLU-based and traditional controller-based state-to-input mappings, ensuring that the system is ultimately bounded within a set with adjustable size and convergence rate. Then, we develop an offline, mixed-integer optimization-based method that allows us to compute that quantity exactly. △ Less

Submitted 23 January, 2024; v1 submitted 27 April, 2022; originally announced April 2022.

Personalized incentives as feedback design in generalized Nash equilibrium problems

Authors: Filippo Fabiani, Andrea Simonetto, Paul J. Goulart

Abstract: We investigate both stationary and time-varying, nonmonotone generalized Nash equilibrium problems that exhibit symmetric interactions among the agents, which are known to be potential. As may happen in practical cases, however, we envision a scenario in which the formal expression of the underlying potential function is not available, and we design a semi-decentralized Nash equilibrium seeking al… ▽ More We investigate both stationary and time-varying, nonmonotone generalized Nash equilibrium problems that exhibit symmetric interactions among the agents, which are known to be potential. As may happen in practical cases, however, we envision a scenario in which the formal expression of the underlying potential function is not available, and we design a semi-decentralized Nash equilibrium seeking algorithm. In the proposed two-layer scheme, a coordinator iteratively integrates the (possibly noisy and sporadic) agents' feedback to learn the pseudo-gradients of the agents, and then design personalized incentives for them. On their side, the agents receive those personalized incentives, compute a solution to an extended game, and then return feedback measurements to the coordinator. In the stationary setting, our algorithm returns a Nash equilibrium in case the coordinator is endowed with standard learning policies, while it returns a Nash equilibrium up to a constant, yet adjustable, error in the time-varying case. As a motivating application, we consider the ridehailing service provided by several companies with mobility as a service orchestration, necessary to both handle competition among firms and avoid traffic congestion, which is also adopted to run numerical experiments verifying our results. △ Less

Submitted 22 May, 2023; v1 submitted 24 March, 2022; originally announced March 2022.

Comments: arXiv admin note: text overlap with arXiv:2111.03854

Reliably-stabilizing piecewise-affine neural network controllers

Authors: Filippo Fabiani, Paul J. Goulart

Abstract: A common problem affecting neural network (NN) approximations of model predictive control (MPC) policies is the lack of analytical tools to assess the stability of the closed-loop system under the action of the NN-based controller. We present a general procedure to quantify the performance of such a controller, or to design minimum complexity NNs with rectified linear units (ReLUs) that preserve t… ▽ More A common problem affecting neural network (NN) approximations of model predictive control (MPC) policies is the lack of analytical tools to assess the stability of the closed-loop system under the action of the NN-based controller. We present a general procedure to quantify the performance of such a controller, or to design minimum complexity NNs with rectified linear units (ReLUs) that preserve the desirable properties of a given MPC scheme. By quantifying the approximation error between NN-based and MPC-based state-to-input mappings, we first establish suitable conditions involving two key quantities, the worst-case error and the Lipschitz constant, guaranteeing the stability of the closed-loop system. We then develop an offline, mixed-integer optimization-based method to compute those quantities exactly. Together these techniques provide conditions sufficient to certify the stability and performance of a ReLU-based approximation of an MPC control law. △ Less

Submitted 26 May, 2022; v1 submitted 13 November, 2021; originally announced November 2021.

Learning equilibria with personalized incentives in a class of nonmonotone games

Authors: Filippo Fabiani, Andrea Simonetto, Paul J. Goulart

Abstract: We consider quadratic, nonmonotone generalized Nash equilibrium problems with symmetric interactions among the agents. Albeit this class of games is known to admit a potential function, its formal expression can be unavailable in several real-world applications. For this reason, we propose a two-layer Nash equilibrium seeking scheme in which a central coordinator exploits noisy feedback from the a… ▽ More We consider quadratic, nonmonotone generalized Nash equilibrium problems with symmetric interactions among the agents. Albeit this class of games is known to admit a potential function, its formal expression can be unavailable in several real-world applications. For this reason, we propose a two-layer Nash equilibrium seeking scheme in which a central coordinator exploits noisy feedback from the agents to design personalized incentives for them. By making use of those incentives, the agents compute a solution to an extended game, and then return feedback measures to the coordinator. We show that our algorithm returns an equilibrium if the coordinator is endowed with standard learning policies, and corroborate our results on a numerical instance of a hypomonotone game. △ Less

Submitted 30 March, 2022; v1 submitted 6 November, 2021; originally announced November 2021.

Model Predictive Control for Electron Beam Stabilization in a Synchrotron

Authors: Idris Kempf, Paul J. Goulart, Stephen R. Duncan

Abstract: Electron beam stabilization in a synchrotron is a disturbance rejection problem, with hundreds of inputs and outputs, that is sampled at frequencies higher than $10$ kHz. In this feasibility study, we focus on the practical issues of an efficient implementation of model predictive control (MPC) for the heavily ill-conditioned plant of the electron beam stabilization problem. To obtain a tractable… ▽ More Electron beam stabilization in a synchrotron is a disturbance rejection problem, with hundreds of inputs and outputs, that is sampled at frequencies higher than $10$ kHz. In this feasibility study, we focus on the practical issues of an efficient implementation of model predictive control (MPC) for the heavily ill-conditioned plant of the electron beam stabilization problem. To obtain a tractable control problem that can be solved using only a few iterations of the fast gradient method, we investigate different methods for preconditioning the resulting optimization problem and relate our findings to standard regularization techniques from cross-directional control. We summarize the single- and multi-core implementations of our control algorithm on a digital signal processor (DSP), and show that MPC can be executed at the rate required for synchrotron control. MPC overcomes various problems of standard electron beam stabilization techniques, and the successful implementation can increase the stability of photon beams in synchrotron light sources. △ Less

Submitted 4 July, 2021; originally announced July 2021.

Comments: 8 pages, 5 figures

ACM Class: C.3; C.4; J.2; J.7

Probabilistic stabilizability certificates for a class of black-box linear systems

Authors: Filippo Fabiani, Kostas Margellos, Paul J. Goulart

Abstract: We provide out-of-sample certificates on the controlled invariance property of a given set with respect to a class of black-box linear systems. Specifically, we consider linear time-invariant models whose state space matrices are known only to belong to a certain family due to a possibly inexact quantification of some parameters. By exploiting a set of realizations of those undetermined parameters… ▽ More We provide out-of-sample certificates on the controlled invariance property of a given set with respect to a class of black-box linear systems. Specifically, we consider linear time-invariant models whose state space matrices are known only to belong to a certain family due to a possibly inexact quantification of some parameters. By exploiting a set of realizations of those undetermined parameters, verifying the controlled invariance property of the given set amounts to a linear program, whose feasibility allows us to establish an a-posteriori probabilistic certificate on the controlled invariance property of such a set with respect to the nominal linear time-invariant dynamics. The proposed framework is applied to the control of a networked multi-agent system with unknown weighted graph. △ Less

Submitted 4 March, 2021; originally announced March 2021.

A Higher-Order Generalized Singular Value Decomposition for Rank Deficient Matrices

Authors: Idris Kempf, Paul J. Goulart, Stephen R. Duncan

Abstract: The higher-order generalized singular value decomposition (HO-GSVD) is a matrix factorization technique that extends the GSVD to $N \ge 2$ data matrices, and can be used to identify shared subspaces in multiple large-scale datasets with different row dimensions. The standard HO-GSVD factors $N$ matrices $A_i\in\mathbb{R}^{m_i\times n}$ as $A_i=U_iΣ_i V^\text{T}$, but requires that each of the matr… ▽ More The higher-order generalized singular value decomposition (HO-GSVD) is a matrix factorization technique that extends the GSVD to $N \ge 2$ data matrices, and can be used to identify shared subspaces in multiple large-scale datasets with different row dimensions. The standard HO-GSVD factors $N$ matrices $A_i\in\mathbb{R}^{m_i\times n}$ as $A_i=U_iΣ_i V^\text{T}$, but requires that each of the matrices $A_i$ has full column rank. We propose a modification of the HO-GSVD that extends its applicability to rank-deficient data matrices $A_i$. If the matrix of stacked $A_i$ has full rank, we show that the properties of the original HO-GSVD extend to our approach. We extend the notion of common subspaces to isolated subspaces, which identify features that are unique to one $A_i$. We also extend our results to the higher-order cosine-sine decomposition (HO-CSD), which is closely related to the HO-GSVD. Our extension of the standard HO-GSVD allows its application to datasets with with $m_i<n$ or $\text{rank}(A_i)<n$, such as are encountered in bioinformatics, neuroscience, control theory or classification problems. △ Less

Submitted 20 June, 2022; v1 submitted 19 February, 2021; originally announced February 2021.

Comments: 18 pages, 4 figures

MSC Class: 65F15; 65F55 ACM Class: G.1.3

Learning ODE Models with Qualitative Structure Using Gaussian Processes

Authors: Steffen Ridderbusch, Christian Offen, Sina Ober-Blöbaum, Paul Goulart

Abstract: Recent advances in learning techniques have enabled the modelling of dynamical systems for scientific and engineering applications directly from data. However, in many contexts explicit data collection is expensive and learning algorithms must be data-efficient to be feasible. This suggests using additional qualitative information about the system, which is often available from prior experiments o… ▽ More Recent advances in learning techniques have enabled the modelling of dynamical systems for scientific and engineering applications directly from data. However, in many contexts explicit data collection is expensive and learning algorithms must be data-efficient to be feasible. This suggests using additional qualitative information about the system, which is often available from prior experiments or domain knowledge. We propose an approach to learning a vector field of differential equations using sparse Gaussian Processes that allows us to combine data and additional structural information, like Lie Group symmetries and fixed points. We show that this combination improves extrapolation performance and long-term behaviour significantly, while also reducing the computational cost. △ Less

Submitted 27 March, 2021; v1 submitted 10 November, 2020; originally announced November 2020.

Journal ref: 2021 60th IEEE Conference on Decision and Control (CDC), 2021, pp. 2896-2896

Stochastic output feedback MPC with intermittent observations

Authors: Shuhao Yan, Mark Cannon, Paul J. Goulart

Abstract: This paper designs a model predictive control (MPC) law for constrained linear systems with stochastic additive disturbances and noisy measurements, minimising a discounted cost subject to a discounted expectation constraint. It is assumed that sensor data is lost with a known probability. Taking into account the data losses modelled by a Bernoulli process, we parameterise the predicted control po… ▽ More This paper designs a model predictive control (MPC) law for constrained linear systems with stochastic additive disturbances and noisy measurements, minimising a discounted cost subject to a discounted expectation constraint. It is assumed that sensor data is lost with a known probability. Taking into account the data losses modelled by a Bernoulli process, we parameterise the predicted control policy as an affine function of future observations and obtain a convex linear-quadratic optimal control problem. Constraint satisfaction and a discounted cost bound are ensured without imposing bounds on the distributions of the disturbance and noise inputs. In addition, the average long-run undiscounted closed loop cost is shown to be finite if the discount factor takes appropriate values. We analyse robustness of the proposed control law with respect to possible uncertainties in the arrival probability of sensor data and we bound the impact of these uncertainties on constraint satisfaction and the discounted cost. Numerical simulations are provided to illustrate these results. △ Less

Submitted 1 March, 2022; v1 submitted 21 September, 2020; originally announced September 2020.

Comments: 13 pages. arXiv admin note: text overlap with arXiv:2004.02591

Multi-Array Electron Beam Stabilization using Block-Circulant Transformation and Generalized Singular Value Decomposition

Authors: Idris Kempf, Stephen R. Duncan, Paul J. Goulart, Guenther Rehm

Abstract: We introduce a novel structured controller design for the electron beam stabilization problem of the UK's national synchrotron light source. Because changes to the synchrotron will not allow the application of existing control approaches, we develop a novel method to diagonalize the multi-input multi-output (MIMO) system. A generalized singular value decomposition (GSVD) is used to simultaneously… ▽ More We introduce a novel structured controller design for the electron beam stabilization problem of the UK's national synchrotron light source. Because changes to the synchrotron will not allow the application of existing control approaches, we develop a novel method to diagonalize the multi-input multi-output (MIMO) system. A generalized singular value decomposition (GSVD) is used to simultaneously diagonalize the actuator response matrices, which is applicable to an arbitrary number of actuator dynamics in a cross-directional setting. The resulting decoupled systems are regulated using mid-ranged control and the controller gains derived as a function of the generalized singular values. In addition, we exploit the inherent block-circulant symmetry of the system. The performance of our controller is demonstrated using simulations that involve machine data. △ Less

Submitted 1 September, 2020; originally announced September 2020.

Symmetry Exploitation in Orbit Feedback Systems of Synchrotron Storage Rings

Authors: Idris Kempf, Paul J. Goulart, Stephen R. Duncan, Guenther Rehm

Abstract: Structural symmetries in the storage ring of synchrotrons are intentionally created during the design phase of the magnetic lattices, but they are not considered in the design of control algorithms that stabilize the beam of accelerated particles. The choice of control algorithm, however, is limited by the speed requirements of the synchrotron. Standard control algorithms for synchrotrons are base… ▽ More Structural symmetries in the storage ring of synchrotrons are intentionally created during the design phase of the magnetic lattices, but they are not considered in the design of control algorithms that stabilize the beam of accelerated particles. The choice of control algorithm, however, is limited by the speed requirements of the synchrotron. Standard control algorithms for synchrotrons are based on a singular value decomposition (SVD) of the orbit response matrix. SVD controllers neither exploit the structural symmetries nor exhibit any speed advantages. Based on the periodicity and the reflection properties of the betatron function, we show that these structural symmetries are inherited by the orbit response matrix. We show that the resulting block-circulant and centrosymmetric properties of the matrix can be used for different computationally efficient decompositions of the controller. We also address the case of broken symmetry due to odd placements of magnets and monitors. Our efficient decomposition could enable the use of more advanced control techniques for synchrotrons, such as control algorithms that require real-time optimization. These advanced control techniques could in turn increase the quality of research in synchrotron light sources. △ Less

Submitted 31 August, 2020; originally announced August 2020.

Approximate Dynamic Programming for Delivery Time Slot Pricing: a Sensitivity Analysis

Authors: Denis Lebedev, Kostas Margellos, Paul Goulart

Abstract: We consider the revenue management problem of finding profit-maximising prices for delivery time slots in the context of attended home delivery. This multi-stage optimal control problem admits a dynamic programming formulation that is intractable for realistic problem sizes due to the so-called "curse of dimensionality". Therefore, we study three approximate dynamic programming algorithms both fro… ▽ More We consider the revenue management problem of finding profit-maximising prices for delivery time slots in the context of attended home delivery. This multi-stage optimal control problem admits a dynamic programming formulation that is intractable for realistic problem sizes due to the so-called "curse of dimensionality". Therefore, we study three approximate dynamic programming algorithms both from a control-theoretical perspective and in a parametric numerical case study. Our numerical analysis is based on real-world data, from which we generate multiple scenarios to stress-test the robustness of the pricing policies to errors in model parameter estimates. Our theoretical analysis and numerical benchmark tests show that one of these algorithms, namely gradient-bounded dynamic programming, dominates the others with respect to computation time and profit-generation capabilities of the delivery slot pricing policies that it generates. Finally, we show that uncertainty in the estimates of the model parameters further increases the profit-generation dominance of this approach. △ Less

Submitted 3 August, 2020; originally announced August 2020.

Comments: 13 pages, 7 figures

Stochastic MPC with Dynamic Feedback Gain Selection and Discounted Probabilistic Constraints

Authors: Shuhao Yan, Paul J. Goulart, Mark Cannon

Abstract: This paper considers linear discrete-time systems with additive disturbances, and designs a Model Predictive Control (MPC) law incorporating a dynamic feedback gain to minimise a quadratic cost function subject to a single chance constraint. The feedback gain is selected online and we provide two selection methods based on minimising upper bounds on predicted costs. The chance constraint is define… ▽ More This paper considers linear discrete-time systems with additive disturbances, and designs a Model Predictive Control (MPC) law incorporating a dynamic feedback gain to minimise a quadratic cost function subject to a single chance constraint. The feedback gain is selected online and we provide two selection methods based on minimising upper bounds on predicted costs. The chance constraint is defined as a discounted sum of violation probabilities on an infinite horizon. By penalising violation probabilities close to the initial time and assigning violation probabilities in the far future with vanishingly small weights, this form of constraints allows for an MPC law with guarantees of recursive feasibility without a boundedness assumption on the disturbance. A computationally convenient MPC optimisation problem is formulated using Chebyshev's inequality and we introduce an online constraint-tightening technique to ensure recursive feasibility. The closed loop system is guaranteed to satisfy the chance constraint and a quadratic stability condition. With dynamic feedback gain selection, the closed loop cost is reduced and conservativeness of Chebyshev's inequality is mitigated. Also, a larger feasible set of initial conditions can be obtained. Numerical simulations are given to show these results. △ Less

Submitted 26 May, 2021; v1 submitted 14 July, 2020; originally announced July 2020.

Comments: 15 pages, 3 figures

Gradient-Bounded Dynamic Programming for Submodular and Concave Extensible Value Functions with Probabilistic Performance Guarantees

Authors: Denis Lebedev, Paul Goulart, Kostas Margellos

Abstract: We consider stochastic dynamic programming problems with high-dimensional, discrete state-spaces and finite, discrete-time horizons that prohibit direct computation of the value function from a given Bellman equation for all states and time steps due to the "curse of dimensionality". For the case where the value function of the dynamic program is concave extensible and submodular in its state-spac… ▽ More We consider stochastic dynamic programming problems with high-dimensional, discrete state-spaces and finite, discrete-time horizons that prohibit direct computation of the value function from a given Bellman equation for all states and time steps due to the "curse of dimensionality". For the case where the value function of the dynamic program is concave extensible and submodular in its state-space, we present a new algorithm that computes deterministic upper and stochastic lower bounds of the value function in the realm of dual dynamic programming. We show that the proposed algorithm terminates after a finite number of iterations. Furthermore, we derive probabilistic guarantees on the value accumulated under the associated policy for a single realisation of the dynamic program and for the expectation of this value. Finally, we demonstrate the efficacy of our approach on a high-dimensional numerical example from delivery slot pricing in attended home delivery. △ Less

Submitted 3 June, 2020; originally announced June 2020.

Comments: 12 pages, 4 figures. arXiv admin note: substantial text overlap with arXiv:2005.11213

Gradient-Bounded Dynamic Programming with Submodular and Concave Extensible Value Functions

Authors: Denis Lebedev, Paul Goulart, Kostas Margellos

Abstract: We consider dynamic programming problems with finite, discrete-time horizons and prohibitively high-dimensional, discrete state-spaces for direct computation of the value function from the Bellman equation. For the case that the value function of the dynamic program is concave extensible and submodular in its state-space, we present a new algorithm that computes deterministic upper and stochastic… ▽ More We consider dynamic programming problems with finite, discrete-time horizons and prohibitively high-dimensional, discrete state-spaces for direct computation of the value function from the Bellman equation. For the case that the value function of the dynamic program is concave extensible and submodular in its state-space, we present a new algorithm that computes deterministic upper and stochastic lower bounds of the value function similar to dual dynamic programming. We then show that the proposed algorithm terminates after a finite number of iterations. Finally, we demonstrate the efficacy of our approach on a high-dimensional numerical example from delivery slot pricing in attended home delivery. △ Less

Submitted 22 May, 2020; originally announced May 2020.

Comments: 6 pages, 2 figures, accepted for IFAC World Congress 2020

Probabilistic feasibility guarantees for solution sets to uncertain variational inequalities

Authors: Filippo Fabiani, Kostas Margellos, Paul J. Goulart

Abstract: We develop a data-driven approach to the computation of a-posteriori feasibility certificates to the solution sets of variational inequalities affected by uncertainty. Specifically, we focus on instances of variational inequalities with a deterministic mapping and an uncertain feasibility set, and represent uncertainty by means of scenarios. Building upon recent advances in the scenario approach l… ▽ More We develop a data-driven approach to the computation of a-posteriori feasibility certificates to the solution sets of variational inequalities affected by uncertainty. Specifically, we focus on instances of variational inequalities with a deterministic mapping and an uncertain feasibility set, and represent uncertainty by means of scenarios. Building upon recent advances in the scenario approach literature, we quantify the robustness properties of the entire set of solutions of a variational inequality, with feasibility set constructed using the scenario approach, against a new unseen realization of the uncertainty. Our results extend existing results that typically impose an assumption that the solution set is a singleton and require certain non-degeneracy properties, and thereby offer probabilistic feasibility guarantees to any feasible solution. We show that assessing the violation probability of an entire set of solutions, rather than of a singleton, requires enumeration of the support constraints that "shape" this set. Additionally, we propose a general procedure to enumerate the support constraints that does not require a closed form description of the solution set, which is unlikely to be available. We show that robust game theory problems can be modelling via uncertain variational inequalities, and illustrate our theoretical results through extensive numerical simulations on a case study involving an electric vehicle charging coordination problem. △ Less

Submitted 4 November, 2020; v1 submitted 19 May, 2020; originally announced May 2020.

On the robustness of equilibria in generalized aggregative games

Authors: Filippo Fabiani, Kostas Margellos, Paul J. Goulart

Abstract: We address the problem of assessing the robustness of the equilibria in uncertain, multi-agent games. Specifically, we focus on generalized Nash equilibrium problems in aggregative form subject to linear coupling constraints affected by uncertainty with a possibly unknown probability distribution. Within a data-driven context, we apply the scenario approach paradigm to provide a-posteriori feasibi… ▽ More We address the problem of assessing the robustness of the equilibria in uncertain, multi-agent games. Specifically, we focus on generalized Nash equilibrium problems in aggregative form subject to linear coupling constraints affected by uncertainty with a possibly unknown probability distribution. Within a data-driven context, we apply the scenario approach paradigm to provide a-posteriori feasibility certificates for the entire set of generalized Nash equilibria of the game. Then, we show that assessing the violation probability of such set merely requires to enumerate the constraints that ``shape'' it. For the class of aggregative games, this results in solving a feasibility problem on each active facet of the feasibility region, for which we propose a semi-decentralized algorithm. We demonstrate our theoretical results by means of an academic example. △ Less

Submitted 19 May, 2020; originally announced May 2020.

The optimal transport paradigm enables data compression in data-driven robust control

Authors: Filippo Fabiani, Paul J. Goulart

Abstract: A new data-enabled control technique for uncertain linear time-invariant systems, recently conceived by Coulson et\ al., builds upon the direct optimization of controllers over input/output pairs drawn from a large dataset. We adopt an optimal transport-based method for compressing such large dataset to a smaller synthetic dataset of representative behaviours, aiming to alleviate the computational… ▽ More A new data-enabled control technique for uncertain linear time-invariant systems, recently conceived by Coulson et\ al., builds upon the direct optimization of controllers over input/output pairs drawn from a large dataset. We adopt an optimal transport-based method for compressing such large dataset to a smaller synthetic dataset of representative behaviours, aiming to alleviate the computational burden of controllers to be implemented online. Specifically, the synthetic data are determined by minimizing the Wasserstein distance between atomic distributions supported on both the original dataset and the compressed one. We show that a distributionally robust control law computed using the compressed data enjoys the same type of performance guarantees as the original dataset, at the price of enlarging the ambiguity set by an easily computable and well-behaved quantity. Numerical simulations confirm that the control performance with the synthetic data is comparable to the one obtained with the original data, but with significantly less computation required. △ Less

Submitted 27 September, 2020; v1 submitted 19 May, 2020; originally announced May 2020.

Output feedback stochastic MPC with packet losses

Authors: Shuhao Yan, Mark Cannon, Paul Goulart

Abstract: The paper considers constrained linear systems with stochastic additive disturbances and noisy measurements transmitted over a lossy communication channel. We propose a model predictive control (MPC) law that minimizes a discounted cost subject to a discounted expectation constraint. Sensor data is assumed to be lost with known probability, and data losses are accounted for by expressing the predi… ▽ More The paper considers constrained linear systems with stochastic additive disturbances and noisy measurements transmitted over a lossy communication channel. We propose a model predictive control (MPC) law that minimizes a discounted cost subject to a discounted expectation constraint. Sensor data is assumed to be lost with known probability, and data losses are accounted for by expressing the predicted control policy as an affine function of future observations, which results in a convex optimal control problem. An online constraint-tightening technique ensures recursive feasibility of the online optimization and satisfaction of the expectation constraint without bounds on the distributions of the noise and disturbance inputs. The cost evaluated along trajectories of the closed loop system is shown to be bounded by the optimal predicted cost. A numerical example is given to illustrate these results. △ Less

Submitted 6 May, 2020; v1 submitted 6 April, 2020; originally announced April 2020.

Fast Gradient Method for Model Predictive Control with Input Rate and Amplitude Constraints

Authors: Idris Kempf, Paul Goulart, Stephen Duncan

Abstract: This paper is concerned with the computing efficiency of model predictive control (MPC) problems for dynamical systems with both rate and amplitude constraints on the inputs. Instead of augmenting the decision variables of the underlying finite-horizon optimal control problem to accommodate the input rate constraints, we propose to solve this problem using the fast gradient method (FGM), where the… ▽ More This paper is concerned with the computing efficiency of model predictive control (MPC) problems for dynamical systems with both rate and amplitude constraints on the inputs. Instead of augmenting the decision variables of the underlying finite-horizon optimal control problem to accommodate the input rate constraints, we propose to solve this problem using the fast gradient method (FGM), where the projection step is solved using Dykstra's algorithm. We show that, relative to the Alternating Direction of Method Multipliers (ADMM), this approach greatly reduces the computation time while halving the memory usage. Our algorithm is implemented in C and its performance demonstrated using several examples. △ Less

Submitted 12 March, 2020; originally announced March 2020.

Comments: Initial IFAC 2020 conference submission

Efficient Semidefinite Programming with approximate ADMM

Authors: Nikitas Rontsis, Paul J. Goulart, Yuji Nakatsukasa

Abstract: Tenfold improvements in computation speed can be brought to the alternating direction method of multipliers (ADMM) for Semidefinite Programming with virtually no decrease in robustness and provable convergence simply by projecting approximately to the Semidefinite cone. Instead of computing the projections via "exact" eigendecompositions that scale cubically with the matrix size and cannot be warm… ▽ More Tenfold improvements in computation speed can be brought to the alternating direction method of multipliers (ADMM) for Semidefinite Programming with virtually no decrease in robustness and provable convergence simply by projecting approximately to the Semidefinite cone. Instead of computing the projections via "exact" eigendecompositions that scale cubically with the matrix size and cannot be warm-started, we suggest using state-of-the-art factorization-free, approximate eigensolvers, thus achieving almost quadratic scaling and the crucial ability of warm-starting. Using a recent result from [Goulart et al., 2020], we are able to circumvent the numerical instability of the eigendecomposition and thus maintain tight control on the projection accuracy. This in turn guarantees convergence, either to a solution or a certificate of infeasibility, of the ADMM algorithm. To achieve this, we extend recent results from [Banjac et al., 2019] to prove that reliable infeasibility detection can be performed with ADMM even in the presence of approximation errors. In all of the considered problems of SDPLIB that "exact" ADMM can solve in a few thousand iterations, our approach brings a significant, up to 20x, speedup without a noticeable increase on ADMM's iterations. △ Less

Submitted 24 December, 2021; v1 submitted 5 December, 2019; originally announced December 2019.

Journal ref: Journal of Optimization Theory and Applications (2021): 1-29

A clique graph based merging strategy for decomposable SDPs

Authors: Michael Garstka, Mark Cannon, Paul Goulart

Abstract: Chordal decomposition techniques are used to reduce large structured positive semidefinite matrix constraints in semidefinite programs (SDPs). The resulting equivalent problem contains multiple smaller constraints on the nonzero blocks (or cliques) of the original problem matrices. This usually leads to a significant reduction in the overall solve time. A further reduction is possible by remerging… ▽ More Chordal decomposition techniques are used to reduce large structured positive semidefinite matrix constraints in semidefinite programs (SDPs). The resulting equivalent problem contains multiple smaller constraints on the nonzero blocks (or cliques) of the original problem matrices. This usually leads to a significant reduction in the overall solve time. A further reduction is possible by remerging cliques with significant overlap. The degree of overlap for which this is effective is dependent on the particular solution algorithm and hardware to be employed. We propose a novel clique merging approach that utilizes the clique graph to identify suitable merge candidates. We show its performance by comparing it with two existing methods on selected problems from a benchmark library. Our approach is implemented in the latest version of the conic ADMM-solver COSMO. △ Less

Submitted 13 May, 2020; v1 submitted 13 November, 2019; originally announced November 2019.

Comments: arXiv admin note: substantial text overlap with arXiv:1901.10887

Dynamic Programming for Optimal Delivery Time Slot Pricing

Authors: Denis Lebedev, Paul Goulart, Kostas Margellos

Abstract: We study the dynamic programming approach to revenue management in the context of attended home delivery. We draw on results from dynamic programming theory for Markov decision problems to show that the underlying Bellman operator has a unique fixed point. We then provide a closed-form expression for the resulting fixed point and show that it admits a natural interpretation. Moreover, we also show… ▽ More We study the dynamic programming approach to revenue management in the context of attended home delivery. We draw on results from dynamic programming theory for Markov decision problems to show that the underlying Bellman operator has a unique fixed point. We then provide a closed-form expression for the resulting fixed point and show that it admits a natural interpretation. Moreover, we also show that -- under certain technical assumptions -- the value function, which has a discrete domain and a continuous codomain, admits a continuous extension, which is a finite-valued, concave function of its state variables, at every time step. This result opens the road for achieving scalable implementations of the proposed formulation in future work, as it allows making informed choices of basis functions in an approximate dynamic programming context. We illustrate our findings on a simple numerical example and provide suggestions on how our results can be exploited to obtain closer approximations of the exact value function. △ Less

Submitted 25 October, 2019; originally announced October 2019.

Comments: 30 pages, 2 figures. arXiv admin note: substantial text overlap with arXiv:1903.06647

Optimal Approximation of Doubly Stochastic Matrices

Authors: Nikitas Rontsis, Paul J. Goulart

Abstract: We consider the least-squares approximation of a matrix C in the set of doubly stochastic matrices with the same sparsity pattern as C. Our approach is based on applying the well-known Alternating Direction Method of Multipliers (ADMM) to a reformulation of the original problem. Our resulting algorithm requires an initial Cholesky factorization of a positive definite matrix that has the same spars… ▽ More We consider the least-squares approximation of a matrix C in the set of doubly stochastic matrices with the same sparsity pattern as C. Our approach is based on applying the well-known Alternating Direction Method of Multipliers (ADMM) to a reformulation of the original problem. Our resulting algorithm requires an initial Cholesky factorization of a positive definite matrix that has the same sparsity pattern as C + I followed by simple iterations whose complexity is linear in the number of nonzeros in C, thus ensuring excellent scalability and speed. We demonstrate the advantages of our approach in a series of experiments on problems with up to 82 million nonzeros; these include normalizing large scale matrices arising from the 3D structure of the human genome, clustering applications, and the SuiteSparse matrix library. Overall, our experiments illustrate the outstanding scalability of our algorithm; matrices with millions of nonzeros can be approximated in a few seconds on modest desktop computing hardware. △ Less

Submitted 11 October, 2019; originally announced October 2019.

Accuracy of approximate projection to the semidefinite cone

Authors: Paul J. Goulart, Yuji Nakatsukasa, Nikitas Rontsis

Abstract: When a projection of a symmetric or Hermitian matrix to the positive semidefinite cone is computed approximately (or to working precision on a computer), a natural question is to quantify its accuracy. A straightforward bound invoking standard eigenvalue perturbation theory (e.g. Davis-Kahan and Weyl bounds) suggests that the accuracy would be inversely proportional to the spectral gap, implying i… ▽ More When a projection of a symmetric or Hermitian matrix to the positive semidefinite cone is computed approximately (or to working precision on a computer), a natural question is to quantify its accuracy. A straightforward bound invoking standard eigenvalue perturbation theory (e.g. Davis-Kahan and Weyl bounds) suggests that the accuracy would be inversely proportional to the spectral gap, implying it can be poor in the presence of small eigenvalues. This work shows that a small gap is not a concern for projection onto the semidefinite cone, by deriving error bounds that are gap-independent. △ Less

Submitted 15 August, 2019; v1 submitted 5 August, 2019; originally announced August 2019.

MSC Class: 65F15; 15A45; 15B48; 90C22

An active-set algorithm for norm constrained quadratic problems

Authors: Nikitas Rontsis, Paul J. Goulart, Yuji Nakatsukasa

Abstract: We present an algorithm for the minimization of a nonconvex quadratic function subject to linear inequality constraints and a two-sided bound on the 2-norm of its solution. The algorithm minimizes the objective using an active-set method by solving a series of Trust-Region Subproblems (TRS). Underpinning the efficiency of this approach is that the global solution of the TRS has been widely studied… ▽ More We present an algorithm for the minimization of a nonconvex quadratic function subject to linear inequality constraints and a two-sided bound on the 2-norm of its solution. The algorithm minimizes the objective using an active-set method by solving a series of Trust-Region Subproblems (TRS). Underpinning the efficiency of this approach is that the global solution of the TRS has been widely studied in the literature, resulting in remarkably efficient algorithms and software. We extend these results by proving that nonglobal minimizers of the TRS, or a certificate of their absence, can also be calculated efficiently by computing the two rightmost eigenpairs of an eigenproblem. We demonstrate the usefulness and scalability of the algorithm in a series of experiments that often outperform state-of-the-art approaches; these include calculation of high-quality search directions arising in Sequential Quadratic Programming on problems of the CUTEst collection, and Sparse Principal Component Analysis on a large text corpus problem (70 million nonzeros) that can help organize documents in a user interpretable way. △ Less

Submitted 24 December, 2021; v1 submitted 10 June, 2019; originally announced June 2019.

MSC Class: 90C26; 65F15; 90C90

Journal ref: Mathematical Programming (2021): 1-37. APA Mathematical Programming (2021): 1-37. APA Mathematical Programming (2021): 1-37

S-duality, entropy function and transport in $AdS_4/CMT_3$

Authors: Luis Alejo, Prieslei Goulart, Horatiu Nastase

Abstract: In this paper we consider Abelian vector plus scalar holographic gravity models for 2+1 dimensional condensed matter transport, and the effect of S-duality on them. We find the transport coefficients from the electric and heat currents via usual membrane paradigm-type calculations, and the effect of S-duality on them. We study the same system also by using the entropy function formalism in the ext… ▽ More In this paper we consider Abelian vector plus scalar holographic gravity models for 2+1 dimensional condensed matter transport, and the effect of S-duality on them. We find the transport coefficients from the electric and heat currents via usual membrane paradigm-type calculations, and the effect of S-duality on them. We study the same system also by using the entropy function formalism in the extremal case, and the formalism of holographic Stokes equations, in the case of one-dimensional lattices. We study a few generalizations that appear when considering a supergravity-inspired model, and apply the entropy function method for them. △ Less

Submitted 7 August, 2019; v1 submitted 13 May, 2019; originally announced May 2019.

Comments: 38 page; added clarifications at the end of subsection 4.2

ADMM for Block Circulant Model Predictive Control

Authors: Idris Kempf, Paul J. Goulart, Stephen Duncan

Abstract: This paper deals with model predictive control problems for large scale dynamical systems with cyclic symmetry. Based on the properties of block circulant matrices, we introduce a complex-valued coordinate transformation that block diagonalizes and truncates the original finite-horizon optimal control problem. Using this coordinate transformation, we develop a modified alternating direction method… ▽ More This paper deals with model predictive control problems for large scale dynamical systems with cyclic symmetry. Based on the properties of block circulant matrices, we introduce a complex-valued coordinate transformation that block diagonalizes and truncates the original finite-horizon optimal control problem. Using this coordinate transformation, we develop a modified alternating direction method of multipliers (ADMM) algorithm for general constrained quadratic programs with block circulant blocks. We test our modified algorithm in two different simulated examples and show that our coordinate transformation significantly increases the computation speed. △ Less

Submitted 8 April, 2019; originally announced April 2019.

A Concave Value Function Extension for the Dynamic Programming Approach to Revenue Management in Attended Home Delivery

Authors: Denis Lebedev, Paul Goulart, Kostas Margellos

Abstract: We study the approximate dynamic programming approach to revenue management in the context of attended home delivery. We draw on results from dynamic programming theory for Markov decision problems, convex optimisation and discrete convex analysis to show that the underlying dynamic programming operator has a unique fixed point. Moreover, we also show that -- under certain assumptions -- for all t… ▽ More We study the approximate dynamic programming approach to revenue management in the context of attended home delivery. We draw on results from dynamic programming theory for Markov decision problems, convex optimisation and discrete convex analysis to show that the underlying dynamic programming operator has a unique fixed point. Moreover, we also show that -- under certain assumptions -- for all time steps in the dynamic program, the value function admits a continuous extension, which is a finite-valued, concave function of its state variables. This result opens the road for achieving scalable implementations of the proposed formulation, as it allows making informed choices of basis functions in an approximate dynamic programming context. We illustrate our findings using a simple numerical example and conclude with suggestions on how our results can be exploited in future work to obtain closer approximations of the value function. △ Less

Submitted 15 March, 2019; originally announced March 2019.

Comments: 16 pages, 3 figures, accepted for ECC 2019

COSMO: A conic operator splitting method for convex conic problems

Authors: Michael Garstka, Mark Cannon, Paul Goulart

Abstract: This paper describes the Conic Operator Splitting Method (COSMO) solver, an operator splitting algorithm for convex optimisation problems with quadratic objective function and conic constraints. At each step the algorithm alternates between solving a quasi-definite linear system with a constant coefficient matrix and a projection onto convex sets. The low per-iteration computational cost makes the… ▽ More This paper describes the Conic Operator Splitting Method (COSMO) solver, an operator splitting algorithm for convex optimisation problems with quadratic objective function and conic constraints. At each step the algorithm alternates between solving a quasi-definite linear system with a constant coefficient matrix and a projection onto convex sets. The low per-iteration computational cost makes the method particularly efficient for large problems, e.g. semidefinite programs that arise in portfolio optimisation, graph theory, and robust control. Moreover, the solver uses chordal decomposition techniques and a new clique merging algorithm to effectively exploit sparsity in large, structured semidefinite programs. A number of benchmarks against other state-of-the-art solvers for a variety of problems show the effectiveness of our approach. Our Julia implementation is open-source, designed to be extended and customised by the user, and is integrated into the Julia optimisation ecosystem. △ Less

Submitted 9 September, 2020; v1 submitted 30 January, 2019; originally announced January 2019.

Comments: 45 pages, 11 figures

Decentralized Resource Allocation via Dual Consensus ADMM

Authors: Goran Banjac, Felix Rey, Paul Goulart, John Lygeros

Abstract: We consider a resource allocation problem over an undirected network of agents, where edges of the network define communication links. The goal is to minimize the sum of agent-specific convex objective functions, while the agents' decisions are coupled via a convex conic constraint. We derive two methods by applying the alternating direction method of multipliers (ADMM) for decentralized consensus… ▽ More We consider a resource allocation problem over an undirected network of agents, where edges of the network define communication links. The goal is to minimize the sum of agent-specific convex objective functions, while the agents' decisions are coupled via a convex conic constraint. We derive two methods by applying the alternating direction method of multipliers (ADMM) for decentralized consensus optimization to the dual of our resource allocation problem. Both methods are fully parallelizable and decentralized in the sense that each agent exchanges information only with its neighbors in the network and requires only its own data for updating its decision. We prove convergence of the proposed methods and demonstrate their effectiveness with a numerical example. △ Less

Submitted 4 March, 2019; v1 submitted 19 September, 2018; originally announced September 2018.

Violation of Weak Cosmic Censorship in Einstein-Maxwell-dilaton theory: singularities connected by traversable wormholes

Authors: Prieslei Goulart

Abstract: We give two new analytical solutions to the low-energy string theory action that violate the weak cosmic censorship conjecture. They are classical charged solutions to the Einstein-Maxwell-dilaton theory in four dimensions that come in two types. The first represents one single naked singularity whose asymptotic region is flat. Its mass respects the positive mass theorem. The absence of horizons a… ▽ More We give two new analytical solutions to the low-energy string theory action that violate the weak cosmic censorship conjecture. They are classical charged solutions to the Einstein-Maxwell-dilaton theory in four dimensions that come in two types. The first represents one single naked singularity whose asymptotic region is flat. Its mass respects the positive mass theorem. The absence of horizons allows us to probe quantum aspects of gravity by direct observation of these singularities. The second represents singularities that come in pairs that "seem to create their own spacetime", and the region connecting them has the geometry of a traversable wormhole. In other words, they represent a pair of singularities connected by a traversable wormhole. The dyonic solution has a limit free of singularities, which we call "extremal limit" in analogy with black holes. This has a geometry given by the AdS$_{2}\times$S$^{2}$ spacetime in global coordinates. We compute the physical charges for the naked singularities and show that the weak gravity conjecture can be respected or violated, depending on the parameters of the solutions. We also compute the light deflection angle. △ Less

Submitted 18 September, 2018; originally announced September 2018.

Comments: 1 figure

Stochastic Model Predictive Control with Discounted Probabilistic Constraints

Authors: Shuhao Yan, Paul Goulart, Mark Cannon

Abstract: This paper considers linear discrete-time systems with additive disturbances, and designs a Model Predictive Control (MPC) law to minimise a quadratic cost function subject to a chance constraint. The chance constraint is defined as a discounted sum of violation probabilities on an infinite horizon. By penalising violation probabilities close to the initial time and ignoring violation probabilitie… ▽ More This paper considers linear discrete-time systems with additive disturbances, and designs a Model Predictive Control (MPC) law to minimise a quadratic cost function subject to a chance constraint. The chance constraint is defined as a discounted sum of violation probabilities on an infinite horizon. By penalising violation probabilities close to the initial time and ignoring violation probabilities in the far future, this form of constraint enables the feasibility of the online optimisation to be guaranteed without an assumption of boundedness of the disturbance. A computationally convenient MPC optimisation problem is formulated using Chebyshev's inequality and we introduce an online constraint-tightening technique to ensure recursive feasibility based on knowledge of a suboptimal solution. The closed loop system is guaranteed to satisfy the chance constraint and a quadratic stability condition. △ Less

Submitted 19 July, 2018; originally announced July 2018.

Comments: 6 pages, Conference Proceedings

Journal ref: In Proceedings of the 2018 European Control Conference

OSQP: An Operator Splitting Solver for Quadratic Programs

Authors: Bartolomeo Stellato, Goran Banjac, Paul Goulart, Alberto Bemporad, Stephen Boyd

Abstract: We present a general-purpose solver for convex quadratic programs based on the alternating direction method of multipliers, employing a novel operator splitting technique that requires the solution of a quasi-definite linear system with the same coefficient matrix at almost every iteration. Our algorithm is very robust, placing no requirements on the problem data such as positive definiteness of t… ▽ More We present a general-purpose solver for convex quadratic programs based on the alternating direction method of multipliers, employing a novel operator splitting technique that requires the solution of a quasi-definite linear system with the same coefficient matrix at almost every iteration. Our algorithm is very robust, placing no requirements on the problem data such as positive definiteness of the objective function or linear independence of the constraint functions. It can be configured to be division-free once an initial matrix factorization is carried out, making it suitable for real-time applications in embedded systems. In addition, our technique is the first operator splitting method for quadratic programs able to reliably detect primal and dual infeasible problems from the algorithm iterates. The method also supports factorization caching and warm starting, making it particularly efficient when solving parametrized problems arising in finance, control, and machine learning. Our open-source C implementation OSQP has a small footprint, is library-free, and has been extensively tested on many problem instances from a wide variety of application areas. It is typically ten times faster than competing interior-point methods, and sometimes much more when factorization caching or warm start is used. OSQP has already shown a large impact with tens of thousands of users both in academia and in large corporations. △ Less

Submitted 12 February, 2020; v1 submitted 21 November, 2017; originally announced November 2017.

Phantom wormholes in Einstein-Maxwell-dilaton theory

Authors: Prieslei Goulart

Abstract: In this paper we give an electrically charged traversable wormhole solution for the Einstein-Maxwell-dilaton theory when the dilaton is a phantom field, i.e. it has flipped sign kinetic term appearing in the action. In the limit when the charge is zero, we recover the anti-Fisher solution, which can be reduced to the Bronnikov-Ellis solution under certain choices of integration constants. The equa… ▽ More In this paper we give an electrically charged traversable wormhole solution for the Einstein-Maxwell-dilaton theory when the dilaton is a phantom field, i.e. it has flipped sign kinetic term appearing in the action. In the limit when the charge is zero, we recover the anti-Fisher solution, which can be reduced to the Bronnikov-Ellis solution under certain choices of integration constants. The equations of motion of this theory share the same S-duality invariance of string theory, so the electrically charged solution is rotated into the magnetically charged one by applying such transformations. The scalar field is topological, so we compute its topological charge, and discuss that under appropriate boundary conditions we can have a lump, a kink, or an anti-kink profile. We determine the position of the throat, and show the embedding diagram of the wormhole. As a physical application, we apply the Gauss-Bonnet theorem to compute the deflection angle of a light-ray that passes close to the wormhole. △ Less

Submitted 6 December, 2017; v1 submitted 2 August, 2017; originally announced August 2017.

Comments: v3: Solution and deflection angle written in terms of asymptotic charges. Matches the published version, Classical and Quantum Gravity (2017)

Chordal decomposition in operator-splitting methods for sparse semidefinite programs

Authors: Yang Zheng, Giovanni Fantuzzi, Antonis Papachristodoulou, Paul Goulart, Andrew Wynn

Abstract: We employ chordal decomposition to reformulate a large and sparse semidefinite program (SDP), either in primal or dual standard form, into an equivalent SDP with smaller positive semidefinite (PSD) constraints. In contrast to previous approaches, the decomposed SDP is suitable for the application of first-order operator-splitting methods, enabling the development of efficient and scalable algorith… ▽ More We employ chordal decomposition to reformulate a large and sparse semidefinite program (SDP), either in primal or dual standard form, into an equivalent SDP with smaller positive semidefinite (PSD) constraints. In contrast to previous approaches, the decomposed SDP is suitable for the application of first-order operator-splitting methods, enabling the development of efficient and scalable algorithms. In particular, we apply the alternating direction method of multipliers (ADMM) to solve decomposed primal- and dual-standard-form SDPs. Each iteration of such ADMM algorithms requires a projection onto an affine subspace, and a set of projections onto small PSD cones that can be computed in parallel. We also formulate the homogeneous self-dual embedding (HSDE) of a primal-dual pair of decomposed SDPs, and extend a recent ADMM-based algorithm to exploit the structure of our HSDE. The resulting HSDE algorithm has the same leading-order computational cost as those for the primal or dual problems only, with the advantage of being able to identify infeasible problems and produce an infeasibility certificate. All algorithms are implemented in the open-source MATLAB solver CDCS. Numerical experiments on a range of large-scale SDPs demonstrate the computational advantages of the proposed methods compared to common state-of-the-art solvers. △ Less

Submitted 12 February, 2019; v1 submitted 17 July, 2017; originally announced July 2017.

Comments: To appear at Math. Prog. 36 pages, 7 figures; Codes available from https://github.com/oxfordcontrol/CDCS (sparse conic solver: CDCS)

Journal ref: Mathematical Programming volume 180, pages 489-532(2020)

Distributionally Ambiguous Optimization Techniques for Batch Bayesian Optimization

Authors: Nikitas Rontsis, Michael A. Osborne, Paul J. Goulart

Abstract: We propose a novel, theoretically-grounded, acquisition function for Batch Bayesian optimization informed by insights from distributionally ambiguous optimization. Our acquisition function is a lower bound on the well-known Expected Improvement function, which requires evaluation of a Gaussian Expectation over a multivariate piecewise affine function. Our bound is computed instead by evaluating th… ▽ More We propose a novel, theoretically-grounded, acquisition function for Batch Bayesian optimization informed by insights from distributionally ambiguous optimization. Our acquisition function is a lower bound on the well-known Expected Improvement function, which requires evaluation of a Gaussian Expectation over a multivariate piecewise affine function. Our bound is computed instead by evaluating the best-case expectation over all probability distributions consistent with the same mean and variance as the original Gaussian distribution. Unlike alternative approaches, including Expected Improvement, our proposed acquisition function avoids multi-dimensional integrations entirely, and can be computed exactly - even on large batch sizes - as the solution of a tractable convex optimization problem. Our suggested acquisition function can also be optimized efficiently, since first and second derivative information can be calculated inexpensively as by-products of the acquisition function calculation itself. We derive various novel theorems that ground our work theoretically and we demonstrate superior performance via simple motivating examples, benchmark functions and real-world problems. △ Less

Submitted 16 April, 2018; v1 submitted 13 July, 2017; originally announced July 2017.