-
Regret Analysis: a control perspective
Authors:
Travis E. Gibson,
Sawal Acharya
Abstract:
Online learning and model reference adaptive control have many interesting intersections. One area where they differ however is in how the algorithms are analyzed and what objective or metric is used to discriminate "good" algorithms from "bad" algorithms. In adaptive control there are usually two objectives: 1) prove that all time varying parameters/states of the system are bounded, and 2) that t…
▽ More
Online learning and model reference adaptive control have many interesting intersections. One area where they differ however is in how the algorithms are analyzed and what objective or metric is used to discriminate "good" algorithms from "bad" algorithms. In adaptive control there are usually two objectives: 1) prove that all time varying parameters/states of the system are bounded, and 2) that the instantaneous error between the adaptively controlled system and a reference system converges to zero over time (or at least a compact set). For online learning the performance of algorithms is often characterized by the regret the algorithm incurs. Regret is defined as the cumulative loss (cost) over time from the online algorithm minus the cumulative loss (cost) of the single optimal fixed parameter choice in hindsight. Another significant difference between the two areas of research is with regard to the assumptions made in order to obtain said results. Adaptive control makes assumptions about the input-output properties of the control problem and derives solutions for a fixed error model or optimization task. In the online learning literature results are derived for classes of loss functions (i.e. convex) while a priori assuming certain signals are bounded. In this work we discuss these differences in detail through the regret based analysis of gradient descent for convex functions and the control based analysis of a streaming regression problem. We close with a discussion about the newly defined paradigm of online adaptive control.
△ Less
Submitted 23 January, 2025; v1 submitted 8 January, 2025;
originally announced January 2025.
-
On the stability of gradient descent with second order dynamics for time-varying cost functions
Authors:
Travis E. Gibson,
Sawal Acharya,
Anjali Parashar,
Joseph E. Gaudio,
Anurdha M. Annaswamy
Abstract:
Gradient based optimization algorithms deployed in Machine Learning (ML) applications are often analyzed and compared by their convergence rates or regret bounds. While these rates and bounds convey valuable information they don't always directly translate to stability guarantees. Stability and similar concepts, like robustness, will become ever more important as we move towards deploying models i…
▽ More
Gradient based optimization algorithms deployed in Machine Learning (ML) applications are often analyzed and compared by their convergence rates or regret bounds. While these rates and bounds convey valuable information they don't always directly translate to stability guarantees. Stability and similar concepts, like robustness, will become ever more important as we move towards deploying models in real-time and safety critical systems. In this work we build upon the results in Gaudio et al. 2021 and Moreu & Annaswamy 2022 for gradient descent with second order dynamics when applied to explicitly time varying cost functions and provide more general stability guarantees. These more general results can aid in the design and certification of these optimization schemes so as to help ensure safe and reliable deployment for real-time learning applications. We also hope that the techniques provided here will stimulate and cross-fertilize the analysis that occurs on the same algorithms from the online learning and stochastic optimization communities.
△ Less
Submitted 3 February, 2025; v1 submitted 22 May, 2024;
originally announced May 2024.
-
Accelerated Learning with Robustness to Adversarial Regressors
Authors:
Joseph E. Gaudio,
Anuradha M. Annaswamy,
José M. Moreu,
Michael A. Bolender,
Travis E. Gibson
Abstract:
High order momentum-based parameter update algorithms have seen widespread applications in training machine learning models. Recently, connections with variational approaches have led to the derivation of new learning algorithms with accelerated learning guarantees. Such methods however, have only considered the case of static regressors. There is a significant need for parameter update algorithms…
▽ More
High order momentum-based parameter update algorithms have seen widespread applications in training machine learning models. Recently, connections with variational approaches have led to the derivation of new learning algorithms with accelerated learning guarantees. Such methods however, have only considered the case of static regressors. There is a significant need for parameter update algorithms which can be proven stable in the presence of adversarial time-varying regressors, as is commonplace in control theory. In this paper, we propose a new discrete time algorithm which 1) provides stability and asymptotic convergence guarantees in the presence of adversarial regressors by leveraging insights from adaptive control theory and 2) provides non-asymptotic accelerated learning guarantees leveraging insights from convex optimization. In particular, our algorithm reaches an $ε$ sub-optimal point in at most $\tilde{\mathcal{O}}(1/\sqrtε)$ iterations when regressors are constant - matching lower bounds due to Nesterov of $Ω(1/\sqrtε)$, up to a $\log(1/ε)$ factor and provides guaranteed bounds for stability when regressors are time-varying. We provide numerical experiments for a variant of Nesterov's provably hard convex optimization problem with time-varying regressors, as well as the problem of recovering an image with a time-varying blur and noise using streaming data.
△ Less
Submitted 4 June, 2021; v1 submitted 4 May, 2020;
originally announced May 2020.
-
Connections Between Adaptive Control and Optimization in Machine Learning
Authors:
Joseph E. Gaudio,
Travis E. Gibson,
Anuradha M. Annaswamy,
Michael A. Bolender,
Eugene Lavretsky
Abstract:
This paper demonstrates many immediate connections between adaptive control and optimization methods commonly employed in machine learning. Starting from common output error formulations, similarities in update law modifications are examined. Concepts in stability, performance, and learning, common to both fields are then discussed. Building on the similarities in update laws and common concepts,…
▽ More
This paper demonstrates many immediate connections between adaptive control and optimization methods commonly employed in machine learning. Starting from common output error formulations, similarities in update law modifications are examined. Concepts in stability, performance, and learning, common to both fields are then discussed. Building on the similarities in update laws and common concepts, new intersections and opportunities for improved algorithm analysis are provided. In particular, a specific problem related to higher order learning is solved through insights obtained from these intersections.
△ Less
Submitted 11 April, 2019;
originally announced April 2019.
-
Provably Correct Learning Algorithms in the Presence of Time-Varying Features Using a Variational Perspective
Authors:
Joseph E. Gaudio,
Travis E. Gibson,
Anuradha M. Annaswamy,
Michael A. Bolender
Abstract:
Features in machine learning problems are often time-varying and may be related to outputs in an algebraic or dynamical manner. The dynamic nature of these machine learning problems renders current higher order accelerated gradient descent methods unstable or weakens their convergence guarantees. Inspired by methods employed in adaptive control, this paper proposes new algorithms for the case when…
▽ More
Features in machine learning problems are often time-varying and may be related to outputs in an algebraic or dynamical manner. The dynamic nature of these machine learning problems renders current higher order accelerated gradient descent methods unstable or weakens their convergence guarantees. Inspired by methods employed in adaptive control, this paper proposes new algorithms for the case when time-varying features are present, and demonstrates provable performance guarantees. In particular, we develop a unified variational perspective within a continuous time algorithm. This variational perspective includes higher order learning concepts and normalization, both of which stem from adaptive control, and allows stability to be established for dynamical machine learning problems where time-varying features are present. These higher order algorithms are also examined for provably correct learning in adaptive control and identification. Simulations are provided to verify the theoretical results.
△ Less
Submitted 27 May, 2019; v1 submitted 11 March, 2019;
originally announced March 2019.
-
Impacts of Network Topology on the Performance of a Distributed Algorithm Solving Linear Equations
Authors:
Hong-Tai Cao,
Travis E. Gibson,
Shaoshuai Mou,
Yang-Yu Liu
Abstract:
Recently a distributed algorithm has been proposed for multi-agent networks to solve a system of linear algebraic equations, by assuming each agent only knows part of the system and is able to communicate with nearest neighbors to update their local solutions. This paper investigates how the network topology impacts exponential convergence of the proposed algorithm. It is found that networks with…
▽ More
Recently a distributed algorithm has been proposed for multi-agent networks to solve a system of linear algebraic equations, by assuming each agent only knows part of the system and is able to communicate with nearest neighbors to update their local solutions. This paper investigates how the network topology impacts exponential convergence of the proposed algorithm. It is found that networks with higher mean degree, smaller diameter, and homogeneous degree distribution tend to achieve faster convergence. Both analytical and numerical results are provided.
△ Less
Submitted 14 March, 2016;
originally announced March 2016.
-
Sign Stability via Root Locus Analysis
Authors:
Travis E. Gibson
Abstract:
With the rise of network science old topics in ecology and economics are resurfacing. One such topic is structural stability (often referred to as qualitative stability or sign stability). A system is deemed structurally stable if the system remains stable for all possible parameter variations so long as the parameters do not change sign. This type of stability analysis is appealing when studying…
▽ More
With the rise of network science old topics in ecology and economics are resurfacing. One such topic is structural stability (often referred to as qualitative stability or sign stability). A system is deemed structurally stable if the system remains stable for all possible parameter variations so long as the parameters do not change sign. This type of stability analysis is appealing when studying real systems as the underlying stability result only requires the scientist or engineer to know the sign of the parameters in the model and not the specific values. The necessary and sufficient conditions for qualitative stability however are opaque. In order to shed light on those conditions root locus analysis is employed. This technique allows us to illustrate the necessary conditions for qualitative stability.
△ Less
Submitted 18 December, 2015;
originally announced December 2015.
-
Convergence Properties of Adaptive Systems and the Definition of Exponential Stability
Authors:
Benjamin M. Jenkins,
Anuradha M. Annaswamy,
Eugene Lavretsky,
Travis E. Gibson
Abstract:
The convergence properties of adaptive systems in terms of excitation conditions on the regressor vector are well known. With persistent excitation of the regressor vector in model reference adaptive control the state error and the adaptation error are globally exponentially stable, or equivalently, exponentially stable in the large. When the excitation condition however is imposed on the referenc…
▽ More
The convergence properties of adaptive systems in terms of excitation conditions on the regressor vector are well known. With persistent excitation of the regressor vector in model reference adaptive control the state error and the adaptation error are globally exponentially stable, or equivalently, exponentially stable in the large. When the excitation condition however is imposed on the reference input or the reference model state it is often incorrectly concluded that the persistent excitation in those signals also implies exponential stability in the large. The definition of persistent excitation is revisited so as to address some possible confusion in the adaptive control literature. It is then shown that persistent excitation of the reference model only implies local persistent excitation (weak persistent excitation). Weak persistent excitation of the regressor is still sufficient for uniform asymptotic stability in the large, but not exponential stability in the large. We show that there exists an infinite region in the state-space of adaptive systems where the state rate is bounded. This infinite region with finite rate of convergence is shown to exist not only in classic open-loop reference model adaptive systems, but also in a new class of closed-loop reference model adaptive systems.
△ Less
Submitted 10 November, 2015;
originally announced November 2015.
-
Adaptive Output Feedback based on Closed-loop Reference Models
Authors:
Travis E. Gibson,
Zheng Qu,
Anuradha M. Annaswamy,
Eugene Lavretsky
Abstract:
This note presents the design and analysis of an adaptive controller for a class of linear plants in the presence of output feedback. This controller makes use of a closed-loop reference model as an observer, and guarantees global stability and asymptotic output tracking.
This note presents the design and analysis of an adaptive controller for a class of linear plants in the presence of output feedback. This controller makes use of a closed-loop reference model as an observer, and guarantees global stability and asymptotic output tracking.
△ Less
Submitted 7 October, 2014;
originally announced October 2014.
-
On Adaptive Control with Closed-loop Reference Models: Transients, Oscillations, and Peaking
Authors:
Travis E. Gibson,
Anuradha M. Annaswamy,
Eugene Lavretsky
Abstract:
One of the main features of adaptive systems is an oscillatory convergence that exacerbates with the speed of adaptation. Recently it has been shown that Closed-loop Reference Models (CRMs) can result in improved transient performance over their open-loop counterparts in model reference adaptive control. In this paper, we quantify both the transient performance in the classical adaptive systems an…
▽ More
One of the main features of adaptive systems is an oscillatory convergence that exacerbates with the speed of adaptation. Recently it has been shown that Closed-loop Reference Models (CRMs) can result in improved transient performance over their open-loop counterparts in model reference adaptive control. In this paper, we quantify both the transient performance in the classical adaptive systems and their improvement with CRMs. In addition to deriving bounds on L-2 norms of the derivatives of the adaptive parameters which are shown to be smaller, an optimal design of CRMs is proposed which minimizes an underlying peaking phenomenon. The analytical tools proposed are shown to be applicable for a range of adaptive control problems including direct control and composite control with observer feedback. The presence of CRMs in adaptive backstepping and adaptive robot control are also discussed. Simulation results are presented throughout the paper to support the theoretical derivations.
△ Less
Submitted 8 August, 2013; v1 submitted 26 April, 2013;
originally announced April 2013.
-
Closed-loop Reference Models for Output-Feedback Adaptive Systems
Authors:
Travis E. Gibson,
Anuradha M. Annaswamy,
Eugene Lavretsky
Abstract:
Closed-loop reference models have recently been proposed for states accessible adaptive systems. They have been shown to have improved transient response over their open loop counter parts. The results in the states accessible case are extended to single input single output plants of arbitrary relative degree.
Closed-loop reference models have recently been proposed for states accessible adaptive systems. They have been shown to have improved transient response over their open loop counter parts. The results in the states accessible case are extended to single input single output plants of arbitrary relative degree.
△ Less
Submitted 27 November, 2012; v1 submitted 30 October, 2012;
originally announced October 2012.
-
Adaptive Systems with Closed-loop Reference Models: Stability, Robustness and Transient Performance
Authors:
Travis E. Gibson,
Anuradha M. Annaswamy,
Eugene Lavretsky
Abstract:
This paper explores the properties of adaptive systems with closed-loop reference models. Using additional design freedom available in closed-loop reference models, we design new adaptive controllers that are (a) stable, and (b) have improved transient properties. Numerical studies that complement theoretical derivations are also reported.
This paper explores the properties of adaptive systems with closed-loop reference models. Using additional design freedom available in closed-loop reference models, we design new adaptive controllers that are (a) stable, and (b) have improved transient properties. Numerical studies that complement theoretical derivations are also reported.
△ Less
Submitted 30 October, 2012; v1 submitted 23 January, 2012;
originally announced January 2012.
-
Projection Operator in Adaptive Systems
Authors:
Eugene Lavretsky,
Travis E. Gibson
Abstract:
The projection algorithm is frequently used in adaptive control and this note presents a detailed analysis of its properties.
The projection algorithm is frequently used in adaptive control and this note presents a detailed analysis of its properties.
△ Less
Submitted 16 October, 2012; v1 submitted 18 December, 2011;
originally announced December 2011.
