-
Parameterizations for Large-Scale Variational System Identification Using Unconstrained Optimization
Authors:
Dimas Abreu Archanjo Dutra
Abstract:
This paper details how to parameterize the posterior distribution of state-space systems to generate improved optimization problems for system identification using variational inference. Three different parameterizations of the assumed state-path posterior distribution are proposed based on this representation: time-varying, steady-state, and convolution smoother; each resulting in a different par…
▽ More
This paper details how to parameterize the posterior distribution of state-space systems to generate improved optimization problems for system identification using variational inference. Three different parameterizations of the assumed state-path posterior distribution are proposed based on this representation: time-varying, steady-state, and convolution smoother; each resulting in a different parameter estimator. In contrast to existing methods for variational system identification, the proposed estimators can be implemented with unconstrained optimization methods. Furthermore, when applied to mini-batches in conjunction with stochastic optimization, the convolution-smoother formulation enables identification of large linear and nonlinear state-space systems from very large datasets. For linear systems, the method achieves the same performance as the inherently sequential prediction-error methods using an embarrassingly parallel algorithm that benefits from large speedups when computed in modern graphical processing units (GPUs). The ability of the proposed estimators to identify large models, work with large datasets split into mini-batches, and work in parallel on GPUs make them well-suited for identifying deep models for applications in systems and control.
△ Less
Submitted 24 September, 2024; v1 submitted 15 April, 2024;
originally announced April 2024.
-
Uncertainty estimation in equality-constrained MAP and maximum likelihood estimation with applications to system identification and state estimation
Authors:
Dimas Abreu Archanjo Dutra
Abstract:
In unconstrained maximum a posteriori (MAP) and maximum likelihood estimation, the inverse of minus the merit-function Hessian matrix is an approximation of the estimate covariance matrix. In the Bayesian context of MAP estimation, it is the covariance of a normal approximation of the posterior around the mode; while in maximum likelihood estimation, it an approximation of the inverse Fisher infor…
▽ More
In unconstrained maximum a posteriori (MAP) and maximum likelihood estimation, the inverse of minus the merit-function Hessian matrix is an approximation of the estimate covariance matrix. In the Bayesian context of MAP estimation, it is the covariance of a normal approximation of the posterior around the mode; while in maximum likelihood estimation, it an approximation of the inverse Fisher information matrix, to which the covariance of efficient estimators converge. These measures are routinely used in system identification to evaluate the estimate uncertainties and diagnose problems such as overparametrization, improper excitation and unidentifiability. A wide variety of estimation problems in systems and control, however, can be formulated as equality-constrained optimizations with additional decision variables to exploit parallelism in computer hardware, simplify implementation and increase the convergence basin and efficiency of the nonlinear program solver. The introduction of the extra variables, however, dissociates the inverse Hessian from the covariance matrix. Instead, submatrices of the inverse Hessian of the constrained-problem's Lagrangian must be used. In this paper, we derive these relationships, showing how the estimates' covariance can be estimated directly from the augmented problem. Application examples are shown in system identification with the output-error method and joint state-path and parameter estimation.
△ Less
Submitted 16 March, 2020; v1 submitted 25 February, 2020;
originally announced February 2020.
-
Collocation-Based Output-Error Method for Aircraft System Identification
Authors:
Dimas Abreu Archanjo Dutra
Abstract:
The output-error method is a mainstay of aircraft system identification from flight-test data. It is the method of choice for a wide range of applications, from the estimation of stability and control derivatives for aerodynamic database generation to sensor bias estimation in flight-path reconstruction. However, notable limitations of the output-error method are that it requires ad hoc modificati…
▽ More
The output-error method is a mainstay of aircraft system identification from flight-test data. It is the method of choice for a wide range of applications, from the estimation of stability and control derivatives for aerodynamic database generation to sensor bias estimation in flight-path reconstruction. However, notable limitations of the output-error method are that it requires ad hoc modifications for applications to unstable systems and it is an iterative method which is particularly sensitive to the initial guess. In this paper, we show how to reformulate the estimation as a collocation problem, an approach common in other disciplines but seldomly used in flight vehicle system identification. Both formulations are equivalent in terms of having the same solution, but the collocation-based can be applied without modifications or loss of efficiency to unstable systems. Examples with simulated and real-world flight-test data also show that convergence to the optimum is obtained even with poor initial guesses.
△ Less
Submitted 4 September, 2019;
originally announced September 2019.
-
Joint Maximum a Posteriori State Path and Parameter Estimation in Stochastic Differential Equations
Authors:
Dimas Abreu Archanjo Dutra,
Bruno Otávio Soares Teixeira,
Luis Antonio Aguirre
Abstract:
In this article, we introduce the joint maximum a posteriori state path and parameter estimator (JME) for continuous-time systems described by stochastic differential equations (SDEs). This estimator can be applied to nonlinear systems with discrete-time (sampled) measurements with a wide range of measurement distributions. We also show that the minimum-energy state path and parameter estimator (M…
▽ More
In this article, we introduce the joint maximum a posteriori state path and parameter estimator (JME) for continuous-time systems described by stochastic differential equations (SDEs). This estimator can be applied to nonlinear systems with discrete-time (sampled) measurements with a wide range of measurement distributions. We also show that the minimum-energy state path and parameter estimator (MEE) obtains the joint maximum a posteriori noise path, initial conditions, and parameters. These estimators are demonstrated in simulated experiments, in which they are compared to the prediction error method (PEM) using the unscented Kalman filter and smoother. The experiments show that the MEE is biased for the damping parameters of the drift function. Furthermore, for robust estimation in the presence of outliers, the JME attains lower state estimation errors than the PEM.
△ Less
Submitted 5 April, 2017;
originally announced April 2017.
