-
New Metrics Between Rational Spectra and their Connection to Optimal Transport
Authors:
Fredrik Bagge Carlson,
Mandar Chitre
Abstract:
We propose a series of metrics between pairs of signals, linear systems or rational spectra, based on optimal transport and linear-systems theory. The metrics operate on the locations of the poles of rational functions and admit very efficient computation of distances, barycenters, displacement interpolation and projections. We establish the connection to the Wasserstein distance between rational…
▽ More
We propose a series of metrics between pairs of signals, linear systems or rational spectra, based on optimal transport and linear-systems theory. The metrics operate on the locations of the poles of rational functions and admit very efficient computation of distances, barycenters, displacement interpolation and projections. We establish the connection to the Wasserstein distance between rational spectra, and demonstrate the use of the metrics in tasks such as signal classification, clustering, detection and approximation.
△ Less
Submitted 20 April, 2020;
originally announced April 2020.
-
MonteCarloMeasurements.jl: Nonlinear Propagation of Arbitrary Multivariate Distributions by means of Method Overloading
Authors:
Fredrik Bagge Carlson
Abstract:
This manuscript outlines a software package that facilitates working with probability distributions by means of Monte-Carlo methods, in a way that allows for propagation of multivariate probability distributions through arbitrary functions. We provide a \emph{type} that represents probability distributions by an internal vector of unweighted samples, \texttt{Particles}, which is a subtype of a \te…
▽ More
This manuscript outlines a software package that facilitates working with probability distributions by means of Monte-Carlo methods, in a way that allows for propagation of multivariate probability distributions through arbitrary functions. We provide a \emph{type} that represents probability distributions by an internal vector of unweighted samples, \texttt{Particles}, which is a subtype of a \texttt{Real} number and behaves just like a regular real number in calculations by means of method overloading. This makes the software easy to work with and presents minimal friction for the user. We highlight how this design facilitates optimal usage of SIMD instructions and showcase the package for uncertainty propagation through an off-the-shelf ODE solver, as well as for robust probabilistic optimization with automatic differentiation.
△ Less
Submitted 21 January, 2020;
originally announced January 2020.
-
Machine Learning and System Identification for Estimation in Physical Systems
Authors:
Fredrik Bagge Carlson
Abstract:
In this thesis, we draw inspiration from both classical system identification and modern machine learning in order to solve estimation problems for real-world, physical systems. The main approach to estimation and learning adopted is optimization based. Concepts such as regularization will be utilized for encoding of prior knowledge and basis-function expansions will be used to add nonlinear model…
▽ More
In this thesis, we draw inspiration from both classical system identification and modern machine learning in order to solve estimation problems for real-world, physical systems. The main approach to estimation and learning adopted is optimization based. Concepts such as regularization will be utilized for encoding of prior knowledge and basis-function expansions will be used to add nonlinear modeling power while keeping data requirements practical. The thesis covers a wide range of applications, many inspired by applications within robotics, but also extending outside this already wide field. Usage of the proposed methods and algorithms are in many cases illustrated in the real-world applications that motivated the research. Topics covered include dynamics modeling and estimation, model-based reinforcement learning, spectral estimation, friction modeling and state estimation and calibration in robotic machining. In the work on modeling and identification of dynamics, we develop regularization strategies that allow us to incorporate prior domain knowledge into flexible, overparameterized models. We make use of classical control theory to gain insight into training and regularization while using flexible tools from modern deep learning. A particular focus of the work is to allow use of modern methods in scenarios where gathering data is associated with a high cost. In the robotics-inspired parts of the thesis, we develop methods that are practically motivated and ensure that they are implementable also outside the research setting. We demonstrate this by performing experiments in realistic settings and providing open-source implementations of all proposed methods and algorithms.
△ Less
Submitted 5 June, 2019;
originally announced June 2019.
-
Tangent-Space Regularization for Neural-Network Models of Dynamical Systems
Authors:
Fredrik Bagge Carlson,
Rolf Johansson,
Anders Robertsson
Abstract:
This work introduces the concept of tangent space regularization for neural-network models of dynamical systems. The tangent space to the dynamics function of many physical systems of interest in control applications exhibits useful properties, e.g., smoothness, motivating regularization of the model Jacobian along system trajectories using assumptions on the tangent space of the dynamics. Without…
▽ More
This work introduces the concept of tangent space regularization for neural-network models of dynamical systems. The tangent space to the dynamics function of many physical systems of interest in control applications exhibits useful properties, e.g., smoothness, motivating regularization of the model Jacobian along system trajectories using assumptions on the tangent space of the dynamics. Without assumptions, large amounts of training data are required for a neural network to learn the full non-linear dynamics without overfitting. We compare different network architectures on one-step prediction and simulation performance and investigate the propensity of different architectures to learn models with correct input-output Jacobian. Furthermore, the influence of $L_2$ weight regularization on the learned Jacobian eigenvalue spectrum, and hence system stability, is investigated.
△ Less
Submitted 26 June, 2018;
originally announced June 2018.
-
Identification of LTV Dynamical Models with Smooth or Discontinuous Time Evolution by means of Convex Optimization
Authors:
Fredrik Bagge Carlson,
Anders Robertsson,
Rolf Johansson
Abstract:
We establish a connection between trend filtering and system identification which results in a family of new identification methods for linear, time-varying (LTV) dynamical models based on convex optimization. We demonstrate how the design of the cost function promotes a model with either a continuous change in dynamics over time, or causes discontinuous changes in model coefficients occurring at…
▽ More
We establish a connection between trend filtering and system identification which results in a family of new identification methods for linear, time-varying (LTV) dynamical models based on convex optimization. We demonstrate how the design of the cost function promotes a model with either a continuous change in dynamics over time, or causes discontinuous changes in model coefficients occurring at a finite (sparse) set of time instances. We further discuss the introduction of priors on the model parameters for situations where excitation is insufficient for identification. The identification problems are cast as convex optimization problems and are applicable to, e.g., ARX models and state-space models with time-varying parameters. We illustrate usage of the methods in simulations of jump-linear systems, a nonlinear robot arm with non-smooth friction and stiff contacts as well as in model-based, trajectory centric reinforcement learning on a smooth nonlinear system.
△ Less
Submitted 27 February, 2018;
originally announced February 2018.
