Showing 1–14 of 14 results for author: Brunton, B W

Network inference via process motifs for lagged correlation in linear stochastic processes

Authors: Alice C. Schwarze, Sara M. Ichinaga, Bingni W. Brunton

Abstract: A major challenge for causal inference from time-series data is the trade-off between computational feasibility and accuracy. Motivated by process motifs for lagged covariance in an autoregressive model with slow mean-reversion, we propose to infer networks of causal relations via pairwise edge measure (PEMs) that one can easily compute from lagged correlation matrices. Motivated by contributions… ▽ More A major challenge for causal inference from time-series data is the trade-off between computational feasibility and accuracy. Motivated by process motifs for lagged covariance in an autoregressive model with slow mean-reversion, we propose to infer networks of causal relations via pairwise edge measure (PEMs) that one can easily compute from lagged correlation matrices. Motivated by contributions of process motifs to covariance and lagged variance, we formulate two PEMs that correct for confounding factors and for reverse causation. To demonstrate the performance of our PEMs, we consider network interference from simulations of linear stochastic processes, and we show that our proposed PEMs can infer networks accurately and efficiently. Specifically, for slightly autocorrelated time-series data, our approach achieves accuracies higher than or similar to Granger causality, transfer entropy, and convergent crossmapping -- but with much shorter computation time than possible with any of these methods. Our fast and accurate PEMs are easy-to-implement methods for network inference with a clear theoretical underpinning. They provide promising alternatives to current paradigms for the inference of linear models from time-series data, including Granger causality, vector-autoregression, and sparse inverse covariance estimation. △ Less

Submitted 21 August, 2022; v1 submitted 18 August, 2022; originally announced August 2022.

Comments: 28 pages, 17 figures

MSC Class: 92C42; 92B20; 94A16

Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control

Authors: Urban Fasel, J. Nathan Kutz, Bingni W. Brunton, Steven L. Brunton

Abstract: Sparse model identification enables the discovery of nonlinear dynamical systems purely from data; however, this approach is sensitive to noise, especially in the low-data limit. In this work, we leverage the statistical approach of bootstrap aggregating (bagging) to robustify the sparse identification of nonlinear dynamics (SINDy) algorithm. First, an ensemble of SINDy models is identified from s… ▽ More Sparse model identification enables the discovery of nonlinear dynamical systems purely from data; however, this approach is sensitive to noise, especially in the low-data limit. In this work, we leverage the statistical approach of bootstrap aggregating (bagging) to robustify the sparse identification of nonlinear dynamics (SINDy) algorithm. First, an ensemble of SINDy models is identified from subsets of limited and noisy data. The aggregate model statistics are then used to produce inclusion probabilities of the candidate functions, which enables uncertainty quantification and probabilistic forecasts. We apply this ensemble-SINDy (E-SINDy) algorithm to several synthetic and real-world data sets and demonstrate substantial improvements to the accuracy and robustness of model discovery from extremely noisy and limited data. For example, E-SINDy uncovers partial differential equations models from data with more than twice as much measurement noise as has been previously reported. Similarly, E-SINDy learns the Lotka Volterra dynamics from remarkably limited data of yearly lynx and hare pelts collected from 1900-1920. E-SINDy is computationally efficient, with similar scaling as standard SINDy. Finally, we show that ensemble statistics from E-SINDy can be exploited for active learning and improved model predictive control. △ Less

Submitted 22 November, 2021; originally announced November 2021.

SINDy with Control: A Tutorial

Authors: Urban Fasel, Eurika Kaiser, J. Nathan Kutz, Bingni W. Brunton, Steven L. Brunton

Abstract: Many dynamical systems of interest are nonlinear, with examples in turbulence, epidemiology, neuroscience, and finance, making them difficult to control using linear approaches. Model predictive control (MPC) is a powerful model-based optimization technique that enables the control of such nonlinear systems with constraints. However, modern systems often lack computationally tractable models, moti… ▽ More Many dynamical systems of interest are nonlinear, with examples in turbulence, epidemiology, neuroscience, and finance, making them difficult to control using linear approaches. Model predictive control (MPC) is a powerful model-based optimization technique that enables the control of such nonlinear systems with constraints. However, modern systems often lack computationally tractable models, motivating the use of system identification techniques to learn accurate and efficient models for real-time control. In this tutorial article, we review emerging data-driven methods for model discovery and how they are used for nonlinear MPC. In particular, we focus on the sparse identification of nonlinear dynamics (SINDy) algorithm and show how it may be used with MPC on an infectious disease control example. We compare the performance against MPC based on a linear dynamic mode decomposition (DMD) model. Code is provided to run the tutorial examples and may be modified to extend this data-driven control framework to arbitrary nonlinear systems. △ Less

Submitted 30 August, 2021; originally announced August 2021.

PySensors: A Python Package for Sparse Sensor Placement

Authors: Brian M. de Silva, Krithika Manohar, Emily Clark, Bingni W. Brunton, Steven L. Brunton, J. Nathan Kutz

Abstract: PySensors is a Python package for selecting and placing a sparse set of sensors for classification and reconstruction tasks. Specifically, PySensors implements algorithms for data-driven sparse sensor placement optimization for reconstruction (SSPOR) and sparse sensor placement optimization for classification (SSPOC). In this work we provide a brief description of the mathematical algorithms and t… ▽ More PySensors is a Python package for selecting and placing a sparse set of sensors for classification and reconstruction tasks. Specifically, PySensors implements algorithms for data-driven sparse sensor placement optimization for reconstruction (SSPOR) and sparse sensor placement optimization for classification (SSPOC). In this work we provide a brief description of the mathematical algorithms and theory for sparse sensor optimization, along with an overview and demonstration of the features implemented in PySensors (with code examples). We also include practical advice for user and a list of potential extensions to PySensors. Software is available at https://github.com/dynamicslab/pysensors. △ Less

Submitted 20 February, 2021; originally announced February 2021.

Structured Time-Delay Models for Dynamical Systems with Connections to Frenet-Serret Frame

Authors: Seth M. Hirsh, Sara M. Ichinaga, Steven L. Brunton, J. Nathan Kutz, Bingni W. Brunton

Abstract: Time-delay embeddings and dimensionality reduction are powerful techniques for discovering effective coordinate systems to represent the dynamics of physical systems. Recently, it has been shown that models identified by dynamic mode decomposition (DMD) on time-delay coordinates provide linear representations of strongly nonlinear systems, in the so-called Hankel alternative view of Koopman (HAVOK… ▽ More Time-delay embeddings and dimensionality reduction are powerful techniques for discovering effective coordinate systems to represent the dynamics of physical systems. Recently, it has been shown that models identified by dynamic mode decomposition (DMD) on time-delay coordinates provide linear representations of strongly nonlinear systems, in the so-called Hankel alternative view of Koopman (HAVOK) approach. Curiously, the resulting linear model has a matrix representation that is approximately antisymmetric and tridiagonal with a zero diagonal; for chaotic systems, there is an additional forcing term in the last component. In this paper, we establish a new theoretical connection between HAVOK and the Frenet-Serret frame from differential geometry, and also develop an improved algorithm to identify more stable and accurate models from less data. In particular, we show that the sub- and super-diagonal entries of the linear model correspond to the intrinsic curvatures in Frenet-Serret frame. Based on this connection, we modify the algorithm to promote this antisymmetric structure, even in the noisy, low-data limit. We demonstrate this improved modeling procedure on data from several nonlinear synthetic and real-world examples. △ Less

Submitted 20 January, 2021; originally announced January 2021.

Comments: 34 pages, 9 figures

Go with the FLOW: Visualizing spatiotemporal dynamics in optical widefield calcium imaging

Authors: Nathaniel J Linden, Dennis R Tabuena, Nicholas A Steinmetz, William J Moody, Steven L Brunton, Bingni W Brunton

Abstract: Widefield calcium imaging has recently emerged as a powerful experimental technique to record coordinated large-scale brain activity. These measurements present a unique opportunity to characterize spatiotemporal coherent structures that underlie neural activity across many regions of the brain. In this work, we leverage analytic techniques from fluid dynamics to develop a visualization framework… ▽ More Widefield calcium imaging has recently emerged as a powerful experimental technique to record coordinated large-scale brain activity. These measurements present a unique opportunity to characterize spatiotemporal coherent structures that underlie neural activity across many regions of the brain. In this work, we leverage analytic techniques from fluid dynamics to develop a visualization framework that highlights features of flow across the cortex, mapping wave fronts that may be correlated with behavioral events. First, we transform the time series of widefield calcium images into time-varying vector fields using optic flow. Next, we extract concise diagrams summarizing the dynamics, which we refer to as FLOW (flow lines in optical widefield imaging) portraits. These FLOW portraits provide an intuitive map of dynamic calcium activity, including regions of initiation and termination, as well as the direction and extent of activity spread. To extract these structures, we use the finite-time Lyapunov exponent (FTLE) technique developed to analyze time-varying manifolds in unsteady fluids. Importantly, our approach captures coherent structures that are poorly represented by traditional modal decomposition techniques. We demonstrate the application of FLOW portraits on three simple synthetic datasets and two widefield calcium imaging datasets, including cortical waves in the developing mouse and spontaneous cortical activity in an adult mouse. △ Less

Submitted 24 June, 2021; v1 submitted 29 September, 2020; originally announced September 2020.

Comments: 25 pages, 8 figures

Numerical differentiation of noisy data: A unifying multi-objective optimization framework

Authors: Floris van Breugel, J. Nathan Kutz, Bingni W. Brunton

Abstract: Computing derivatives of noisy measurement data is ubiquitous in the physical, engineering, and biological sciences, and it is often a critical step in developing dynamic models or designing control. Unfortunately, the mathematical formulation of numerical differentiation is typically ill-posed, and researchers often resort to an \textit{ad hoc} process for choosing one of many computational metho… ▽ More Computing derivatives of noisy measurement data is ubiquitous in the physical, engineering, and biological sciences, and it is often a critical step in developing dynamic models or designing control. Unfortunately, the mathematical formulation of numerical differentiation is typically ill-posed, and researchers often resort to an \textit{ad hoc} process for choosing one of many computational methods and its parameters. In this work, we take a principled approach and propose a multi-objective optimization framework for choosing parameters that minimize a loss function to balance the faithfulness and smoothness of the derivative estimate. Our framework has three significant advantages. First, the task of selecting multiple parameters is reduced to choosing a single hyper-parameter. Second, where ground-truth data is unknown, we provide a heuristic for automatically selecting this hyper-parameter based on the power spectrum and temporal resolution of the data. Third, the optimal value of the hyper-parameter is consistent across different differentiation methods, thus our approach unifies vastly different numerical differentiation methods and facilitates unbiased comparison of their results. Finally, we provide an extensive open-source Python library \texttt{pynumdiff} to facilitate easy application to diverse datasets (https://github.com/florisvb/PyNumDiff). △ Less

Submitted 7 September, 2020; v1 submitted 3 September, 2020; originally announced September 2020.

Comments: 14 pages, 7 figures

Inferring Causal Networks of Dynamical Systems through Transient Dynamics and Perturbation

Authors: George Stepaniants, Bingni W. Brunton, J. Nathan Kutz

Abstract: Inferring causal relations from time series measurements is an ill-posed mathematical problem, where typically an infinite number of potential solutions can reproduce the given data. We explore in depth a strategy to disambiguate between possible underlying causal networks by perturbing the network, where the actuations are either targeted or applied at random. The resulting transient dynamics pro… ▽ More Inferring causal relations from time series measurements is an ill-posed mathematical problem, where typically an infinite number of potential solutions can reproduce the given data. We explore in depth a strategy to disambiguate between possible underlying causal networks by perturbing the network, where the actuations are either targeted or applied at random. The resulting transient dynamics provide the critical information necessary to infer causality. Two methods are shown to provide accurate causal reconstructions: Granger causality (GC) with perturbations, and our proposed perturbation cascade inference (PCI). Perturbed GC is capable of inferring smaller networks under low coupling strength regimes. Our proposed PCI method demonstrated consistently strong performance in inferring causal relations for small (2-5 node) and large (10-20 node) networks, with both linear and nonlinear dynamics. Thus the ability to apply a large and diverse set of perturbations/actuations to the network is critical for successfully and accurately determining causal relations and disambiguating between various viable networks. △ Less

Submitted 24 June, 2020; v1 submitted 23 June, 2020; originally announced June 2020.

Comments: 11 pages, 9 figures

MSC Class: 37M10; 62D20; 62M10

Journal ref: Phys. Rev. E 102, 042309 (2020)

Centering Data Improves the Dynamic Mode Decomposition

Authors: Seth M. Hirsh, Kameron Decker Harris, J. Nathan Kutz, Bingni W. Brunton

Abstract: Dynamic mode decomposition (DMD) is a data-driven method that models high-dimensional time series as a sum of spatiotemporal modes, where the temporal modes are constrained by linear dynamics. For nonlinear dynamical systems exhibiting strongly coherent structures, DMD can be a useful approximation to extract dominant, interpretable modes. In many domains with large spatiotemporal data---including… ▽ More Dynamic mode decomposition (DMD) is a data-driven method that models high-dimensional time series as a sum of spatiotemporal modes, where the temporal modes are constrained by linear dynamics. For nonlinear dynamical systems exhibiting strongly coherent structures, DMD can be a useful approximation to extract dominant, interpretable modes. In many domains with large spatiotemporal data---including fluid dynamics, video processing, and finance---the dynamics of interest are often perturbations about fixed points or equilibria, which motivates the application of DMD to centered (i.e. mean-subtracted) data. In this work, we show that DMD with centered data is equivalent to incorporating an affine term in the dynamic model and is not equivalent to computing a discrete Fourier transform. Importantly, DMD with centering can always be used to compute eigenvalue spectra of the dynamics. However, in many cases DMD without centering cannot model the corresponding dynamics, most notably if the dynamics have full effective rank. Additionally, we generalize the notion of centering to extracting arbitrary, but known, fixed frequencies from the data. We corroborate these theoretical results numerically on three nonlinear examples: the Lorenz system, a surveillance video, and brain recordings. Since centering the data is simple and computationally efficient, we recommend it as a preprocessing step before DMD; furthermore, we suggest that it can be readily used in conjunction with many other popular implementations of the DMD algorithm. △ Less

Submitted 13 June, 2019; originally announced June 2019.

Data-driven Spatiotemporal Modal Decomposition for Time Frequency Analysis

Authors: Seth M. Hirsh, Bingni W. Brunton, J. Nathan Kutz

Abstract: We propose a new solution to the blind source separation problem that factors mixed time-series signals into a sum of spatiotemporal modes, with the constraint that the temporal components are intrinsic mode functions (IMF's). The key motivation is that IMF's allow the computation of meaningful Hilbert transforms of non-stationary data, from which instantaneous time-frequency representations may b… ▽ More We propose a new solution to the blind source separation problem that factors mixed time-series signals into a sum of spatiotemporal modes, with the constraint that the temporal components are intrinsic mode functions (IMF's). The key motivation is that IMF's allow the computation of meaningful Hilbert transforms of non-stationary data, from which instantaneous time-frequency representations may be derived. Our spatiotemporal intrinsic mode decomposition (STIMD) method leverages spatial correlations to generalize the extraction of IMF's from one-dimensional signals, commonly performed using the empirical mode decomposition (EMD), to multi-dimensional signals. Further, this data-driven method enables future-state prediction. We demonstrate STIMD on several synthetic examples, comparing it to common matrix factorization techniques, namely singular value decomposition (SVD), independent component analysis (ICA), and dynamic mode decomposition (DMD). We show that STIMD outperforms these methods at reconstruction and extracting interpretable modes. Next, we apply STIMD to analyze two real-world datasets, gravitational wave data and neural recordings from the rodent hippocampus. △ Less

Submitted 22 June, 2018; originally announced June 2018.

Comments: 23 pages, 10 figures

Neural-inspired sensors enable sparse, efficient classification of spatiotemporal data

Authors: Thomas L. Mohren, Thomas L. Daniel, Steven L. Brunton, Bingni W. Brunton

Abstract: Sparse sensor placement is a central challenge in the efficient characterization of complex systems when the cost of acquiring and processing data is high. Leading sparse sensing methods typically exploit either spatial or temporal correlations, but rarely both. This work introduces a new sparse sensor optimization that is designed to leverage the rich spatiotemporal coherence exhibited by many sy… ▽ More Sparse sensor placement is a central challenge in the efficient characterization of complex systems when the cost of acquiring and processing data is high. Leading sparse sensing methods typically exploit either spatial or temporal correlations, but rarely both. This work introduces a new sparse sensor optimization that is designed to leverage the rich spatiotemporal coherence exhibited by many systems. Our approach is inspired by the remarkable performance of flying insects, which use a few embedded strain-sensitive neurons to achieve rapid and robust flight control despite large gust disturbances. Specifically, we draw on nature to identify targeted neural-inspired sensors on a flapping wing to detect body rotation. This task is particularly challenging as the rotational twisting mode is three orders-of-magnitude smaller than the flapping modes. We show that nonlinear filtering in time, built to mimic strain-sensitive neurons, is essential to detect rotation, whereas instantaneous measurements fail. Optimized sparse sensor placement results in efficient classification with approximately ten sensors, achieving the same accuracy and noise robustness as full measurements consisting of hundreds of sensors. Sparse sensing with neural inspired encoding establishes a new paradigm in hyper-efficient, embodied sensing of spatiotemporal data and sheds light on principles of biological sensing for agile flight control. △ Less

Submitted 20 April, 2018; originally announced April 2018.

Comments: 21 pages, 19 figures

Data-Driven Sparse Sensor Placement for Reconstruction

Authors: Krithika Manohar, Bingni W. Brunton, J. Nathan Kutz, Steven L. Brunton

Abstract: Optimal sensor placement is a central challenge in the design, prediction, estimation, and control of high-dimensional systems. High-dimensional states can often leverage a latent low-dimensional representation, and this inherent compressibility enables sparse sensing. This article explores optimized sensor placement for signal reconstruction based on a tailored library of features extracted from… ▽ More Optimal sensor placement is a central challenge in the design, prediction, estimation, and control of high-dimensional systems. High-dimensional states can often leverage a latent low-dimensional representation, and this inherent compressibility enables sparse sensing. This article explores optimized sensor placement for signal reconstruction based on a tailored library of features extracted from training data. Sparse point sensors are discovered using the singular value decomposition and QR pivoting, which are two ubiquitous matrix computations that underpin modern linear dimensionality reduction. Sparse sensing in a tailored basis is contrasted with compressed sensing, a universal signal recovery method in which an unknown signal is reconstructed via a sparse representation in a universal basis. Although compressed sensing can recover a wider class of signals, we demonstrate the benefits of exploiting known patterns in data with optimized sensing. In particular, drastic reductions in the required number of sensors and improved reconstruction are observed in examples ranging from facial images to fluid vorticity fields. Principled sensor placement may be critically enabling when sensors are costly and provides faster state estimation for low-latency, high-bandwidth control. MATLAB code is provided for all examples. △ Less

Submitted 18 August, 2017; v1 submitted 25 January, 2017; originally announced January 2017.

Comments: 34 pages, 17 figures

Journal ref: IEEE Control Syst. Mag. 38 (2018) 63-86

Chaos as an Intermittently Forced Linear System

Authors: Steven L. Brunton, Bingni W. Brunton, Joshua L. Proctor, Eurika Kaiser, J. Nathan Kutz

Abstract: Understanding the interplay of order and disorder in chaotic systems is a central challenge in modern quantitative science. We present a universal, data-driven decomposition of chaos as an intermittently forced linear system. This work combines Takens' delay embedding with modern Koopman operator theory and sparse regression to obtain linear representations of strongly nonlinear dynamics. The resu… ▽ More Understanding the interplay of order and disorder in chaotic systems is a central challenge in modern quantitative science. We present a universal, data-driven decomposition of chaos as an intermittently forced linear system. This work combines Takens' delay embedding with modern Koopman operator theory and sparse regression to obtain linear representations of strongly nonlinear dynamics. The result is a decomposition of chaotic dynamics into a linear model in the leading delay coordinates with forcing by low energy delay coordinates; we call this the Hankel alternative view of Koopman (HAVOK) analysis. This analysis is applied to the canonical Lorenz system, as well as to real-world examples such as the Earth's magnetic field reversal, and data from electrocardiogram, electroencephalogram, and measles outbreaks. In each case, the forcing statistics are non-Gaussian, with long tails corresponding to rare events that trigger intermittent switching and bursting phenomena; this forcing is highly predictive, providing a clear signature that precedes these events. Moreover, the activity of the forcing signal demarcates large coherent regions of phase space where the dynamics are approximately linear from those that are strongly nonlinear. △ Less

Submitted 18 August, 2016; originally announced August 2016.

Comments: 34 Pages, 23 Figures, 4 Tables

MSC Class: 37XX; 70G60; 65Pxx; 34C28; 37D45; 93B30

Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control

Authors: Steven L. Brunton, Bingni W. Brunton, Joshua L. Proctor, J. Nathan Kutz

Abstract: In this work, we explore finite-dimensional linear representations of nonlinear dynamical systems by restricting the Koopman operator to an invariant subspace. The Koopman operator is an infinite-dimensional linear operator that evolves observable functions of the state-space of a dynamical system [Koopman 1931, PNAS]. Dominant terms in the Koopman expansion are typically computed using dynamic mo… ▽ More In this work, we explore finite-dimensional linear representations of nonlinear dynamical systems by restricting the Koopman operator to an invariant subspace. The Koopman operator is an infinite-dimensional linear operator that evolves observable functions of the state-space of a dynamical system [Koopman 1931, PNAS]. Dominant terms in the Koopman expansion are typically computed using dynamic mode decomposition (DMD). DMD uses linear measurements of the state variables, and it has recently been shown that this may be too restrictive for nonlinear systems [Williams et al. 2015, JNLS]. Choosing nonlinear observable functions to form an invariant subspace where it is possible to obtain linear models, especially those that are useful for control, is an open challenge. Here, we investigate the choice of observable functions for Koopman analysis that enable the use of optimal linear control techniques on nonlinear problems. First, to include a cost on the state of the system, as in linear quadratic regulator (LQR) control, it is helpful to include these states in the observable subspace, as in DMD. However, we find that this is only possible when there is a single isolated fixed point, as systems with multiple fixed points or more complicated attractors are not globally topologically conjugate to a finite-dimensional linear system, and cannot be represented by a finite-dimensional linear Koopman subspace that includes the state. We then present a data-driven strategy to identify relevant observable functions for Koopman analysis using a new algorithm to determine terms in a dynamical system by sparse regression of the data in a nonlinear function space [Brunton et al. 2015, arxiv]; we show how this algorithm is related to DMD. Finally, we demonstrate how to design optimal control laws for nonlinear systems using techniques from linear optimal control on Koopman invariant subspaces. △ Less

Submitted 23 December, 2015; v1 submitted 10 October, 2015; originally announced October 2015.

Comments: 20 pages, 5 figures, 2 codes