Showing 1–50 of 72 results for author: Gottwald, G A

Choosing observables that capture critical slowing down before tipping points: A Fokker-Planck operator approach

Authors: Johannes Lohmann, Georg A. Gottwald

Abstract: Tipping points (TP) are abrupt transitions between metastable states in complex systems, most often described by a bifurcation or crisis of a multistable system induced by a slowly changing control parameter. An avenue for predicting TPs in real-world systems is critical slowing down (CSD), which is a decrease in the relaxation rate after perturbations prior to a TP that can be measured by statist… ▽ More Tipping points (TP) are abrupt transitions between metastable states in complex systems, most often described by a bifurcation or crisis of a multistable system induced by a slowly changing control parameter. An avenue for predicting TPs in real-world systems is critical slowing down (CSD), which is a decrease in the relaxation rate after perturbations prior to a TP that can be measured by statistical early warning signals (EWS) in the autocovariance of observational time series. In high-dimensional systems, we cannot expect a priori chosen scalar observables to show significant EWS, and some may even show an opposite signal. Thus, to avoid false negative or positive early warnings, it is desirable to monitor fluctuations only in observables that are designed to capture CSD. Here we propose that a natural observable for this purpose can be obtained by a data-driven approximation of the first non-trivial eigenfunction of the backward Fokker-Planck (or Kolmogorov) operator, using the diffusion map algorithm. △ Less

Submitted 13 June, 2025; originally announced June 2025.

Learning Chaotic Dynamics with Neuromorphic Network Dynamics

Authors: Yinhao Xu, Georg A. Gottwald, Zdenka Kuncic

Abstract: This study investigates how dynamical systems may be learned and modelled with a neuromorphic network which is itself a dynamical system. The neuromorphic network used in this study is based on a complex electrical circuit comprised of memristive elements that produce neuro-synaptic nonlinear responses to input electrical signals. To determine how computation may be performed using the physics of… ▽ More This study investigates how dynamical systems may be learned and modelled with a neuromorphic network which is itself a dynamical system. The neuromorphic network used in this study is based on a complex electrical circuit comprised of memristive elements that produce neuro-synaptic nonlinear responses to input electrical signals. To determine how computation may be performed using the physics of the underlying system, the neuromorphic network was simulated and evaluated on autonomous prediction of a multivariate chaotic time series, implemented with a reservoir computing framework. Through manipulating only input electrodes and voltages, optimal nonlinear dynamical responses were found when input voltages maximise the number of memristive components whose internal dynamics explore the entire dynamical range of the memristor model. Increasing the network coverage with the input electrodes was found to suppress other nonlinear responses that are less conducive to learning. These results provide valuable insights into how a practical neuromorphic network device can be optimised for learning complex dynamical systems using only external control parameters. △ Less

Submitted 12 June, 2025; originally announced June 2025.

Comments: 37 pages, 22 figures

UNO: Unlearning via Orthogonalization in Generative models

Authors: Pinak Mandal, Georg A. Gottwald

Abstract: As generative models become increasingly powerful and pervasive, the ability to unlearn specific data, whether due to privacy concerns, legal requirements, or the correction of harmful content, has become increasingly important. Unlike in conventional training, where data are accumulated and knowledge is reinforced, unlearning aims to selectively remove the influence of particular data points with… ▽ More As generative models become increasingly powerful and pervasive, the ability to unlearn specific data, whether due to privacy concerns, legal requirements, or the correction of harmful content, has become increasingly important. Unlike in conventional training, where data are accumulated and knowledge is reinforced, unlearning aims to selectively remove the influence of particular data points without costly retraining from scratch. To be effective and reliable, such algorithms need to achieve (i) forgetting of the undesired data, (ii) preservation of the quality of the generation, (iii) preservation of the influence of the desired training data on the model parameters, and (iv) small number of training steps. We propose fast unlearning algorithms based on loss gradient orthogonalization. We show that our algorithms are able to forget data while maintaining the fidelity of the original model. Using MNIST and CelebA data, we demonstrate that our algorithms achieve orders of magnitude faster unlearning times than their predecessors, such as gradient surgery. △ Less

Submitted 5 June, 2025; originally announced June 2025.

Dynamic Reservoir Computing with Physical Neuromorphic Networks

Authors: Yinhao Xu, Georg A. Gottwald, Zdenka Kuncic

Abstract: Reservoir Computing (RC) with physical systems requires an understanding of the underlying structure and internal dynamics of the specific physical reservoir. In this study, physical nano-electronic networks with neuromorphic dynamics are investigated for their use as physical reservoirs in an RC framework. These neuromorphic networks operate as dynamic reservoirs, with node activities in general… ▽ More Reservoir Computing (RC) with physical systems requires an understanding of the underlying structure and internal dynamics of the specific physical reservoir. In this study, physical nano-electronic networks with neuromorphic dynamics are investigated for their use as physical reservoirs in an RC framework. These neuromorphic networks operate as dynamic reservoirs, with node activities in general coupled to the edge dynamics through nonlinear nano-electronic circuit elements, and the reservoir outputs influenced by the underlying network connectivity structure. This study finds that networks with varying degrees of sparsity generate more useful nonlinear temporal outputs for dynamic RC compared to dense networks. Dynamic RC is also tested on an autonomous multivariate chaotic time series prediction task with networks of varying densities, which revealed the importance of network sparsity in maintaining network activity and overall dynamics, that in turn enabled the learning of the chaotic Lorenz63 system's attractor behavior. △ Less

Submitted 22 May, 2025; originally announced May 2025.

Comments: 8 pages, 8 figures, IJCNN 2025, accepted

Localized Diffusion Models for High Dimensional Distributions Generation

Authors: Georg A. Gottwald, Shuigen Liu, Youssef Marzouk, Sebastian Reich, Xin T. Tong

Abstract: Diffusion models are the state-of-the-art tools for various generative tasks. However, estimating high-dimensional score functions makes them potentially suffer from the curse of dimensionality (CoD). This underscores the importance of better understanding and exploiting low-dimensional structure in the target distribution. In this work, we consider locality structure, which describes sparse depen… ▽ More Diffusion models are the state-of-the-art tools for various generative tasks. However, estimating high-dimensional score functions makes them potentially suffer from the curse of dimensionality (CoD). This underscores the importance of better understanding and exploiting low-dimensional structure in the target distribution. In this work, we consider locality structure, which describes sparse dependencies between model components. Under locality structure, the score function is effectively low-dimensional, so that it can be estimated by a localized neural network with significantly reduced sample complexity. This motivates the localized diffusion model, where a localized score matching loss is used to train the score function within a localized hypothesis space. We prove that such localization enables diffusion models to circumvent CoD, at the price of additional localization error. Under realistic sample size scaling, we show both theoretically and numerically that a moderate localization radius can balance the statistical and localization error, leading to a better overall performance. The localized structure also facilitates parallel training of diffusion models, making it potentially more efficient for large-scale applications. △ Less

Submitted 7 May, 2025; originally announced May 2025.

Learning dynamical systems with hit-and-run random feature maps

Authors: Pinak Mandal, Georg A. Gottwald

Abstract: We show how random feature maps can be used to forecast dynamical systems with excellent forecasting skill. We consider the tanh activation function and judiciously choose the internal weights in a data-driven manner such that the resulting features explore the nonlinear, non-saturated regions of the activation function. We introduce skip connections and construct a deep variant of random feature… ▽ More We show how random feature maps can be used to forecast dynamical systems with excellent forecasting skill. We consider the tanh activation function and judiciously choose the internal weights in a data-driven manner such that the resulting features explore the nonlinear, non-saturated regions of the activation function. We introduce skip connections and construct a deep variant of random feature maps by combining several units. To mitigate the curse of dimensionality, we introduce localization where we learn local maps, employing conditional independence. Our modified random feature maps provide excellent forecasting skill for both single trajectory forecasts as well as long-time estimates of statistical properties, for a range of chaotic dynamical systems with dimensions up to 512. In contrast to other methods such as reservoir computers which require extensive hyperparameter tuning, we effectively need to tune only a single hyperparameter, and are able to achieve state-of-the-art forecast skill with much smaller networks. △ Less

Submitted 30 March, 2025; v1 submitted 11 January, 2025; originally announced January 2025.

A modified Hegselmann-Krause model for interacting voters and political parties

Authors: Patrick H. Cahill, Georg A. Gottwald

Abstract: The Hegselmann--Krause model is a prototypical model for opinion dynamics. It models the stochastic time evolution of an agent's or voter's opinion in response to the opinion of other like-minded agents. The Hegselmann--Krause model only considers the opinions of voters; we extend it here by incorporating the dynamics of political parties which influence and are influenced by the voters. We show i… ▽ More The Hegselmann--Krause model is a prototypical model for opinion dynamics. It models the stochastic time evolution of an agent's or voter's opinion in response to the opinion of other like-minded agents. The Hegselmann--Krause model only considers the opinions of voters; we extend it here by incorporating the dynamics of political parties which influence and are influenced by the voters. We show in numerical simulations for $1$- and $2$-dimensional opinion spaces that, as for the original Hegselmann--Krause model, the modified model exhibits opinion cluster formation as well as a phase transition from disagreement to consensus. We provide an analytical sufficient condition for the formation of unanimous consensus in which voters and parties collapse to the same point in opinion space in the deterministic case. Using mean-field theory, we further derive an approximation for the critical noise strength delineating consensus from non-consensus in the stochastically driven modified Hegselmann--Krause model. We compare our analytical findings with simulations of the modified Hegselmann--Krause model. △ Less

Submitted 24 February, 2025; v1 submitted 17 October, 2024; originally announced October 2024.

Localized Schrödinger Bridge Sampler

Authors: Georg A. Gottwald, Sebastian Reich

Abstract: We consider the problem of sampling from an unknown distribution for which only a sufficiently large number of training samples are available. In this paper, we build on previous work combining Schrödinger bridges and plug & play Langevin samplers. A key bottleneck of these approaches is the exponential dependence of the required training samples on the dimension, $d$, of the ambient state space.… ▽ More We consider the problem of sampling from an unknown distribution for which only a sufficiently large number of training samples are available. In this paper, we build on previous work combining Schrödinger bridges and plug & play Langevin samplers. A key bottleneck of these approaches is the exponential dependence of the required training samples on the dimension, $d$, of the ambient state space. We propose a localization strategy which exploits conditional independence of conditional expectation values. Localization thus replaces a single high-dimensional Schrödinger bridge problem by $d$ low-dimensional Schrödinger bridge problems over the available training samples. In this context, a connection to multi-head self attention transformer architectures is established. As for the original Schrödinger bridge sampling approach, the localized sampler is stable and geometric ergodic. The sampler also naturally extends to conditional sampling and to Bayesian inference. We demonstrate the performance of our proposed scheme through experiments on a high-dimensional Gaussian problem, on a temporal stochastic process, and on a stochastic subgrid-scale parametrization conditional sampling problem. We also extend the idea of localization to plug & play Langevin samplers using kernel-based denoising in combination with Tweedie's formula. △ Less

Submitted 17 November, 2024; v1 submitted 12 September, 2024; originally announced September 2024.

MSC Class: 60H10; 62F15; 62F30; 65C05; 65C40

On the choice of the non-trainable internal weights in random feature maps

Authors: Pinak Mandal, Georg A. Gottwald, Nicholas Cranch

Abstract: The computationally cheap machine learning architecture of random feature maps can be viewed as a single-layer feedforward network in which the weights of the hidden layer are random but fixed and only the outer weights are learned via linear regression. The internal weights are typically chosen from a prescribed distribution. The choice of the internal weights significantly impacts the accuracy o… ▽ More The computationally cheap machine learning architecture of random feature maps can be viewed as a single-layer feedforward network in which the weights of the hidden layer are random but fixed and only the outer weights are learned via linear regression. The internal weights are typically chosen from a prescribed distribution. The choice of the internal weights significantly impacts the accuracy of random feature maps. We address here the task of how to best select the internal weights. In particular, we consider the forecasting problem whereby random feature maps are used to learn a one-step propagator map for a dynamical system. We provide a computationally cheap hit-and-run algorithm to select good internal weights which lead to good forecasting skill. We show that the number of good features is the main factor controlling the forecasting skill of random feature maps and acts as an effective feature dimension. Lastly, we compare random feature maps with single-layer feedforward neural networks in which the internal weights are now learned using gradient descent. We find that random feature maps have superior forecasting capabilities whilst having several orders of magnitude lower computational cost. △ Less

Submitted 28 April, 2025; v1 submitted 7 August, 2024; originally announced August 2024.

Comments: 27 pages, 23 figures

Stable generative modeling using Schrödinger bridges

Authors: Georg A. Gottwald, Fengyi Li, Youssef Marzouk, Sebastian Reich

Abstract: We consider the problem of sampling from an unknown distribution for which only a sufficiently large number of training samples are available. Such settings have recently drawn considerable interest in the context of generative modelling and Bayesian inference. In this paper, we propose a generative model combining Schrödinger bridges and Langevin dynamics. Schrödinger bridges over an appropriate… ▽ More We consider the problem of sampling from an unknown distribution for which only a sufficiently large number of training samples are available. Such settings have recently drawn considerable interest in the context of generative modelling and Bayesian inference. In this paper, we propose a generative model combining Schrödinger bridges and Langevin dynamics. Schrödinger bridges over an appropriate reversible reference process are used to approximate the conditional transition probability from the available training samples, which is then implemented in a discrete-time reversible Langevin sampler to generate new samples. By setting the kernel bandwidth in the reference process to match the time step size used in the unadjusted Langevin algorithm, our method effectively circumvents any stability issues typically associated with the time-stepping of stiff stochastic differential equations. Moreover, we introduce a novel split-step scheme, ensuring that the generated samples remain within the convex hull of the training samples. Our framework can be naturally extended to generate conditional samples and to Bayesian inference problems. We demonstrate the performance of our proposed scheme through experiments on synthetic datasets with increasing dimensions and on a stochastic subgrid-scale parametrization conditional sampling problem as well as generating sample trajectories of a dynamical system using conditional sampling. △ Less

Submitted 23 October, 2024; v1 submitted 9 January, 2024; originally announced January 2024.

MSC Class: 60H10; 62F15; 62F30; 65C05; 65C40

A stochastic approximation for the finite-size Kuramoto-Sakaguchi model

Authors: Wenqi Yue, Georg A. Gottwald

Abstract: We perform a stochastic model reduction of the Kuramoto-Sakaguchi model for finitely many coupled phase oscillators with phase frustration. Whereas in the thermodynamic limit coupled oscillators exhibit stationary states and a constant order parameter, finite-size networks exhibit persistent temporal fluctuations of the order parameter. These fluctuations are caused by the interaction of the synch… ▽ More We perform a stochastic model reduction of the Kuramoto-Sakaguchi model for finitely many coupled phase oscillators with phase frustration. Whereas in the thermodynamic limit coupled oscillators exhibit stationary states and a constant order parameter, finite-size networks exhibit persistent temporal fluctuations of the order parameter. These fluctuations are caused by the interaction of the synchronized oscillators with the non-entrained oscillators. We present numerical results suggesting that the collective effect of the non-entrained oscillators on the synchronized cluster can be approximated by a Gaussian process. This allows for an effective closed evolution equation for the synchronized oscillators driven by a Gaussian process which we approximate by a two-dimensional Ornstein-Uhlenbeck process. Our reduction reproduces the stochastic fluctuations of the order parameter and leads to a simple stochastic differential equation for the order parameter. △ Less

Submitted 11 May, 2024; v1 submitted 30 October, 2023; originally announced October 2023.

Data assimilation for networks of coupled oscillators: Inferring unknown model parameters from partial observations

Authors: Lauren D. Smith, Georg A. Gottwald

Abstract: Inferring the state and unknown parameters of a network of coupled oscillators is of utmost importance. This task is made harder when only partial and noisy observations are available, which is a typical scenario in realistic high-dimensional systems. The general task of inference falls under data assimilation, and a commonly used assimilation method is the Ensemble Kalman Filter. Employing networ… ▽ More Inferring the state and unknown parameters of a network of coupled oscillators is of utmost importance. This task is made harder when only partial and noisy observations are available, which is a typical scenario in realistic high-dimensional systems. The general task of inference falls under data assimilation, and a commonly used assimilation method is the Ensemble Kalman Filter. Employing network-specific localization of the forecast covariance, an Ensemble Kalman Filter with state space augmentation is shown to yield highly accurate estimates of both the oscillator phases and unknown model parameters in the case where only a subset of oscillator phases are observed. In contrast, standard data assimilation methods yield poor results. We demonstrate the effectiveness of our approach for Kuramoto oscillators and for networks of theta neurons, using a variety of network topologies. △ Less

Submitted 3 April, 2025; v1 submitted 7 September, 2023; originally announced September 2023.

Canards in modified equations for Euler discretizations

Authors: Maximilian Engel, Georg A. Gottwald

Abstract: Canards are a well-studied phenomenon in fast-slow ordinary differential equations implying the delayed loss of stability after the slow passage through a singularity. Recent studies have shown that the corresponding maps stemming from explicit Runge-Kutta discretizations, in particular the forward Euler scheme, exhibit significant distinctions to the continuous-time behavior: for folds, the delay… ▽ More Canards are a well-studied phenomenon in fast-slow ordinary differential equations implying the delayed loss of stability after the slow passage through a singularity. Recent studies have shown that the corresponding maps stemming from explicit Runge-Kutta discretizations, in particular the forward Euler scheme, exhibit significant distinctions to the continuous-time behavior: for folds, the delay in loss of stability is typically shortened whereas, for transcritical singularities, it is arbitrarily prolonged. We employ the method of modified equations, which correspond with the fixed discretization schemes up to higher order, to understand and quantify these effects directly from a fast-slow ODE, yielding consistent results with the discrete-time behavior and opening a new perspective on the wide range of (de-)stabilization phenomena along canards. △ Less

Submitted 18 April, 2023; originally announced April 2023.

Lévy flights as an emergent phenomenon in a spatially extended system

Authors: Chunxi Jiao, Georg A. Gottwald

Abstract: Anomalous diffusion and Lévy flights, which are characterized by the occurrence of random discrete jumps of all scales, have been observed in a plethora of natural and engineered systems, ranging from the motion of molecules to climate signals. Mathematicians have recently unveiled mechanisms to generate anomalous diffusion, both stochastically and deterministically. However, there exists to the b… ▽ More Anomalous diffusion and Lévy flights, which are characterized by the occurrence of random discrete jumps of all scales, have been observed in a plethora of natural and engineered systems, ranging from the motion of molecules to climate signals. Mathematicians have recently unveiled mechanisms to generate anomalous diffusion, both stochastically and deterministically. However, there exists to the best of our knowledge no explicit example of a spatially extended system which exhibits anomalous diffusion without being explicitly driven by Lévy noise. We show here that the Landau-Lifshitz-Gilbert equation, a stochastic partial differential equation (SPDE), despite only driven by Gaussian white noise, exhibits superdiffusive behaviour. The anomalous diffusion is an entirely emergent behaviour and manifests itself in jumps in the location of its travelling front solution. Using a collective coordinate approach we reduce the SPDE to a set of stochastic differential equations (SDEs) driven by Gaussian white noise. This allows us to identify the mechanism giving rise to the anomalous diffusion as random widening events of the front interface. △ Less

Submitted 1 September, 2023; v1 submitted 3 September, 2022; originally announced September 2022.

SINDy for delay-differential equations: application to model bacterial zinc response

Authors: Antoine Sandoz, Verena Ducret, Georg A. Gottwald, Gilles Vilmart, Karl Perron

Abstract: We extend the data-driven method of Sparse Identification of Nonlinear Dynamics (SINDy) developed by Brunton et al, Proc. Natl. Acad. Sci USA 113 (2016) to the case of delay differential equations (DDEs). This is achieved in a bilevel optimization procedure by first applying SINDy for fixed delay and then subsequently optimizing the error of the reconstructed SINDy model over delay times. We test… ▽ More We extend the data-driven method of Sparse Identification of Nonlinear Dynamics (SINDy) developed by Brunton et al, Proc. Natl. Acad. Sci USA 113 (2016) to the case of delay differential equations (DDEs). This is achieved in a bilevel optimization procedure by first applying SINDy for fixed delay and then subsequently optimizing the error of the reconstructed SINDy model over delay times. We test the SINDy-delay method on a noisy short data set from a toy delay differential equation and show excellent agreement. We then apply the method to experimental data of gene expressions in the bacterium {\it Pseudomonas aeruginosa} subject to the influence of zinc. The derived SINDy model suggests that the increase of zinc concentration mainly affects the time delay and not the strengths of the interactions between the different agents controlling the zinc export mechanism. △ Less

Submitted 13 December, 2022; v1 submitted 23 August, 2022; originally announced August 2022.

Comments: 21 pages. To appear in Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences

MSC Class: 37N25

Heterogeneous nucleation in finite size adaptive dynamical networks

Authors: Jan Fialkowski, Serhiy Yanchuk, Igor M. Sokolov, Eckehard Schöll, Georg A. Gottwald, Rico Berner

Abstract: Phase transitions in equilibrium and nonequilibrium systems play a major role in the natural sciences. In dynamical networks, phase transitions organize qualitative changes in the collective behavior of coupled dynamical units. Adaptive dynamical networks feature a connectivity structure that changes over time, co-evolving with the nodes' dynamical state. In this Letter, we show the emergence of t… ▽ More Phase transitions in equilibrium and nonequilibrium systems play a major role in the natural sciences. In dynamical networks, phase transitions organize qualitative changes in the collective behavior of coupled dynamical units. Adaptive dynamical networks feature a connectivity structure that changes over time, co-evolving with the nodes' dynamical state. In this Letter, we show the emergence of two distinct first-order nonequilibrium phase transitions in a finite size adaptive network of heterogeneous phase oscillators. Depending on the nature of defects in the internal frequency distribution, we observe either an abrupt single-step transition to full synchronization or a more gradual multi-step transition. This observation has a striking resemblance to heterogeneous nucleation. We develop a mean-field approach to study the interplay between adaptivity and nodal heterogeneity and describe the dynamics of multicluster states and their role in determining the character of the phase transition. Our work provides a theoretical framework for studying the interplay between adaptivity and nodal heterogeneity. △ Less

Submitted 21 October, 2022; v1 submitted 6 July, 2022; originally announced July 2022.

On financial market correlation structures and diversification benefits across and within equity sectors

Authors: Nick James, Max Menzies, Georg A. Gottwald

Abstract: We study how to assess the potential benefit of diversifying an equity portfolio by investing within and across equity sectors. We analyse 20 years of US stock price data, which includes the global financial crisis (GFC) and the COVID-19 market crash, as well as periods of financial stability, to determine the `all weather' nature of equity portfolios. We establish that one may use the leading eig… ▽ More We study how to assess the potential benefit of diversifying an equity portfolio by investing within and across equity sectors. We analyse 20 years of US stock price data, which includes the global financial crisis (GFC) and the COVID-19 market crash, as well as periods of financial stability, to determine the `all weather' nature of equity portfolios. We establish that one may use the leading eigenvalue of the cross-correlation matrix of log returns as well as graph-theoretic diagnostics such as modularity to quantify the collective behaviour of the market or a subset of it. We confirm that financial crises are characterised by a high degree of collective behaviour of equities, whereas periods of financial stability exhibit less collective behaviour. We argue that during times of increased collective behaviour, risk reduction via sector-based portfolio diversification is ineffective. Using the degree of collectivity as a proxy for the benefit of diversification, we perform an extensive sampling of equity portfolios to confirm the old financial adage that 30-40 stocks provide sufficient diversification. Using hierarchical clustering, we discover a `best value' equity portfolio for diversification consisting of 36 equities sampled uniformly from 9 sectors. We further show that it is typically more beneficial to diversify across sectors rather than within. Our findings have implications for cost-conscious retail investors seeking broad diversification across equity markets. △ Less

Submitted 13 June, 2022; v1 submitted 21 February, 2022; originally announced February 2022.

Comments: Accepted manuscript. Minor edits and typos fixed since v1. Equal contribution

Journal ref: Physica A: Statistical Mechanics and its Applications 604 (2022) 127682

Combining machine learning and data assimilation to forecast dynamical systems from noisy partial observations

Authors: Georg A. Gottwald, Sebastian Reich

Abstract: We present a supervised learning method to learn the propagator map of a dynamical system from partial and noisy observations. In our computationally cheap and easy-to-implement framework a neural network consisting of random feature maps is trained sequentially by incoming observations within a data assimilation procedure. By employing Takens' embedding theorem, the network is trained on delay co… ▽ More We present a supervised learning method to learn the propagator map of a dynamical system from partial and noisy observations. In our computationally cheap and easy-to-implement framework a neural network consisting of random feature maps is trained sequentially by incoming observations within a data assimilation procedure. By employing Takens' embedding theorem, the network is trained on delay coordinates. We show that the combination of random feature maps and data assimilation, called RAFDA, outperforms standard random feature maps for which the dynamics is learned using batch data. △ Less

Submitted 2 September, 2021; v1 submitted 7 August, 2021; originally announced August 2021.

Mesoscopic model reduction for the collective dynamics of sparse coupled oscillator networks

Authors: Lauren D Smith, Georg A Gottwald

Abstract: The behavior at bifurcation from global synchronization to partial synchronization in finite networks of coupled oscillators is a complex phenomenon, involving the intricate dynamics of one or more oscillators with the remaining synchronized oscillators. This is not captured well by standard macroscopic model reduction techniques which capture only the collective behavior of synchronized oscillato… ▽ More The behavior at bifurcation from global synchronization to partial synchronization in finite networks of coupled oscillators is a complex phenomenon, involving the intricate dynamics of one or more oscillators with the remaining synchronized oscillators. This is not captured well by standard macroscopic model reduction techniques which capture only the collective behavior of synchronized oscillators in the thermodynamic limit. We introduce two mesoscopic model reductions for finite sparse networks of coupled oscillators to quantitatively capture the dynamics close to bifurcation from global to partial synchronization. Our model reduction builds upon the method of collective coordinates. We first show that standard collective coordinate reduction has difficulties capturing this bifurcation. We identify a particular topological structure at bifurcation consisting of a main synchronized cluster, the oscillator that desynchronizes at bifurcation, and an intermediary node connecting them. Utilizing this structure and ensemble averages we derive an analytic expression for the mismatch between the true bifurcation from global to partial synchronization and its estimate calculated via the collective coordinate approach. This allows to calibrate the standard collective coordinate approach without prior knowledge of which node will desynchronize. We introduce a second mesoscopic reduction, utilizing the same particular topological structure, which allows for a quantitative dynamical description of the phases near bifurcation. The mesoscopic reductions significantly reduce the computational complexity of the collective coordinate approach, reducing from $\mathcal{O}(N^2)$ to $\mathcal{O}(1)$. We perform numerical simulations for Erdős-Rényi networks and for modified Barabási-Albert networks demonstrating excellent quantitative agreement at and close to bifurcation. △ Less

Submitted 7 July, 2021; v1 submitted 15 April, 2021; originally announced April 2021.

Journal ref: Chaos, 31, 073116 (2021)

A collective coordinate framework to study solitary waves in stochastically perturbed Korteweg-de Vries equations

Authors: Madeleine Cartwright, Georg A. Gottwald

Abstract: Stochastically perturbed Korteweg-de Vries (KdV) equations are widely used to describe the effect of random perturbations on coherent solitary waves. We present a collective coordinate approach to describe the effect on coherent solitary waves in stochastically perturbed KdV equations. The collective coordinate approach allows one to reduce the infinite-dimensional stochastic partial differential… ▽ More Stochastically perturbed Korteweg-de Vries (KdV) equations are widely used to describe the effect of random perturbations on coherent solitary waves. We present a collective coordinate approach to describe the effect on coherent solitary waves in stochastically perturbed KdV equations. The collective coordinate approach allows one to reduce the infinite-dimensional stochastic partial differential equation (SPDE) to a finite-dimensional stochastic differential equation for the amplitude, width and location of the solitary wave. The reduction provides a remarkably good quantitative description of the shape of the solitary waves and its location. Moreover, the collective coordinate framework can be used to estimate the time-scale of validity of stochastically perturbed KdV equations for which they can be used to describe coherent solitary waves. We describe loss of coherence by blow-up as well as by radiation into linear waves. We corroborate our analytical results with numerical simulations of the full SPDE. △ Less

Submitted 19 July, 2021; v1 submitted 9 April, 2021; originally announced April 2021.

Journal ref: Phys. Rev. E 104, 024201 (2021)

Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation

Authors: Georg A. Gottwald, Sebastian Reich

Abstract: Data-driven prediction and physics-agnostic machine-learning methods have attracted increased interest in recent years achieving forecast horizons going well beyond those to be expected for chaotic dynamical systems. In a separate strand of research data-assimilation has been successfully used to optimally combine forecast models and their inherent uncertainty with incoming noisy observations. The… ▽ More Data-driven prediction and physics-agnostic machine-learning methods have attracted increased interest in recent years achieving forecast horizons going well beyond those to be expected for chaotic dynamical systems. In a separate strand of research data-assimilation has been successfully used to optimally combine forecast models and their inherent uncertainty with incoming noisy observations. The key idea in our work here is to achieve increased forecast capabilities by judiciously combining machine-learning algorithms and data assimilation. We combine the physics-agnostic data-driven approach of random feature maps as a forecast model within an ensemble Kalman filter data assimilation procedure. The machine-learning model is learned sequentially by incorporating incoming noisy observations. We show that the obtained forecast model has remarkably good forecast skill while being computationally cheap once trained. Going beyond the task of forecasting, we show that our method can be used to generate reliable ensembles for probabilistic forecasting as well as to learn effective model closure in multi-scale systems. △ Less

Submitted 8 March, 2021; v1 submitted 14 July, 2020; originally announced July 2020.

A model for Dansgaard-Oeschger events and millennial-scale abrupt climate change without external forcing

Authors: Georg A. Gottwald

Abstract: We propose a conceptual model which generates abrupt climate changes akin to Dansgaard-Oeschger events. In the model these abrupt climate changes are not triggered by external perturbations but rather emerge in a dynamic self-consistent model through complex interactions of the ocean, the atmosphere and an intermittent process. The abrupt climate changes are caused in our model by intermittencies… ▽ More We propose a conceptual model which generates abrupt climate changes akin to Dansgaard-Oeschger events. In the model these abrupt climate changes are not triggered by external perturbations but rather emerge in a dynamic self-consistent model through complex interactions of the ocean, the atmosphere and an intermittent process. The abrupt climate changes are caused in our model by intermittencies in the sea-ice cover. The ocean is represented by a Stommel two-box model, the atmosphere by a Lorenz-84 model and the sea-ice cover by a deterministic approximation of correlated additive and multiplicative noise (CAM) process. The key dynamical ingredients of the model are given by stochastic limits of deterministic multi-scale systems and recent results in deterministic homogenisation theory. The deterministic model reproduces statistical features of actual ice-core data such as non-Gaussian $α$-stable behaviour. The proposed mechanism for abrupt millenial-scale climate change only relies on the existence of a quantity, which exhibits intermittent dynamics on an intermediate time scale. We consider as a particular mechanism intermittent sea-ice cover where the intermittency is generated by emergent atmospheric noise. However, other mechanisms such as freshwater influxes may also be formulated within the proposed framework. △ Less

Submitted 23 September, 2020; v1 submitted 30 May, 2020; originally announced June 2020.

Simulation of non-Lipschitz stochastic differential equations driven by $α$-stable noise: a method based on deterministic homogenisation

Authors: Georg A. Gottwald, Ian Melbourne

Abstract: We devise an explicit method to integrate $α$-stable stochastic differential equations (SDEs) with non-Lipschitz coefficients. To mitigate against numerical instabilities caused by unbounded increments of the Lévy noise, we use a deterministic map which has the desired SDE as its homogenised limit. Moreover, our method naturally overcomes difficulties in expressing the Marcus integral explicitly.… ▽ More We devise an explicit method to integrate $α$-stable stochastic differential equations (SDEs) with non-Lipschitz coefficients. To mitigate against numerical instabilities caused by unbounded increments of the Lévy noise, we use a deterministic map which has the desired SDE as its homogenised limit. Moreover, our method naturally overcomes difficulties in expressing the Marcus integral explicitly. We present an example of an SDE with a natural boundary showing that our method respects the boundary whereas Euler-Maruyama discretisation fails to do so. As a by-product we devise an entirely deterministic method to construct $α$-stable laws. △ Less

Submitted 21 April, 2020; originally announced April 2020.

Journal ref: Multiscale Model. Simul. 19 (2021) 665-687

Model Reduction for the Kuramoto-Sakaguchi Model: The Importance of Non-entrained Rogue Oscillators

Authors: Wenqi Yue, Lachlan D. Smith, Georg A. Gottwald

Abstract: The Kuramoto-Sakaguchi model for coupled phase oscillators with phase-frustration is often studied in the thermodynamic limit of infinitely many oscillators. Here we extend a model reduction method based on collective coordinates to capture the collective dynamics of finite size Kuramoto-Sakaguchi models. We find that the inclusion of the effects of rogue oscillators is essential to obtain an accu… ▽ More The Kuramoto-Sakaguchi model for coupled phase oscillators with phase-frustration is often studied in the thermodynamic limit of infinitely many oscillators. Here we extend a model reduction method based on collective coordinates to capture the collective dynamics of finite size Kuramoto-Sakaguchi models. We find that the inclusion of the effects of rogue oscillators is essential to obtain an accurate description, in contrast to the original Kuramoto model where we show that their effects can be ignored. We further introduce a more accurate ansatz function to describe the shape of synchronized oscillators. Our results from this extended collective coordinate approach reduce in the thermodynamic limit to the well-known mean-field consistency relations. For finite networks we show that our model reduction describes the collective behavior accurately, reproducing the order parameter, the mean frequency of the synchronized cluster, and the size of the cluster at given coupling strength, as well as the critical coupling strength for partial and for global synchronization. △ Less

Submitted 1 June, 2020; v1 submitted 14 April, 2020; originally announced April 2020.

Journal ref: Phys. Rev. E 101, 062213 (2020)

Stochastically perturbed bred vectors

Authors: Brent Giggins, Georg A. Gottwald

Abstract: The breeding method is a computationally cheap procedure to generate initial conditions for ensemble forecasting which project onto relevant synoptic growing modes. Ensembles of bred vectors, however, often lack diversity and align with the leading Lyapunov vector, which severely impacts their statistical reliability. In previous work we developed stochastically perturbed bred vectors (SPBVs) and… ▽ More The breeding method is a computationally cheap procedure to generate initial conditions for ensemble forecasting which project onto relevant synoptic growing modes. Ensembles of bred vectors, however, often lack diversity and align with the leading Lyapunov vector, which severely impacts their statistical reliability. In previous work we developed stochastically perturbed bred vectors (SPBVs) and random draw bred vectors (RDBVs) in the context of multi-scale systems. Here we explore when this method can be extended to systems without scale separation, and examine the performance of the stochastically modified bred vectors in the single scale Lorenz 96 model. In particular, we show that the performance of SPBVs crucially depends on the degree of localisation of the bred vectors. It is found that, contrary to the case of multi-scale systems, localisation is detrimental for applications of SPBVs in systems without scale-separation when initialised from assimilated data. In the case of weakly localised bred vectors, however, ensembles of SPBVs constitute a reliable ensemble with improved ensemble forecasting skills compared to classical bred vectors, while still preserving the low computational cost of the breeding method. RDBVs are shown to have superior forecast skill and form a reliable ensemble in weakly localised situations, but in situations when they are strongly localised they do not constitute a reliable ensemble and are over-dispersive. △ Less

Submitted 9 August, 2020; v1 submitted 10 March, 2020; originally announced March 2020.

Model reduction for the collective dynamics of globally coupled oscillators: From finite networks to the thermodynamic limit

Authors: Lachlan D Smith, Georg A Gottwald

Abstract: Model reduction techniques have been widely used to study the collective behavior of globally coupled oscillators. However, most approaches assume that there are infinitely many oscillators. Here we propose a new ansatz, based on the collective coordinate approach, that reproduces the collective dynamics of the Kuramoto model for finite networks to high accuracy, yields the same bifurcation struct… ▽ More Model reduction techniques have been widely used to study the collective behavior of globally coupled oscillators. However, most approaches assume that there are infinitely many oscillators. Here we propose a new ansatz, based on the collective coordinate approach, that reproduces the collective dynamics of the Kuramoto model for finite networks to high accuracy, yields the same bifurcation structure in the thermodynamic limit of infinitely many oscillators as previous approaches, and additionally captures the dynamics of the order parameter in the thermodynamic limit, including critical slowing down that results from a cascade of saddle-node bifurcations. △ Less

Submitted 6 August, 2020; v1 submitted 1 October, 2019; originally announced October 2019.

Linear response for macroscopic observables in high-dimensional systems

Authors: Caroline L. Wormell, Georg A. Gottwald

Abstract: The long-term average response of observables of chaotic systems to dynamical perturbations can often be predicted using linear response theory, but not all chaotic systems possess a linear response. Macroscopic observables of complex dissipative chaotic systems, however, are widely assumed to have a linear response even if the microscopic variables do not, but the mechanism for this is not well-u… ▽ More The long-term average response of observables of chaotic systems to dynamical perturbations can often be predicted using linear response theory, but not all chaotic systems possess a linear response. Macroscopic observables of complex dissipative chaotic systems, however, are widely assumed to have a linear response even if the microscopic variables do not, but the mechanism for this is not well-understood. We present a comprehensive picture for the linear response of macroscopic observables in high-dimensional coupled deterministic dynamical systems, where the coupling is via a mean field and the microscopic subsystems may or may not obey linear response theory. We derive stochastic reductions of the dynamics of these observables from statistics of the microscopic system, and provide conditions for linear response theory to hold in finite dimensional systems and in the thermodynamic limit. In particular, we show that for large systems of finite size, linear response is induced via self-generated noise. We present examples in the thermodynamic limit where the macroscopic observable satisfies LRT, although the microscopic subsystems individually violate LRT, as well a converse example where the macroscopic observable does not satisfy LRT despite all microscopic subsystems satisfying LRT when uncoupled. This latter, maybe surprising, example is associated with emergent non-trivial dynamics of the macroscopic observable. We provide numerical evidence for our results on linear response as well as some analytical intuition. △ Less

Submitted 23 October, 2019; v1 submitted 31 July, 2019; originally announced July 2019.

MSC Class: 37F15 37L40

Chaos in networks of coupled oscillators with multimodal natural frequency distributions

Authors: Lachlan D. Smith, Georg A. Gottwald

Abstract: We explore chaos in the Kuramoto model with multimodal distributions of the natural frequencies of oscillators and provide a comprehensive description under what conditions chaos occurs. For a natural frequency distribution with $M$ peaks it is typical that there is a range of coupling strengths such that oscillators belonging to each peak form a synchronized cluster, but the clusters do not globa… ▽ More We explore chaos in the Kuramoto model with multimodal distributions of the natural frequencies of oscillators and provide a comprehensive description under what conditions chaos occurs. For a natural frequency distribution with $M$ peaks it is typical that there is a range of coupling strengths such that oscillators belonging to each peak form a synchronized cluster, but the clusters do not globally synchronize. We use collective coordinates to describe the inter- and intra-cluster dynamics, which reduces the Kuramoto model to $2M-1$ degrees of freedom. We show that under some assumptions, there is a time-scale splitting between the slow intracluster dynamics and fast intercluster dynamics, which reduces the collective coordinate model to an $M-1$ degree of freedom rescaled Kuramoto model. Therefore, four or more clusters are required to yield the three degrees of freedom necessary for chaos. However, the time-scale splitting breaks down if a cluster intermittently desynchronizes. We show that this intermittent desynchronization provides a mechanism for chaos for trimodal natural frequency distributions. In addition, we use collective coordinates to show analytically that chaos cannot occur for bimodal frequency distributions, even if they are asymmetric and if intermittent desynchronization occurs. △ Less

Submitted 4 October, 2019; v1 submitted 7 May, 2019; originally announced May 2019.

Journal ref: Chaos 29(9) 093127 (2019)

Detecting regime transitions in time series using dynamic mode decomposition

Authors: Georg A. Gottwald, Federica Gugole

Abstract: We employ the framework of the Koopman operator and dynamic mode decomposition to devise a computationally cheap and easily implementable method to detect transient dynamics and regime changes in time series. We argue that typically transient dynamics experiences the full state space dimension with subsequent fast relaxation towards the attractor. In equilibrium, on the other hand, the dynamics ev… ▽ More We employ the framework of the Koopman operator and dynamic mode decomposition to devise a computationally cheap and easily implementable method to detect transient dynamics and regime changes in time series. We argue that typically transient dynamics experiences the full state space dimension with subsequent fast relaxation towards the attractor. In equilibrium, on the other hand, the dynamics evolves on a slower time scale on a lower dimensional attractor. The reconstruction error of a dynamic mode decomposition is used to monitor the inability of the time series to resolve the fast relaxation towards the attractor as well as the effective dimension of the dynamics. We illustrate our method by detecting transient dynamics in the Kuramoto-Sivashinsky equation. We further apply our method to atmospheric reanalysis data; our diagnostics detects the transition from a predominantly negative North Atlantic Oscillation (NAO) to a predominantly positive NAO around 1970, as well as the recently found regime change in the Southern Hemisphere atmospheric circulation around 1970. △ Less

Submitted 3 September, 2019; v1 submitted 19 April, 2019; originally announced April 2019.

On the impossibility of solitary Rossby waves in meridionally unbounded domains

Authors: Georg A. Gottwald, Dmitry E. Pelinovsky

Abstract: Evolution of weakly nonlinear and slowly varying Rossby waves in planetary atmospheres and oceans is considered within the quasi-geostrophic equation on unbounded domains. When the mean flow profile has a jump in the ambient potential vorticity, localized eigenmodes are trapped by the mean flow with a non-resonant speed of propagation. We address amplitude equations for these modes. Whereas the li… ▽ More Evolution of weakly nonlinear and slowly varying Rossby waves in planetary atmospheres and oceans is considered within the quasi-geostrophic equation on unbounded domains. When the mean flow profile has a jump in the ambient potential vorticity, localized eigenmodes are trapped by the mean flow with a non-resonant speed of propagation. We address amplitude equations for these modes. Whereas the linear problem is suggestive of a two-dimensional Zakharov-Kuznetsov equation, we found that the dynamics of Rossby waves is effectively linear and moreover confined to zonal waveguides of the mean flow. This eliminates even the ubiquitous Korteweg-de Vries equations as underlying models for spatially localized coherent structures in these geophysical flows. △ Less

Submitted 7 August, 2018; originally announced August 2018.

A collective coordinate framework to study the dynamics of travelling waves in stochastic partial differential equations

Authors: Madeleine C. Cartwright, Georg A. Gottwald

Abstract: We propose a formal framework based on collective coordinates to reduce infinite-dimensional stochastic partial differential equations (SPDEs) with symmetry to a set of finite-dimensional stochastic differential equations which describe the shape of the solution and the dynamics along the symmetry group. We study SPDEs arising in population dynamics with multiplicative noise and additive symmetry… ▽ More We propose a formal framework based on collective coordinates to reduce infinite-dimensional stochastic partial differential equations (SPDEs) with symmetry to a set of finite-dimensional stochastic differential equations which describe the shape of the solution and the dynamics along the symmetry group. We study SPDEs arising in population dynamics with multiplicative noise and additive symmetry breaking noise. The collective coordinate approach provides a remarkably good quantitative description of the shape of the travelling front as well as its diffusive behaviour, which would otherwise only be available through costly computational experiments. We corroborate our analytical results with numerical simulations of the full SPDE. △ Less

Submitted 25 March, 2019; v1 submitted 19 June, 2018; originally announced June 2018.

Comments: includes now pathwise matching

Stochastic model reduction for slow-fast systems with moderate time-scale separation

Authors: Jeroen Wouters, Georg A. Gottwald

Abstract: We propose a stochastic model reduction strategy for deterministic and stochastic slow-fast systems with finite time-scale separation. The stochastic model reduction relaxes the assumption of infinite time-scale separation of classical homogenization theory by incorporating deviations from this limit as described by an Edgeworth expansion. A surrogate system is constructed the parameters of which… ▽ More We propose a stochastic model reduction strategy for deterministic and stochastic slow-fast systems with finite time-scale separation. The stochastic model reduction relaxes the assumption of infinite time-scale separation of classical homogenization theory by incorporating deviations from this limit as described by an Edgeworth expansion. A surrogate system is constructed the parameters of which are matched to produce the same Edgeworth expansions up to any desired order of the original multi-scale system. We corroborate our analytical findings by numerical examples, showing significant improvements to classical homogenized model reduction. △ Less

Submitted 16 April, 2018; originally announced April 2018.

Stochastically perturbed bred vectors in multi-scale systems

Authors: Brent Giggins, Georg A. Gottwald

Abstract: The breeding method is a computationally cheap way to generate flow-adapted ensembles to be used in probabilistic forecasts. Its main disadvantage is that the ensemble may lack diversity and collapse to a low-dimensional subspace. To still benefit from the breeding method's simplicity and its low computational cost, approaches are needed to increase the diversity of these bred vector (BV) ensemble… ▽ More The breeding method is a computationally cheap way to generate flow-adapted ensembles to be used in probabilistic forecasts. Its main disadvantage is that the ensemble may lack diversity and collapse to a low-dimensional subspace. To still benefit from the breeding method's simplicity and its low computational cost, approaches are needed to increase the diversity of these bred vector (BV) ensembles. We present here such a method tailored for multi-scale systems. We describe how to judiciously introduce stochastic perturbations to the standard bred vectors leading to stochastically perturbed bred vectors. The increased diversity leads to a better forecast skill as measured by the RMS error, as well as to more reliable ensembles quantified by the error-spread relationship, the continuous ranked probability score and reliability diagrams. Our approach is dynamically informed and in effect generates random draws from the fast equilibrium measure conditioned on the slow variables. We illustrate the advantage of stochastically perturbed bred vectors over standard BVs in numerical simulations of a multi-scale Lorenz 96 model. △ Less

Submitted 9 December, 2018; v1 submitted 24 April, 2018; originally announced April 2018.

Comments: accepted for publication in Q.J.R. Meteorolog. Soc

Model reduction for Kuramoto models with complex topologies

Authors: Edward J. Hancock, Georg A. Gottwald

Abstract: Synchronisation of coupled oscillators is a ubiquitous phenomenon, occurring in topics ranging from biology and physics, to social networks and technology. A fundamental and long-time goal in the study of synchronisation has been to find low-order descriptions of complex oscillator networks and their collective dynamics. However, for the Kuramoto model - the most widely used model of coupled oscil… ▽ More Synchronisation of coupled oscillators is a ubiquitous phenomenon, occurring in topics ranging from biology and physics, to social networks and technology. A fundamental and long-time goal in the study of synchronisation has been to find low-order descriptions of complex oscillator networks and their collective dynamics. However, for the Kuramoto model - the most widely used model of coupled oscillators - this goal has remained surprisingly challenging, in particular for finite-size networks. Here, we propose a model reduction framework that effectively captures synchronisation behaviour in complex network topologies. This framework generalises a collective coordinates approach for all-to-all networks [Gottwald (2015) Chaos 25, 053111] by incorporating the graph Laplacian matrix in the collective coordinates. We first derive low dimensional evolution equations for both clustered and non-clustered oscillator networks. We then demonstrate in numerical simulations for Erdos-Renyi (ER) networks that the collective coordinates capture the synchronisation behaviour in both finite-size networks as well as in the thermodynamic limit, even in the presence of interacting clusters. △ Less

Submitted 27 June, 2018; v1 submitted 19 April, 2018; originally announced April 2018.

Journal ref: Phys. Rev. E 98, 012307 (2018)

A note on statistical consistency of numerical integrators for multi-scale dynamics

Authors: J. Frank, G. A. Gottwald

Abstract: A minimal requirement for simulating multi-scale systems is to reproduce the statistical behavior of the slow variables. In particular, a good numerical method should accurately aproximate the probability density function of the continuous-time slow variables. In this note we use results from homogenization and from backward error analysis to quantify how errors of time integrators affect the mean… ▽ More A minimal requirement for simulating multi-scale systems is to reproduce the statistical behavior of the slow variables. In particular, a good numerical method should accurately aproximate the probability density function of the continuous-time slow variables. In this note we use results from homogenization and from backward error analysis to quantify how errors of time integrators affect the mean behavior of trajectories. We show that numerical simulations converge, not to the exact probability density function (pdf) of the homogenized multi-scale system, but rather to that of the homogenized modified equations following from backward error analysis. Using homogenization theory we find that the observed statistical bias is exacerbated for multi-scale systems driven by fast chaotic dynamics that decorrelate insufficiently rapidly. This suggests that to resolve the statistical behavior of trajectories in certain multi-scale systems solvers of sufficiently high order are necessary. Alternatively, backward error analysis suggests the form of an amended vector field that corrects the lowest order bias in Euler's method. The resulting scheme, a second order Taylor method, avoids any statistical drift bias. We corroborate our analysis with a numerical example. △ Less

Submitted 11 April, 2018; originally announced April 2018.

Comments: accepted for publication in SIAM MMS

On the validity of linear response theory in high-dimensional deterministic dynamical systems

Authors: Caroline L. Wormell, Georg A. Gottwald

Abstract: This theoretical work considers the following conundrum: linear response theory is successfully used by scientists in numerous fields, but mathematicians have shown that typical low-dimensional dynamical systems violate the theory's assumptions. Here we provide a proof of concept for the validity of linear response theory in high-dimensional deterministic systems for large-scale observables. We in… ▽ More This theoretical work considers the following conundrum: linear response theory is successfully used by scientists in numerous fields, but mathematicians have shown that typical low-dimensional dynamical systems violate the theory's assumptions. Here we provide a proof of concept for the validity of linear response theory in high-dimensional deterministic systems for large-scale observables. We introduce an exemplary model in which observables of resolved degrees of freedom are weakly coupled to a large, inhomogeneous collection of unresolved chaotic degrees of freedom. By employing statistical limit laws we give conditions under which such systems obey linear response theory even if all the degrees of freedom individually violate linear response. We corroborate our result with numerical simulations. △ Less

Submitted 19 June, 2018; v1 submitted 29 January, 2018; originally announced January 2018.

Comments: 13 pages plus appendices

MSC Class: 37F15; 37H99; 37A10

Finite-size effects in a stochastic Kuramoto model

Authors: Georg A. Gottwald

Abstract: We present a collective coordinate approach to study the collective behaviour of a finite ensemble of $N$ stochastic Kuramoto oscillators using two degrees of freedom; one describing the shape dynamics of the oscillators and one describing their mean phase. Contrary to the thermodynamic limit $N\to\infty$ in which the mean phase of the cluster of globally synchronized oscillators is constant in ti… ▽ More We present a collective coordinate approach to study the collective behaviour of a finite ensemble of $N$ stochastic Kuramoto oscillators using two degrees of freedom; one describing the shape dynamics of the oscillators and one describing their mean phase. Contrary to the thermodynamic limit $N\to\infty$ in which the mean phase of the cluster of globally synchronized oscillators is constant in time, the mean phase of a finite-size cluster experiences Brownian diffusion with a variance proportional to $1/N$. This finite-size effect is quantitatively well captured by our collective coordinate approach. △ Less

Submitted 8 September, 2017; originally announced September 2017.

Optimal balance via adiabatic invariance of approximate slow manifolds

Authors: Georg A. Gottwald, Haidar Mohamad, Marcel Oliver

Abstract: We analyze the method of optimal balance which was introduced by Viudez and Dritschel (J. Fluid Mech. 521, 2004, pp. 343-352) to provide balanced initializations for two-dimensional and three-dimensional geophysical flows, here in the simpler context of a finite dimensional Hamiltonian two-scale system with strong gyroscopic forces. It is well known that when the potential is analytic, such system… ▽ More We analyze the method of optimal balance which was introduced by Viudez and Dritschel (J. Fluid Mech. 521, 2004, pp. 343-352) to provide balanced initializations for two-dimensional and three-dimensional geophysical flows, here in the simpler context of a finite dimensional Hamiltonian two-scale system with strong gyroscopic forces. It is well known that when the potential is analytic, such systems have an approximate slow manifold that is defined up to terms that are exponentially small with respect to the scale separation parameter. The method of optimal balance relies on the observation that the approximate slow manifold remains an adiabatic invariant under slow deformations of the nonlinear interactions. The method is formulated as a boundary value problem for a homotopic deformation of the system from a linear regime, where the slow-fast splitting is known exactly, to the full nonlinear regime. We show that, providing the ramp function which defines the homotopy is of Gevrey class 2 and satisfies vanishing conditions to all orders at the temporal end points, the solution of the optimal balance boundary value problem yields a point on the approximate slow manifold that is exponentially close to the approximation to the slow manifold via exponential asymptotics, albeit with a smaller power of the small parameter in the exponent. In general, the order of accuracy of optimal balance is limited by the order of vanishing derivatives of the ramp function at the temporal end points. We also give a numerical demonstration of the efficacy of optimal balance, showing the dependence of accuracy on the ramp time and the ramp function. △ Less

Submitted 28 August, 2017; originally announced August 2017.

Comments: accepted for publication in SIAM Multiscale Modeling and Simulation

Edgeworth expansions for slow-fast systems with finite time scale separation

Authors: Jeroen Wouters, Georg A. Gottwald

Abstract: We derive Edgeworth expansions that describe corrections to the Gaussian limiting behaviour of slow-fast systems. The Edgeworth expansion is achieved using a semi-group formalism for the transfer operator, where a Duhamel-Dyson series is used to asymptotically determine the corrections at any desired order of the time scale parameter $\varepsilon$. The corrections involve integrals over higher-ord… ▽ More We derive Edgeworth expansions that describe corrections to the Gaussian limiting behaviour of slow-fast systems. The Edgeworth expansion is achieved using a semi-group formalism for the transfer operator, where a Duhamel-Dyson series is used to asymptotically determine the corrections at any desired order of the time scale parameter $\varepsilon$. The corrections involve integrals over higher-order auto-correlation functions. We develop a diagrammatic representation of the series to control the combinatorial wealth of the asymptotic expansion in $\varepsilon$ and provide explicit expressions for the first two orders. At a formal level, the expressions derived are valid in the case when the fast dynamics is stochastic as well as when the fast dynamics is entirely deterministic. We corroborate our analytical results with numerical simulations and show that our method provides an improvement on the classical homogenization limit which is restricted to the limit of infinite time scale separation. △ Less

Submitted 4 February, 2019; v1 submitted 23 August, 2017; originally announced August 2017.

Comments: accepted for publication in Proceedings of the Royal Society A

MSC Class: 70K70; 65C20; 37A50; 60F05

Stochastic partial differential fluid equations as a diffusive limit of deterministic Lagrangian multi-time dynamics

Authors: Colin J Cotter, Georg A Gottwald, Darryl D Holm

Abstract: In {\em{Holm}, Proc. Roy. Soc. A 471 (2015)} stochastic fluid equations were derived by employing a variational principle with an assumed stochastic Lagrangian particle dynamics. Here we show that the same stochastic Lagrangian dynamics naturally arises in a multi-scale decomposition of the deterministic Lagrangian flow map into a slow large-scale mean and a rapidly fluctuating small scale map. We… ▽ More In {\em{Holm}, Proc. Roy. Soc. A 471 (2015)} stochastic fluid equations were derived by employing a variational principle with an assumed stochastic Lagrangian particle dynamics. Here we show that the same stochastic Lagrangian dynamics naturally arises in a multi-scale decomposition of the deterministic Lagrangian flow map into a slow large-scale mean and a rapidly fluctuating small scale map. We employ homogenization theory to derive effective slow stochastic particle dynamics for the resolved mean part, thereby justifying stochastic fluid partial equations in the Eulerian formulation. To justify the application of rigorous homogenization theory, we assume mildly chaotic fast small-scale dynamics, as well as a centering condition. The latter requires that the mean of the fluctuating deviations is small, when pulled back to the mean flow. △ Less

Submitted 1 June, 2017; originally announced June 2017.

Comments: 11 pages, geometric mechanics, stochastic fluid models, stochastic processes, multi-scale fluid dynamics, symmetry reduced variational principles, homogenisation

Comparison of variational balance models for the rotating shallow water equations

Authors: David G. Dritschel, Georg A. Gottwald, Marcel Oliver

Abstract: We present an extensive numerical comparison of a family of balance models appropriate to the semi-geostrophic limit of the rotating shallow water equations, and derived by variational asymptotics in Oliver (2006) for small Rossby numbers ${\mathrm{Ro}}$. This family of generalized large-scale semi-geostrophic (GLSG) models contains the $L_1$-model introduced by Salmon (1983) as a special case. We… ▽ More We present an extensive numerical comparison of a family of balance models appropriate to the semi-geostrophic limit of the rotating shallow water equations, and derived by variational asymptotics in Oliver (2006) for small Rossby numbers ${\mathrm{Ro}}$. This family of generalized large-scale semi-geostrophic (GLSG) models contains the $L_1$-model introduced by Salmon (1983) as a special case. We use these models to produce balanced initial states for the full shallow water equations. We then numerically investigate how well these models capture the dynamics of an initially balanced shallow water flow. It is shown that, whereas the $L_1$-member of the GLSG family is able to reproduce the balanced dynamics of the full shallow water equations on time scales of ${\mathcal{O}}(1/{\mathrm{Ro}})$ very well, all other members develop significant unphysical high wavenumber contributions in the ageostrophic vorticity which spoil the dynamics. △ Less

Submitted 16 May, 2017; originally announced May 2017.

Comments: accepted for publication in Journal of Fluid Mechanics

Stochastic Climate Theory

Authors: Georg A. Gottwald, Daan T. Crommelin, Christian L. E. Franzke

Abstract: In this chapter we review stochastic modelling methods in climate science. First we provide a conceptual framework for stochastic modelling of deterministic dynamical systems based on the Mori-Zwanzig formalism. The Mori-Zwanzig equations contain a Markov term, a memory term and a term suggestive of stochastic noise. Within this framework we express standard model reduction methods such as averagi… ▽ More In this chapter we review stochastic modelling methods in climate science. First we provide a conceptual framework for stochastic modelling of deterministic dynamical systems based on the Mori-Zwanzig formalism. The Mori-Zwanzig equations contain a Markov term, a memory term and a term suggestive of stochastic noise. Within this framework we express standard model reduction methods such as averaging and homogenization which eliminate the memory term. We further discuss ways to deal with the memory term and how the type of noise depends on the underlying deterministic chaotic system. Secondly, we review current approaches in stochastic data-driven models. We discuss how the drift and diffusion coefficients of models in the form of stochastic differential equations can be estimated from observational data. We pay attention to situations where the data stems from multi scale systems, a relevant topic in the context of data from the climate system. Furthermore, we discuss the use of discrete stochastic processes (Markov chains) for e.g. stochastic subgrid-scale modeling and other topics in climate science. △ Less

Submitted 22 December, 2016; originally announced December 2016.

Comments: book chapter in Nonlinear and Stochastic Climate Dynamics, Cambridge University Press (C. Franzke and T. O'Kane, eds.) (2016)

On the detection of superdiffusive behaviour in time series

Authors: Georg A. Gottwald, Ian Melbourne

Abstract: We present a new method for detecting superdiffusive behaviour and for determining rates of superdiffusion in time series data. Our method applies equally to stochastic and deterministic time series data (with no prior knowledge required of the nature of the data) and relies on one realisation (ie one sample path) of the process. Linear drift effects are automatically removed without any preproces… ▽ More We present a new method for detecting superdiffusive behaviour and for determining rates of superdiffusion in time series data. Our method applies equally to stochastic and deterministic time series data (with no prior knowledge required of the nature of the data) and relies on one realisation (ie one sample path) of the process. Linear drift effects are automatically removed without any preprocessing. We show numerical results for time series constructed from i.i.d. $α$-stable random variables and from deterministic weakly chaotic maps. We compare our method with the standard method of estimating the growth rate of the mean-square displacement as well as the $p$-variation method, maximum likelihood, quantile matching and linear regression of the empirical characteristic function. △ Less

Submitted 21 December, 2016; v1 submitted 25 July, 2016; originally announced July 2016.

Journal ref: J Stat Mech (12) (2016) 123205

On spurious detection of linear response and misuse of the fluctuation-dissipation theorem in finite time series

Authors: Georg A. Gottwald, Caroline L. Wormell, Jeroen Wouters

Abstract: Using a sensitive statistical test we determine whether or not one can detect the breakdown of linear response given observations of deterministic dynamical systems. A goodness-of-fit statistics is developed for a linear statistical model of the observations, based on results on central limit theorems for deterministic dynamical systems, and used to detect linear response breakdown. We apply the m… ▽ More Using a sensitive statistical test we determine whether or not one can detect the breakdown of linear response given observations of deterministic dynamical systems. A goodness-of-fit statistics is developed for a linear statistical model of the observations, based on results on central limit theorems for deterministic dynamical systems, and used to detect linear response breakdown. We apply the method to discrete maps which do not obey linear response and show that the successful detection of breakdown depends on the length of the time series, the magnitude of the perturbation and on the choice of the observable. We find that in order to reliably reject the assumption of linear response for typical observables sufficiently large data sets are needed. Even for simple systems such as the logistic map, one needs of the order of $10^6$ observations to reliably detect the breakdown, if less observations are available one may be falsely led to conclude that linear response theory is valid. The amount of data required is larger the smaller the applied perturbation. For judiciously chosen observables the necessary amount of data can be drastically reduced, but requires detailed {\em{a priori}} knowledge about the invariant measure which is typically not available for complex dynamical systems. Furthermore we explore the use of the fluctuation-dissipation theorem (FDT) in cases with limited data length or coarse-graining of observations. The FDT, if applied naively to a system without linear response, is shown to be very sensitive to the details of the sampling method, resulting in erroneous predictions of the response. △ Less

Submitted 9 September, 2021; v1 submitted 12 January, 2016; originally announced January 2016.

Comments: Published in Physica D (2016)

Broadband nature of power spectra for intermittent Maps with summable and nonsummable decay of correlations

Authors: Georg A. Gottwald, Ian Melbourne

Abstract: We present results on the broadband nature of the power spectrum $S(ω)$, $ω\in(0,2π)$, for a large class of nonuniformly expanding maps with summable and nonsummable decay of correlations. In particular, we consider a class of intermittent maps $f:[0,1]\to[0,1]$ with $f(x)\approx x^{1+γ}$ for $x\approx 0$, where $γ\in(0,1)$. Such maps have summable decay of correlations when $γ\in(0,\frac12)$, and… ▽ More We present results on the broadband nature of the power spectrum $S(ω)$, $ω\in(0,2π)$, for a large class of nonuniformly expanding maps with summable and nonsummable decay of correlations. In particular, we consider a class of intermittent maps $f:[0,1]\to[0,1]$ with $f(x)\approx x^{1+γ}$ for $x\approx 0$, where $γ\in(0,1)$. Such maps have summable decay of correlations when $γ\in(0,\frac12)$, and $S(ω)$ extends to a continuous function on $[0,2π]$ by the classical Wiener-Khintchine Theorem. We show that $S(ω)$ is typically bounded away from zero for Hölder observables. Moreover, in the nonsummable case $γ\in[\frac12,1)$, we show that $S(ω)$ is defined almost everywhere with a continuous extension $\tilde S(ω)$ defined on $(0,2π)$, and $\tilde S(ω)$ is typically nonvanishing. △ Less

Submitted 30 March, 2016; v1 submitted 31 October, 2015; originally announced November 2015.

Comments: Final version

Journal ref: J. Phys. A: Math. Theor. 49 (2016) 174003 (17pp)

A data-driven method for the stochastic parametrisation of subgrid-scale tropical convective area fraction

Authors: Georg A. Gottwald, Karsten Peters, Laura Davies

Abstract: Observations of tropical convection from precipitation radar and the concurring large-scale atmospheric state at two locations (Darwin and Kwajalein) are used to establish effective stochastic models to parameterise subgrid-scale tropical convective activity. Two approaches are presented which rely on the assumption that tropical convection induces a stationary equilibrium distribution. In the fir… ▽ More Observations of tropical convection from precipitation radar and the concurring large-scale atmospheric state at two locations (Darwin and Kwajalein) are used to establish effective stochastic models to parameterise subgrid-scale tropical convective activity. Two approaches are presented which rely on the assumption that tropical convection induces a stationary equilibrium distribution. In the first approach we parameterise convection variables such as convective area fraction as an instantaneous random realisation conditioned on the large-scale vertical velocities according to a probability density function estimated from the observations. In the second approach convection variables are generated in a Markov process conditioned on the large-scale vertical velocity, allowing for non-trivial temporal correlations. Despite the different prevalent atmospheric and oceanic regimes at the two locations, with Kwajalein being exposed to a purely oceanic weather regime and Darwin exhibiting land-sea interaction, we establish that the empirical measure for the convective variables conditioned on large-scale mid-level vertical velocities for the two locations are close. This allows us to train the stochastic models at one location and then generate time series of convective activity at the other location. The proposed stochastic subgrid-scale models adequately reproduce the statistics of the observed convective variables and we discuss how they may be used in future scale-independent mass-flux convection parameterisations. △ Less

Submitted 16 August, 2015; originally announced August 2015.

Comments: accepted for publication in Quarterly Journal of the Royal Meteorological Society (QJRMS) (2015)

Model reduction for networks of coupled oscillators

Authors: Georg A. Gottwald

Abstract: We present a collective coordinate approach to describe coupled phase oscillators. We apply the method to study synchronisation in a Kuramoto model. In our approach an N-dimensional Kuramoto model is reduced to an n-dimensional ordinary differential equation with n<<N, constituting an immense reduction in complexity. The onset of both local and global synchronisation is reproduced to good numerica… ▽ More We present a collective coordinate approach to describe coupled phase oscillators. We apply the method to study synchronisation in a Kuramoto model. In our approach an N-dimensional Kuramoto model is reduced to an n-dimensional ordinary differential equation with n<<N, constituting an immense reduction in complexity. The onset of both local and global synchronisation is reproduced to good numerical accuracy, and we are able to describe both soft and hard transitions. By introducing 2 collective coordinates the approach is able to describe the interaction of two partially synchronised clusters in the case of bimodally distributed native frequencies. Furthermore, our approach allows us to accurately describe finite size scalings of the critical coupling strength. We corroborate our analytical results by comparing with numerical simulations of the Kuramoto model with all-to-all coupling networks for several distributions of the native frequencies. △ Less

Submitted 20 May, 2015; originally announced May 2015.

Journal ref: Chaos 25, 053111 (2015)

On convergence of higher order schemes for the projective integration method for stiff ordinary differential equations

Authors: John Maclean, Georg A. Gottwald

Abstract: We present a convergence proof for higher order implementations of the projective integration method (PI) for a class of deterministic multi-scale systems in which fast variables quickly settle on a slow manifold. The error is shown to contain contributions associated with the length of the microsolver, the numerical accuracy of the macrosolver and the distance from the slow manifold caused by the… ▽ More We present a convergence proof for higher order implementations of the projective integration method (PI) for a class of deterministic multi-scale systems in which fast variables quickly settle on a slow manifold. The error is shown to contain contributions associated with the length of the microsolver, the numerical accuracy of the macrosolver and the distance from the slow manifold caused by the combined effect of micro- and macrosolvers, respectively. We also provide stability conditions for the PI methods under which the fast variables will not diverge from the slow manifold. We corroborate our results by numerical simulations. △ Less

Submitted 6 April, 2015; originally announced April 2015.

Comments: 43 pages, 7 figures; accepted for publication in the Journal of Computational and Applied Mathematics

MSC Class: 65LXX; 65PXX; 34E13; 37MXX

A trajectory-free framework for analysing multiscale systems

Authors: Gary Froyland, Georg A. Gottwald, Andy Hammerlindl

Abstract: We develop algorithms built around properties of the transfer operator and Koopman operator which 1) test for possible multiscale dynamics in a given dynamical system, 2) estimate the magnitude of the time-scale separation, and finally 3) distill the reduced slow dynamics on a suitably designed subspace. By avoiding trajectory integration, the developed techniques are highly computationally effici… ▽ More We develop algorithms built around properties of the transfer operator and Koopman operator which 1) test for possible multiscale dynamics in a given dynamical system, 2) estimate the magnitude of the time-scale separation, and finally 3) distill the reduced slow dynamics on a suitably designed subspace. By avoiding trajectory integration, the developed techniques are highly computationally efficient. We corroborate our findings with numerical simulations of a test problem. △ Less

Submitted 23 December, 2014; originally announced December 2014.

Comments: 20 pages

MSC Class: 37xx; 37Mxx; 65Pxx

Central limit theorems and suppression of anomalous diffusion for systems with symmetry

Authors: Georg A. Gottwald, Ian Melbourne

Abstract: We give general conditions for the central limit theorem and weak convergence to Brownian motion (the weak invariance principle / functional central limit theorem) to hold for observables of compact group extensions of nonuniformly expanding maps. In particular, our results include situations where the central limit theorem would fail, and anomalous behaviour would prevail, if the compact group we… ▽ More We give general conditions for the central limit theorem and weak convergence to Brownian motion (the weak invariance principle / functional central limit theorem) to hold for observables of compact group extensions of nonuniformly expanding maps. In particular, our results include situations where the central limit theorem would fail, and anomalous behaviour would prevail, if the compact group were not present. This has important consequences for systems with noncompact Euclidean symmetry and provides the rigorous proof for a conjecture made in our paper: A Huygens principle for diffusion and anomalous diffusion in spatially extended systems. Proc. Natl. Acad. Sci. USA 110 (2013) 8411-8416. △ Less

Submitted 22 February, 2015; v1 submitted 3 April, 2014; originally announced April 2014.

Comments: Minor revisions

MSC Class: 37D25 (Primary) 37A50; 60F17; 37C80 (Secondary)

Journal ref: Nonlinearity 29 (2016) 2941-2960