Showing 1–16 of 16 results for author: Hallerberg, S

On the weight dynamics of learning networks

Authors: Nahal Sharafi, Christoph Martin, Sarah Hallerberg

Abstract: Neural networks have become a widely adopted tool for tackling a variety of problems in machine learning and artificial intelligence. In this contribution we use the mathematical framework of local stability analysis to gain a deeper understanding of the learning dynamics of feed forward neural networks. Therefore, we derive equations for the tangent operator of the learning dynamics of three-laye… ▽ More Neural networks have become a widely adopted tool for tackling a variety of problems in machine learning and artificial intelligence. In this contribution we use the mathematical framework of local stability analysis to gain a deeper understanding of the learning dynamics of feed forward neural networks. Therefore, we derive equations for the tangent operator of the learning dynamics of three-layer networks learning regression tasks. The results are valid for an arbitrary numbers of nodes and arbitrary choices of activation functions. Applying the results to a network learning a regression task, we investigate numerically, how stability indicators relate to the final training-loss. Although the specific results vary with different choices of initial conditions and activation functions, we demonstrate that it is possible to predict the final training loss, by monitoring finite-time Lyapunov exponents or covariant Lyapunov vectors during the training process. △ Less

Submitted 30 April, 2024; originally announced May 2024.

Estimating covariant Lyapunov vectors from data

Authors: Christoph Martin, Nahal Sharafi, Sarah Hallerberg

Abstract: Covariant Lyapunov vectors characterize the directions along which perturbations in dynamical systems grow. They have also been studied as predictors of critical transitions and extreme events. For many applications like, for example, prediction, it is necessary to estimate the vectors from data since model equations are unknown for many interesting phenomena. We propose a novel method for estimat… ▽ More Covariant Lyapunov vectors characterize the directions along which perturbations in dynamical systems grow. They have also been studied as predictors of critical transitions and extreme events. For many applications like, for example, prediction, it is necessary to estimate the vectors from data since model equations are unknown for many interesting phenomena. We propose a novel method for estimating covariant Lyapunov vectors based on data records without knowing the underlying equations of the system. In contrast to previous approaches, our approach can be applied to high-dimensional data-sets. We demonstrate that this purely data-driven approach can accurately estimate covariant Lyapunpov vectors from data records generated by low and high-dimensional dynamical systems. The highest dimension of a time-series from which covariant Lyapunov vectors were estimated in this contribution is 128. Being able to infer covariant Lyapunov vectors from data-records could encourage numerous future applications in data-analysis and data-based predictions. △ Less

Submitted 11 October, 2021; v1 submitted 16 July, 2021; originally announced July 2021.

Fluctuation-induced Distributed Resonances in Oscillatory Networks

Authors: Xiaozhu Zhang, Sarah Hallerberg, Moritz Matthiae, Dirk Witthaut, Marc Timme

Abstract: Self-organized network dynamics prevails for systems across physics, biology and engineering. How external signals generate distributed responses in networked systems fundamentally underlies their function, yet is far from fully understood. Here we analyze the dynamic response patterns of oscillatory networks to fluctuating input signals. We disentangle the impact of the signal distribution across… ▽ More Self-organized network dynamics prevails for systems across physics, biology and engineering. How external signals generate distributed responses in networked systems fundamentally underlies their function, yet is far from fully understood. Here we analyze the dynamic response patterns of oscillatory networks to fluctuating input signals. We disentangle the impact of the signal distribution across the network, the signals' frequency contents and the network topology. We analytically derive qualitatively different dynamic response patterns and find three frequency regimes: homogeneous responses at low frequencies, topology-dependent resonances at intermediate frequencies, and localized responses at high frequencies. The theory faithfully predicts the network-wide collective responses to regular and irregular, localized and distributed simulated signals, as well as to real input signals to power grids recorded from renewable-energy supplies. These results not only provide general insights into the formation of dynamic response patterns in networked systems but also suggest regime- and topology-specific design principles underlying network function. △ Less

Submitted 9 September, 2018; originally announced September 2018.

Comments: 7 pages, 4 figures

Journal ref: Science advances 5.7(2019): eaav1027

Model-free inference of direct network interactions from nonlinear collective dynamics

Authors: Jose Casadiego, Mor Nitzan, Sarah Hallerberg, Marc Timme

Abstract: The topology of interactions in network dynamical systems fundamentally underlies their function. Accelerating technological progress creates massively available data about collective nonlinear dynamics in physical, biological, and technological systems. Detecting direct interaction patterns from those dynamics still constitutes a major open problem. In particular, current nonlinear dynamics appro… ▽ More The topology of interactions in network dynamical systems fundamentally underlies their function. Accelerating technological progress creates massively available data about collective nonlinear dynamics in physical, biological, and technological systems. Detecting direct interaction patterns from those dynamics still constitutes a major open problem. In particular, current nonlinear dynamics approaches mostly require to know a priori a model of the (often high dimensional) system dynamics. Here we develop a model-independent framework for inferring direct interactions solely from recording the nonlinear collective dynamics generated. Introducing an explicit dependency matrix in combination with a block-orthogonal regression algorithm, the approach works reliably across many dynamical regimes, including transient dynamics toward steady states, periodic and non-periodic dynamics, and chaos. Together with its capabilities to reveal network (two point) as well as hypernetwork (e.g., three point) interactions, this framework may thus open up nonlinear dynamics options of inferring direct interaction patterns across systems where no model is known. △ Less

Submitted 17 January, 2018; originally announced January 2018.

Comments: 10 pages, 7 figures

Journal ref: Nature Communications 8, Article number: 2192 (2017)

Critical Transitions and Perturbation Growth Directions

Authors: Nahal Sharafi, Marc Timme, Sarah Hallerberg

Abstract: Critical transitions occur in a variety of dynamical systems. Here, we employ quantifiers of chaos to identify changes in the dynamical structure of complex systems preceding critical transitions. As suitable indicator variables for critical transitions, we consider changes in growth rates and directions of covariant Lyapunov vectors. Studying critical transitions in several models of fast-slow sy… ▽ More Critical transitions occur in a variety of dynamical systems. Here, we employ quantifiers of chaos to identify changes in the dynamical structure of complex systems preceding critical transitions. As suitable indicator variables for critical transitions, we consider changes in growth rates and directions of covariant Lyapunov vectors. Studying critical transitions in several models of fast-slow systems, i.e., a network of coupled FitzHugh-Nagumo oscillators, models for Josephson junctions and the Hindmarsh-Rose model, we find that tangencies between covariant Lyapunov vectors are a common and maybe generic feature during critical transitions. We further demonstrate that this deviation from hyperbolic dynamics is linked to the occurrence of critical transitions by using it as an indicator variable and evaluating the prediction success through receiver operating characteristic curves. In the presence of noise, we find the alignment of covariant Lyapunov vectors and changes in finite-time Lyapunov exponents to be more successful in announcing critical transitions than common indicator variables as, e.g., finite-time estimates of the variance. Additionally, we propose a new method for estimating approximations of covariant Lyapunov vectors without knowledge of the future trajectory of the system. We find that these approximated covariant Lyapunov vectors can also be applied to predict critical transitions. △ Less

Submitted 25 July, 2017; originally announced July 2017.

Journal ref: Phys. Rev. E 96, 032220 (2017)

High-Intensity Discharge Lamp and Duffing Oscillator - Similarities and Differences

Authors: Bernd Baumann, Joerg Schwieger, Ulrich Stein, Sarah Hallerberg, Marcus Wolff

Abstract: The processes inside the arc tube of high-intensity discharge lamps are investigated by finite element simulations. The behavior of the gas mixture inside the arc tube is governed by differential equations describing mass, energy and charge conservation as well as the Helmholtz equation for the acoustic pressure and the Navier-Stokes equation for the flow driven by the buoyancy and the acoustic st… ▽ More The processes inside the arc tube of high-intensity discharge lamps are investigated by finite element simulations. The behavior of the gas mixture inside the arc tube is governed by differential equations describing mass, energy and charge conservation as well as the Helmholtz equation for the acoustic pressure and the Navier-Stokes equation for the flow driven by the buoyancy and the acoustic streaming force. The model is highly nonlinear and requires a recursion procedure to account for the impact of acoustic streaming on the temperature and other fields. The investigations reveal the presence of a hysteresis and the corresponding jump phenomenon, quite similar to a Duffing oscillator. The similarities and, in particular, the differences of the nonlinear behavior of the high-intensity discharge lamp to that of a Duffing oscillator are discussed. For large amplitudes the high-intensity discharge lamp exhibits a stiffening effect in contrast to the Duffing oscillator. △ Less

Submitted 8 May, 2017; originally announced May 2017.

Comments: 14 pages, 8 figures

Network susceptibilities: theory and applications

Authors: Debsankha Manik, Martin Rohden, Henrik Ronellenfitsch, Xiaozhu Zhang, Sarah Hallerberg, Dirk Witthaut, Marc Timme

Abstract: We introduce the concept of network susceptibilities quantifying the response of the collective dy- namics of a network to small parameter changes. We distinguish two types of susceptibilities: vertex susceptibilities and edge susceptibilities, measuring the responses due to changes in the properties of units and their interactions, respectively. We derive explicit forms of network susceptibilitie… ▽ More We introduce the concept of network susceptibilities quantifying the response of the collective dy- namics of a network to small parameter changes. We distinguish two types of susceptibilities: vertex susceptibilities and edge susceptibilities, measuring the responses due to changes in the properties of units and their interactions, respectively. We derive explicit forms of network susceptibilities for oscillator networks close to steady states and offer example applications for Kuramoto-type phase- oscillator models, power grid models and generic flow models. Focusing on the role of the network topology implies that these ideas can be easily generalized to other types of networks, in particular those characterizing flow, transport, or spreading phenomena. The concept of network susceptibil- ities is broadly applicable and may straightforwardly be transferred to all settings where networks responses of the collective dynamics to topological changes are essential. △ Less

Submitted 14 September, 2016; originally announced September 2016.

Journal ref: Phys. Rev. E 95, 012319 (2017)

Critical links and nonlocal rerouting in complex supply networks

Authors: Dirk Witthaut, Martin Rohden, Xiaozhu Zhang, Sarah Hallerberg, Marc Timme

Abstract: Link failures repeatedly induce large-scale outages in power grids and other supply networks. Yet, it is still not well understood, which links are particularly prone to inducing such outages. Here we analyze how the nature and location of each link impact the network's capability to maintain stable supply. We propose two criteria to identify critical links on the basis of the topology and the loa… ▽ More Link failures repeatedly induce large-scale outages in power grids and other supply networks. Yet, it is still not well understood, which links are particularly prone to inducing such outages. Here we analyze how the nature and location of each link impact the network's capability to maintain stable supply. We propose two criteria to identify critical links on the basis of the topology and the load distribution of the network prior to link failure. They are determined via a link's redundant capacity and a renormalized linear response theory we derive. These criteria outperform critical link prediction based on local measures such as loads. The results not only further our understanding of the physics of supply networks in general. As both criteria are available before any outage from the state of normal operation, they may also help real-time monitoring of grid operation, employing counter-measures and support network planning and design. △ Less

Submitted 30 October, 2015; originally announced October 2015.

Comments: 21 pages, 19 figures

Journal ref: Phys. Rev. Lett. 116, 138701 (2016)

Predictability of Critical Transitions

Authors: Xiaozhu Zhang, Christian Kuehn, Sarah Hallerberg

Abstract: Critical transitions in multistable systems have been discussed as models for a variety of phenomena ranging from the extinctions of species to socio-economic changes and climate transitions between ice-ages and warm-ages. From bifurcation theory we can expect certain critical transitions to be preceded by a decreased recovery from external perturbations. The consequences of this critical slowing… ▽ More Critical transitions in multistable systems have been discussed as models for a variety of phenomena ranging from the extinctions of species to socio-economic changes and climate transitions between ice-ages and warm-ages. From bifurcation theory we can expect certain critical transitions to be preceded by a decreased recovery from external perturbations. The consequences of this critical slowing down have been observed as an increase in variance and autocorrelation prior to the transition. However especially in the presence of noise it is not clear, whether these changes in observation variables are statistically relevant such that they could be used as indicators for critical transitions. In this contribution we investigate the predictability of critical transitions in conceptual models. We study the quadratic integrate-and-fire model and the van der Pol model, under the influence of external noise. We focus especially on the statistical analysis of the success of predictions and the overall predictability of the system. The performance of different indicator variables turns out to be dependent on the specific model under study and the conditions of accessing it. Furthermore, we study the influence of the magnitude of transitions on the predictive performance. △ Less

Submitted 3 November, 2015; v1 submitted 21 May, 2015; originally announced May 2015.

Journal ref: Physical Review E, Vol. 92, 052905, 2015

Bag-of-calls analysis reveals group-specific vocal repertoire in long-finned pilot whales

Authors: Heike Vester, Kurt Hammerschmidt, Marc Timme, Sarah Hallerberg

Abstract: Besides humans, several marine mammal species exhibit prerequisites to evolve language: high cognitive abilities, flexibility in vocal production and advanced social interactions. Here, we describe and analyse the vocal repertoire of long-finned pilot whales (Globicephalus melas) recorded in northern Norway. Observer based analysis reveals a complex vocal repertoire with 140 different call types,… ▽ More Besides humans, several marine mammal species exhibit prerequisites to evolve language: high cognitive abilities, flexibility in vocal production and advanced social interactions. Here, we describe and analyse the vocal repertoire of long-finned pilot whales (Globicephalus melas) recorded in northern Norway. Observer based analysis reveals a complex vocal repertoire with 140 different call types, call sequences, call repetitions and group-specific differences in the usage of call types. Developing and applying a new automated analysis method, the bag-of-calls approach, we find that groups of pilot whales can be distinguished purely by statistical properties of their vocalisations. Comparing inter-and intra-group differences of ensembles of calls allows to identify and quantify group-specificity. Consequently, the bag-of-calls approach is a valid method to specify difference and concordance in acoustic communication in the absence of exact knowledge about signalers, which is common observing marine mammals under natural conditions. △ Less

Submitted 12 June, 2015; v1 submitted 17 October, 2014; originally announced October 2014.

Comments: under review

Understanding and Controlling Regime Switching in Molecular Diffusion

Authors: S. Hallerberg, A. S. de Wijn

Abstract: Diffusion can be strongly affected by ballistic flights (long jumps) as well as long-lived sticking trajectories (long sticks). Using statistical inference techniques in the spirit of Granger causality, we investigate the appearance of long jumps and sticks in molecular-dynamics simulations of diffusion in a prototype system, a benzene molecule on a graphite substrate. We find that specific fluctu… ▽ More Diffusion can be strongly affected by ballistic flights (long jumps) as well as long-lived sticking trajectories (long sticks). Using statistical inference techniques in the spirit of Granger causality, we investigate the appearance of long jumps and sticks in molecular-dynamics simulations of diffusion in a prototype system, a benzene molecule on a graphite substrate. We find that specific fluctuations in certain, but not all, internal degrees of freedom of the molecule can be linked to either long jumps or sticks. Furthermore, by changing the prevalence of these predictors with an outside influence, the diffusion of the molecule can be controlled. The approach presented in this proof of concept study is very generic, and can be applied to larger and more complex molecules. Additionally, the predictor variables can be chosen in a general way so as to be accessible in experiments, making the method feasible for control of diffusion in applications. Our results also demonstrate that data-mining techniques can be used to investigate the phase-space structure of high-dimensional nonlinear dynamical systems. △ Less

Submitted 5 November, 2014; v1 submitted 8 October, 2013; originally announced October 2013.

Comments: accepted for publication by PRE

Journal ref: Phys. Rev. E 90, 062901, 2014

Predicting Failures of Point Forecasts

Authors: S. Hallerberg, J. Bröcker, H. Kantz, L. A. Smith

Abstract: The predictability of errors in deterministic temperature forecasts is investigated. More precisely, the aim is to issue warnings whenever the differences between forecast and verification exceed a given threshold. The warnings are generated by analyzing the output of an ensemble forecast system in terms of a decision making approach. The quality of the resulting predictions is evaluated by comput… ▽ More The predictability of errors in deterministic temperature forecasts is investigated. More precisely, the aim is to issue warnings whenever the differences between forecast and verification exceed a given threshold. The warnings are generated by analyzing the output of an ensemble forecast system in terms of a decision making approach. The quality of the resulting predictions is evaluated by computing receiver operating characteristics, the Brier score, and the Ignorance score. Special emphasis is also given to the question whether rare events are better predictable. △ Less

Submitted 7 December, 2011; originally announced December 2011.

Comments: 19 pages, 13 figures

Logarithmic bred vectors in spatiotemporal chaos: structure and growth

Authors: Sarah Hallerberg, Diego Pazó, Juan M. López, Miguel A. Rodríguez

Abstract: Bred vectors are a type of finite perturbation used in prediction studies of atmospheric models that exhibit spatially extended chaos. We study the structure, spatial correlations, and the growth- rates of logarithmic bred vectors (which are constructed by using a given norm). We find that, after a suitable transformation, logarithmic bred vectors are roughly piecewise copies of the leading Lyapun… ▽ More Bred vectors are a type of finite perturbation used in prediction studies of atmospheric models that exhibit spatially extended chaos. We study the structure, spatial correlations, and the growth- rates of logarithmic bred vectors (which are constructed by using a given norm). We find that, after a suitable transformation, logarithmic bred vectors are roughly piecewise copies of the leading Lyapunov vector. This fact allows us to deduce a scaling law for the bred vector growth rate as a function of their amplitude. In addition, we relate growth rates with the spectrum of Lyapunov exponents corresponding to the most expanding directions. We illustrate our results with simulations of the Lorenz '96 model. △ Less

Submitted 25 May, 2010; originally announced May 2010.

Comments: 8 pages, 8 figures

Journal ref: Phys. Rev. E 81, 066204 (2010)

When are Extreme Events the better predictable, the larger they are?

Authors: S. Hallerberg, H. Kantz

Abstract: We investigate the predictability of extreme events in time series. The focus of this work is to understand under which circumstances large events are better predictable than smaller events. Therefore we use a simple prediction algorithm based on precursory structures which are identified using the maximum likelihood principle. Using the receiver operator characteristic curve as a measure for th… ▽ More We investigate the predictability of extreme events in time series. The focus of this work is to understand under which circumstances large events are better predictable than smaller events. Therefore we use a simple prediction algorithm based on precursory structures which are identified using the maximum likelihood principle. Using the receiver operator characteristic curve as a measure for the quality of predictions we find that the dependence on the event magnitude is closely linked to the probability distribution function of the underlying stochastic process. We evaluate this dependence on the probability distribution function analytically and numerically. If we assume that the optimal precursory structures are used to make the predictions, we find that large increments are better predictable if the underlying stochastic process has a Gaussian probability distribution function, whereas larger increments are harder to predict if the underlying probability distribution function has a power law tail. In the case of an exponential distribution function we find no significant dependence on the event magnitude. Furthermore we compare these results with predictions of increments in correlated data, namely, velocity increments of a free jet flow. The velocity increments in the free jet flow are in dependence on the time scale either asymptotically Gaussian or asymptotically exponential distributed. The numerical results for predictions within free jet data are in good agreement with the previous analytical considerations for random numbers. △ Less

Submitted 29 January, 2008; originally announced January 2008.

Journal ref: Phys. Rev. E 77, 011108 (2008)

Precursors of extreme increments

Authors: Sarah Hallerberg, Eduardo G. Altmann, Detlef Holstein, Holger Kantz

Abstract: We investigate precursors and predictability of extreme increments in a time series. The events we are focusing on consist in large increments within successive time steps. We are especially interested in understanding how the quality of the predictions depends on the strategy to choose precursors, on the size of the event and on the correlation strength. We study the prediction of extreme incre… ▽ More We investigate precursors and predictability of extreme increments in a time series. The events we are focusing on consist in large increments within successive time steps. We are especially interested in understanding how the quality of the predictions depends on the strategy to choose precursors, on the size of the event and on the correlation strength. We study the prediction of extreme increments analytically in an AR(1) process, and numerically in wind speed recordings and long-range correlated ARMA data. We evaluate the success of predictions via receiver operator characteristics (ROC-curves). Furthermore, we observe an increase of the quality of predictions with increasing event size and with decreasing correlation in all examples. Both effects can be understood by using the likelihood ratio as a summary index for smooth ROC-curves. △ Less

Submitted 12 September, 2006; v1 submitted 20 April, 2006; originally announced April 2006.

Journal ref: Phys. Rev. E 75, 016706 (2007)

Reactions to extreme events: moving threshold model

Authors: Eduardo G. Altmann, Sarah Hallerberg, Holger Kantz

Abstract: In spite of precautions to avoid the harmful effects of extreme events, we experience recurrently phenomena that overcome the preventive barriers. These barriers usually increase drastically right after the occurrence of such extreme events, but steadily decay in their absence. In this paper we consider a simple model that mimics the evolution of the protection barriers to study the efficiency o… ▽ More In spite of precautions to avoid the harmful effects of extreme events, we experience recurrently phenomena that overcome the preventive barriers. These barriers usually increase drastically right after the occurrence of such extreme events, but steadily decay in their absence. In this paper we consider a simple model that mimics the evolution of the protection barriers to study the efficiency of the system's reaction to extreme events and how it changes our perception of the sequence of extreme events itself. We obtain that the usual method of fighting extreme events introduces a periodicity in their occurrence and is generally less efficient than the use of a constant barrier. On the other hand, it shows a good adaptation to the presence of slow non-stationarities. △ Less

Submitted 23 August, 2005; originally announced August 2005.

Comments: 14 pages and 7 figures

Journal ref: Physica A 364 p. 435 (2006)