-
Statistics for Differential Topological Properties between Data Sets with an Application to Reservoir Computers
Authors:
Louis Pecora,
Thomas Carroll
Abstract:
It is common for researchers to record long, multiple time series from experiments or calculations. But sometimes there are no good models for the systems or no applicable mathematical theorems that can tell us when there are basic relationships between subsets of the time series data such as continuity, differentiability, embeddings, etc. The data is often higher dimensional and simple plotting w…
▽ More
It is common for researchers to record long, multiple time series from experiments or calculations. But sometimes there are no good models for the systems or no applicable mathematical theorems that can tell us when there are basic relationships between subsets of the time series data such as continuity, differentiability, embeddings, etc. The data is often higher dimensional and simple plotting will not guide us. At that point fitting the data to polynomials, Fourier series, etc. becomes uncertain. Even at the simplest level, having data that shows there is a function between the data subsets is useful and a negative answer means that more particular data fitting or analysis will be suspect and probably fail. We show here statistics that test time series subsets for basic mathematical properties and relations between them that not only indicate when more specific analyses are safe to do, but whether the systems are operating correctly. We apply these statistics to examples from reservoir computing where an important property of reservoir computers is that the reservoir system establishes an embedding of the drive system in order to make any other calculations with the reservoir computer successful.
△ Less
Submitted 22 November, 2024; v1 submitted 6 September, 2024;
originally announced September 2024.
-
Creating New Chaotic Signals with Reservoir Computers
Authors:
Thomas L. Carroll
Abstract:
While there have been many publications on potential applications of chaos to fields such as communications, radar, sonar, random signal generation, channel equalization and others, designing continuous chaotic systems is still an unsolved problem. There are a number of well known chaotic systems used for applications, but if any application is to become widely used, some way of generating many di…
▽ More
While there have been many publications on potential applications of chaos to fields such as communications, radar, sonar, random signal generation, channel equalization and others, designing continuous chaotic systems is still an unsolved problem. There are a number of well known chaotic systems used for applications, but if any application is to become widely used, some way of generating many different chaotic signals is necessary. This work shows that one may use a reservoir computer to create a set of chaotic signals that are correlated but easily distinguishable from one chaotic signal with desirable properties. The ability to distinguish the new signals is demonstrated with a simple communications example.
△ Less
Submitted 9 September, 2022;
originally announced October 2022.
-
Do Reservoir Computers Work Best at the Edge of Chaos?
Authors:
Thomas L. Carroll
Abstract:
It has been demonstrated that cellular automata had the highest computational capacity at the edge of chaos, the parameter at which their behavior transitioned from ordered to chaotic. This same concept has been applied to reservoir computers; a number of researchers have stated that the highest computational capacity for a reservoir computer is at the edge of chaos, although others have suggested…
▽ More
It has been demonstrated that cellular automata had the highest computational capacity at the edge of chaos, the parameter at which their behavior transitioned from ordered to chaotic. This same concept has been applied to reservoir computers; a number of researchers have stated that the highest computational capacity for a reservoir computer is at the edge of chaos, although others have suggested that this rule is not universally true. Because many reservoir computers do not show chaotic behavior but merely become unstable, it is felt that a more accurate term for this instability transition is the "edge of stability"Here I find two examples where the computational capacity of a reservoir computer decreases as the edge of stability is approached; in one case, because generalized synchronization breaks down, and in the other case because the reservoir computer is a poor match to the problem being solved. The edge of stability as an optimal operating point for a reservoir computer is not in general true, although it may be true in some cases.
△ Less
Submitted 2 December, 2020;
originally announced December 2020.
-
Dimension of Reservoir Computers
Authors:
Thomas L. Carroll
Abstract:
A reservoir computer is a complex dynamical system, often created by coupling nonlinear nodes in a network. The nodes are all driven by a common driving signal. In this work, three dimension estimation methods, false nearest neighbor, covariance and Kaplan-Yorke dimensions, are used to estimate the dimension of the reservoir dynamical system. It is shown that the signals in the reservoir system ex…
▽ More
A reservoir computer is a complex dynamical system, often created by coupling nonlinear nodes in a network. The nodes are all driven by a common driving signal. In this work, three dimension estimation methods, false nearest neighbor, covariance and Kaplan-Yorke dimensions, are used to estimate the dimension of the reservoir dynamical system. It is shown that the signals in the reservoir system exist on a relatively low dimensional surface. Changing the spectral radius of the reservoir network can increase the fractal dimension of the reservoir signals, leading to an increase in testing error.
△ Less
Submitted 10 December, 2019;
originally announced December 2019.
-
Mutual Information and the Edge of Chaos in Reservoir Computers
Authors:
Thomas L. Carroll
Abstract:
A reservoir computer is a dynamical system that may be used to perform computations. A reservoir computer usually consists of a set of nonlinear nodes coupled together in a network so that there are feedback paths. Training the reservoir computer consists of inputing a signal of interest and fitting the time series signals of the reservoir computer nodes to a training signal that is related to the…
▽ More
A reservoir computer is a dynamical system that may be used to perform computations. A reservoir computer usually consists of a set of nonlinear nodes coupled together in a network so that there are feedback paths. Training the reservoir computer consists of inputing a signal of interest and fitting the time series signals of the reservoir computer nodes to a training signal that is related to the input signal. It is believed that dynamical systems function most efficiently as computers at the "edge of chaos", the point at which the largest Lyapunov exponent of the dynamical system transitions from negative to positive. In this work I simulate several different reservoir computers and ask if the best performance really does come at this edge of chaos. I find that while it is possible to get optimum performance at the edge of chaos, there may also be parameter values where the edge of chaos regime produces poor performance. This ambiguous parameter dependance has implications for building reservoir computers from analog physical systems, where the parameter range is restricted.
△ Less
Submitted 19 July, 2019; v1 submitted 6 June, 2019;
originally announced June 2019.
-
Network Structure Effects in Reservoir Computers
Authors:
Thomas L. Carroll,
Louis M. Pecora
Abstract:
A reservoir computer is a complex nonlinear dynamical system that has been shown to be useful for solving certain problems, such as prediction of chaotic signals, speech recognition or control of robotic systems. Typically a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network, driving the nodes with an input signal and using the node outputs to fit a trai…
▽ More
A reservoir computer is a complex nonlinear dynamical system that has been shown to be useful for solving certain problems, such as prediction of chaotic signals, speech recognition or control of robotic systems. Typically a reservoir computer is constructed by connecting a large number of nonlinear nodes in a network, driving the nodes with an input signal and using the node outputs to fit a training signal. In this work, we set up reservoirs where the edges (or connections) between all the network nodes are either +1 or 0, and proceed to alter the network structure by flipping some of these edges from +1 to -1. We use this simple network because it turns out to be easy to characterize; we may use the fraction of edges flipped as a measure of how much we have altered the network. In some cases, the network can be rearranged in a finite number of ways without changing its structure; these rearrangements are symmetries of the network, and the number of symmetries is also useful for characterizing the network. We find that changing the number of edges flipped in the network changes the rank of the covariance of a matrix consisting of the time series from the different nodes in the network, and speculate that this rank is important for understanding the reservoir computer performance.
△ Less
Submitted 7 August, 2019; v1 submitted 28 March, 2019;
originally announced March 2019.
-
Using reservoir computers to distinguish chaotic signals
Authors:
Thomas L. Carroll
Abstract:
Several recent papers have shown that reservoir computers are useful for analyzing and predicting dynamical systems. Reservoir computers have also been shown to be useful for various classification problems. In this work, a reservoir computer is used to identify one out of the 19 different Sprott systems. An advantage of reservoir computers for this problem is that no embedding is necessary. Some…
▽ More
Several recent papers have shown that reservoir computers are useful for analyzing and predicting dynamical systems. Reservoir computers have also been shown to be useful for various classification problems. In this work, a reservoir computer is used to identify one out of the 19 different Sprott systems. An advantage of reservoir computers for this problem is that no embedding is necessary. Some guidance on choosing the reservoir computer parameters is given. The dependance on number of points, number of reservoir nodes and noise in identifying the Sprott systems is explored.
△ Less
Submitted 21 September, 2018;
originally announced October 2018.
-
Testing Dynamical System Variables for Reconstruction
Authors:
Thomas L. Carroll
Abstract:
Analyzing data from dynamical systems often begins with creating a reconstruction of the trajectory based on one or more variables, but not all variables are suitable for reconstructing the trajectory. The concept of nonlinear observability has been investigated as a way to determine if a dynamical system can be reconstructed from one signal or a combination of signals, however nonlinear observabi…
▽ More
Analyzing data from dynamical systems often begins with creating a reconstruction of the trajectory based on one or more variables, but not all variables are suitable for reconstructing the trajectory. The concept of nonlinear observability has been investigated as a way to determine if a dynamical system can be reconstructed from one signal or a combination of signals, however nonlinear observability can be difficult to calculate for a high dimensional system. In this work I compare the results from nonlinear observability to a continuity statistic that indicates the likelihood that there is a continuous function between two sets of multidimensional points- in this case two different reconstructions of the same attractor from different signals simultaneously measured.
Without a metric against which to test the ability to reconstruct a system, the predictions of nonlinear observability and continuity are ambiguous. As a additional test how well different signals can predict the ability to reconstruct a dynamical system I use the fitting error from training a reservoir computer.
△ Less
Submitted 21 September, 2018; v1 submitted 27 August, 2018;
originally announced August 2018.
-
A Unified Approach to Attractor Reconstruction
Authors:
Louis M. Pecora,
Linda Moniz,
Jonathan Nichols,
Thomas L. Carroll
Abstract:
In the analysis of complex, nonlinear time series, scientists in a variety of disciplines have relied on a time delayed embedding of their data, i.e. attractor reconstruction. The process has focused primarily on heuristic and empirical arguments for selection of the key embedding parameters, delay and embedding dimension. This approach has left several long-standing, but common problems unresol…
▽ More
In the analysis of complex, nonlinear time series, scientists in a variety of disciplines have relied on a time delayed embedding of their data, i.e. attractor reconstruction. The process has focused primarily on heuristic and empirical arguments for selection of the key embedding parameters, delay and embedding dimension. This approach has left several long-standing, but common problems unresolved in which the standard approaches produce inferior results or give no guidance at all. We view the current reconstruction process as unnecessarily broken into separate problems. We propose an alternative approach that views the problem of choosing all embedding parameters as being one and the same problem addressable using a single statistical test formulated directly from the reconstruction theorems. This allows for varying time delays appropriate to the data and simultaneously helps decide on embedding dimension. A second new statistic, undersampling, acts as a check against overly long time delays and overly large embedding dimension. Our approach is more flexible than those currently used, but is more directly connected with the mathematical requirements of embedding. In addition, the statistics developed guide the user by allowing optimization and warning when embedding parameters are chosen beyond what the data can support. We demonstrate our approach on uni- and multivariate data, data possessing multiple time scales, and chaotic data. This unified approach resolves all the main issues in attractor reconstruction.
△ Less
Submitted 30 June, 2006; v1 submitted 23 February, 2006;
originally announced February 2006.
