-
A Neural Programming Language for the Reservoir Computer
Authors:
Jason Z. Kim,
Dani S. Bassett
Abstract:
From logical reasoning to mental simulation, biological and artificial neural systems possess an incredible capacity for computation. Such neural computers offer a fundamentally novel computing paradigm by representing data continuously and processing information in a natively parallel and distributed manner. To harness this computation, prior work has developed extensive training techniques to un…
▽ More
From logical reasoning to mental simulation, biological and artificial neural systems possess an incredible capacity for computation. Such neural computers offer a fundamentally novel computing paradigm by representing data continuously and processing information in a natively parallel and distributed manner. To harness this computation, prior work has developed extensive training techniques to understand existing neural networks. However, the lack of a concrete and low-level programming language for neural networks precludes us from taking full advantage of a neural computing framework. Here, we provide such a programming language using reservoir computing -- a simple recurrent neural network -- and close the gap between how we conceptualize and implement neural computers and silicon computers. By decomposing the reservoir's internal representation and dynamics into a symbolic basis of its inputs, we define a low-level neural machine code that we use to program the reservoir to solve complex equations and store chaotic dynamical systems as random access memory (dRAM). Using this representation, we provide a fully distributed neural implementation of software virtualization and logical circuits, and even program a playable game of pong inside of a reservoir computer. Taken together, we define a concrete, practical, and fully generalizable implementation of neural computation.
△ Less
Submitted 9 March, 2022;
originally announced March 2022.
-
Excitation and inhibition imbalance affects dynamical complexity through symmetries
Authors:
Mathieu Ouellet,
Jason Z. Kim,
Harmange Guillaume,
Sydney M. Shaffer,
Lee C. Bassett,
Dani S. Bassett
Abstract:
From the perfect radial symmetries of radiolarian mineral skeletons to the broken symmetry of homochirality, the logic of Nature's regularities has fascinated scientists for centuries. Some of Nature's symmetries are clearly visible in morphology and physical structure, whereas others are hidden in the network of interactions among system components. Just as visible symmetries and asymmetries cont…
▽ More
From the perfect radial symmetries of radiolarian mineral skeletons to the broken symmetry of homochirality, the logic of Nature's regularities has fascinated scientists for centuries. Some of Nature's symmetries are clearly visible in morphology and physical structure, whereas others are hidden in the network of interactions among system components. Just as visible symmetries and asymmetries contribute to the system's beauty, might hidden symmetries contribute to the system's functional harmony? And if so, how? Here we demonstrate that the interaction networks of biological systems from cell signaling to cancer display a form of dynamical reflection symmetry that serves to expand their dynamical complexity. The expansion proceeds according to precise rules regarding the lengths of dynamical cycles, made possible by a peculiar imbalance between excitation and inhibition. To probe the conditions under which this reflection symmetry evolves, we use a multi-objective genetic algorithm to produce networks with long dynamical cycles. We find that local structural motifs that break the reflection symmetry are deleted in the evolutionary process, providing evidence for symmetry's causal role in dynamical complexity. Finally, we identify symmetries in real continuous-valued scRNA-seq data of gene expression in drug-resistant cancer cells. Broadly, our work reveals a class of hidden symmetries in biology, present in the network of interactions among system components, and it underscores their functional importance. Mathematically simple and computational inexpensive, our approach is applicable to the types of biological data commonly acquired in modern experiments.
△ Less
Submitted 11 January, 2025; v1 submitted 23 February, 2022;
originally announced February 2022.
-
Learning Continuous Chaotic Attractors with a Reservoir Computer
Authors:
Lindsay M. Smith,
Jason Z. Kim,
Zhixin Lu,
Dani S. Bassett
Abstract:
Neural systems are well known for their ability to learn and store information as memories. Even more impressive is their ability to abstract these memories to create complex internal representations, enabling advanced functions such as the spatial manipulation of mental representations. While recurrent neural networks (RNNs) are capable of representing complex information, the exact mechanisms of…
▽ More
Neural systems are well known for their ability to learn and store information as memories. Even more impressive is their ability to abstract these memories to create complex internal representations, enabling advanced functions such as the spatial manipulation of mental representations. While recurrent neural networks (RNNs) are capable of representing complex information, the exact mechanisms of how dynamical neural systems perform abstraction are still not well-understood, thereby hindering the development of more advanced functions. Here, we train a 1000-neuron RNN -- a reservoir computer (RC) -- to abstract a continuous dynamical attractor memory from isolated examples of dynamical attractor memories. Further, we explain the abstraction mechanism with new theory. By training the RC on isolated and shifted examples of either stable limit cycles or chaotic Lorenz attractors, the RC learns a continuum of attractors, as quantified by an extra Lyapunov exponent equal to zero. We propose a theoretical mechanism of this abstraction by combining ideas from differentiable generalized synchronization and feedback dynamics. Our results quantify abstraction in simple neural systems, enabling us to design artificial RNNs for abstraction, and leading us towards a neural basis of abstraction.
△ Less
Submitted 16 October, 2021;
originally announced October 2021.
-
Teaching Recurrent Neural Networks to Modify Chaotic Memories by Example
Authors:
Jason Z. Kim,
Zhixin Lu,
Erfan Nozari,
George J. Pappas,
Danielle S. Bassett
Abstract:
The ability to store and manipulate information is a hallmark of computational systems. Whereas computers are carefully engineered to represent and perform mathematical operations on structured data, neurobiological systems perform analogous functions despite flexible organization and unstructured sensory input. Recent efforts have made progress in modeling the representation and recall of informa…
▽ More
The ability to store and manipulate information is a hallmark of computational systems. Whereas computers are carefully engineered to represent and perform mathematical operations on structured data, neurobiological systems perform analogous functions despite flexible organization and unstructured sensory input. Recent efforts have made progress in modeling the representation and recall of information in neural systems. However, precisely how neural systems learn to modify these representations remains far from understood. Here we demonstrate that a recurrent neural network (RNN) can learn to modify its representation of complex information using only examples, and we explain the associated learning mechanism with new theory. Specifically, we drive an RNN with examples of translated, linearly transformed, or pre-bifurcated time series from a chaotic Lorenz system, alongside an additional control signal that changes value for each example. By training the network to replicate the Lorenz inputs, it learns to autonomously evolve about a Lorenz-shaped manifold. Additionally, it learns to continuously interpolate and extrapolate the translation, transformation, and bifurcation of this representation far beyond the training data by changing the control signal. Finally, we provide a mechanism for how these computations are learned, and demonstrate that a single network can simultaneously learn multiple computations. Together, our results provide a simple but powerful mechanism by which an RNN can learn to manipulate internal representations of complex information, allowing for the principled study and precise design of RNNs.
△ Less
Submitted 3 May, 2020;
originally announced May 2020.
-
Supervised Chaotic Source Separation by a Tank of Water
Authors:
Zhixin Lu,
Jason Z. Kim,
Danielle S. Bassett
Abstract:
Whether listening to overlapping conversations in a crowded room or recording the simultaneous electrical activity of millions of neurons, the natural world abounds with sparse measurements of complex overlapping signals that arise from dynamical processes. While tools that separate mixed signals into linear sources have proven necessary and useful, the underlying equational forms of most natural…
▽ More
Whether listening to overlapping conversations in a crowded room or recording the simultaneous electrical activity of millions of neurons, the natural world abounds with sparse measurements of complex overlapping signals that arise from dynamical processes. While tools that separate mixed signals into linear sources have proven necessary and useful, the underlying equational forms of most natural signals are unknown and nonlinear. Hence, there is a need for a framework that is general enough to extract sources without knowledge of their generating equations, and flexible enough to accommodate nonlinear, even chaotic, sources. Here we provide such a framework, where the sources are chaotic trajectories from independently evolving dynamical systems. We consider the mixture signal as the sum of two chaotic trajectories, and propose a supervised learning scheme that extracts the chaotic trajectories from their mixture. Specifically, we recruit a complex dynamical system as an intermediate processor that is constantly driven by the mixture. We then obtain the separated chaotic trajectories based on this intermediate system by training the proper output functions. To demonstrate the generalizability of this framework \emph{in silico}, we employ a tank of water as the intermediate system, and show its success in separating two-part mixtures of various chaotic trajectories. Finally, we relate the underlying mechanism of this method to the state-observer problem. This relation provides a quantitative theory that explains the performance of our method, such as why separation is difficult when two source signals are trajectories from the same chaotic system.
△ Less
Submitted 6 January, 2020; v1 submitted 12 December, 2019;
originally announced December 2019.
