-
convSeq: Fast and Scalable Method for Detecting Patterns in Spike Data
Authors:
Roman Koshkin,
Tomoki Fukai
Abstract:
Spontaneous neural activity, crucial in memory, learning, and spatial navigation, often manifests itself as repetitive spatiotemporal patterns. Despite their importance, analyzing these patterns in large neural recordings remains challenging due to a lack of efficient and scalable detection methods. Addressing this gap, we introduce convSeq, an unsupervised method that employs backpropagation for…
▽ More
Spontaneous neural activity, crucial in memory, learning, and spatial navigation, often manifests itself as repetitive spatiotemporal patterns. Despite their importance, analyzing these patterns in large neural recordings remains challenging due to a lack of efficient and scalable detection methods. Addressing this gap, we introduce convSeq, an unsupervised method that employs backpropagation for optimizing spatiotemporal filters that effectively identify these neural patterns. Our method's performance is validated on various synthetic data and real neural recordings, revealing spike sequences with unprecedented scalability and efficiency. Significantly surpassing existing methods in speed, convSeq sets a new standard for analyzing spontaneous neural activity, potentially advancing our understanding of information processing in neural circuits.
△ Less
Submitted 13 May, 2024; v1 submitted 1 February, 2024;
originally announced February 2024.
-
Simplicial Hopfield networks
Authors:
Thomas F Burns,
Tomoki Fukai
Abstract:
Hopfield networks are artificial neural networks which store memory patterns on the states of their neurons by choosing recurrent connection weights and update rules such that the energy landscape of the network forms attractors around the memories. How many stable, sufficiently-attracting memory patterns can we store in such a network using $N$ neurons? The answer depends on the choice of weights…
▽ More
Hopfield networks are artificial neural networks which store memory patterns on the states of their neurons by choosing recurrent connection weights and update rules such that the energy landscape of the network forms attractors around the memories. How many stable, sufficiently-attracting memory patterns can we store in such a network using $N$ neurons? The answer depends on the choice of weights and update rule. Inspired by setwise connectivity in biology, we extend Hopfield networks by adding setwise connections and embedding these connections in a simplicial complex. Simplicial complexes are higher dimensional analogues of graphs which naturally represent collections of pairwise and setwise relationships. We show that our simplicial Hopfield networks increase memory storage capacity. Surprisingly, even when connections are limited to a small random subset of equivalent size to an all-pairwise network, our networks still outperform their pairwise counterparts. Such scenarios include non-trivial simplicial topology. We also test analogous modern continuous Hopfield networks, offering a potentially promising avenue for improving the attention mechanism in Transformer models.
△ Less
Submitted 9 May, 2023;
originally announced May 2023.
-
Extended temporal association memory by inhibitory Hebbian learning
Authors:
Tatsuya Haga,
Tomoki Fukai
Abstract:
Hebbian learning of excitatory synapses plays a central role in storing activity patterns in associative memory models. Furthermore, interstimulus Hebbian learning associates multiple items in the brain by converting temporal correlation to spatial correlation between attractors. However, growing experimental evidence suggests that learning of inhibitory synapses creates "inhibitory engrams", whic…
▽ More
Hebbian learning of excitatory synapses plays a central role in storing activity patterns in associative memory models. Furthermore, interstimulus Hebbian learning associates multiple items in the brain by converting temporal correlation to spatial correlation between attractors. However, growing experimental evidence suggests that learning of inhibitory synapses creates "inhibitory engrams", which presumably balance with the patterns encoded in the excitatory network. Controlling inhibitory engrams may modify the behavior of associative memory in neural networks, but the consequence of such control has not been theoretically understood. Noting that Hebbian learning of inhibitory synapses yields an anti-Hebbian effect, we show that the combination of Hebbian and anti-Hebbian learning can increase the span of temporal association between the correlated attractors. The balance of targetted and global inhibition regulates this span of association in the network. Our results suggest a nontrivial role of anti-Hebbian learning and inhibitory engrams in associative memory.
△ Less
Submitted 14 September, 2018;
originally announced September 2018.
-
Discrete-attractor-like Tracking in Continuous Attractor Neural Networks
Authors:
Chi Chung Alan Fung,
Tomoki Fukai
Abstract:
Continuous attractor neural networks generate a set of smoothly connected attractor states. In memory systems of the brain, these attractor states may represent continuous pieces of information such as spatial locations and head directions of animals. However, during the replay of previous experiences, hippocampal neurons show a discontinuous sequence in which discrete transitions of neural state…
▽ More
Continuous attractor neural networks generate a set of smoothly connected attractor states. In memory systems of the brain, these attractor states may represent continuous pieces of information such as spatial locations and head directions of animals. However, during the replay of previous experiences, hippocampal neurons show a discontinuous sequence in which discrete transitions of neural state are phase-locked with the slow-gamma (30-40 Hz) oscillation. Here, we explored the underlying mechanisms of the discontinuous sequence generation. We found that a continuous attractor neural network has several phases depending on the interactions between external input and local inhibitory feedback. The discrete-attractor-like behavior naturally emerges in one of these phases without any discreteness assumption. We propose that the dynamics of continuous attractor neural networks is the key to generate discontinuous state changes phase-locked to the brain rhythm.
△ Less
Submitted 21 August, 2018;
originally announced August 2018.
-
Reliability of temporal coding on pulse-coupled networks of oscillators
Authors:
Jun-nosuke Teramae,
Tomoki Fukai
Abstract:
We study the reliability of spike output in a general class of pulse-coupled oscillators receiving a fluctuating input. Showing that this problem is equivalent to noise-induced synchronization between identical networks of oscillators, we employ the phase reduction method to analytically derive the average Lyapunov exponent of the synchronized state. We show that a transition occurs between reli…
▽ More
We study the reliability of spike output in a general class of pulse-coupled oscillators receiving a fluctuating input. Showing that this problem is equivalent to noise-induced synchronization between identical networks of oscillators, we employ the phase reduction method to analytically derive the average Lyapunov exponent of the synchronized state. We show that a transition occurs between reliable and unreliable responses at a critical coupling strength, which is determined through the competition between the external input and recurrent input. To our surprise, the critical value does not depend on intrinsic properties of oscillators.
△ Less
Submitted 6 August, 2007;
originally announced August 2007.
