-
Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity
Authors:
Vladimir A. Ivanov,
Konstantinos P. Michmizos
Abstract:
The liquid state machine (LSM) combines low training complexity and biological plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as a model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks.…
▽ More
The liquid state machine (LSM) combines low training complexity and biological plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as a model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks. Recent findings in neuroscience suggest that astrocytes, a long-neglected non-neuronal brain cell, modulate synaptic plasticity and brain dynamics, tuning brain networks to the vicinity of the computationally optimal critical phase transition between order and chaos. Inspired by this disruptive understanding of how brain networks self-tune, we propose the neuron-astrocyte liquid state machine (NALSM) that addresses under-performance through self-organized near-critical dynamics. Similar to its biological counterpart, the astrocyte model integrates neuronal activity and provides global feedback to spike-timing-dependent plasticity (STDP), which self-organizes NALSM dynamics around a critical branching factor that is associated with the edge-of-chaos. We demonstrate that NALSM achieves state-of-the-art accuracy versus comparable LSM methods, without the need for data-specific hand-tuning. With a top accuracy of 97.61% on MNIST, 97.51% on N-MNIST, and 85.84% on Fashion-MNIST, NALSM achieved comparable performance to current fully-connected multi-layer spiking neural networks trained via backpropagation. Our findings suggest that the further development of brain-inspired machine learning methods has the potential to reach the performance of deep learning, with the added benefits of supporting robust and energy-efficient neuromorphic computing on the edge.
△ Less
Submitted 26 October, 2021;
originally announced November 2021.
-
Introducing Astrocytes on a Neuromorphic Processor: Synchronization, Local Plasticity and Edge of Chaos
Authors:
Guangzhi Tang,
Ioannis E. Polykretis,
Vladimir A. Ivanov,
Arpit Shah,
Konstantinos P. Michmizos
Abstract:
While there is still a lot to learn about astrocytes and their neuromodulatory role in the spatial and temporal integration of neuronal activity, their introduction to neuromorphic hardware is timely, facilitating their computational exploration in basic science questions as well as their exploitation in real-world applications. Here, we present an astrocytic module that enables the development of…
▽ More
While there is still a lot to learn about astrocytes and their neuromodulatory role in the spatial and temporal integration of neuronal activity, their introduction to neuromorphic hardware is timely, facilitating their computational exploration in basic science questions as well as their exploitation in real-world applications. Here, we present an astrocytic module that enables the development of a spiking Neuronal-Astrocytic Network (SNAN) into Intel's Loihi neuromorphic chip. The basis of the Loihi module is an end-to-end biophysically plausible compartmental model of an astrocyte that simulates the intracellular activity in response to the synaptic activity in space and time. To demonstrate the functional role of astrocytes in SNAN, we describe how an astrocyte may sense and induce activity-dependent neuronal synchronization, switch on and off spike-time-dependent plasticity (STDP) to introduce single-shot learning, and monitor the transition between ordered and chaotic activity at the synaptic space. Our module may serve as an extension for neuromorphic hardware, by either replicating or exploring the distinct computational roles that astrocytes have in forming biological intelligence.
△ Less
Submitted 19 September, 2019; v1 submitted 2 July, 2019;
originally announced July 2019.
-
Axonal Conduction Velocity Impacts Neuronal Network Oscillations
Authors:
Vladimir A. Ivanov,
Ioannis E. Polykretis,
Konstantinos P. Michmizos
Abstract:
Increasing experimental evidence suggests that axonal action potential conduction velocity is a highly adaptive parameter in the adult central nervous system. Yet, the effects of this newfound plasticity on global brain dynamics is poorly understood. In this work, we analyzed oscillations in biologically plausible neuronal networks with different conduction velocity distributions. Changes of 1-2 (…
▽ More
Increasing experimental evidence suggests that axonal action potential conduction velocity is a highly adaptive parameter in the adult central nervous system. Yet, the effects of this newfound plasticity on global brain dynamics is poorly understood. In this work, we analyzed oscillations in biologically plausible neuronal networks with different conduction velocity distributions. Changes of 1-2 (ms) in network mean signal transmission time resulted in substantial network oscillation frequency changes ranging in 0-120 (Hz). Our results suggest that changes in axonal conduction velocity may significantly affect both the frequency and synchrony of brain rhythms, which have well established connections to learning, memory, and other cognitive processes.
△ Less
Submitted 22 March, 2019;
originally announced March 2019.
-
Computational Astrocyence: Astrocytes encode inhibitory activity into the frequency and spatial extent of their calcium elevations
Authors:
Ioannis E. Polykretis,
Vladimir A. Ivanov,
Konstantinos P. Michmizos
Abstract:
Deciphering the complex interactions between neurotransmission and astrocytic $Ca^{2+}$ elevations is a target promising a comprehensive understanding of brain function. While the astrocytic response to excitatory synaptic activity has been extensively studied, how inhibitory activity results to intracellular $Ca^{2+}$ waves remains elusive. In this study, we developed a compartmental astrocytic m…
▽ More
Deciphering the complex interactions between neurotransmission and astrocytic $Ca^{2+}$ elevations is a target promising a comprehensive understanding of brain function. While the astrocytic response to excitatory synaptic activity has been extensively studied, how inhibitory activity results to intracellular $Ca^{2+}$ waves remains elusive. In this study, we developed a compartmental astrocytic model that exhibits distinct levels of responsiveness to inhibitory activity. Our model suggested that the astrocytic coverage of inhibitory terminals defines the spatial and temporal scale of their $Ca^{2+}$ elevations. Understanding the interplay between the synaptic pathways and the astrocytic responses will help us identify how astrocytes work independently and cooperatively with neurons, in health and disease.
△ Less
Submitted 18 March, 2019;
originally announced March 2019.
-
Coil-Globule Transition for Regular, Random and Specially Designed Copolymers: Monte Carlo Simulation and Self-Consistent Field Theory
Authors:
J. M. P. van den Oever,
F. A. M. Leermakers,
G. J. Fleer,
V. A. Ivanov,
N. P. Shusharina,
A. R. Khokhlov,
P. G. Khalatur
Abstract:
The coil-globule transition has been studied for A-B copolymer chains both by means of lattice Monte Carlo simulations using bond fluctuation algorithm and by a numerical self-consistent field method. Copolymer chains of fixed length with A and B monomeric units with regular, random and specially designed (protein-like) primary sequences have been investigated. The dependence of the transition t…
▽ More
The coil-globule transition has been studied for A-B copolymer chains both by means of lattice Monte Carlo simulations using bond fluctuation algorithm and by a numerical self-consistent field method. Copolymer chains of fixed length with A and B monomeric units with regular, random and specially designed (protein-like) primary sequences have been investigated. The dependence of the transition temperature on the AB sequence has been analyzed. A protein-like copolymer is more stable than a copolymer with statistically random sequence. The transition is more sharpe for random copolymers. It is found that there exist a temperature below which the chain appears to be in the lowest energy state (ground state). Both for random and protein-like sequences and for regular copolymers with a relatively long repeating block, a molten globule regime is found between the ground state temperature and the transition temperature. For regular block copolymers the transition temperature increases with block size. Qualitatively, the results from both methods are in agreement. Differences between the methods result from approximations in the SCF theory and equilibration problems in MC simulations. The two methods are thus complementary.
△ Less
Submitted 21 September, 2001;
originally announced September 2001.
-
Primary Sequences of Protein-Like Copolymers: Levy Flight Type Long Range Correlations
Authors:
Elena N. Govorun,
Victor A. Ivanov,
Alexei R. Khokhlov,
Pavel G. Khalatur,
Alexander L. Borovinsky,
Alexander Yu. Grosberg
Abstract:
We consider the statistical properties of primary sequences of two-letter HP copolymers (H for hydrophobic and P for polar) designed to have water soluble globular conformations with H monomers shielded from water inside the shell of P monomers. We show, both by computer simulations and by exact analytical calculation, that for large globules and flexible polymers such sequences exhibit long-ran…
▽ More
We consider the statistical properties of primary sequences of two-letter HP copolymers (H for hydrophobic and P for polar) designed to have water soluble globular conformations with H monomers shielded from water inside the shell of P monomers. We show, both by computer simulations and by exact analytical calculation, that for large globules and flexible polymers such sequences exhibit long-range correlations which can be described by Levy-flight statistics.
△ Less
Submitted 22 March, 2001; v1 submitted 9 February, 2001;
originally announced February 2001.
