Showing 1–14 of 14 results for author: Schneidman, E

The computational and learning benefits of Daleian neural networks

Authors: Adam Haber, Elad Schneidman

Abstract: Dale's principle implies that biological neural networks are composed of neurons that are either excitatory or inhibitory. While the number of possible architectures of such Daleian networks is exponentially smaller than non-Daleian ones, the computational and functional implications of using Daleian networks by the brain are mostly unknown. Here, we use models of recurrent spiking neural networks… ▽ More Dale's principle implies that biological neural networks are composed of neurons that are either excitatory or inhibitory. While the number of possible architectures of such Daleian networks is exponentially smaller than non-Daleian ones, the computational and functional implications of using Daleian networks by the brain are mostly unknown. Here, we use models of recurrent spiking neural networks and rate-based networks to show, surprisingly, that despite the structural limitations on Daleian networks, they can approximate the computation performed by non-Daleian networks to a very high degree of accuracy. Moreover, we find that Daleian networks are more functionally robust to synaptic noise. We then show that unlike non-Daleian networks, Daleian ones can learn efficiently by tuning single neuron features, nearly as well as learning by tuning individual synaptic weights - suggesting a simpler and more biologically plausible learning mechanism. We thus suggest that in addition to architectural simplicity, Dale's principle confers computational and learning benefits for biological networks, and offers new directions for constructing and training biologically-inspired artificial neural networks △ Less

Submitted 12 October, 2022; originally announced October 2022.

Probabilistic models of individual and collective animal behavior

Authors: Katarina Bodova, Gabriel J. Mitchell, Roy Harpaz, Elad Schneidman, Gasper Tkacik

Abstract: Recent developments in automated tracking allow uninterrupted, high-resolution recording of animal trajectories, sometimes coupled with the identification of stereotyped changes of body pose or other behaviors of interest. Analysis and interpretation of such data represents a challenge: the timing of animal behaviors may be stochastic and modulated by kinematic variables, by the interaction with t… ▽ More Recent developments in automated tracking allow uninterrupted, high-resolution recording of animal trajectories, sometimes coupled with the identification of stereotyped changes of body pose or other behaviors of interest. Analysis and interpretation of such data represents a challenge: the timing of animal behaviors may be stochastic and modulated by kinematic variables, by the interaction with the environment or with the conspecifics within the animal group, and dependent on internal cognitive or behavioral state of the individual. Existing models for collective motion typically fail to incorporate the discrete, stochastic, and internal-state-dependent aspects of behavior, while models focusing on individual animal behavior typically ignore the spatial aspects of the problem. Here we propose a probabilistic modeling framework to address this gap. Each animal can switch stochastically between different behavioral states, with each state resulting in a possibly different law of motion through space. Switching rates for behavioral transitions can depend in a very general way, which we seek to identify from data, on the effects of the environment as well as the interaction between the animals. We represent the switching dynamics as a Generalized Linear Model and show that: (i) forward simulation of multiple interacting animals is possible using a variant of the Gillespie's Stochastic Simulation Algorithm; (ii) formulated properly, the maximum likelihood inference of switching rate functions is tractably solvable by gradient descent; (iii) model selection can be used to identify factors that modulate behavioral state switching and to appropriately adjust model complexity to data. To illustrate our framework, we apply it to two synthetic models of animal motion and to real zebrafish tracking data. △ Less

Submitted 31 July, 2017; originally announced August 2017.

Comments: 26 pages, 11 figures

Discrete modes of social information processing predict individual behavior of fish in a group

Authors: Roy Harpaz, Gašper Tkačik, Elad Schneidman

Abstract: Individual computations and social interactions underlying collective behavior in groups of animals are of great ethological, behavioral, and theoretical interest. While complex individual behaviors have successfully been parsed into small dictionaries of stereotyped behavioral modes, studies of collective behavior largely ignored these findings; instead, their focus was on inferring single, mode-… ▽ More Individual computations and social interactions underlying collective behavior in groups of animals are of great ethological, behavioral, and theoretical interest. While complex individual behaviors have successfully been parsed into small dictionaries of stereotyped behavioral modes, studies of collective behavior largely ignored these findings; instead, their focus was on inferring single, mode-independent social interaction rules that reproduced macroscopic and often qualitative features of group behavior. Here we bring these two approaches together to predict individual swimming patterns of adult zebrafish in a group. We show that fish alternate between an active mode in which they are sensitive to the swimming patterns of conspecifics, and a passive mode where they ignore them. Using a model that accounts for these two modes explicitly, we predict behaviors of individual fish with high accuracy, outperforming previous approaches that assumed a single continuous computation by individuals and simple metric or topological weighing of neighbors behavior. At the group level, switching between active and passive modes is uncorrelated among fish, yet correlated directional swimming behavior still emerges. Our quantitative approach for studying complex, multi-modal individual behavior jointly with emergent group behavior is readily extensible to additional behavioral modes and their neural correlates, as well as to other species. △ Less

Submitted 8 March, 2017; originally announced March 2017.

Searching for collective behavior in a network of real neurons

Authors: Gašper Tkačik, Olivier Marre, Dario Amodei, Elad Schneidman, William Bialek, Michael J Berry II

Abstract: Maximum entropy models are the least structured probability distributions that exactly reproduce a chosen set of statistics measured in an interacting network. Here we use this principle to construct probabilistic models which describe the correlated spiking activity of populations of up to 120 neurons in the salamander retina as it responds to natural movies. Already in groups as small as 10 neur… ▽ More Maximum entropy models are the least structured probability distributions that exactly reproduce a chosen set of statistics measured in an interacting network. Here we use this principle to construct probabilistic models which describe the correlated spiking activity of populations of up to 120 neurons in the salamander retina as it responds to natural movies. Already in groups as small as 10 neurons, interactions between spikes can no longer be regarded as small perturbations in an otherwise independent system; for 40 or more neurons pairwise interactions need to be supplemented by a global interaction that controls the distribution of synchrony in the population. Here we show that such "K-pairwise" models--being systematic extensions of the previously used pairwise Ising models--provide an excellent account of the data. We explore the properties of the neural vocabulary by: 1) estimating its entropy, which constrains the population's capacity to represent visual information; 2) classifying activity patterns into a small set of metastable collective modes; 3) showing that the neural codeword ensembles are extremely inhomogenous; 4) demonstrating that the state of individual neurons is highly predictable from the rest of the population, allowing the capacity for error correction. △ Less

Submitted 13 June, 2013; originally announced June 2013.

Comments: 24 pages, 19 figures

Journal ref: PLOS Comput Biol 10 (2014): e1003408

Retinal metric: a stimulus distance measure derived from population neural responses

Authors: Gašper Tkačik, Einat Granot-Atedgi, Ronen Segev, Elad Schneidman

Abstract: The ability of the organism to distinguish between various stimuli is limited by the structure and noise in the population code of its sensory neurons. Here we infer a distance measure on the stimulus space directly from the recorded activity of 100 neurons in the salamander retina. In contrast to previously used measures of stimulus similarity, this "neural metric" tells us how distinguishable a… ▽ More The ability of the organism to distinguish between various stimuli is limited by the structure and noise in the population code of its sensory neurons. Here we infer a distance measure on the stimulus space directly from the recorded activity of 100 neurons in the salamander retina. In contrast to previously used measures of stimulus similarity, this "neural metric" tells us how distinguishable a pair of stimulus clips is to the retina, given the noise in the neural population response. We show that the retinal distance strongly deviates from Euclidean, or any static metric, yet has a simple structure: we identify the stimulus features that the neural population is jointly sensitive to, and show the SVM-like kernel function relating the stimulus and neural response spaces. We show that the non-Euclidean nature of the retinal distance has important consequences for neural decoding. △ Less

Submitted 8 December, 2012; v1 submitted 30 May, 2012; originally announced May 2012.

Comments: 5 pages, 4 figures, to appear in Phys Rev Lett

Journal ref: Phys Rev Lett 110 (2013): 058104

Stimulus-dependent maximum entropy models of neural population codes

Authors: Einat Granot-Atedgi, Gašper Tkačik, Ronen Segev, Elad Schneidman

Abstract: Neural populations encode information about their stimulus in a collective fashion, by joint activity patterns of spiking and silence. A full account of this mapping from stimulus to neural activity is given by the conditional probability distribution over neural codewords given the sensory input. To be able to infer a model for this distribution from large-scale neural recordings, we introduce a… ▽ More Neural populations encode information about their stimulus in a collective fashion, by joint activity patterns of spiking and silence. A full account of this mapping from stimulus to neural activity is given by the conditional probability distribution over neural codewords given the sensory input. To be able to infer a model for this distribution from large-scale neural recordings, we introduce a stimulus-dependent maximum entropy (SDME) model---a minimal extension of the canonical linear-nonlinear model of a single neuron, to a pairwise-coupled neural population. The model is able to capture the single-cell response properties as well as the correlations in neural spiking due to shared stimulus and due to effective neuron-to-neuron connections. Here we show that in a population of 100 retinal ganglion cells in the salamander retina responding to temporal white-noise stimuli, dependencies between cells play an important encoding role. As a result, the SDME model gives a more accurate account of single cell responses and in particular outperforms uncoupled models in reproducing the distributions of codewords emitted in response to a stimulus. We show how the SDME model, in conjunction with static maximum entropy models of population vocabulary, can be used to estimate information-theoretic quantities like surprise and information transmission in a neural population. △ Less

Submitted 29 May, 2012; originally announced May 2012.

Comments: 11 pages, 7 figures

Journal ref: PLoS Comput Biol 9 (2013): e1002922

Retinal adaptation and invariance to changes in higher-order stimulus statistics

Authors: Gašper Tkačik, Anandamohan Ghosh, Elad Schneidman, Ronen Segev

Abstract: Adaptation in the retina is thought to optimize the encoding of natural light signals into sequences of spikes sent to the brain. However, adaptation also entails computational costs: adaptive code is intrinsically ambiguous, because output symbols cannot be trivially mapped back to the stimuli without the knowledge of the adaptive state of the encoding neuron. It is thus important to learn which… ▽ More Adaptation in the retina is thought to optimize the encoding of natural light signals into sequences of spikes sent to the brain. However, adaptation also entails computational costs: adaptive code is intrinsically ambiguous, because output symbols cannot be trivially mapped back to the stimuli without the knowledge of the adaptive state of the encoding neuron. It is thus important to learn which statistical changes in the input do, and which do not, invoke adaptive responses, and ask about the reasons for potential limits to adaptation. We measured the ganglion cell responses in the tiger salamander retina to controlled changes in the second (contrast), third (skew) and fourth (kurtosis) moments of the light intensity distribution of spatially uniform temporally independent stimuli. The skew and kurtosis of the stimuli were chosen to cover the range observed in natural scenes. We quantified adaptation in ganglion cells by studying two-dimensional linear-nonlinear models that capture well the retinal encoding properties across all stimuli. We found that the retinal ganglion cells adapt to contrast, but exhibit remarkably invariant behavior to changes in higher-order statistics. Finally, by theoretically analyzing optimal coding in LN-type models, we showed that the neural code can maintain a high information rate without dynamic adaptation despite changes in stimulus skew and kurtosis. △ Less

Submitted 17 January, 2012; originally announced January 2012.

Journal ref: PLOS One 9 (2014): e85841

Spin glass models for a network of real neurons

Authors: Gasper Tkacik, Elad Schneidman, Michael J. Berry II, William Bialek

Abstract: Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins. Here we use this equivalence to construct Ising models that describe the correlated spiking activity of populations of 40 neurons in the salamander retina responding to natural movies. We show that pairwise interactio… ▽ More Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins. Here we use this equivalence to construct Ising models that describe the correlated spiking activity of populations of 40 neurons in the salamander retina responding to natural movies. We show that pairwise interactions between neurons account for observed higher-order correlations, and that for groups of 10 or more neurons pairwise interactions can no longer be regarded as small perturbations in an independent system. We then construct network ensembles that generalize the network instances observed in the experiment, and study their thermodynamic behavior and coding capacity. Based on this construction, we can also create synthetic networks of 120 neurons, and find that with increasing size the networks operate closer to a critical point and start exhibiting collective behaviors reminiscent of spin glasses. We examine closely two such behaviors that could be relevant for neural code: tuning of the network to the critical point to maximize the ability to encode diverse stimuli, and using the metastable states of the Ising Hamiltonian as neural code words. △ Less

Submitted 29 December, 2009; originally announced December 2009.

Comments: This is an extended version of arXiv:q-bio.NC/0611072

Ising models for networks of real neurons

Authors: Gasper Tkacik, Elad Schneidman, Michael J Berry II, William Bialek

Abstract: Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins. Here we use this equivalence to construct Ising models that describe the correlated spiking activity of populations of 40 neurons in the retina, and show that pairwise interactions account for observed higher-order co… ▽ More Ising models with pairwise interactions are the least structured, or maximum-entropy, probability distributions that exactly reproduce measured pairwise correlations between spins. Here we use this equivalence to construct Ising models that describe the correlated spiking activity of populations of 40 neurons in the retina, and show that pairwise interactions account for observed higher-order correlations. By first finding a representative ensemble for observed networks we can create synthetic networks of 120 neurons, and find that with increasing size the networks operate closer to a critical point and start exhibiting collective behaviors reminiscent of spin glasses. △ Less

Submitted 22 November, 2006; originally announced November 2006.

Comments: 4 pages, 3 figures

Synergy from Silence in a Combinatorial Neural Code

Authors: Elad Schneidman, Jason L. Puchalla, Ronen Segev, Robert A. Harris, William Bialek, Michael J. Berry II

Abstract: The manner in which groups of neurons represent events in the external world is fundamental to neuroscience. Here, we analyze the population code of the retina during naturalistic stimulation and show that the information conveyed by specific multi-neuronal firing patterns can be very different from the sum of the pattern's parts. Synchronous spikes convey more information than either of the par… ▽ More The manner in which groups of neurons represent events in the external world is fundamental to neuroscience. Here, we analyze the population code of the retina during naturalistic stimulation and show that the information conveyed by specific multi-neuronal firing patterns can be very different from the sum of the pattern's parts. Synchronous spikes convey more information than either of the participating cells, but almost always less than their sum, making them redundant coding symbols. Surprisingly, patterns of spiking and silence are mostly synergistic - carrying information that is accessible only by observing the whole pattern of activity, rather than its components, and signifying unique features in the stimulus. These results demonstrate that the retina uses a combinatorial code and that the brain can benefit significantly from recognizing multi-neuronal firing patterns. △ Less

Submitted 13 July, 2006; originally announced July 2006.

Comments: 9 pages, 6 figures

Weak pairwise correlations imply strongly correlated network states in a neural population

Authors: Elad Schneidman, Michael J. Berry II, Ronen Segev, William Bialek

Abstract: Biological networks have so many possible states that exhaustive sampling is impossible. Successful analysis thus depends on simplifying hypotheses, but experiments on many systems hint that complicated, higher order interactions among large groups of elements play an important role. In the vertebrate retina, we show that weak correlations between pairs of neurons coexist with strongly collectiv… ▽ More Biological networks have so many possible states that exhaustive sampling is impossible. Successful analysis thus depends on simplifying hypotheses, but experiments on many systems hint that complicated, higher order interactions among large groups of elements play an important role. In the vertebrate retina, we show that weak correlations between pairs of neurons coexist with strongly collective behavior in the responses of ten or more neurons. Surprisingly, we find that this collective behavior is described quantitatively by models that capture the observed pairwise correlations but assume no higher order interactions. These maximum entropy models are equivalent to Ising models, and predict that larger networks are completely dominated by correlation effects. This suggests that the neural code has associative or error-correcting properties, and we provide preliminary evidence for such behavior. As a first test for the generality of these ideas, we show that similar results are obtained from networks of cultured cortical neurons. △ Less

Submitted 6 December, 2005; originally announced December 2005.

Comments: Full account of work presented at the conference on Computational and Systems Neuroscience (COSYNE), 17-20 March 2005, in Salt Lake City, Utah (http://cosyne.org)

Network information and connected correlations

Authors: Elad Schneidman, Susanne Still, Michael J. Berry II, William Bialek

Abstract: Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease in entropy for the joint distribution of $N$ variables relative to the maximum entropy allowed by all the observed $N-1$ variable distributions. We calculate… ▽ More Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease in entropy for the joint distribution of $N$ variables relative to the maximum entropy allowed by all the observed $N-1$ variable distributions. We calculate the ``connected information'' terms for several examples, and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source. △ Less

Submitted 15 July, 2003; originally announced July 2003.

Comments: 4 pages, 3 figures

Journal ref: Phys. Rev. Lett. 91, 238701:1-4 (2003)

An information theoretic approach to the functional classification of neurons

Authors: Elad Schneidman, William Bialek, Michael J. Berry II

Abstract: A population of neurons typically exhibits a broad diversity of responses to sensory inputs. The intuitive notion of functional classification is that cells can be clustered so that most of the diversity is captured in the identity of the clusters rather than by individuals within clusters. We show how this intuition can be made precise using information theory, without any need to introduce a m… ▽ More A population of neurons typically exhibits a broad diversity of responses to sensory inputs. The intuitive notion of functional classification is that cells can be clustered so that most of the diversity is captured in the identity of the clusters rather than by individuals within clusters. We show how this intuition can be made precise using information theory, without any need to introduce a metric on the space of stimuli or responses. Applied to the retinal ganglion cells of the salamander, this approach recovers classical results, but also provides clear evidence for subclasses beyond those identified previously. Further, we find that each of the ganglion cells is functionally unique, and that even within the same subclass only a few spikes are needed to reliably distinguish between cells. △ Less

Submitted 31 December, 2002; originally announced December 2002.

Comments: 13 pages, 4 figures. To appear in Advances in Neural Information Processing Systems (NIPS) 15

Universality and individuality in a neural code

Authors: Elad Schneidman, Naama Brenner, Naftali Tishby, Rob R. de Ruyter van Steveninck, William Bialek

Abstract: The problem of neural coding is to understand how sequences of action potentials (spikes) are related to sensory stimuli, motor outputs, or (ultimately) thoughts and intentions. One clear question is whether the same coding rules are used by different neurons, or by corresponding neurons in different individuals. We present a quantitative formulation of this problem using ideas from information… ▽ More The problem of neural coding is to understand how sequences of action potentials (spikes) are related to sensory stimuli, motor outputs, or (ultimately) thoughts and intentions. One clear question is whether the same coding rules are used by different neurons, or by corresponding neurons in different individuals. We present a quantitative formulation of this problem using ideas from information theory, and apply this approach to the analysis of experiments in the fly visual system. We find significant individual differences in the structure of the code, particularly in the way that temporal patterns of spikes are used to convey information beyond that available from variations in spike rate. On the other hand, all the flies in our ensemble exhibit a high coding efficiency, so that every spike carries the same amount of information in all the individuals. Thus the neural code has a quantifiable mixture of individuality and universality. △ Less

Submitted 16 May, 2000; originally announced May 2000.

Comments: 16 pages, 4 figures