Showing 1–23 of 23 results for author: Panzeri, S

Theory of enhanced-by-coincidence neural information transmission

Authors: Miguel Ibáñez-Berganza, Giulio Bondanelli, Stefano Panzeri

Abstract: The activity of neurons within brain circuits has been ubiquitously reported to be correlated. The impact of these correlations on brain function has been extensively investigated. Correlations can in principle increase or decrease the information that neural populations carry about sensory stimuli, but experiments in cortical areas have mostly reported information-limiting correlations, which dec… ▽ More The activity of neurons within brain circuits has been ubiquitously reported to be correlated. The impact of these correlations on brain function has been extensively investigated. Correlations can in principle increase or decrease the information that neural populations carry about sensory stimuli, but experiments in cortical areas have mostly reported information-limiting correlations, which decrease the information encoded in the population. However, a second stream of evidence suggests that temporal correlations between the spiking activity of different neurons may increase the impact of neural activity downstream, implying that temporal correlations affect both the encoding of information and its downstream readout. The principle of how encoding and readout combine are still unclear. Here, we consider a model of transmission of stimulus information encoded in pre-synaptic input spike trains with information-limiting time-correlations to the output firing of a post-synaptic biologicaly-plausible leaky integrate and fire (LIF) readout neuron. We derive an analytical solution of the model in the diffusion approximation, in which the encoding spiking activity is treated as a continuous-time stochastic variable. An ansatz based on a separation of timescales allows us compute the stimulus information transmitted to the readout over a broad range of parameters. Our analytical results reveal that, for sufficiently low input firing rates, large enough difference in input stimulus-specific activity, and moderately large input temporal correlations, the stimulus discriminability of the firing of the LIF readout neuron can be enhanced by the presence of input time correlations, despite they decrease the stimulus information encoded in its inputs. △ Less

Submitted 14 February, 2025; originally announced February 2025.

Comments: 17 pages, 10 figures (with appendices: 41 pages and 25 figures)

Constraints on the design of neuromorphic circuits set by the properties of neural population codes

Authors: Stefano Panzeri, Ella Janotte, Alejandro Pequeño-Zurro, Jacopo Bonato, Chiara Bartolozzi

Abstract: In the brain, information is encoded, transmitted and used to inform behaviour at the level of timing of action potentials distributed over population of neurons. To implement neural-like systems in silico, to emulate neural function, and to interface successfully with the brain, neuromorphic circuits need to encode information in a way compatible to that used by populations of neuron in the brain… ▽ More In the brain, information is encoded, transmitted and used to inform behaviour at the level of timing of action potentials distributed over population of neurons. To implement neural-like systems in silico, to emulate neural function, and to interface successfully with the brain, neuromorphic circuits need to encode information in a way compatible to that used by populations of neuron in the brain. To facilitate the cross-talk between neuromorphic engineering and neuroscience, in this Review we first critically examine and summarize emerging recent findings about how population of neurons encode and transmit information. We examine the effects on encoding and readout of information for different features of neural population activity, namely the sparseness of neural representations, the heterogeneity of neural properties, the correlations among neurons, and the time scales (from short to long) at which neurons encode information and maintain it consistently over time. Finally, we critically elaborate on how these facts constrain the design of information coding in neuromorphic circuits. We focus primarily on the implications for designing neuromorphic circuits that communicate with the brain, as in this case it is essential that artificial and biological neurons use compatible neural codes. However, we also discuss implications for the design of neuromorphic systems for implementation or emulation of neural computation. △ Less

Submitted 8 December, 2022; originally announced December 2022.

Comments: Review paper

Biologically plausible solutions for spiking networks with efficient coding

Authors: Veronika Koren, Stefano Panzeri

Abstract: Understanding how the dynamics of neural networks is shaped by the computations they perform is a fundamental question in neuroscience. Recently, the framework of efficient coding proposed a theory of how spiking neural networks can compute low-dimensional stimulus signals with high efficiency. Efficient spiking networks are based on time-dependent minimization of a loss function related to inform… ▽ More Understanding how the dynamics of neural networks is shaped by the computations they perform is a fundamental question in neuroscience. Recently, the framework of efficient coding proposed a theory of how spiking neural networks can compute low-dimensional stimulus signals with high efficiency. Efficient spiking networks are based on time-dependent minimization of a loss function related to information coding with spikes. To inform the understanding of the function and dynamics of biological networks in the brain, however, the mathematical models have to be informed by biology and obey the same constraints as biological networks. Currently, spiking network models of efficient coding have been extended to include some features of biological plausibility, such as architectures with excitatory and inhibitory neurons. However, biological realism of efficient coding theories is still limited to simple cases and does not include single neuron and network properties that are known to be key in biological circuits. Here, we revisit the theory of efficient coding with spikes to develop spiking neural networks that are closer to biological circuits. Namely, we find a biologically plausible spiking model realizing efficient coding in the case of a generalized leaky integrate-and-fire network with excitatory and inhibitory units, equipped with fast and slow synaptic currents, local homeostatic currents such as spike-triggered adaptation, hyperpolarization-activated rebound current, heterogeneous firing thresholds and resets, heterogeneous postsynaptic potentials, and structured, low-rank connectivity. We show how the complexity of E-E connectivity matrix shapes network responses. △ Less

Submitted 24 October, 2022; v1 submitted 13 October, 2022; originally announced October 2022.

Comments: in press at NeurIPS 2022- Thirty-sixth Conference on Neural Information Processing Systems

Mathematical studies of the dynamics of finite-size binary neural networks: A review of recent progress

Authors: Diego Fasoli, Stefano Panzeri

Abstract: Traditional mathematical approaches to studying analytically the dynamics of neural networks rely on the mean-field approximation, which is rigorously applicable only to networks of infinite size. However, all existing real biological networks have finite size, and many of them, such as microscopic circuits in invertebrates, are composed only of a few tens of neurons. Thus, it is important to be a… ▽ More Traditional mathematical approaches to studying analytically the dynamics of neural networks rely on the mean-field approximation, which is rigorously applicable only to networks of infinite size. However, all existing real biological networks have finite size, and many of them, such as microscopic circuits in invertebrates, are composed only of a few tens of neurons. Thus, it is important to be able to extend to small-size networks our ability to study analytically neural dynamics. Analytical solutions of the dynamics of finite-size neural networks have remained elusive for many decades, because the powerful methods of statistical analysis, such as the central limit theorem and the law of large numbers, do not apply to small networks. In this article, we critically review recent progress on the study of the dynamics of small networks composed of binary neurons. In particular, we review the mathematical techniques we developed for studying the bifurcations of the network dynamics, the dualism between neural activity and membrane potentials, cross-neuron correlations, and pattern storage in stochastic networks. Finally, we highlight key challenges that remain open, future directions for further progress, and possible implications of our results for neuroscience. △ Less

Submitted 29 April, 2019; originally announced April 2019.

Comments: 27 pages, 7 figures

Stationary-State Statistics of a Binary Neural Network Model with Quenched Disorder

Authors: Diego Fasoli, Stefano Panzeri

Abstract: We study the statistical properties of the stationary firing-rate states of a neural network model with quenched disorder. The model has arbitrary size, discrete-time evolution equations and binary firing rates, while the topology and the strength of the synaptic connections are randomly generated from known, generally arbitrary, probability distributions. We derived semi-analytical expressions of… ▽ More We study the statistical properties of the stationary firing-rate states of a neural network model with quenched disorder. The model has arbitrary size, discrete-time evolution equations and binary firing rates, while the topology and the strength of the synaptic connections are randomly generated from known, generally arbitrary, probability distributions. We derived semi-analytical expressions of the occurrence probability of the stationary states and the mean multistability diagram of the model, in terms of the distribution of the synaptic connections and of the external stimuli to the network. Our calculations rely on the probability distribution of the bifurcation points of the stationary states with respect to the external stimuli, which can be calculated in terms of the permanent of special matrices, according to extreme value theory. While our semi-analytical expressions are exact for any size of the network and for any distribution of the synaptic connections, we also specialized our calculations to the case of statistically-homogeneous multi-population networks. In the specific case of this network topology, we calculated analytically the permanent, obtaining a compact formula that outperforms of several orders of magnitude the Balasubramanian-Bax-Franklin-Glynn algorithm. To conclude, by applying the Fisher-Tippett-Gnedenko theorem, we derived asymptotic expressions of the stationary-state statistics of multi-population networks in the large-network-size limit, in terms of the Gumbel (double exponential) distribution. We also provide a Python implementation of our formulas and some examples of the results generated by the code. △ Less

Submitted 29 November, 2018; originally announced November 2018.

Comments: 30 pages, 6 figures, 2 supplemental Python scripts

Information estimation using nonparametric copulas

Authors: Houman Safaai, Arno Onken, Christopher D. Harvey, Stefano Panzeri

Abstract: Estimation of mutual information between random variables has become crucial in a range of fields, from physics to neuroscience to finance. Estimating information accurately over a wide range of conditions relies on the development of flexible methods to describe statistical dependencies among variables, without imposing potentially invalid assumptions on the data. Such methods are needed in cases… ▽ More Estimation of mutual information between random variables has become crucial in a range of fields, from physics to neuroscience to finance. Estimating information accurately over a wide range of conditions relies on the development of flexible methods to describe statistical dependencies among variables, without imposing potentially invalid assumptions on the data. Such methods are needed in cases that lack prior knowledge of their statistical properties and that have limited sample numbers. Here we propose a powerful and generally applicable information estimator based on non-parametric copulas. This estimator, called the non-parametric copula-based estimator (NPC), is tailored to take into account detailed stochastic relationships in the data independently of the data's marginal distributions. The NPC estimator can be used both for continuous and discrete numerical variables and thus provides a single framework for the mutual information estimation of both continuous and discrete data. By extensive validation on artificial samples drawn from various statistical distributions, we found that the NPC estimator compares well against commonly used alternatives. Unlike methods not based on copulas, it allows an estimation of information that is robust to changes of the details of the marginal distributions. Unlike parametric copula methods, it remains accurate regardless of the precise form of the interactions between the variables. In addition, the NPC estimator had accurate information estimates even at low sample numbers, in comparison to alternative estimators. The NPC estimator therefore provides a good balance between general applicability to arbitrarily shaped statistical dependencies in the data and shows accurate and robust performance when working with small sample sizes. △ Less

Submitted 25 October, 2018; v1 submitted 20 July, 2018; originally announced July 2018.

Comments: 17 pages, 13 figures

Journal ref: Phys. Rev. E 98, 053302 (2018)

Synthesizing realistic neural population activity patterns using Generative Adversarial Networks

Authors: Manuel Molano-Mazon, Arno Onken, Eugenio Piasini, Stefano Panzeri

Abstract: The ability to synthesize realistic patterns of neural activity is crucial for studying neural information processing. Here we used the Generative Adversarial Networks (GANs) framework to simulate the concerted activity of a population of neurons. We adapted the Wasserstein-GAN variant to facilitate the generation of unconstrained neural population activity patterns while still benefiting from par… ▽ More The ability to synthesize realistic patterns of neural activity is crucial for studying neural information processing. Here we used the Generative Adversarial Networks (GANs) framework to simulate the concerted activity of a population of neurons. We adapted the Wasserstein-GAN variant to facilitate the generation of unconstrained neural population activity patterns while still benefiting from parameter sharing in the temporal domain. We demonstrate that our proposed GAN, which we termed Spike-GAN, generates spike trains that match accurately the first- and second-order statistics of datasets of tens of neurons and also approximates well their higher-order statistics. We applied Spike-GAN to a real dataset recorded from salamander retina and showed that it performs as well as state-of-the-art approaches based on the maximum entropy and the dichotomized Gaussian frameworks. Importantly, Spike-GAN does not require to specify a priori the statistics to be matched by the model, and so constitutes a more flexible method than these alternative approaches. Finally, we show how to exploit a trained Spike-GAN to construct 'importance maps' to detect the most relevant statistical structures present in a spike train. Spike-GAN provides a powerful, easy-to-use technique for generating realistic spiking neural activity and for describing the most relevant features of the large-scale neural population recordings studied in modern systems neuroscience. △ Less

Submitted 17 April, 2018; v1 submitted 1 March, 2018; originally announced March 2018.

Comments: Published as a conference paper at ICLR 2018 V2: minor changes in supp. material

Quantifying how much sensory information in a neural code is relevant for behavior

Authors: Giuseppe Pica, Eugenio Piasini, Houman Safaai, Caroline A. Runyan, Mathew E. Diamond, Tommaso Fellin, Christoph Kayser, Christopher D. Harvey, Stefano Panzeri

Abstract: Determining how much of the sensory information carried by a neural code contributes to behavioral performance is key to understand sensory function and neural information flow. However, there are as yet no analytical tools to compute this information that lies at the intersection between sensory coding and behavioral readout. Here we develop a novel measure, termed the information-theoretic inter… ▽ More Determining how much of the sensory information carried by a neural code contributes to behavioral performance is key to understand sensory function and neural information flow. However, there are as yet no analytical tools to compute this information that lies at the intersection between sensory coding and behavioral readout. Here we develop a novel measure, termed the information-theoretic intersection information $I_{II}(S;R;C)$, that quantifies how much of the sensory information carried by a neural response R is used for behavior during perceptual discrimination tasks. Building on the Partial Information Decomposition framework, we define $I_{II}(S;R;C)$ as the part of the mutual information between the stimulus S and the response R that also informs the consequent behavioral choice C. We compute $I_{II}(S;R;C)$ in the analysis of two experimental cortical datasets, to show how this measure can be used to compare quantitatively the contributions of spike timing and spike rates to task performance, and to identify brain areas or neural populations that specifically transform sensory information into choice. △ Less

Submitted 6 December, 2017; originally announced December 2017.

Journal ref: Advances in Neural Information Processing Systems 30, 3689--3699, 2017

The identity of information: how deterministic dependencies constrain information synergy and redundancy

Authors: Daniel Chicharro, Giuseppe Pica, Stefano Panzeri

Abstract: Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) which separates the mut… ▽ More Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) which separates the mutual information that a set of sources contains about a set of targets into nonnegative terms interpretable as these pieces. Quantifying redundancy requires assigning an identity to different information pieces, to assess when information is common across sources. Harder et al. (2013) proposed an identity axiom stating that there cannot be redundancy between two independent sources about a copy of themselves. However, Bertschinger et al. (2012) showed that with a deterministically related sources-target copy this axiom is incompatible with ensuring PID nonnegativity. Here we study systematically the effect of deterministic target-sources dependencies. We introduce two synergy stochasticity axioms that generalize the identity axiom, and we derive general expressions separating stochastic and deterministic PID components. Our analysis identifies how negative terms can originate from deterministic dependencies and shows how different assumptions on information identity, implicit in the stochasticity and identity axioms, determine the PID structure. The implications for studying neural coding are discussed. △ Less

Submitted 13 November, 2017; originally announced November 2017.

MSC Class: 94A15; 94A17

Optimized brute-force algorithms for the bifurcation analysis of a spin-glass-like neural network model

Authors: Diego Fasoli, Stefano Panzeri

Abstract: Bifurcation theory is a powerful tool for studying how the dynamics of a neural network model depends on its underlying neurophysiological parameters. However, bifurcation theory has been developed mostly for smooth dynamical systems and for continuous-time non-smooth models, which prevents us from understanding the changes of dynamics in some widely used classes of artificial neural network model… ▽ More Bifurcation theory is a powerful tool for studying how the dynamics of a neural network model depends on its underlying neurophysiological parameters. However, bifurcation theory has been developed mostly for smooth dynamical systems and for continuous-time non-smooth models, which prevents us from understanding the changes of dynamics in some widely used classes of artificial neural network models. This article is an attempt to fill this gap, through the introduction of algorithms that perform a semi-analytical bifurcation analysis of a spin-glass-like neural network model with binary firing rates and discrete-time evolution. Our approach is based on a numerical brute-force search of the stationary and oscillatory solutions of the spin-glass model, from which we derive analytical expressions of its bifurcation structure by means of the state-to-state transition probability matrix. The algorithms determine how the network parameters affect the degree of multistability, the emergence and the period of the neural oscillations, and the formation of symmetry-breaking in the neural populations. While this technique can be applied to networks with arbitrary (generally asymmetric) connectivity matrices, in particular we introduce a highly efficient algorithm for the bifurcation analysis of sparse networks. We also provide some examples of the obtained bifurcation diagrams and a Python implementation of the algorithms. △ Less

Submitted 16 May, 2017; originally announced May 2017.

Comments: 22 pages, 5 figures, 4 Python scripts

Journal ref: Phys. Rev. E 99, 012316 (2019)

Pattern Storage, Bifurcations and Higher-Order Correlation Structure of an Exactly Solvable Asymmetric Neural Network Model

Authors: Diego Fasoli, Anna Cattani, Stefano Panzeri

Abstract: Exactly solvable neural network models with asymmetric weights are rare, and exact solutions are available only in some mean-field approaches. In this article we find exact analytical solutions of an asymmetric spin-glass-like model of arbitrary size and we perform a complete study of its dynamical and statistical properties. The network has discrete-time evolution equations, binary firing rates a… ▽ More Exactly solvable neural network models with asymmetric weights are rare, and exact solutions are available only in some mean-field approaches. In this article we find exact analytical solutions of an asymmetric spin-glass-like model of arbitrary size and we perform a complete study of its dynamical and statistical properties. The network has discrete-time evolution equations, binary firing rates and can be driven by noise with any distribution. We find analytical expressions of the conditional and stationary joint probability distributions of the membrane potentials and the firing rates. The conditional probability distribution of the firing rates allows us to introduce a new learning rule to store safely, under the presence of noise, point and cyclic attractors, with important applications in the field of content-addressable memories. Furthermore, we study the neuronal dynamics in terms of the bifurcation structure of the network. We derive analytically examples of the codimension one and codimension two bifurcation diagrams of the network, which describe how the neuronal dynamics changes with the external stimuli. In particular, we find that the network may undergo transitions among multistable regimes, oscillatory behavior elicited by asymmetric synaptic connections, and various forms of spontaneous symmetry-breaking. On the other hand, the joint probability distributions allow us to calculate analytically the higher-order correlation structure of the network, which reveals neuronal regimes where, statistically, the membrane potentials and the firing rates are either synchronous or asynchronous. Our results are valid for networks composed of an arbitrary number of neurons, but for completeness we also derive the network equations in the mean-field limit and we study analytically their local bifurcations. All the analytical results are extensively validated by numerical simulations. △ Less

Submitted 15 February, 2017; v1 submitted 10 February, 2017; originally announced February 2017.

Comments: Main Text: 25 pages, 10 figures. Supplementary Materials: 25 pages, 3 figures. In version 1, we plotted Fig. 7 with an inhibitory current that was different from that stated in the figure caption. We fixed this in version 2, by replotting Fig. 7 with the correct value of the current. Results unchanged

Redundancy and synergy in dual decompositions of mutual information gain and information loss

Authors: Daniel Chicharro, Stefano Panzeri

Abstract: Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of information gain lattices, which allows separating the information that a set of variables contains about another into components interpretable as the unique information of one variable, or redundant and synergy components. In this work we extend the framework of Williams and Beer (2010)… ▽ More Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of information gain lattices, which allows separating the information that a set of variables contains about another into components interpretable as the unique information of one variable, or redundant and synergy components. In this work we extend the framework of Williams and Beer (2010) focusing on the lattices that underpin the decomposition. We generalize the type of constructible lattices and examine the relations between the terms in different lattices, for example relating bivariate and trivariate decompositions. We point out that, in information gain lattices, redundancy components are invariant across decompositions, but unique and synergy components are decomposition-dependent. Exploiting the connection between different lattices we propose a procedure to construct, in the general multivariate case, information decompositions from measures of synergy or unique information. We introduce an alternative type of mutual information decompositions based on information loss lattices, with the role and invariance properties of redundancy and synergy components exchanged with respect to gain lattices. We study the correspondence between information gain and information loss lattices and we define dual decompositions that allow overcoming the intrinsic asymmetry between invariant and decomposition-dependent components, which hinders the consistent joint characterization of synergy and redundancy. △ Less

Submitted 30 December, 2016; originally announced December 2016.

MSC Class: 94A15; 94A17

From Local Chaos to Critical Slowing Down: A Theory of the Functional Connectivity of Small Neural Circuits

Authors: Diego Fasoli, Anna Cattani, Stefano Panzeri

Abstract: Functional connectivity is a fundamental property of neural networks that quantifies the segregation and integration of information between cortical areas. Due to mathematical complexity, a theory that could explain how the parameters of mesoscopic networks composed of a few tens of neurons affect the functional connectivity is still to be formulated. Yet, many interesting problems in neuroscience… ▽ More Functional connectivity is a fundamental property of neural networks that quantifies the segregation and integration of information between cortical areas. Due to mathematical complexity, a theory that could explain how the parameters of mesoscopic networks composed of a few tens of neurons affect the functional connectivity is still to be formulated. Yet, many interesting problems in neuroscience involve the study of networks composed of a small number of neurons. Based on a recent study of the dynamics of small neural circuits, we combine the analysis of local bifurcations of multi-population neural networks of arbitrary size with the analytical calculation of the functional connectivity. We study the functional connectivity in different regimes, showing that external stimuli cause the network to switch from asynchronous states characterized by weak correlation and low variability (local chaos), to synchronous states characterized by strong correlations and wide temporal fluctuations (critical slowing down). Local chaos typically occurs in large networks, but here we show that it can also be generated by strong stimuli in small neural circuits. On the other side, critical slowing down is expected to occur when the stimulus moves the network close to a local bifurcation. In particular, strongly positive correlations occur at the saddle-node and Andronov-Hopf bifurcations of the network, while strongly negative correlations occur when the network undergoes a spontaneous symmetry-breaking at the branching-point bifurcations. These results prove that the functional connectivity of firing-rate network models is strongly affected by the external stimuli even if the anatomical connections are fixed, and suggest an effective mechanism through which biological networks can dynamically modulate the encoding and integration of sensory information. △ Less

Submitted 26 May, 2016; v1 submitted 24 May, 2016; originally announced May 2016.

Comments: 24 pages, 5 figures; compiled version of the Supplementary Materials added (12 pages)

The complexity of dynamics in small neural circuits

Authors: Diego Fasoli, Anna Cattani, Stefano Panzeri

Abstract: Mean-field theory is a powerful tool for studying large neural networks. However, when the system is composed of a few neurons, macroscopic differences between the mean-field approximation and the real behavior of the network can arise. Here we introduce a study of the dynamics of a small firing-rate network with excitatory and inhibitory populations, in terms of local and global bifurcations of t… ▽ More Mean-field theory is a powerful tool for studying large neural networks. However, when the system is composed of a few neurons, macroscopic differences between the mean-field approximation and the real behavior of the network can arise. Here we introduce a study of the dynamics of a small firing-rate network with excitatory and inhibitory populations, in terms of local and global bifurcations of the neural activity. Our approach is analytically tractable in many respects, and sheds new light on the finite-size effects of the system. In particular, we focus on the formation of multiple branching solutions of the neural equations through spontaneous symmetry-breaking, since this phenomenon increases considerably the complexity of the dynamical behavior of the network. For these reasons, branching points may reveal important mechanisms through which neurons interact and process information, which are not accounted for by the mean-field approximation. △ Less

Submitted 22 July, 2015; v1 submitted 30 June, 2015; originally announced June 2015.

Comments: 34 pages, 11 figures. Supplementary materials added, colors of figures 8 and 9 fixed, results unchanged

Phase-of-firing code

Authors: Anna Cattani, Gaute T. Einevoll, Stefano Panzeri

Abstract: Definition. The phase-of-firing code is a neural coding scheme whereby neurons encode information using the time at which they fire spikes within a cycle of the ongoing oscillatory pattern of network activity. This coding scheme may allow neurons to use their temporal pattern of spikes to encode information that is not encoded in their firing rate. Definition. The phase-of-firing code is a neural coding scheme whereby neurons encode information using the time at which they fire spikes within a cycle of the ongoing oscillatory pattern of network activity. This coding scheme may allow neurons to use their temporal pattern of spikes to encode information that is not encoded in their firing rate. △ Less

Submitted 15 April, 2015; originally announced April 2015.

Comments: In press, Encyclopedia of Computational Neuroscience, 2015

Estimating Information-Theoretic Quantities

Authors: Robin A. A. Ince, Simon R. Schultz, Stefano Panzeri

Abstract: Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. It has a number of useful properties: it is a general measure sensitive to any relationship, not only linear effects; it… ▽ More Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. It has a number of useful properties: it is a general measure sensitive to any relationship, not only linear effects; it has meaningful units which in many cases allow direct comparison between different experiments; and it can be used to study how much information can be gained by observing neural responses in single trials, rather than in averages over multiple trials. A variety of information theoretic quantities are in common use in neuroscience - (see entry "Summary of Information-Theoretic Quantities"). Estimating these quantities in an accurate and unbiased way from real neurophysiological data frequently presents challenges, which are explained in this entry. △ Less

Submitted 8 January, 2015; originally announced January 2015.

Comments: 16 pages, 3 figures

Journal ref: Encyclopaedia of Computational Neuroscience 2014, pp 1-13

Applications of Information Theory to Analysis of Neural Data

Authors: Simon R. Schultz, Robin A. A. Ince, Stefano Panzeri

Abstract: Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. It has a number of useful properties: it is a general measure sensitive to any relationship, not only linear effects; it… ▽ More Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. It has a number of useful properties: it is a general measure sensitive to any relationship, not only linear effects; it has meaningful units which in many cases allow direct comparison between different experiments; and it can be used to study how much information can be gained by observing neural responses in single trials, rather than in averages over multiple trials. A variety of information theoretic quantities are commonly used in neuroscience - (see entry "Definitions of Information-Theoretic Quantities"). In this entry we review some applications of information theory in neuroscience to study encoding of information in both single neurons and neuronal populations. △ Less

Submitted 8 January, 2015; originally announced January 2015.

Comments: 8 pages, 2 figures

Journal ref: Encyclopedia of Computational Neuroscience 2014, pp 1-6

Summary of Information Theoretic Quantities

Authors: Robin A. A. Ince, Stefano Panzeri, Simon R. Schultz

Abstract: Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. As a framework it has a number of useful properties: it provides a general measure sensitive to any relationship, not onl… ▽ More Information theory is a practical and theoretical framework developed for the study of communication over noisy channels. Its probabilistic basis and capacity to relate statistical structure to function make it ideally suited for studying information flow in the nervous system. As a framework it has a number of useful properties: it provides a general measure sensitive to any relationship, not only linear effects; its quantities have meaningful units which in many cases allow direct comparison between different experiments; and it can be used to study how much information can be gained by observing neural responses in single experimental trials, rather than in averages over multiple trials. A variety of information theoretic quantities are in common use in neuroscience - including the Shannon entropy, Kullback-Leibler divergence, and mutual information. In this entry, we introduce and define these quantities. Further details on how these quantities can be estimated in practice are provided in the entry "Estimation of Information-Theoretic Quantities" and examples of application of these techniques in neuroscience can be found in the entry "Applications of Information-Theoretic Quantities in Neuroscience". △ Less

Submitted 8 January, 2015; originally announced January 2015.

Comments: 7 pages

Journal ref: Encyclopaedia of Computational Neuroscience 2014, pp 1-6

The information content of Local Field Potentials: experiments and models

Authors: Alberto Mazzoni, Nikos K. Logothetis, Stefano Panzeri

Abstract: The LFPs is a broadband signal that captures variations of neural population activity over a wide range of time scales. The range of time scales available in LFPs is particularly interesting from the neural coding point of view because it opens up the possibility to investigate whether there are privileged time scales for information processing, a question that has been hotly debated over the last… ▽ More The LFPs is a broadband signal that captures variations of neural population activity over a wide range of time scales. The range of time scales available in LFPs is particularly interesting from the neural coding point of view because it opens up the possibility to investigate whether there are privileged time scales for information processing, a question that has been hotly debated over the last one or two decades.It is possible that information is represented by only a small number of specific frequency ranges, each carrying a separate contribution to the information representation. To shed light on this issue, it is important to quantify the information content of each frequency range of neural activity, and understand which ranges carry complementary or similar information. △ Less

Submitted 4 June, 2012; originally announced June 2012.

Comments: To appear in Quian Quiroga and Panzeri (Eds) Principles of Neural Coding, CRC Press, 2012

Synchronisation, binding and the role of correlated firing in fast information transmission

Authors: Simon R. Schultz, Huw D. R. Golledge, Stefano Panzeri

Abstract: Does synchronization between action potentials from different neurons in the visual system play a substantial role in solving the binding problem? The binding problem can be studied quantitatively in the broader framework of the information contained in neural spike trains about some external correlate, which in this case is object configurations in the visual field. We approach this problem by… ▽ More Does synchronization between action potentials from different neurons in the visual system play a substantial role in solving the binding problem? The binding problem can be studied quantitatively in the broader framework of the information contained in neural spike trains about some external correlate, which in this case is object configurations in the visual field. We approach this problem by using a mathematical formalism that quantifies the impact of correlated firing in short time scales. Using a power series expansion, the mutual information an ensemble of neurons conveys about external stimuli is broken down into firing rate and correlation components. This leads to a new quantification procedure directly applicable to simultaneous multiple neuron recordings. It theoretically constrains the neural code, showing that correlations contribute less significantly than firing rates to rapid information processing. By using this approach to study the limits upon the amount of information that an ideal observer is able to extract from a synchrony code, it may be possible to determine whether the available amount of information is sufficient to support computational processes such as feature binding. △ Less

Submitted 30 June, 2000; originally announced June 2000.

Comments: 15 pages, to appear in Emergent Neural Computational Architectures Based on Neuroscience, Lecture Notes in Computer Science, Springer-Verlag, Heidelberg

Temporal correlations and neural spike train entropy

Authors: Simon R. Schultz, Stefano Panzeri

Abstract: Sampling considerations limit the experimental conditions under which information theoretic analyses of neurophysiological data yield reliable results. We develop a procedure for computing the full temporal entropy and information of ensembles of neural spike trains, which performs reliably for limited samples of data. This approach also yields insight upon the role of correlations between spike… ▽ More Sampling considerations limit the experimental conditions under which information theoretic analyses of neurophysiological data yield reliable results. We develop a procedure for computing the full temporal entropy and information of ensembles of neural spike trains, which performs reliably for limited samples of data. This approach also yields insight upon the role of correlations between spikes in temporal coding mechanisms. The method, when applied to recordings from complex cells of the monkey primary visual cortex, results in lower RMS error information estimates in comparison to a `brute force' approach. △ Less

Submitted 11 April, 2001; v1 submitted 4 January, 2000; originally announced January 2000.

Comments: 4 pages, 3 figures; final published version. In press, Physical Review Letters

A unified approach to the study of temporal, correlational and rate coding

Authors: S. Panzeri, S. R. Schultz

Abstract: We demonstrate that the information contained in the spike occurrence times of a population of neurons can be broken up into a series of terms, each of which reflect something about potential coding mechanisms. This is possible in the coding r{é}gime in which few spikes are emitted in the relevant time window. This approach allows us to study the additional information contributed by spike tim… ▽ More We demonstrate that the information contained in the spike occurrence times of a population of neurons can be broken up into a series of terms, each of which reflect something about potential coding mechanisms. This is possible in the coding r{é}gime in which few spikes are emitted in the relevant time window. This approach allows us to study the additional information contributed by spike timing beyond that present in the spike counts; to examine the contributions to the whole information of different statistical properties of spike trains, such as firing rates and correlation functions; and forms the basis for a new quantitative procedure for the analysis of simultaneous multiple neuron recordings. It also provides theoretical constraints upon neural coding strategies. We find a transition between two coding r{é}gimes, depending upon the size of the relevant observation timescale. For time windows shorter than the timescale of the stimulus-induced response fluctuations, there exists a spike count coding phase, where the purely temporal information is of third order in time. For time windows much longer than the characteristic timescale, there can be additional timing information of first order, leading to a temporal coding phase in which timing information may affect the instantaneous information rate. We study the relative contributions of the dynamic firing rate and correlation variables to the full temporal information; the interaction of signal and noise correlations in temporal coding; synergy between spikes and between cells; and the effect of refractoriness. We illustrate the utility of the technique by analysis of a few cells from the rat barrel cortex. △ Less

Submitted 23 August, 2000; v1 submitted 13 August, 1999; originally announced August 1999.

Comments: 33 pages, 7 figures

Quantitative analysis of a Schaffer collateral model

Authors: S Schultz, S Panzeri, ET Rolls, A Treves

Abstract: Advances in techniques for the formal analysis of neural networks have introduced the possibility of detailed quantitative analyses of brain circuitry. This paper applies a method for calculating mutual information to the analysis of the Schaffer collateral connections between regions CA3 and CA1 of the hippocampus. Attention is given to the introduction of further details of anatomy and physiol… ▽ More Advances in techniques for the formal analysis of neural networks have introduced the possibility of detailed quantitative analyses of brain circuitry. This paper applies a method for calculating mutual information to the analysis of the Schaffer collateral connections between regions CA3 and CA1 of the hippocampus. Attention is given to the introduction of further details of anatomy and physiology to the calculation: in particular, the distribution of the number of connections that CA1 neurons receive from CA3, and the graded nature of the firing-rate distribution in region CA3. △ Less

Submitted 19 September, 1997; v1 submitted 28 April, 1997; originally announced April 1997.

Comments: Revised version. 16 pages LaTeX, 5 figs. To be published in Baddeley et al (Eds) Information Theory and the Brain, Cambridge Univ. Press 1998