-
Available observation time regulates optimal balance between sensitivity and confidence
Authors:
Sahel Azizpour,
Viola Priesemann,
Johannes Zierenberg,
Anna Levina
Abstract:
Tasks that require information about the world imply a trade-off between the time spent on observation and the variance of the response. In particular, fast decisions need to rely on uncertain information. However, standard estimates of information processing capabilities, such as the dynamic range, are defined based on mean values that assume infinite observation times. Here, we show that limitin…
▽ More
Tasks that require information about the world imply a trade-off between the time spent on observation and the variance of the response. In particular, fast decisions need to rely on uncertain information. However, standard estimates of information processing capabilities, such as the dynamic range, are defined based on mean values that assume infinite observation times. Here, we show that limiting the observation time results in distributions of responses whose variance increases with the temporal correlations in a system and, importantly, affects a system's confidence in distinguishing inputs and thereby making decisions. To quantify the ability to distinguish features of an input, we propose several measures and demonstrate them on the prime example of a recurrent neural network that represents an input rate by a response firing averaged over a finite observation time. We show analytically and in simulations that the optimal tuning of the network depends on the available observation time, implying that tasks require a ``useful'' rather than maximal sensitivity. Interestingly, this shifts the optimal dynamic regime from critical to subcritical for finite observation times and highlights the importance of incorporating the finite observation times concept in future studies of information processing capabilities in a principled manner.
△ Less
Submitted 15 July, 2023;
originally announced July 2023.
-
Signatures of hierarchical temporal processing in the mouse visual system
Authors:
Lucas Rudelt,
Daniel González Marx,
F. Paul Spitzner,
Benjamin Cramer,
Johannes Zierenberg,
Viola Priesemann
Abstract:
A core challenge for the brain is to process information across various timescales. This could be achieved by a hierarchical organization of temporal processing through intrinsic mechanisms (e.g., recurrent coupling or adaptation), but recent evidence from spike recordings of the rodent visual system seems to conflict with this hypothesis. Here, we used an optimized information-theoretic and class…
▽ More
A core challenge for the brain is to process information across various timescales. This could be achieved by a hierarchical organization of temporal processing through intrinsic mechanisms (e.g., recurrent coupling or adaptation), but recent evidence from spike recordings of the rodent visual system seems to conflict with this hypothesis. Here, we used an optimized information-theoretic and classical autocorrelation analysis to show that information- and intrinsic timescales of spiking activity increase along the anatomical hierarchy of the mouse visual system, while information-theoretic predictability decreases. Moreover, the timescale hierarchy was invariant to the stimulus condition, whereas the decrease in predictability was strongest under natural movie stimulation. We could reproduce this effect in a basic recurrent network model with correlated sensory input. Our findings suggest that the rodent visual system indeed employs intrinsic mechanisms to achieve longer integration for higher cortical areas, while simultaneously reducing predictability for an efficient neural code.
△ Less
Submitted 17 January, 2024; v1 submitted 22 May, 2023;
originally announced May 2023.
-
Tackling the subsampling problem to infer collective properties from limited data
Authors:
Anna Levina,
Viola Priesemann,
Johannes Zierenberg
Abstract:
Complex systems are fascinating because their rich macroscopic properties emerge from the interaction of many simple parts. Understanding the building principles of these emergent phenomena in nature requires assessing natural complex systems experimentally. However, despite the development of large-scale data-acquisition techniques, experimental observations are often limited to a tiny fraction o…
▽ More
Complex systems are fascinating because their rich macroscopic properties emerge from the interaction of many simple parts. Understanding the building principles of these emergent phenomena in nature requires assessing natural complex systems experimentally. However, despite the development of large-scale data-acquisition techniques, experimental observations are often limited to a tiny fraction of the system. This spatial subsampling is particularly severe in neuroscience, where only a tiny fraction of millions or even billions of neurons can be individually recorded. Spatial subsampling may lead to significant systematic biases when inferring the collective properties of the entire system naively from a subsampled part. To overcome such biases, powerful mathematical tools have been developed in the past. In this perspective, we overview some issues arising from subsampling and review recently developed approaches to tackle the subsampling problem. These approaches enable one to assess, e.g., graph structures, collective dynamics of animals, neural network activity, or the spread of disease correctly from observing only a tiny fraction of the system. However, our current approaches are still far from having solved the subsampling problem in general, and hence we conclude by outlining what we believe are the main open challenges. Solving these challenges alongside the development of large-scale recording techniques will enable further fundamental insights into the working of complex and living systems.
△ Less
Submitted 17 September, 2022; v1 submitted 12 September, 2022;
originally announced September 2022.
-
Autocorrelations from emergent bistability in homeostatic spiking neural networks on neuromorphic hardware
Authors:
Benjamin Cramer,
Markus Kreft,
Sebastian Billaudelle,
Vitali Karasenko,
Aron Leibfried,
Eric Müller,
Philipp Spilger,
Johannes Weis,
Johannes Schemmel,
Miguel A. Muñoz,
Viola Priesemann,
Johannes Zierenberg
Abstract:
A unique feature of neuromorphic computing is that memory is an implicit part of processing through traces of past information in the system's collective dynamics. The extent of memory about past inputs is commonly quantified by the autocorrelation time of collective dynamics. Based on past experimental evidence, a potential explanation for the underlying autocorrelations are close-to-critical flu…
▽ More
A unique feature of neuromorphic computing is that memory is an implicit part of processing through traces of past information in the system's collective dynamics. The extent of memory about past inputs is commonly quantified by the autocorrelation time of collective dynamics. Based on past experimental evidence, a potential explanation for the underlying autocorrelations are close-to-critical fluctuations. Here, we show for self-organized networks of excitatory and inhibitory leaky integrate-and-fire neurons that autocorrelations can originate from emergent bistability upon reducing external input strength. We identify the bistability as a fluctuation-induced stochastic switching between metastable active and quiescent states in the vicinity of a non-equilibrium phase transition. This bistability occurs for networks with fixed heterogeneous weights as a consequence of homeostatic self-organization during development. Specifically, in our experiments on neuromorphic hardware and in computer simulations, the emergent bistability gives rise to autocorrelation times exceeding 500 ms despite single-neuron timescales of only 20 ms. Our results provide the first verification of biologically compatible autocorrelation times in networks of leaky integrate-and-fire neurons, which here are not generated by close-to-critical fluctuations but by emergent bistability in homeostatically regulated networks. Our results thereby constitute a new, complementary mechanism for emergent autocorrelations in networks of spiking neurons, with implications for biological and artificial networks, and introduces the general paradigm of fluctuation-induced bistability for driven systems with absorbing states.
△ Less
Submitted 17 August, 2022;
originally announced August 2022.
-
Modular architecture facilitates noise-driven control of synchrony in neuronal networks
Authors:
Hideaki Yamamoto,
F. Paul Spitzner,
Taiki Takemuro,
Victor Buendía,
Carla Morante,
Tomohiro Konno,
Shigeo Sato,
Ayumi Hirano-Iwata,
Viola Priesemann,
Miguel A. Muñoz,
Johannes Zierenberg,
Jordi Soriano
Abstract:
Brain functions require both segregated processing of information in specialized circuits, as well as integration across circuits to perform high-level information processing. One possible way to implement these seemingly opposing demands is by flexibly switching between synchronous and less synchronous states. Understanding how complex synchronization patterns are controlled by the interaction of…
▽ More
Brain functions require both segregated processing of information in specialized circuits, as well as integration across circuits to perform high-level information processing. One possible way to implement these seemingly opposing demands is by flexibly switching between synchronous and less synchronous states. Understanding how complex synchronization patterns are controlled by the interaction of network architecture and external perturbations is thus a central challenge in neuroscience, but the mechanisms behind such interactions remain elusive. Here, we utilise precision neuroengineering to manipulate cultured neuronal networks and show that a modular architecture facilitates desynchronization upon asynchronous stimulation, making external noise a control parameter of synchrony. Using spiking neuron models, we then demonstrate that external noise can reduce the level of available synaptic resources, which make intermodular interactions more stochastic and thereby facilitates the breakdown of synchrony. Finally, the phenomenology of stochastic intermodular interactions is formulated into a mesoscopic model that incorporates a state-dependent gating mechanism for signal propagation. Taken together, our results demonstrate a network mechanism by which asynchronous inputs tune the inherent dynamical state in structured networks of excitable units.
△ Less
Submitted 21 May, 2022;
originally announced May 2022.
-
How contact patterns destabilize and modulate epidemic outbreaks
Authors:
Johannes Zierenberg,
F. Paul Spitzner,
Jonas Dehning,
Viola Priesemann,
Martin Weigel,
Michael Wilczek
Abstract:
The spread of a contagious disease clearly depends on when infected individuals come into contact with susceptible ones. Such effects, however, have remained largely unexplored in the study of epidemic outbreaks. In particular, it remains unclear how the timing of contacts interacts with the latent and infectious stages of the disease. Here, we use real-world physical proximity data to study this…
▽ More
The spread of a contagious disease clearly depends on when infected individuals come into contact with susceptible ones. Such effects, however, have remained largely unexplored in the study of epidemic outbreaks. In particular, it remains unclear how the timing of contacts interacts with the latent and infectious stages of the disease. Here, we use real-world physical proximity data to study this interaction and find that the temporal statistics of actual human contact patterns i) destabilize epidemic outbreaks and ii) modulate the basic reproduction number. We explain both observations by distinct aspects of the observed contact patterns. On the one hand, we find the destabilization of outbreaks to be caused by temporal clustering of contacts leading to over-dispersed offspring distributions and increased probabilities of otherwise rare events (zero- and super-spreading). Notably, our analysis enables us to disentangle previously elusive sources of over-dispersion in empirical offspring distributions. On the other hand, we find the modulation of R0 to be caused by a periodically varying contact rate. Both mechanisms are a direct consequence of the memory in contact behavior, and we showcase a generative process that reproduces these non-Markovian statistics. Our results point to the importance of including non-Markovian contact timings into studies of epidemic outbreaks.
△ Less
Submitted 3 May, 2023; v1 submitted 24 September, 2021;
originally announced September 2021.
-
The challenges of containing SARS-CoV-2 via test-trace-and-isolate
Authors:
Sebastian Contreras,
Jonas Dehning,
Matthias Loidolt,
F. Paul Spitzner,
Jorge H. Urrea-Quintero,
Sebastian B. Mohr,
Michael Wilczek,
Johannes Zierenberg,
Michael Wibral,
Viola Priesemann
Abstract:
Without a cure, vaccine, or proven long-term immunity against SARS-CoV-2, test-trace-and-isolate (TTI) strategies present a promising tool to contain its spread. For any TTI strategy, however, mitigation is challenged by pre- and asymptomatic transmission, TTI-avoiders, and undetected spreaders, who strongly contribute to hidden infection chains. Here, we studied a semi-analytical model and identi…
▽ More
Without a cure, vaccine, or proven long-term immunity against SARS-CoV-2, test-trace-and-isolate (TTI) strategies present a promising tool to contain its spread. For any TTI strategy, however, mitigation is challenged by pre- and asymptomatic transmission, TTI-avoiders, and undetected spreaders, who strongly contribute to hidden infection chains. Here, we studied a semi-analytical model and identified two tipping points between controlled and uncontrolled spread: (1) the behavior-driven reproduction number of the hidden chains becomes too large to be compensated by the TTI capabilities, and (2) the number of new infections exceeds the tracing capacity. Both trigger a self-accelerating spread. We investigated how these tipping points depend on challenges like limited cooperation, missing contacts, and imperfect isolation. Our model results suggest that TTI alone is insufficient to contain an otherwise unhindered spread of SARS-CoV-2, implying that complementary measures like social distancing and improved hygiene remain necessary.
△ Less
Submitted 10 November, 2020; v1 submitted 12 September, 2020;
originally announced September 2020.
-
MR. Estimator, a toolbox to determine intrinsic timescales from subsampled spiking activity
Authors:
F. P. Spitzner,
J. Dehning,
J. Wilting,
A. Hagemann,
J. P. Neto,
J. Zierenberg,
V. Priesemann
Abstract:
Here we present our Python toolbox "MR. Estimator" to reliably estimate the intrinsic timescale from electrophysiologal recordings of heavily subsampled systems. Originally intended for the analysis of time series from neuronal spiking activity, our toolbox is applicable to a wide range of systems where subsampling -- the difficulty to observe the whole system in full detail -- limits our capabili…
▽ More
Here we present our Python toolbox "MR. Estimator" to reliably estimate the intrinsic timescale from electrophysiologal recordings of heavily subsampled systems. Originally intended for the analysis of time series from neuronal spiking activity, our toolbox is applicable to a wide range of systems where subsampling -- the difficulty to observe the whole system in full detail -- limits our capability to record. Applications range from epidemic spreading to any system that can be represented by an autoregressive process.
In the context of neuroscience, the intrinsic timescale can be thought of as the duration over which any perturbation reverberates within the network; it has been used as a key observable to investigate a functional hierarchy across the primate cortex and serves as a measure of working memory. It is also a proxy for the distance to criticality and quantifies a system's dynamic working point.
△ Less
Submitted 7 May, 2021; v1 submitted 7 July, 2020;
originally announced July 2020.
-
Characterizing spreading dynamics of subsampled systems with non-stationary external input
Authors:
Jorge de Heuvel,
Jens Wilting,
Moritz Becker,
Viola Priesemann,
Johannes Zierenberg
Abstract:
Many systems with propagation dynamics, such as spike propagation in neural networks and spreading of infectious diseases, can be approximated by autoregressive models. The estimation of model parameters can be complicated by the experimental limitation that one observes only a fraction of the system (subsampling) and potentially time-dependent parameters, leading to incorrect estimates. We show a…
▽ More
Many systems with propagation dynamics, such as spike propagation in neural networks and spreading of infectious diseases, can be approximated by autoregressive models. The estimation of model parameters can be complicated by the experimental limitation that one observes only a fraction of the system (subsampling) and potentially time-dependent parameters, leading to incorrect estimates. We show analytically how to overcome the subsampling bias when estimating the propagation rate for systems with certain non-stationary external input. This approach is readily applicable to trial-based experimental setups and seasonal fluctuations, as demonstrated on spike recordings from monkey prefrontal cortex and spreading of norovirus and measles.
△ Less
Submitted 24 April, 2020;
originally announced May 2020.
-
Inferring change points in the COVID-19 spreading reveals the effectiveness of interventions
Authors:
Jonas Dehning,
Johannes Zierenberg,
F. Paul Spitzner,
Michael Wibral,
Joao Pinheiro Neto,
Michael Wilczek,
Viola Priesemann
Abstract:
As COVID-19 is rapidly spreading across the globe, short-term modeling forecasts provide time-critical information for decisions on containment and mitigation strategies. A main challenge for short-term forecasts is the assessment of key epidemiological parameters and how they change when first interventions show an effect. By combining an established epidemiological model with Bayesian inference,…
▽ More
As COVID-19 is rapidly spreading across the globe, short-term modeling forecasts provide time-critical information for decisions on containment and mitigation strategies. A main challenge for short-term forecasts is the assessment of key epidemiological parameters and how they change when first interventions show an effect. By combining an established epidemiological model with Bayesian inference, we analyze the time dependence of the effective growth rate of new infections. Focusing on the COVID-19 spread in Germany, we detect change points in the effective growth rate that correlate well with the times of publicly announced interventions. Thereby, we can quantify the effect of interventions, and we can incorporate the corresponding change points into forecasts of future scenarios and case numbers. Our code is freely available and can be readily adapted to any country or region.
△ Less
Submitted 4 May, 2020; v1 submitted 2 April, 2020;
originally announced April 2020.
-
Description of spreading dynamics by microscopic network models and macroscopic branching processes can differ due to coalescence
Authors:
Johannes Zierenberg,
Jens Wilting,
Viola Priesemann,
Anna Levina
Abstract:
Spreading processes are conventionally monitored on a macroscopic level by counting the number of incidences over time. The spreading process can then be modeled either on the microscopic level, assuming an underlying interaction network, or directly on the macroscopic level, assuming that microscopic contributions are negligible. The macroscopic characteristics of both descriptions are commonly a…
▽ More
Spreading processes are conventionally monitored on a macroscopic level by counting the number of incidences over time. The spreading process can then be modeled either on the microscopic level, assuming an underlying interaction network, or directly on the macroscopic level, assuming that microscopic contributions are negligible. The macroscopic characteristics of both descriptions are commonly assumed to be identical. In this work, we show that these characteristics of microscopic and macroscopic descriptions can be different due to coalescence, i.e., a node being activated at the same time by multiple sources. In particular, we consider a (microscopic) branching network (probabilistic cellular automaton) with annealed connectivity disorder, record the macroscopic activity, and then approximate this activity by a (macroscopic) branching process. In this framework, we analytically calculate the effect of coalescence on the collective dynamics. We show that coalescence leads to a universal non-linear scaling function for the conditional expectation value of successive network activity. This allows us to quantify the difference between the microscopic model parameter and established macroscopic estimates. To overcome this difference, we propose a non-linear estimator that correctly infers the model branching parameter for all system sizes.
△ Less
Submitted 24 May, 2019;
originally announced May 2019.
-
Tailored ensembles of neural networks optimize sensitivity to stimulus statistics
Authors:
Johannes Zierenberg,
Jens Wilting,
Viola Priesemann,
Anna Levina
Abstract:
The dynamic range of stimulus processing in living organisms is much larger than a single neural network can explain. For a generic, tunable spiking network we derive that while the dynamic range is maximal at criticality, the interval of discriminable intensities is very similar for any network tuning due to coalescence. Compensating coalescence enables adaptation of discriminable intervals. Thus…
▽ More
The dynamic range of stimulus processing in living organisms is much larger than a single neural network can explain. For a generic, tunable spiking network we derive that while the dynamic range is maximal at criticality, the interval of discriminable intensities is very similar for any network tuning due to coalescence. Compensating coalescence enables adaptation of discriminable intervals. Thus, we can tailor an ensemble of networks optimized to the distribution of stimulus intensities, e.g., extending the dynamic range arbitrarily. We discuss potential applications in machine learning.
△ Less
Submitted 24 May, 2019;
originally announced May 2019.
-
Dynamic Adaptive Computation: Tuning network states to task requirements
Authors:
Jens Wilting,
Jonas Dehning,
Joao Pinheiro Neto,
Lucas Rudelt,
Michael Wibral,
Johannes Zierenberg,
Viola Priesemann
Abstract:
Neural circuits are able to perform computations under very diverse conditions and requirements. The required computations impose clear constraints on their fine-tuning: a rapid and maximally informative response to stimuli in general requires decorrelated baseline neural activity. Such network dynamics is known as asynchronous-irregular. In contrast, spatio-temporal integration of information req…
▽ More
Neural circuits are able to perform computations under very diverse conditions and requirements. The required computations impose clear constraints on their fine-tuning: a rapid and maximally informative response to stimuli in general requires decorrelated baseline neural activity. Such network dynamics is known as asynchronous-irregular. In contrast, spatio-temporal integration of information requires maintenance and transfer of stimulus information over extended time periods. This can be realized at criticality, a phase transition where correlations, sensitivity and integration time diverge. Being able to flexibly switch, or even combine the above properties in a task-dependent manner would present a clear functional advantage. We propose that cortex operates in a "reverberating regime" because it is particularly favorable for ready adaptation of computational properties to context and task. This reverberating regime enables cortical networks to interpolate between the asynchronous-irregular and the critical state by small changes in effective synaptic strength or excitation-inhibition ratio. These changes directly adapt computational properties, including sensitivity, amplification, integration time and correlation length within the local network. We review recent converging evidence that cortex in vivo operates in the reverberating regime, and that various cortical areas have adapted their integration times to processing requirements. In addition, we propose that neuromodulation enables a fine-tuning of the network, so that local circuits can either decorrelate or integrate, and quench or maintain their input depending on task. We argue that this task-dependent tuning, which we call "dynamic adaptive computation", presents a central organization principle of cortical networks and discuss first experimental evidence.
△ Less
Submitted 20 September, 2018;
originally announced September 2018.
-
Homeostatic plasticity and external input shape neural network dynamics
Authors:
Johannes Zierenberg,
Jens Wilting,
Viola Priesemann
Abstract:
In vitro and in vivo spiking activity clearly differ. Whereas networks in vitro develop strong bursts separated by periods of very little spiking activity, in vivo cortical networks show continuous activity. This is puzzling considering that both networks presumably share similar single-neuron dynamics and plasticity rules. We propose that the defining difference between in vitro and in vivo dynam…
▽ More
In vitro and in vivo spiking activity clearly differ. Whereas networks in vitro develop strong bursts separated by periods of very little spiking activity, in vivo cortical networks show continuous activity. This is puzzling considering that both networks presumably share similar single-neuron dynamics and plasticity rules. We propose that the defining difference between in vitro and in vivo dynamics is the strength of external input. In vitro, networks are virtually isolated, whereas in vivo every brain area receives continuous input. We analyze a model of spiking neurons in which the input strength, mediated by spike rate homeostasis, determines the characteristics of the dynamical state. In more detail, our analytical and numerical results on various network topologies show consistently that under increasing input, homeostatic plasticity generates distinct dynamic states, from bursting, to close-to-critical, reverberating and irregular states. This implies that the dynamic state of a neural network is not fixed but can readily adapt to the input strengths. Indeed, our results match experimental spike recordings in vitro and in vivo: the in vitro bursting behavior is consistent with a state generated by very low network input (< 0.1%), whereas in vivo activity suggests that on the order of 1% recorded spikes are input-driven, resulting in reverberating dynamics. Importantly, this predicts that one can abolish the ubiquitous bursts of in vitro preparations, and instead impose dynamics comparable to in vivo activity by exposing the system to weak long-term stimulation, thereby opening new paths to establish an in vivo-like assay in vitro for basic as well as neurological studies.
△ Less
Submitted 4 July, 2018;
originally announced July 2018.
