-
POCO: Scalable Neural Forecasting through Population Conditioning
Authors:
Yu Duan,
Hamza Tahir Chaudhry,
Misha B. Ahrens,
Christopher D Harvey,
Matthew G Perich,
Karl Deisseroth,
Kanaka Rajan
Abstract:
Predicting future neural activity is a core challenge in modeling brain dynamics, with applications ranging from scientific investigation to closed-loop neurotechnology. While recent models of population activity emphasize interpretability and behavioral decoding, neural forecasting-particularly across multi-session, spontaneous recordings-remains underexplored. We introduce POCO, a unified foreca…
▽ More
Predicting future neural activity is a core challenge in modeling brain dynamics, with applications ranging from scientific investigation to closed-loop neurotechnology. While recent models of population activity emphasize interpretability and behavioral decoding, neural forecasting-particularly across multi-session, spontaneous recordings-remains underexplored. We introduce POCO, a unified forecasting model that combines a lightweight univariate forecaster with a population-level encoder to capture both neuron-specific and brain-wide dynamics. Trained across five calcium imaging datasets spanning zebrafish, mice, and C. elegans, POCO achieves state-of-the-art accuracy at cellular resolution in spontaneous behaviors. After pre-training, POCO rapidly adapts to new recordings with minimal fine-tuning. Notably, POCO's learned unit embeddings recover biologically meaningful structure-such as brain region clustering-without any anatomical labels. Our comprehensive analysis reveals several key factors influencing performance, including context length, session diversity, and preprocessing. Together, these results position POCO as a scalable and adaptable approach for cross-session neural forecasting and offer actionable insights for future model design. By enabling accurate, generalizable forecasting models of neural dynamics across individuals and species, POCO lays the groundwork for adaptive neurotechnologies and large-scale efforts for neural foundation models.
△ Less
Submitted 17 June, 2025;
originally announced June 2025.
-
Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks
Authors:
Ann Huang,
Satpreet H. Singh,
Flavio Martinelli,
Kanaka Rajan
Abstract:
Task-trained recurrent neural networks (RNNs) are widely used in neuroscience and machine learning to model dynamical computations. To gain mechanistic insight into how neural systems solve tasks, prior work often reverse-engineers individual trained networks. However, different RNNs trained on the same task and achieving similar performance can exhibit strikingly different internal solutions-a ph…
▽ More
Task-trained recurrent neural networks (RNNs) are widely used in neuroscience and machine learning to model dynamical computations. To gain mechanistic insight into how neural systems solve tasks, prior work often reverse-engineers individual trained networks. However, different RNNs trained on the same task and achieving similar performance can exhibit strikingly different internal solutions-a phenomenon known as solution degeneracy. Here, we develop a unified framework to systematically quantify and control solution degeneracy across three levels: behavior, neural dynamics, and weight space. We apply this framework to 3,400 RNNs trained on four neuroscience-relevant tasks-flip-flop memory, sine wave generation, delayed discrimination, and path integration-while systematically varying task complexity, learning regime, network size, and regularization. We find that higher task complexity and stronger feature learning reduce degeneracy in neural dynamics but increase it in weight space, with mixed effects on behavior. In contrast, larger networks and structural regularization reduce degeneracy at all three levels. These findings empirically validate the Contravariance Principle and provide practical guidance for researchers aiming to tailor RNN solutions-whether to uncover shared neural mechanisms or to model individual variability observed in biological systems. This work provides a principled framework for quantifying and controlling solution degeneracy in task-trained RNNs, offering new tools for building more interpretable and biologically grounded models of neural computation.
△ Less
Submitted 28 May, 2025; v1 submitted 4 October, 2024;
originally announced October 2024.
-
The time is ripe to reverse engineer an entire nervous system: simulating behavior from neural interactions
Authors:
Gal Haspel,
Ben Baker,
Isabel Beets,
Edward S Boyden,
Jeffrey Brown,
George Church,
Netta Cohen,
Daniel Colon-Ramos,
Eva Dyer,
Christopher Fang-Yen,
Steven Flavell,
Miriam B Goodman,
Anne C Hart,
Eduardo J Izquierdo,
Konstantinos Kagias,
Shawn Lockery,
Yangning Lu,
Adam Marblestone,
Jordan Matelsky,
Brett Mensh,
Talmo D Pereira,
Hanspeter Pfister,
Kanaka Rajan,
Horacio G Rotstein,
Monika Scholz
, et al. (12 additional authors not shown)
Abstract:
Just like electrical engineers understand how microprocessors execute programs in terms of how transistor currents are affected by their inputs, neuroscientists want to understand behavior production in terms of how neuronal outputs are affected by their inputs and internal states. This dependency of neuronal outputs on inputs can be described by a state-dependent input-output (IO)-function. Howev…
▽ More
Just like electrical engineers understand how microprocessors execute programs in terms of how transistor currents are affected by their inputs, neuroscientists want to understand behavior production in terms of how neuronal outputs are affected by their inputs and internal states. This dependency of neuronal outputs on inputs can be described by a state-dependent input-output (IO)-function. However, to reliably identify these IO-functions, we need to perturb each input and combinations of inputs while observing all the outputs. Here, we argue that such completeness is possible in C. elegans; a complete description that goes all the way from the activity of every neuron to predict behavior. The established and growing toolkit of optophysiology can non-invasively capture and control every neuron's activity and scale to countless experiments. The information from many such experiments can be pooled while capturing the inter-individual variability because neuronal identity and function are largely conserved across individuals. Just like electrical engineers use transistor IO-functions to simulate program execution, we argue that neuronal IO-functions could be used to simulate the impressive breadth of brain states and behaviors of C. elegans.
△ Less
Submitted 18 September, 2024; v1 submitted 12 August, 2023;
originally announced August 2023.
-
Efficient and robust multi-task learning in the brain with modular latent primitives
Authors:
Christian David Márton,
Léo Gagnon,
Guillaume Lajoie,
Kanaka Rajan
Abstract:
Biological agents do not have infinite resources to learn new things. For this reason, a central aspect of human learning is the ability to recycle previously acquired knowledge in a way that allows for faster, less resource-intensive acquisition of new skills. In spite of that, how neural networks in the brain leverage existing knowledge to learn new computations is not well understood. In this w…
▽ More
Biological agents do not have infinite resources to learn new things. For this reason, a central aspect of human learning is the ability to recycle previously acquired knowledge in a way that allows for faster, less resource-intensive acquisition of new skills. In spite of that, how neural networks in the brain leverage existing knowledge to learn new computations is not well understood. In this work, we study this question in artificial recurrent neural networks (RNNs) trained on a corpus of commonly used neuroscience tasks. Combining brain-inspired inductive biases we call functional and structural, we propose a system that learns new tasks by building on top of pre-trained latent dynamics organised into separate recurrent modules. These modules, acting as prior knowledge acquired previously through evolution or development, are pre-trained on the statistics of the full corpus of tasks so as to be independent and maximally informative. The resulting model, we call a Modular Latent Primitives (MoLaP) network, allows for learning multiple tasks while keeping parameter counts, and updates, low. We also show that the skills acquired with our approach are more robust to a broad range of perturbations compared to those acquired with other multi-task learning strategies, and that generalisation to new tasks is facilitated. This work offers a new perspective on achieving efficient multi-task learning in the brain, illustrating the benefits of leveraging pre-trained latent dynamical primitives.
△ Less
Submitted 25 May, 2022; v1 submitted 28 May, 2021;
originally announced May 2021.
-
A brain basis of dynamical intelligence for AI and computational neuroscience
Authors:
Joseph D. Monaco,
Kanaka Rajan,
Grace M. Hwang
Abstract:
The deep neural nets of modern artificial intelligence (AI) have not achieved defining features of biological intelligence, including abstraction, causal learning, and energy-efficiency. While scaling to larger models has delivered performance improvements for current applications, more brain-like capacities may demand new theories, models, and methods for designing artificial learning systems. He…
▽ More
The deep neural nets of modern artificial intelligence (AI) have not achieved defining features of biological intelligence, including abstraction, causal learning, and energy-efficiency. While scaling to larger models has delivered performance improvements for current applications, more brain-like capacities may demand new theories, models, and methods for designing artificial learning systems. Here, we argue that this opportunity to reassess insights from the brain should stimulate cooperation between AI research and theory-driven computational neuroscience (CN). To motivate a brain basis of neural computation, we present a dynamical view of intelligence from which we elaborate concepts of sparsity in network structure, temporal dynamics, and interactive learning. In particular, we suggest that temporal dynamics, as expressed through neural synchrony, nested oscillations, and flexible sequences, provide a rich computational layer for reading and updating hierarchical models distributed in long-term memory networks. Moreover, embracing agent-centered paradigms in AI and CN will accelerate our understanding of the complex dynamics and behaviors that build useful world models. A convergence of AI/CN theories and objectives will reveal dynamical principles of intelligence for brains and engineered learning systems. This article was inspired by our symposium on dynamical neuroscience and machine learning at the 6th Annual US/NIH BRAIN Initiative Investigators Meeting.
△ Less
Submitted 21 May, 2021; v1 submitted 15 May, 2021;
originally announced May 2021.
-
full-FORCE: A Target-Based Method for Training Recurrent Networks
Authors:
Brian DePasquale,
Christopher J. Cueva,
Kanaka Rajan,
G. Sean Escola,
L. F. Abbott
Abstract:
Trained recurrent networks are powerful tools for modeling dynamic neural computations. We present a target-based method for modifying the full connectivity matrix of a recurrent network to train it to perform tasks involving temporally complex input/output transformations. The method introduces a second network during training to provide suitable "target" dynamics useful for performing the task.…
▽ More
Trained recurrent networks are powerful tools for modeling dynamic neural computations. We present a target-based method for modifying the full connectivity matrix of a recurrent network to train it to perform tasks involving temporally complex input/output transformations. The method introduces a second network during training to provide suitable "target" dynamics useful for performing the task. Because it exploits the full recurrent connectivity, the method produces networks that perform tasks with fewer neurons and greater noise robustness than traditional least-squares (FORCE) approaches. In addition, we show how introducing additional input signals into the target-generating network, which act as task hints, greatly extends the range of tasks that can be learned and provides control over the complexity and nature of the dynamics of the trained, task-performing network.
△ Less
Submitted 9 October, 2017;
originally announced October 2017.
-
Recurrent Network Models Of Sequence Generation And Memory
Authors:
Kanaka Rajan,
Christopher D Harvey,
David W Tank
Abstract:
Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in which a principled mechanism is pre-wired into their connectivity. Here we demonstrate that, starting from random connectivity and modifying a small fraction of…
▽ More
Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in which a principled mechanism is pre-wired into their connectivity. Here we demonstrate that, starting from random connectivity and modifying a small fraction of connections, a largely disordered recur- rent network can produce sequences and implement working memory efficiently. We use this process, called Partial In-Network Training (PINning), to model and match cellular resolution imaging data from the posterior parietal cortex during a virtual memory- guided two-alternative forced-choice task. Analysis of the connectivity reveals that sequences propagate by the cooperation between recurrent synaptic interactions and external inputs, rather than through feedforward or asymmetric connections. Together our results suggest that neural sequences may emerge through learning from largely unstructured network architectures.
△ Less
Submitted 14 March, 2016;
originally announced March 2016.
-
Learning quadratic receptive fields from neural responses to natural stimuli
Authors:
Kanaka Rajan,
Olivier Marre,
Gašper Tkačik
Abstract:
Models of neural responses to stimuli with complex spatiotemporal correlation structure often assume that neurons are only selective for a small number of linear projections of a potentially high-dimensional input. Here we explore recent modeling approaches where the neural response depends on the quadratic form of the input rather than on its linear projection, that is, the neuron is sensitive to…
▽ More
Models of neural responses to stimuli with complex spatiotemporal correlation structure often assume that neurons are only selective for a small number of linear projections of a potentially high-dimensional input. Here we explore recent modeling approaches where the neural response depends on the quadratic form of the input rather than on its linear projection, that is, the neuron is sensitive to the local covariance structure of the signal preceding the spike. To infer this quadratic dependence in the presence of arbitrary (e.g. naturalistic) stimulus distribution, we review several inference methods, focussing in particular on two information-theory-based approaches (maximization of stimulus energy or of noise entropy) and a likelihood-based approach (Bayesian spike-triggered covariance, extensions of generalized linear models). We analyze the formal connection between the likelihood-based and information-based approaches to show how they lead to consistent inference. We demonstrate the practical feasibility of these procedures by using model neurons responding to a flickering variance stimulus.
△ Less
Submitted 1 September, 2012;
originally announced September 2012.
-
Maximally informative "stimulus energies" in the analysis of neural responses to natural signals
Authors:
Kanaka Rajan,
William Bialek
Abstract:
The concept of feature selectivity in sensory signal processing can be formalized as dimensionality reduction: in a stimulus space of very high dimensions, neurons respond only to variations within some smaller, relevant subspace. But if neural responses exhibit invariances, then the relevant subspace typically cannot be reached by a Euclidean projection of the original stimulus. We argue that, in…
▽ More
The concept of feature selectivity in sensory signal processing can be formalized as dimensionality reduction: in a stimulus space of very high dimensions, neurons respond only to variations within some smaller, relevant subspace. But if neural responses exhibit invariances, then the relevant subspace typically cannot be reached by a Euclidean projection of the original stimulus. We argue that, in several cases, we can make progress by appealing to the simplest nonlinear construction, identifying the relevant variables as quadratic forms, or "stimulus energies." Natural examples include non-phase-locked cells in the auditory system, complex cells in visual cortex, and motion-sensitive neurons in the visual system. Generalizing the idea of maximally informative dimensions, we show that one can search for the kernels of the relevant quadratic forms by maximizing the mutual information between the stimulus energy and the arrival times of action potentials. Simple implementations of this idea successfully recover the underlying properties of model neurons even when the number of parameters in the kernel is comparable to the number of action potentials and stimuli are completely natural. We explore several generalizations that allow us to incorporate plausible structure into the kernel and thereby restrict the number of parameters. We hope that this approach will add significantly to the set of tools available for the analysis of neural responses to complex, naturalistic stimuli.
△ Less
Submitted 31 December, 2011;
originally announced January 2012.
-
Interactions between Intrinsic and Stimulus-Evoked Activity in Recurrent Neural Networks
Authors:
L F Abbott,
Kanaka Rajan,
Haim Sompolinsky
Abstract:
Trial-to-trial variability is an essential feature of neural responses, but its source is a subject of active debate. Response variability (Mast and Victor, 1991; Arieli et al., 1995 & 1996; Anderson et al., 2000 & 2001; Kenet et al., 2003; Petersen et al., 2003a & b; Fiser, Chiu and Weliky, 2004; MacLean et al., 2005; Yuste et al., 2005; Vincent et al., 2007) is often treated as random noise, gen…
▽ More
Trial-to-trial variability is an essential feature of neural responses, but its source is a subject of active debate. Response variability (Mast and Victor, 1991; Arieli et al., 1995 & 1996; Anderson et al., 2000 & 2001; Kenet et al., 2003; Petersen et al., 2003a & b; Fiser, Chiu and Weliky, 2004; MacLean et al., 2005; Yuste et al., 2005; Vincent et al., 2007) is often treated as random noise, generated either by other brain areas, or by stochastic processes within the circuitry being studied. We call such sources of variability external to stress the independence of this form of noise from activity driven by the stimulus. Variability can also be generated internally by the same network dynamics that generates responses to a stimulus. How can we distinguish between external and internal sources of response variability? Here we show that internal sources of variability interact nonlinearly with stimulus-induced activity, and this interaction yields a suppression of noise in the evoked state. This provides a theoretical basis and potential mechanism for the experimental observation that, in many brain areas, stimuli cause significant suppression of neuronal variability (Werner and Mountcastle, 1963; Fortier, Smith and Kalaska, 1993; Anderson et al., 2000; Friedrich and Laurent, 2004; Churchland et al., 2006; Finn, Priebe and Ferster, 2007; Mitchell, Sundberg and Reynolds, 2007; Churchland et al., 2009). The combined theoretical and experimental results suggest that internally generated activity is a significant contributor to response variability in neural circuits.
△ Less
Submitted 2 August, 2010; v1 submitted 18 December, 2009;
originally announced December 2009.
-
Stimulus-Dependent Suppression of Chaos in Recurrent Neural Networks
Authors:
Kanaka Rajan,
L F Abbott,
Haim Sompolinsky
Abstract:
Neuronal activity arises from an interaction between ongoing firing generated spontaneously by neural circuits and responses driven by external stimuli. Using mean-field analysis, we ask how a neural network that intrinsically generates chaotic patterns of activity can remain sensitive to extrinsic input. We find that inputs not only drive network responses, they also actively suppress ongoing act…
▽ More
Neuronal activity arises from an interaction between ongoing firing generated spontaneously by neural circuits and responses driven by external stimuli. Using mean-field analysis, we ask how a neural network that intrinsically generates chaotic patterns of activity can remain sensitive to extrinsic input. We find that inputs not only drive network responses, they also actively suppress ongoing activity, ultimately leading to a phase transition in which chaos is completely eliminated. The critical input intensity at the phase transition is a non-monotonic function of stimulus frequency, revealing a "resonant" frequency at which the input is most effective at suppressing chaos even though the power spectrum of the spontaneous activity peaks at zero and falls exponentially. A prediction of our analysis is that the variance of neural responses should be most strongly suppressed at frequencies matching the range over which many sensory systems operate.
△ Less
Submitted 2 August, 2010; v1 submitted 17 December, 2009;
originally announced December 2009.
