-
Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos
Authors:
Polina Turishcheva,
Paul G. Fahey,
Michaela Vystrčilová,
Laura Hansel,
Rachel Froebe,
Kayla Ponder,
Yongrong Qiu,
Konstantin F. Willeke,
Mohammad Bashiri,
Ruslan Baikulov,
Yu Zhu,
Lei Ma,
Shan Yu,
Tiejun Huang,
Bryan M. Li,
Wolf De Wulf,
Nina Kudryashova,
Matthias H. Hennig,
Nathalie L. Rochefort,
Arno Onken,
Eric Wang,
Zhiwei Ding,
Andreas S. Tolias,
Fabian H. Sinz,
Alexander S Ecker
Abstract:
Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Machine learning has benefited tremendously from benchmarks that compare different model on the same ta…
▽ More
Understanding how biological visual systems process information is challenging because of the nonlinear relationship between visual input and neuronal responses. Artificial neural networks allow computational neuroscientists to create predictive models that connect biological and machine vision. Machine learning has benefited tremendously from benchmarks that compare different model on the same task under standardized conditions. However, there was no standardized benchmark to identify state-of-the-art dynamic models of the mouse visual system. To address this gap, we established the Sensorium 2023 Benchmark Competition with dynamic input, featuring a new large-scale dataset from the primary visual cortex of ten mice. This dataset includes responses from 78,853 neurons to 2 hours of dynamic stimuli per neuron, together with the behavioral measurements such as running speed, pupil dilation, and eye movements. The competition ranked models in two tracks based on predictive performance for neuronal responses on a held-out test set: one focusing on predicting in-domain natural stimuli and another on out-of-distribution (OOD) stimuli to assess model generalization. As part of the NeurIPS 2023 competition track, we received more than 160 model submissions from 22 teams. Several new architectures for predictive models were proposed, and the winning teams improved the previous state-of-the-art model by 50%. Access to the dataset as well as the benchmarking infrastructure will remain online at www.sensorium-competition.net.
△ Less
Submitted 12 July, 2024;
originally announced July 2024.
-
V1T: large-scale mouse V1 response prediction using a Vision Transformer
Authors:
Bryan M. Li,
Isabel M. Cornacchia,
Nathalie L. Rochefort,
Arno Onken
Abstract:
Accurate predictive models of the visual cortex neural response to natural visual stimuli remain a challenge in computational neuroscience. In this work, we introduce V1T, a novel Vision Transformer based architecture that learns a shared visual and behavioral representation across animals. We evaluate our model on two large datasets recorded from mouse primary visual cortex and outperform previou…
▽ More
Accurate predictive models of the visual cortex neural response to natural visual stimuli remain a challenge in computational neuroscience. In this work, we introduce V1T, a novel Vision Transformer based architecture that learns a shared visual and behavioral representation across animals. We evaluate our model on two large datasets recorded from mouse primary visual cortex and outperform previous convolution-based models by more than 12.7% in prediction performance. Moreover, we show that the self-attention weights learned by the Transformer correlate with the population receptive fields. Our model thus sets a new benchmark for neural response prediction and can be used jointly with behavioral and neural recordings to reveal meaningful characteristic features of the visual cortex.
△ Less
Submitted 5 September, 2023; v1 submitted 6 February, 2023;
originally announced February 2023.
-
High-fidelity multimode fibre-based endoscopy for deep-brain in vivo imaging
Authors:
Sergey Turtaev,
Ivo T. Leite,
Tristan Altwegg-Boussac,
Janelle M. P. Pakan,
Nathalie L. Rochefort,
Tomáš Čižmár
Abstract:
Progress in neuroscience constantly relies on the development of new techniques to investigate the complex dynamics of neuronal networks. An ongoing challenge is to achieve minimally-invasive and high-resolution observations of neuronal activity in vivo inside deep brain areas. A perspective strategy is to utilise holographic control of light propagation in complex media, which allows converting a…
▽ More
Progress in neuroscience constantly relies on the development of new techniques to investigate the complex dynamics of neuronal networks. An ongoing challenge is to achieve minimally-invasive and high-resolution observations of neuronal activity in vivo inside deep brain areas. A perspective strategy is to utilise holographic control of light propagation in complex media, which allows converting a hair-thin multimode optical fibre into an ultra-narrow imaging tool. Compared to current endoscopes based on GRIN lenses or fibre bundles, this concept offers a footprint reduction exceeding an order of magnitude, together with a significant enhancement in resolution. We designed a compact and high-speed system for fluorescent imaging at the tip of a fibre, achieving micron-scale resolution across a 50 um field of view, and yielding 7-kilopixel images at a rate of 3.5 frames/s. Furthermore, we demonstrate in vivo observations of cell bodies and processes of inhibitory neurons within deep layers of the visual cortex and hippocampus of anesthetised mice. This study forms the basis for several perspective techniques of modern microscopy to be delivered deep inside the tissue of living animal models while causing minimal impact on its structural and functional properties.
△ Less
Submitted 4 June, 2018;
originally announced June 2018.
