-
How does the primate brain combine generative and discriminative computations in vision?
Authors:
Benjamin Peters,
James J. DiCarlo,
Todd Gureckis,
Ralf Haefner,
Leyla Isik,
Joshua Tenenbaum,
Talia Konkle,
Thomas Naselaris,
Kimberly Stachenfeld,
Zenna Tavares,
Doris Tsao,
Ilker Yildirim,
Nikolaus Kriegeskorte
Abstract:
Vision is widely understood as an inference problem. However, two contrasting conceptions of the inference process have each been influential in research on biological vision as well as the engineering of machine vision. The first emphasizes bottom-up signal flow, describing vision as a largely feedforward, discriminative inference process that filters and transforms the visual information to remo…
▽ More
Vision is widely understood as an inference problem. However, two contrasting conceptions of the inference process have each been influential in research on biological vision as well as the engineering of machine vision. The first emphasizes bottom-up signal flow, describing vision as a largely feedforward, discriminative inference process that filters and transforms the visual information to remove irrelevant variation and represent behaviorally relevant information in a format suitable for downstream functions of cognition and behavioral control. In this conception, vision is driven by the sensory data, and perception is direct because the processing proceeds from the data to the latent variables of interest. The notion of "inference" in this conception is that of the engineering literature on neural networks, where feedforward convolutional neural networks processing images are said to perform inference. The alternative conception is that of vision as an inference process in Helmholtz's sense, where the sensory evidence is evaluated in the context of a generative model of the causal processes giving rise to it. In this conception, vision inverts a generative model through an interrogation of the evidence in a process often thought to involve top-down predictions of sensory data to evaluate the likelihood of alternative hypotheses. The authors include scientists rooted in roughly equal numbers in each of the conceptions and motivated to overcome what might be a false dichotomy between them and engage the other perspective in the realm of theory and experiment. The primate brain employs an unknown algorithm that may combine the advantages of both conceptions. We explain and clarify the terminology, review the key empirical evidence, and propose an empirical research program that transcends the dichotomy and sets the stage for revealing the mysterious hybrid algorithm of primate vision.
△ Less
Submitted 11 January, 2024;
originally announced January 2024.
-
Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution
Authors:
Anthony Zador,
Sean Escola,
Blake Richards,
Bence Ölveczky,
Yoshua Bengio,
Kwabena Boahen,
Matthew Botvinick,
Dmitri Chklovskii,
Anne Churchland,
Claudia Clopath,
James DiCarlo,
Surya Ganguli,
Jeff Hawkins,
Konrad Koerding,
Alexei Koulakov,
Yann LeCun,
Timothy Lillicrap,
Adam Marblestone,
Bruno Olshausen,
Alexandre Pouget,
Cristina Savin,
Terrence Sejnowski,
Eero Simoncelli,
Sara Solla,
David Sussillo
, et al. (2 additional authors not shown)
Abstract:
Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts. The embodied Turing test shifts…
▽ More
Neuroscience has long been an essential driver of progress in artificial intelligence (AI). We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI. A core component of this is the embodied Turing test, which challenges AI animal models to interact with the sensorimotor world at skill levels akin to their living counterparts. The embodied Turing test shifts the focus from those capabilities like game playing and language that are especially well-developed or uniquely human to those capabilities, inherited from over 500 million years of evolution, that are shared with all animals. Building models that can pass the embodied Turing test will provide a roadmap for the next generation of AI.
△ Less
Submitted 22 February, 2023; v1 submitted 15 October, 2022;
originally announced October 2022.
-
Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons
Authors:
Irina Higgins,
Le Chang,
Victoria Langston,
Demis Hassabis,
Christopher Summerfield,
Doris Tsao,
Matthew Botvinick
Abstract:
Deep supervised neural networks trained to classify objects have emerged as popular models of computation in the primate ventral stream. These models represent information with a high-dimensional distributed population code, implying that inferotemporal (IT) responses are also too complex to interpret at the single-neuron level. We challenge this view by modelling neural responses to faces in the…
▽ More
Deep supervised neural networks trained to classify objects have emerged as popular models of computation in the primate ventral stream. These models represent information with a high-dimensional distributed population code, implying that inferotemporal (IT) responses are also too complex to interpret at the single-neuron level. We challenge this view by modelling neural responses to faces in the macaque IT with a deep unsupervised generative model, beta-VAE. Unlike deep classifiers, beta-VAE "disentangles" sensory data into interpretable latent factors, such as gender or hair length. We found a remarkable correspondence between the generative factors discovered by the model and those coded by single IT neurons. Moreover, we were able to reconstruct face images using the signals from just a handful of cells. This suggests that the ventral visual stream may be optimising the disentangling objective, producing a neural code that is low-dimensional and semantically interpretable at the single-unit level.
△ Less
Submitted 25 June, 2020;
originally announced June 2020.
