-
Object-based active inference
Authors:
Ruben S. van Bergen,
Pablo L. Lanillos
Abstract:
The world consists of objects: distinct entities possessing independent properties and dynamics. For agents to interact with the world intelligently, they must translate sensory inputs into the bound-together features that describe each object. These object-based representations form a natural basis for planning behavior. Active inference (AIF) is an influential unifying account of perception and…
▽ More
The world consists of objects: distinct entities possessing independent properties and dynamics. For agents to interact with the world intelligently, they must translate sensory inputs into the bound-together features that describe each object. These object-based representations form a natural basis for planning behavior. Active inference (AIF) is an influential unifying account of perception and action, but existing AIF models have not leveraged this important inductive bias. To remedy this, we introduce 'object-based active inference' (OBAI), marrying AIF with recent deep object-based neural networks. OBAI represents distinct objects with separate variational beliefs, and uses selective attention to route inputs to their corresponding object slots. Object representations are endowed with independent action-based dynamics. The dynamics and generative model are learned from experience with a simple environment (active multi-dSprites). We show that OBAI learns to correctly segment the action-perturbed objects from video input, and to manipulate these objects towards arbitrary goals.
△ Less
Submitted 2 September, 2022;
originally announced September 2022.
-
Going in circles is the way forward: the role of recurrence in visual inference
Authors:
Ruben S. van Bergen,
Nikolaus Kriegeskorte
Abstract:
Biological visual systems exhibit abundant recurrent connectivity. State-of-the-art neural network models for visual recognition, by contrast, rely heavily or exclusively on feedforward computation. Any finite-time recurrent neural network (RNN) can be unrolled along time to yield an equivalent feedforward neural network (FNN). This important insight suggests that computational neuroscientists may…
▽ More
Biological visual systems exhibit abundant recurrent connectivity. State-of-the-art neural network models for visual recognition, by contrast, rely heavily or exclusively on feedforward computation. Any finite-time recurrent neural network (RNN) can be unrolled along time to yield an equivalent feedforward neural network (FNN). This important insight suggests that computational neuroscientists may not need to engage recurrent computation, and that computer-vision engineers may be limiting themselves to a special case of FNN if they build recurrent models. Here we argue, to the contrary, that FNNs are a special case of RNNs and that computational neuroscientists and engineers should engage recurrence to understand how brains and machines can (1) achieve greater and more flexible computational depth, (2) compress complex computations into limited hardware, (3) integrate priors and priorities into visual inference through expectation and attention, (4) exploit sequential dependencies in their data for better inference and prediction, and (5) leverage the power of iterative computation.
△ Less
Submitted 16 November, 2020; v1 submitted 26 March, 2020;
originally announced March 2020.
-
Modeling correlated noise is necessary to decode uncertainty
Authors:
R. S. van Bergen,
J. F. M. Jehee
Abstract:
Brain decoding algorithms form an important part of the arsenal of analysis tools available to neuroscientists, allowing for a more detailed study of the kind of information represented in patterns of cortical activity. While most current decoding algorithms focus on estimating a single, most likely stimulus from the pattern of noisy fMRI responses, the presence of noise causes this estimate to be…
▽ More
Brain decoding algorithms form an important part of the arsenal of analysis tools available to neuroscientists, allowing for a more detailed study of the kind of information represented in patterns of cortical activity. While most current decoding algorithms focus on estimating a single, most likely stimulus from the pattern of noisy fMRI responses, the presence of noise causes this estimate to be uncertain. This uncertainty in stimulus estimates is a potentially highly relevant aspect of cortical stimulus processing, and features prominently in Bayesian or probabilistic models of neural coding. Here, we focus on sensory uncertainty and how best to extract this information with fMRI. We first demonstrate in simulations that decoding algorithms that take into account correlated noise between fMRI voxels better recover the amount of uncertainty (quantified as the width of a probability distribution over possible stimuli) associated with the decoded estimate. Furthermore, we show that not all correlated variability should be treated equally, as modeling tuning-dependent correlations has the greatest impact on decoding performance. Next, we examine actual noise correlations in human visual cortex, and find that shared variability in areas V1-V3 depends on the tuning properties of fMRI voxels. In line with our simulations, accounting for this shared noise between similarly tuned voxels produces important benefits in decoding. Our findings underscore the importance of accurate noise models in fMRI decoding approaches, and suggest a statistically feasible method to incorporate the most relevant forms of shared noise.
△ Less
Submitted 16 August, 2017;
originally announced August 2017.
