-
Deep Predictive Learning: A Comprehensive Model of Three Visual Streams
Authors:
Randall C. O'Reilly,
Dean R. Wyatte,
John Rohrlich
Abstract:
How does the neocortex learn and develop the foundations of all our high-level cognitive abilities? We present a comprehensive framework spanning biological, computational, and cognitive levels, with a clear theoretical continuity between levels, providing a coherent answer directly supported by extensive data at each level. Learning is based on making predictions about what the senses will report…
▽ More
How does the neocortex learn and develop the foundations of all our high-level cognitive abilities? We present a comprehensive framework spanning biological, computational, and cognitive levels, with a clear theoretical continuity between levels, providing a coherent answer directly supported by extensive data at each level. Learning is based on making predictions about what the senses will report at 100 msec (alpha frequency) intervals, and adapting synaptic weights to improve prediction accuracy. The pulvinar nucleus of the thalamus serves as a projection screen upon which predictions are generated, through deep-layer 6 corticothalamic inputs from multiple brain areas and levels of abstraction. The sparse driving inputs from layer 5 intrinsic bursting neurons provide the target signal, and the temporal difference between it and the prediction reverberates throughout the cortex, driving synaptic changes that approximate error backpropagation, using only local activation signals in equations derived directly from a detailed biophysical model. In vision, predictive learning requires a carefully-organized developmental progression and anatomical organization of three pathways (What, Where, and What * Where), according to two central principles: top-down input from compact, high-level, abstract representations is essential for accurate prediction of low-level sensory inputs; and the collective, low-level prediction error must be progressively and opportunistically partitioned to enable extraction of separable factors that drive the learning of further high-level abstractions. Our model self-organized systematic invariant object representations of 100 different objects from simple movies, accounts for a wide range of data, and makes many testable predictions.
△ Less
Submitted 14 September, 2017;
originally announced September 2017.
-
What happens next and when "next" happens: Mechanisms of spatial and temporal prediction
Authors:
Dean Wyatte
Abstract:
The physics of the environment provide a rich spatiotemporal structure for our experience. Objects move in predictable ways and their features and identity remain stable across time and space. How does the brain leverage this structure to make predictions about and learn from the environment? This thesis describes research centered around a mechanistic description of sensory prediction called Leab…
▽ More
The physics of the environment provide a rich spatiotemporal structure for our experience. Objects move in predictable ways and their features and identity remain stable across time and space. How does the brain leverage this structure to make predictions about and learn from the environment? This thesis describes research centered around a mechanistic description of sensory prediction called LeabraTI (TI: Temporal Integration) that explains precisely how predictive processing is accomplished in neocortical microcircuits. The fundamental prediction of LeabraTI is that predictions and sensations are interleaved across the same neural tissue at an overall rate of 10 Hz, corresponding to the widely studied alpha rhythm of posterior cortex. Experiments described herein tested this prediction by manipulating the spatiotemporal properties of three-dimensional object stimuli in a laboratory setting. EEG results indicated that predictions were subserved by ~10 Hz oscillations that reliably tracked the onset of stimuli and differentiated between spatially predictable and unpredictable object sequences. There was a behavioral advantage for combined spatial and temporal predictability for discrimination of unlearned objects, but prolonged study of objects under this combined predictability context impaired discriminability relative to other learning contexts. This counterintuitive pattern of results was accounted for by a neural network model that learned three-dimensional viewpoint invariance with LeabraTI's spatiotemporal prediction rule. Synaptic weight scaling from prolonged learning built viewpoint invariance, but led to confusion between ambiguous views of objects, producing slightly lower performance on average. Overall, this work advances a biological architecture for sensory prediction accompanied by empirical evidence that supports learning of realistic time- and space-varying inputs.
△ Less
Submitted 20 July, 2014;
originally announced July 2014.
-
Learning Through Time in the Thalamocortical Loops
Authors:
Randall C. O'Reilly,
Dean Wyatte,
John Rohrlich
Abstract:
We present a comprehensive, novel framework for understanding how the neocortex, including the thalamocortical loops through the deep layers, can support a temporal context representation in the service of predictive learning. Many have argued that predictive learning provides a compelling, powerful source of learning signals to drive the development of human intelligence: if we constantly predict…
▽ More
We present a comprehensive, novel framework for understanding how the neocortex, including the thalamocortical loops through the deep layers, can support a temporal context representation in the service of predictive learning. Many have argued that predictive learning provides a compelling, powerful source of learning signals to drive the development of human intelligence: if we constantly predict what will happen next, and learn based on the discrepancies from our predictions (error-driven learning), then we can learn to improve our predictions by developing internal representations that capture the regularities of the environment (e.g., physical laws governing the time-evolution of object motions). Our version of this idea builds upon existing work with simple recurrent networks (SRN's), which have a discretely-updated temporal context representations that are a direct copy of the prior internal state representation. We argue that this discretization of temporal context updating has a number of important computational and functional advantages, and further show how the strong alpha-frequency (10hz, 100ms cycle time) oscillations in the posterior neocortex could reflect this temporal context updating. We examine a wide range of data from biology to behavior through the lens of this LeabraTI model, and find that it provides a unified account of a number of otherwise disconnected findings, all of which converge to support this new model of neocortical learning and processing. We describe an implemented model showing how predictive learning of tumbling object trajectories can facilitate object recognition with cluttered backgrounds.
△ Less
Submitted 13 July, 2014;
originally announced July 2014.
