-
Shedding light on social learning
Authors:
Kingsley J. A. Cox,
Paul R. Adams
Abstract:
Culture involves the origination and transmission of ideas, but the conditions in which culture can emerge and evolve are unclear. We constructed and studied a highly simplified neural-network model of these processes. In this model ideas originate by individual learning from the environment and are transmitted by communication between individuals. Individuals (or "agents") comprise a single neuro…
▽ More
Culture involves the origination and transmission of ideas, but the conditions in which culture can emerge and evolve are unclear. We constructed and studied a highly simplified neural-network model of these processes. In this model ideas originate by individual learning from the environment and are transmitted by communication between individuals. Individuals (or "agents") comprise a single neuron which receives structured data from the environment via plastic synaptic connections. The data are generated in the simplest possible way: linear mixing of independently fluctuating sources and the goal of learning is to unmix the data. To make this problem tractable we assume that at least one of the sources fluctuates in a nonGaussian manner. Linear mixing creates structure in the data, and agents attempt to learn (from the data and possibly from other individuals) synaptic weights that will unmix, i.e., to "understand" the agent's world. For a variety of reasons even this goal can be difficult for a single agent to achieve; we studied one particular type of difficulty (created by imperfection in synaptic plasticity), though our conclusions should carry over to many other types of difficulty. We previously studied whether a small population of communicating agents, learning from each other, could more easily learn unmixing coefficients than isolated individuals, learning only from their environment. We found, unsurprisingly, that if agents learn indiscriminately from any other agent (whether or not they have learned good solutions), communication does not enhance understanding. Here we extend the model slightly, by allowing successful learners to be more effective teachers, and find that now a population of agents can learn more effectively than isolated individuals. We suggest that a key factor in the onset of culture might be the development of selective learning.
△ Less
Submitted 13 October, 2023;
originally announced October 2023.
-
Hocus-Socus: An Error Catastrophe for Complex Hebbian Learning Implies Neocortical Proofreading
Authors:
Kingsley J. A. Cox,
Paul R. Adams
Abstract:
The neocortex is widely believed to be the seat of intelligence and "mind". However, it's unclear what "mind" is, or how the special features of neocortex enable it, though likely "connectionist" principles are involved *A. The key to intelligence1 is learning relationships between large numbers of signals (such as pixel values), rather than memorizing explicit patterns. Causes (such as objects) c…
▽ More
The neocortex is widely believed to be the seat of intelligence and "mind". However, it's unclear what "mind" is, or how the special features of neocortex enable it, though likely "connectionist" principles are involved *A. The key to intelligence1 is learning relationships between large numbers of signals (such as pixel values), rather than memorizing explicit patterns. Causes (such as objects) can then be inferred from a learned internal model. These relationships fall into 2 classes: simple pairwise or second-order correlations (socs), and complex, and vastly more numerous, higher-order correlations (hocsB), such as the product of 3 or more pixels averaged over a set of images. Thus if 3 pixels correlate, they may give an "edge". Neurons with "Hebbian" synapses (changing strength in response to input-output spike-coincidences) are sensitive to such correlations, and it's likely that learned internal models use such neurons. Because output firing depends on input firing via the relevant connection strengths, Hebbian learning provides, in a feedback manner, sensitivity to input correlations. Hocs are vital, since they express "interesting" structure2 (e.g. edges), but their detection requires nonlinear rules operating at synapses of individual neurons. Here we report that in single model neurons learning from hocs fails, and defaults to socs, if nonlinear Hebbian rules are not sufficiently connection-specific. Such failure would inevitably occur if a neuron's input synapses were too crowded, and would undermine biological connectionism. Since the cortex must be hoc-sensitive to achieve the type of learning enabling mind, we propose it uses known, detailed but poorly understood circuitry and physiology to "proofread" Hebbian connections. Analogous DNA proofreading allows evolution of complex genomes (i.e. "life").
△ Less
Submitted 4 December, 2010;
originally announced December 2010.
-
Hebbian Crosstalk Prevents Nonlinear Unsupervised Learning
Authors:
Kingsley J. A. Cox,
Paul R. Adams
Abstract:
Learning is thought to occur by localized, experience-induced changes in the strength of synaptic connections between neurons. Recent work has shown that activity-dependent changes at one connection can affect changes at others (crosstalk). We studied the role of such crosstalk in nonlinear Hebbian learning using a neural network implementation of Independent Components Analysis (ICA). We find t…
▽ More
Learning is thought to occur by localized, experience-induced changes in the strength of synaptic connections between neurons. Recent work has shown that activity-dependent changes at one connection can affect changes at others (crosstalk). We studied the role of such crosstalk in nonlinear Hebbian learning using a neural network implementation of Independent Components Analysis (ICA). We find that there is a sudden qualitative change in the performance of the network at a critical crosstalk level and discuss the implications of this for nonlinear learning from higher-order correlations in the neocortex.
△ Less
Submitted 20 February, 2008;
originally announced February 2008.
