-
Phase transitions in the three-state Ising spin-glass model with finite connectivity
Authors:
R. Erichsen Jr.,
W. K. Theumann
Abstract:
The statistical mechanics of a two-state Ising spin-glass model with finite random connectivity, in which each site is connected to a finite number of other sites, is extended in this work within the replica technique to study the phase transitions in the three-state Ghatak-Sherrington (or random Blume-Capel) model of a spin glass with a crystal field term. The replica symmetry ansatz for the orde…
▽ More
The statistical mechanics of a two-state Ising spin-glass model with finite random connectivity, in which each site is connected to a finite number of other sites, is extended in this work within the replica technique to study the phase transitions in the three-state Ghatak-Sherrington (or random Blume-Capel) model of a spin glass with a crystal field term. The replica symmetry ansatz for the order function is expressed in terms of a two-dimensional effective-field distribution which is determined numerically by means of a population dynamics procedure. Phase diagrams are obtained exhibiting phase boundaries which have a reentrance with both a continuous and a genuine first-order transition with a discontinuity in the entropy. This may be seen as "inverse freezing", which has been studied extensively lately, as a process either with or without exchange of latent heat.
△ Less
Submitted 16 June, 2011;
originally announced June 2011.
-
Symmetric sequence processing in a recurrent neural network model with a synchronous dynamics
Authors:
F. L. Metz,
W. K. Theumann
Abstract:
The synchronous dynamics and the stationary states of a recurrent attractor neural network model with competing synapses between symmetric sequence processing and Hebbian pattern reconstruction is studied in this work allowing for the presence of a self-interaction for each unit. Phase diagrams of stationary states are obtained exhibiting phases of retrieval, symmetric and period-two cyclic stat…
▽ More
The synchronous dynamics and the stationary states of a recurrent attractor neural network model with competing synapses between symmetric sequence processing and Hebbian pattern reconstruction is studied in this work allowing for the presence of a self-interaction for each unit. Phase diagrams of stationary states are obtained exhibiting phases of retrieval, symmetric and period-two cyclic states as well as correlated and frozen-in states, in the absence of noise. The frozen-in states are destabilised by synaptic noise and well separated regions of correlated and cyclic states are obtained. Excitatory or inhibitory self-interactions yield enlarged phases of fixed-point or cyclic behaviour.
△ Less
Submitted 11 August, 2009;
originally announced August 2009.
-
Instability of frozen-in states in synchronous Hebbian neural networks
Authors:
F. L. Metz,
W. K. Theumann
Abstract:
The full dynamics of a synchronous recurrent neural network model with Ising binary units and a Hebbian learning rule with a finite self-interaction is studied in order to determine the stability to synaptic and stochastic noise of frozen-in states that appear in the absence of both kinds of noise. Both, the numerical simulation procedure of Eissfeller and Opper and a new alternative procedure t…
▽ More
The full dynamics of a synchronous recurrent neural network model with Ising binary units and a Hebbian learning rule with a finite self-interaction is studied in order to determine the stability to synaptic and stochastic noise of frozen-in states that appear in the absence of both kinds of noise. Both, the numerical simulation procedure of Eissfeller and Opper and a new alternative procedure that allows to follow the dynamics over larger time scales have been used in this work. It is shown that synaptic noise destabilizes the frozen-in states and yields either retrieval or paramagnetic states for not too large stochastic noise. The indications are that the same results may follow in the absence of synaptic noise, for low stochastic noise.
△ Less
Submitted 13 May, 2008;
originally announced May 2008.
-
Period-two cycles in a feed-forward layered neural network model with symmetric sequence processing
Authors:
F. L. Metz,
W. K. Theumann
Abstract:
The effects of dominant sequential interactions are investigated in an exactly solvable feed-forward layered neural network model of binary units and patterns near saturation in which the interaction consists of a Hebbian part and a symmetric sequential term. Phase diagrams of stationary states are obtained and a new phase of cyclic correlated states of period two is found for a weak Hebbian ter…
▽ More
The effects of dominant sequential interactions are investigated in an exactly solvable feed-forward layered neural network model of binary units and patterns near saturation in which the interaction consists of a Hebbian part and a symmetric sequential term. Phase diagrams of stationary states are obtained and a new phase of cyclic correlated states of period two is found for a weak Hebbian term, independently of the number of condensed patterns $c$.
△ Less
Submitted 19 April, 2007;
originally announced April 2007.
-
Feed-forward chains of recurrent attractor neural networks with finite dilution near saturation
Authors:
F. L. Metz,
W. K. Theumann
Abstract:
A stationary state replica analysis for a dual neural network model that interpolates between a fully recurrent symmetric attractor network and a strictly feed-forward layered network, studied by Coolen and Viana, is extended in this work to account for finite dilution of the recurrent Hebbian interactions between binary Ising units within each layer. Gradual dilution is found to suppress part o…
▽ More
A stationary state replica analysis for a dual neural network model that interpolates between a fully recurrent symmetric attractor network and a strictly feed-forward layered network, studied by Coolen and Viana, is extended in this work to account for finite dilution of the recurrent Hebbian interactions between binary Ising units within each layer. Gradual dilution is found to suppress part of the phase transitions that arise from the competition between recurrent and feed-forward operation modes of the network. Despite that, a long chain of layers still exhibits a relatively good performance under finite dilution for a balanced ratio between inter-layer and intra-layer interactions.
△ Less
Submitted 9 November, 2005;
originally announced November 2005.
-
Pattern reconstruction and sequence processing in feed-forward layered neural networks near saturation
Authors:
F. L. Metz,
W. K. Theumann
Abstract:
The dynamics and the stationary states for the competition between pattern reconstruction and asymmetric sequence processing are studied here in an exactly solvable feed-forward layered neural network model of binary units and patterns near saturation. Earlier work by Coolen and Sherrington on a parallel dynamics far from saturation is extended here to account for finite stochastic noise due to…
▽ More
The dynamics and the stationary states for the competition between pattern reconstruction and asymmetric sequence processing are studied here in an exactly solvable feed-forward layered neural network model of binary units and patterns near saturation. Earlier work by Coolen and Sherrington on a parallel dynamics far from saturation is extended here to account for finite stochastic noise due to a Hebbian and a sequential learning rule. Phase diagrams are obtained with stationary states and quasi-periodic non-stationary solutions. The relevant dependence of these diagrams and of the quasi-periodic solutions on the stochastic noise and on initial inputs for the overlaps is explicitly discussed.
△ Less
Submitted 1 July, 2005;
originally announced July 2005.
-
The three-state layered neural network with finite dilution
Authors:
W. K. Theumann,
R. Erichsen Jr
Abstract:
The dynamics and the stationary states of an exactly solvable three-state layered feed-forward neural network model with asymmetric synaptic connections, finite dilution and low pattern activity are studied in extension of a recent work on a recurrent network. Detailed phase diagrams are obtained for the stationary states and for the time evolution of the retrieval overlap with a single pattern.…
▽ More
The dynamics and the stationary states of an exactly solvable three-state layered feed-forward neural network model with asymmetric synaptic connections, finite dilution and low pattern activity are studied in extension of a recent work on a recurrent network. Detailed phase diagrams are obtained for the stationary states and for the time evolution of the retrieval overlap with a single pattern. It is shown that the network develops instabilities for low thresholds and that there is a gradual improvement in network performance with increasing threshold up to an optimal stage. The robustness to synaptic noise is checked and the effects of dilution and of variable threshold on the information content of the network are also established.
△ Less
Submitted 19 January, 2004;
originally announced January 2004.
-
A layered neural network with three-state neurons optimizing the mutual information
Authors:
D. Bolle,
R. Erichsen, Jr.,
W. K. Theumann
Abstract:
The time evolution of an exactly solvable layered feedforward neural network with three-state neurons and optimizing the mutual information is studied for arbitrary synaptic noise (temperature). Detailed stationary temperature-capacity and capacity-activity phase diagrams are obtained. The model exhibits pattern retrieval, pattern-fluctuation retrieval and spin-glass phases. It is found that the…
▽ More
The time evolution of an exactly solvable layered feedforward neural network with three-state neurons and optimizing the mutual information is studied for arbitrary synaptic noise (temperature). Detailed stationary temperature-capacity and capacity-activity phase diagrams are obtained. The model exhibits pattern retrieval, pattern-fluctuation retrieval and spin-glass phases. It is found that there is an improved performance in the form of both a larger critical capacity and information content compared with three-state Ising-type layered network models. Flow diagrams reveal that saddle-point solutions associated with fluctuation overlaps slow down considerably the flow of the network states towards the stable fixed-points.
△ Less
Submitted 26 May, 2003;
originally announced May 2003.
-
Mean-field dynamics of sequence processing neural networks with finite connectivity
Authors:
W. K. Theumann
Abstract:
A recent dynamic mean-field theory for sequence processing in fully connected neural networks of Hopfield-type (During, Coolen and Sherrington, 1998) is extended and analized here for a symmetrically diluted network with finite connectivity near saturation. Equations for the dynamics and the stationary states are obtained for the macroscopic observables and the precise equivalence is established…
▽ More
A recent dynamic mean-field theory for sequence processing in fully connected neural networks of Hopfield-type (During, Coolen and Sherrington, 1998) is extended and analized here for a symmetrically diluted network with finite connectivity near saturation. Equations for the dynamics and the stationary states are obtained for the macroscopic observables and the precise equivalence is established with the single-pattern retrieval problem in a layered feed-forward network with finite connectivity.
△ Less
Submitted 20 March, 2003;
originally announced March 2003.
-
Time evolution of the extremely diluted Blume-Emery-Griffiths neural network
Authors:
D. Bolle',
D. R. C. Dominguez,
R. Erichsen Jr.,
E. Korutcheva,
W. K. Theumann
Abstract:
The time evolution of the extremely diluted Blume-Emery-Griffiths neural network model is studied, and a detailed equilibrium phase diagram is obtained exhibiting pattern retrieval, fluctuation retrieval and self-sustained activity phases. It is shown that saddle-point solutions associated with fluctuation overlaps slow down considerably the flow of the network states towards the retrieval fixed…
▽ More
The time evolution of the extremely diluted Blume-Emery-Griffiths neural network model is studied, and a detailed equilibrium phase diagram is obtained exhibiting pattern retrieval, fluctuation retrieval and self-sustained activity phases. It is shown that saddle-point solutions associated with fluctuation overlaps slow down considerably the flow of the network states towards the retrieval fixed points. A comparison of the performance with other three-state networks is also presented.
△ Less
Submitted 14 August, 2002; v1 submitted 14 August, 2002;
originally announced August 2002.
-
Retrieval behavior and thermodynamic properties of symmetrically diluted Q-Ising neural networks
Authors:
W. K. Theumann,
R. Erichsen Jr
Abstract:
The retrieval behavior and thermodynamic properties of symmetrically diluted Q-Ising neural networks are derived and studied in replica-symmetric mean-field theory generalizing earlier works on either the fully connected or the symmetrical extremely diluted network. Capacity-gain parameter phase diagrams are obtained for the Q=3, Q=4 and $Q=\infty$ state networks with uniformly distributed patte…
▽ More
The retrieval behavior and thermodynamic properties of symmetrically diluted Q-Ising neural networks are derived and studied in replica-symmetric mean-field theory generalizing earlier works on either the fully connected or the symmetrical extremely diluted network. Capacity-gain parameter phase diagrams are obtained for the Q=3, Q=4 and $Q=\infty$ state networks with uniformly distributed patterns of low activity in order to search for the effects of a gradual dilution of the synapses. It is shown that enlarged regions of continuous changeover into a region of optimal performance are obtained for finite stochastic noise and small but finite connectivity. The de Almeida-Thouless lines of stability are obtained for arbitrary connectivity, and the resulting phase diagrams are used to draw conclusions on the behavior of symmetrically diluted networks with other pattern distributions of either high or low activity.
△ Less
Submitted 28 August, 2001; v1 submitted 6 July, 2001;
originally announced July 2001.
-
Categorization in fully connected multi-state neural network models
Authors:
R. Erichsen,
W. K. Theumann,
D. R. C. Dominguez
Abstract:
The categorization ability of fully connected neural network models, with either discrete or continuous Q-state units, is studied in this work in replica symmetric mean-field theory. Hierarchically correlated multi-state patterns in a two level structure of ancestors and descendents (examples) are embedded in the network and the categorization task consists in recognizing the ancestors when the…
▽ More
The categorization ability of fully connected neural network models, with either discrete or continuous Q-state units, is studied in this work in replica symmetric mean-field theory. Hierarchically correlated multi-state patterns in a two level structure of ancestors and descendents (examples) are embedded in the network and the categorization task consists in recognizing the ancestors when the network is trained exclusively with their descendents. Explicit results for the dependence of the equilibrium properties of a Q=3-state model and a $Q=\infty$-state model are obtained in the form of phase diagrams and categorization curves. A strong improvement of the categorization ability is found when the network is trained with examples of low activity. The categorization ability is found to be robust to finite threshold and synaptic noise. The Almeida-Thouless lines that limit the validity of the replica-symmetric results, are also obtained.
△ Less
Submitted 1 October, 1999;
originally announced October 1999.
-
Optimally adapted multi-state neural networks trained with noise
Authors:
R. Erichsen Jr.,
W. K. Theumann
Abstract:
The principle of adaptation in a noisy retrieval environment is extended here to a diluted attractor neural network of Q-state neurons trained with noisy data. The network is adapted to an appropriate noisy training overlap and training activity which are determined self-consistently by the optimized retrieval attractor overlap and activity. The optimized storage capacity and the corresponding r…
▽ More
The principle of adaptation in a noisy retrieval environment is extended here to a diluted attractor neural network of Q-state neurons trained with noisy data. The network is adapted to an appropriate noisy training overlap and training activity which are determined self-consistently by the optimized retrieval attractor overlap and activity. The optimized storage capacity and the corresponding retriever overlap are considerably enhanced by an adequate threshold in the states. Explicit results for improved optimal performance and new retriever phase diagrams are obtained for Q=3 and Q=4, with coexisting phases over a wide range of thresholds. Most of the interesting results are stable to replica-symmetry-breaking fluctuations.
△ Less
Submitted 20 October, 1998;
originally announced October 1998.
