-
Mixed memories in Hopfield networks
Authors:
Véronique Gayrard
Abstract:
We consider the class of Hopfield models of associative memory with activation function $F$ and state space $\{-1,1\}^N$, where each vertex of the cube describes a configuration of $N$ binary neurons. $M$ randomly chosen configurations, called patterns, are stored using an energy function designed to make them local minima. If they are, which is known to depend on how $M$ scales with $N$, then the…
▽ More
We consider the class of Hopfield models of associative memory with activation function $F$ and state space $\{-1,1\}^N$, where each vertex of the cube describes a configuration of $N$ binary neurons. $M$ randomly chosen configurations, called patterns, are stored using an energy function designed to make them local minima. If they are, which is known to depend on how $M$ scales with $N$, then they can be retrieved using a dynamics that decreases the energy. However, storing the patterns in the energy function also creates unintended local minima, and thus false memories. Although this has been known since the earliest work on the subject, it has only been supported by numerical simulations and non-rigorous calculations, except in elementary cases.
Our results are twofold. For a generic function $F$, we explicitly construct a set of configurations, called mixed memories, whose properties are intended to characterise the local minima of the energy function. For three prominent models, namely the classical, the dense and the modern Hopfield models, obtained for quadratic, polynomial and exponential functions $F$ respectively, we give conditions on the growth rate of $M$ which guarantee that, as $N$ diverges, mixed memories are fixed points of the retrieval dynamics and thus exact minima of the energy. We conjecture that in this regime, all local minima are mixed memories.
△ Less
Submitted 7 April, 2025;
originally announced April 2025.
-
Emergence of near-TAP free energy functional in the SK model at high temperature
Authors:
Véronique Gayrard
Abstract:
We study the SK model at inverse temperature $β>0$ and strictly positive field $h>0$ in the region of $(β,h)$ where the replica-symmetric formula is valid. An integral representation of the partition function derived from the Hubbard-Stratonovitch transformation combined with a duality formula is used to prove that the infinite volume free energy of the SK model can be expressed as a variational f…
▽ More
We study the SK model at inverse temperature $β>0$ and strictly positive field $h>0$ in the region of $(β,h)$ where the replica-symmetric formula is valid. An integral representation of the partition function derived from the Hubbard-Stratonovitch transformation combined with a duality formula is used to prove that the infinite volume free energy of the SK model can be expressed as a variational formula on the space of magnetisations, $m$. The resulting free energy functional differs from that of Thouless, Anderson and Palmer (TAP) by the term $-\frac{β^2}{4}\left(q-q_{\text{EA}}(m)\right)^2$ where $q_{\text{EA}}(m)$ is the Edwards-Anderson parameter and $q$ is the minimiser of the replica-symmetric formula. Thus, both functionals have the same critical points and take the same value on the subspace of magnetisations satisfying $q_{\text{EA}}(m)=q$. This result is based on an in-depth study of the global maximum of this near-TAP free energy functional using Bolthausen's solutions of the TAP equations, Bandeira & van Handel's bounds on the spectral norm of non-homogeneous Wigner-type random matrices, and Gaussian comparison techniques. It holds for $(β,h)$ in a large subregion of the de Almeida and Thouless high-temperature stability region.
△ Less
Submitted 4 June, 2023;
originally announced June 2023.
-
Dynamic phase diagram of the REM
Authors:
Véronique Gayrard,
Lisa Hartung
Abstract:
By studying the two-time overlap correlation function, we give a comprehensive analysis of the phase diagram of the Random Hopping Dynamics of the Random Energy Model (REM) on time-scales that are exponential in the volume. These results are derived from the convergence properties of the clock process associated to the dynamics and fine properties of the simple random walk in the $n$-dimensional d…
▽ More
By studying the two-time overlap correlation function, we give a comprehensive analysis of the phase diagram of the Random Hopping Dynamics of the Random Energy Model (REM) on time-scales that are exponential in the volume. These results are derived from the convergence properties of the clock process associated to the dynamics and fine properties of the simple random walk in the $n$-dimensional discrete cube.
△ Less
Submitted 11 March, 2019;
originally announced March 2019.
-
Asymptotic behavior and aging of a low temperature cascading 2-GREM dynamics at extreme time scales
Authors:
Luiz Renato Fontes,
Véronique Gayrard
Abstract:
We derive scaling limit results for the Random Hopping Dynamics for the cascading two-level GREM at low temperature at extreme time scales. It is known that in the cascading regime there are two static critical temperatures. We show that there exists a (narrow) set of fine tuning temperatures; when they lie below the static lowest critical temperature, three distinct dynamical phases emerge below…
▽ More
We derive scaling limit results for the Random Hopping Dynamics for the cascading two-level GREM at low temperature at extreme time scales. It is known that in the cascading regime there are two static critical temperatures. We show that there exists a (narrow) set of fine tuning temperatures; when they lie below the static lowest critical temperature, three distinct dynamical phases emerge below the lowest critical temperature, with three different types of limiting dynamics depending on whether the temperature is (well) above or below, or at a fine tuning temperature, all of which are given in terms of K processes. We also derive scaling limit results for temperatures between the lowest and he highest critical ones, as well as aging results for all the limiting processes mentioned above, by taking a second small time limit.
△ Less
Submitted 26 January, 2018;
originally announced January 2018.
-
Aging in Metropolis dynamics of the REM: a proof
Authors:
Véronique Gayrard
Abstract:
We study the aging behavior of the Random Energy Model (REM) evolving under Metropolis dynamics. We prove that a classical two-time correlation function converges almost surely to the arcsine law distribution function that characterizes activated aging, as predicted in the physics literature, in the optimal domain of the time-scale and temperature parameters where this result can be expected to ho…
▽ More
We study the aging behavior of the Random Energy Model (REM) evolving under Metropolis dynamics. We prove that a classical two-time correlation function converges almost surely to the arcsine law distribution function that characterizes activated aging, as predicted in the physics literature, in the optimal domain of the time-scale and temperature parameters where this result can be expected to hold. In the course of the proof we establish that a certain continuous time clock process, after proper rescaling, converges almost surely to a stable subordinator, improving upon the result of Cerny and Wassmer (2015) where a closely related clock is shown to converge in probability only, and in a restricted region of the time-scale and temperature parameters. The random rescaling involved in this convergence is controlled at the fine level of fluctuations. As a byproduct, we refine and prove a conjecture made in Cerny and Wassmer (2015).
△ Less
Submitted 19 February, 2016;
originally announced February 2016.
-
Aging in the GREM-like trap model
Authors:
Véronique Gayrard,
Onur Gün
Abstract:
The GREM-like trap model is a continuous time Markov jump process on the leaves of a finite volume $L$-level tree whose transition rates depend on a trapping landscape built on the vertices of the whole tree. We prove that the natural two-time correlation function of the dynamics ages in the infinite volume limit and identify the limiting function. Moreover, we take the limit $L\to\infty$ of the t…
▽ More
The GREM-like trap model is a continuous time Markov jump process on the leaves of a finite volume $L$-level tree whose transition rates depend on a trapping landscape built on the vertices of the whole tree. We prove that the natural two-time correlation function of the dynamics ages in the infinite volume limit and identify the limiting function. Moreover, we take the limit $L\to\infty$ of the two-time correlation function of the infinite volume $L$-level tree. The aging behavior of the dynamics is characterized by a collection of clock processes, one for each level of the tree. We show that for any $L$, the joint law of the clock processes converges. Furthermore, any such limit can be expressed through Neveu's continuous state branching process. Hence, the latter contains all the information needed to describe aging in the GREM-like trap model both for finite and infinite levels.
△ Less
Submitted 13 January, 2015;
originally announced January 2015.
-
Convergence of clock processes and aging in Metropolis dynamics of a truncated REM
Authors:
Véronique Gayrard
Abstract:
We study the aging behavior of a truncated version of the Random Energy Model evolving under Metropolis dynamics. We prove that the natural time-time correlation function defined through the overlap function converges to an arcsine law distribution function, almost surely in the random environment and in the full range of time scales and temperatures for which such a result can be expected to hold…
▽ More
We study the aging behavior of a truncated version of the Random Energy Model evolving under Metropolis dynamics. We prove that the natural time-time correlation function defined through the overlap function converges to an arcsine law distribution function, almost surely in the random environment and in the full range of time scales and temperatures for which such a result can be expected to hold. This establishes that the dynamics ages in the same way as Bouchaud's REM-like trap model, thus extending the universality class of the latter model. The proof relies on a clock process convergence result of a new type where the number of summands is itself a clock process. This reflects the fact that the exploration process of Metropolis dynamics is itself an aging process, governed by its own clock. Both clock processes are shown to converge to stable subordinators below certain critical lines in their time-scale and temperature domains, almost surely in the random environment.
△ Less
Submitted 3 February, 2014;
originally announced February 2014.
-
Convergence of clock processes on infinite graphs and aging in Bouchaud's asymmetric trap model on ${\Bbb Z}^d$
Authors:
Véronique Gayrard,
Adela Svejda
Abstract:
Using a method developed by Durrett and Resnick [22] we establish general criteria for the convergence of properly rescaled clock processes of random dynamics in random environments on infinite graphs. This complements the results of [26], [19], and [20]: put together these results provide a unified framework for proving convergence of clock processes. As a first application we prove that Bouchaud…
▽ More
Using a method developed by Durrett and Resnick [22] we establish general criteria for the convergence of properly rescaled clock processes of random dynamics in random environments on infinite graphs. This complements the results of [26], [19], and [20]: put together these results provide a unified framework for proving convergence of clock processes. As a first application we prove that Bouchaud's asymmetric trap model on ${\Bbb Z}^d$ exhibits a normal aging behavior for all $d\geq 2$. Namely, we show that certain two-time correlation functions, among which the classical probability to find the process at the same site at two time points, converge, as the age of the process diverges, to the distribution function of the arcsine law. As a byproduct we prove that the fractional kinetics process ages.
△ Less
Submitted 13 January, 2015; v1 submitted 12 September, 2013;
originally announced September 2013.
-
The K-process on a tree as a scaling limit of the GREM-like trap model
Authors:
L. R. G. Fontes,
R. J. Gava,
V. Gayrard
Abstract:
We introduce trap models on a finite volume $k$-level tree as a class of Markov jump processes with state space the leaves of that tree. They serve to describe the GREM-like trap model of Sasaki and Nemoto. Under suitable conditions on the parameters of the trap model, we establish its infinite volume limit, given by what we call a $K$-process in an infinite $k$-level tree. From this we deduce tha…
▽ More
We introduce trap models on a finite volume $k$-level tree as a class of Markov jump processes with state space the leaves of that tree. They serve to describe the GREM-like trap model of Sasaki and Nemoto. Under suitable conditions on the parameters of the trap model, we establish its infinite volume limit, given by what we call a $K$-process in an infinite $k$-level tree. From this we deduce that the $K$-process also is the scaling limit of the GREM-like trap model on extreme time scales under a fine tuning assumption on the volumes.
△ Less
Submitted 21 March, 2014; v1 submitted 18 February, 2012;
originally announced February 2012.
-
Scaling limits and aging for asymmetric trap models on the complete graph and K-processes
Authors:
S. C. Bezerra,
L. R. G. Fontes,
R. J. Gava,
V. Gayrard,
P. Mathieu
Abstract:
We obtain scaling limit results for asymmetric trap models and their infinite volume counterparts, namely asymmetric K processes. Aging results for the latter processes are derived therefrom.
We obtain scaling limit results for asymmetric trap models and their infinite volume counterparts, namely asymmetric K processes. Aging results for the latter processes are derived therefrom.
△ Less
Submitted 16 May, 2012; v1 submitted 18 February, 2012;
originally announced February 2012.
-
Convergence to extremal processes in random environments and extremal ageing in SK models
Authors:
Anton Bovier,
Veronique Gayrard,
Adela Svejda
Abstract:
This paper extends recent results on aging in mean field spin glasses on short time scales, obtained by Ben Arous and Gun [2] in law with respect to the environment, to results that hold almost surely, respectively in probability, with respect to the environment. It is based on the methods put forward in Gayrard [8,9] and naturally complements Bovier and Gayrard [6].
This paper extends recent results on aging in mean field spin glasses on short time scales, obtained by Ben Arous and Gun [2] in law with respect to the environment, to results that hold almost surely, respectively in probability, with respect to the environment. It is based on the methods put forward in Gayrard [8,9] and naturally complements Bovier and Gayrard [6].
△ Less
Submitted 2 October, 2012; v1 submitted 10 January, 2012;
originally announced January 2012.
-
Aging in reversible dynamics of disordered systems. I. Emergence of the arcsine law in Bouchaud's asymmetric trap model on the complete graph
Authors:
Véronique Gayrard
Abstract:
In this paper the celebrated arcsine aging scheme of G. Ben Arous and J. Cerný is taken up. Using a brand new approach based on point processes and weak convergence techniques, this scheme is implemented in a wide class of Markov processes that can best be described as Glauber dynamics of discrete disordered systems. More specifically, conditions are given for the underlying clock process (a parti…
▽ More
In this paper the celebrated arcsine aging scheme of G. Ben Arous and J. Cerný is taken up. Using a brand new approach based on point processes and weak convergence techniques, this scheme is implemented in a wide class of Markov processes that can best be described as Glauber dynamics of discrete disordered systems. More specifically, conditions are given for the underlying clock process (a partial sum process that measures the total time elapsed along paths of a given length) to converge to a subordinator, and this subordinator is constructed explicitly. This approach is illustrated on Bouchaud's asymmetric trap model on the complete graph for which aging is for the first time proved, and the full, optimal picture, obtained.
△ Less
Submitted 23 August, 2010;
originally announced August 2010.
-
Aging in reversible dynamics of disordered systems. II. Emergence of the arcsine law in the random hopping time dynamics of the REM
Authors:
Véronique Gayrard
Abstract:
Applying the new tools developed in [G1], we investigate the arcsine aging regime of the random hopping time dynamics of the REM. Our results are optimal in several ways. They cover the full time-scale and temperature domain where this phenomenon occurs. On this domain the limiting clock process and associated time correlation function are explicitly constructed. Finally, all convergence statement…
▽ More
Applying the new tools developed in [G1], we investigate the arcsine aging regime of the random hopping time dynamics of the REM. Our results are optimal in several ways. They cover the full time-scale and temperature domain where this phenomenon occurs. On this domain the limiting clock process and associated time correlation function are explicitly constructed. Finally, all convergence statements w.r.t. the law of the random environment are obtained in the strongest sense possible, except perhaps on the very last scales before equilibrium.
△ Less
Submitted 23 August, 2010;
originally announced August 2010.
-
Convergence of clock processes in random environments and ageing in the p-spin SK model
Authors:
Anton Bovier,
Véronique Gayrard
Abstract:
We derive a general criterion for the convergence of clock processes in random dynamics in random environments that is applicable in cases when correlations are not negligible, extending recent results by Gayrard [(2010), (2011), forthcoming], based on general criterion for convergence of sums of dependent random variables due to Durrett and Resnick [Ann. Probab. 6 (1978) 829-846]. We demonstrate…
▽ More
We derive a general criterion for the convergence of clock processes in random dynamics in random environments that is applicable in cases when correlations are not negligible, extending recent results by Gayrard [(2010), (2011), forthcoming], based on general criterion for convergence of sums of dependent random variables due to Durrett and Resnick [Ann. Probab. 6 (1978) 829-846]. We demonstrate the power of this criterion by applying it to the case of random hopping time dynamics of the p-spin SK model. We prove that on a wide range of time scales, the clock process converges to a stable subordinator almost surely with respect to the environment. We also show that a time-time correlation function converges to the arcsine law for this subordinator, almost surely. This improves recent results of Ben Arous, Bovier and Cerny [Comm. Math. Phys. 282 (2008) 663-695] that obtained similar convergence results in law, with respect to the random environment.
△ Less
Submitted 15 March, 2013; v1 submitted 23 August, 2010;
originally announced August 2010.
-
A new REM conjecture
Authors:
Gerard Ben Arous,
Veronique Gayrard,
Alexey Kuptsov
Abstract:
We introduce here a new universality conjecture for levels of random Hamiltonians, in the same spirit as the local REM conjecture made by S. Mertens and H. Bauke. We establish our conjecture for a wide class of Gaussian and non-Gaussian Hamiltonians, which include the $p$-spin models, the Sherrington-Kirkpatrick model and the number partitioning problem. We prove that our universality result is…
▽ More
We introduce here a new universality conjecture for levels of random Hamiltonians, in the same spirit as the local REM conjecture made by S. Mertens and H. Bauke. We establish our conjecture for a wide class of Gaussian and non-Gaussian Hamiltonians, which include the $p$-spin models, the Sherrington-Kirkpatrick model and the number partitioning problem. We prove that our universality result is optimal for the last two models by showing when this universality breaks down.
△ Less
Submitted 13 December, 2006;
originally announced December 2006.
-
Hitting time of large subsets of the hypercube
Authors:
Jiri Cerny,
Veronique Gayrard
Abstract:
We study the simple random walk on the $n$-dimensional hypercube, in particular its hitting times of large (possibly random) sets. We give simple conditions on these sets ensuring that the properly-rescaled hitting time is asymptotically exponentially distributed, uniformly in the starting position of the walk. These conditions are then verified for percolation clouds with densities that are muc…
▽ More
We study the simple random walk on the $n$-dimensional hypercube, in particular its hitting times of large (possibly random) sets. We give simple conditions on these sets ensuring that the properly-rescaled hitting time is asymptotically exponentially distributed, uniformly in the starting position of the walk. These conditions are then verified for percolation clouds with densities that are much smaller than $(n \log n)^{-1}$. A main motivation behind this paper is the study of the so-called aging phenomenon in the Random Energy Model (REM), the simplest model of a mean-field spin glass. Our results allow us to prove aging in the REM for all temperatures, thereby extending earlier results to their optimal temperature domain.
△ Less
Submitted 8 November, 2006;
originally announced November 2006.
-
Elementary potential theory on the hypercube
Authors:
Gerard Ben Arous,
Veronique Gayrard
Abstract:
This work addresses potential theoretic questions for the standard nearest neighbor random walk on the hypercube $\{-1,+1\}^N$. For a large class of subsets $A\subset\{-1,+1\}^N$ we give precise estimates for the harmonic measure of $A$, the mean hitting time of $A$, and the Laplace transform of this hitting time. In particular, we give precise sufficient conditions for the harmonic measure to b…
▽ More
This work addresses potential theoretic questions for the standard nearest neighbor random walk on the hypercube $\{-1,+1\}^N$. For a large class of subsets $A\subset\{-1,+1\}^N$ we give precise estimates for the harmonic measure of $A$, the mean hitting time of $A$, and the Laplace transform of this hitting time. In particular, we give precise sufficient conditions for the harmonic measure to be asymptotically uniform, and for the hitting time to be asymptotically exponentially distributed, as $N\to\infty$. Our approach relies on a $d$-dimensional extension of the Ehrenfest urn scheme called lumping and covers the case where $d$ is allowed to diverge with $N$ as long as $d\leqα_0\frac{N}{\log N}$ for some constant $0<α_0<1$.
△ Less
Submitted 7 November, 2006;
originally announced November 2006.
-
Aging in the random energy model
Authors:
G. Ben Arous,
A. Bovier,
V. Gayrard
Abstract:
In this letter we announce rigorous results on the phenomenon of aging in the Glauber dynamics of the random energy model and their relation to Bouchaud's 'REM-like' trap model. We show that, below the critical temperature, if we consider a time-scale that diverges with the system size in such a way that equilibrium is almost, but not quite reached on that scale, a suitably defined autocorrelati…
▽ More
In this letter we announce rigorous results on the phenomenon of aging in the Glauber dynamics of the random energy model and their relation to Bouchaud's 'REM-like' trap model. We show that, below the critical temperature, if we consider a time-scale that diverges with the system size in such a way that equilibrium is almost, but not quite reached on that scale, a suitably defined autocorrelation function has the same asymptotic behaviour than its analog in the trap model.
△ Less
Submitted 11 October, 2001;
originally announced October 2001.
-
Metastability and low lying spectra in reversible Markov chains
Authors:
A. Bovier,
M. Eckhoff,
V. Gayrard,
M. Klein
Abstract:
We study a large class of reversible Markov chains with discrete state space and transition matrix $P_N$. We define the notion of a set of {\it metastable points} as a subset of the state space $\G_N$ such that (i) this set is reached from any point $x\in \G_N$ without return to x with probability at least $b_N$, while (ii) for any two point x,y in the metastable set, the probability…
▽ More
We study a large class of reversible Markov chains with discrete state space and transition matrix $P_N$. We define the notion of a set of {\it metastable points} as a subset of the state space $\G_N$ such that (i) this set is reached from any point $x\in \G_N$ without return to x with probability at least $b_N$, while (ii) for any two point x,y in the metastable set, the probability $T^{-1}_{x,y}$ to reach y from x without return to x is smaller than $a_N^{-1}\ll b_N$. Under some additional non-degeneracy assumption, we show that in such a situation: \item{(i)} To each metastable point corresponds a metastable state, whose mean exit time can be computed precisely. \item{(ii)} To each metastable point corresponds one simple eigenvalue of $1-P_N$ which is essentially equal to the inverse mean exit time from this state. The corresponding eigenfunctions are close to the indicator function of the support of the metastable state. Moreover, these results imply very sharp uniform control of the deviation of the probability distribution of metastable exit times from the exponential distribution.
△ Less
Submitted 26 July, 2000;
originally announced July 2000.
-
Metastability and small eigenvalues in Markov chains
Authors:
A. Bovier,
M. Eckhoff,
V. Gayrard,
M. Klein
Abstract:
In this letter we announce rigorous results that elucidate the relation between metastable states and low-lying eigenvalues in Markov chains in a much more general setting and with considerable greater precision as was so far available. This includes a sharp uncertainty principle relating all low-lying eigenvalues to mean times of metastable transitions, a relation between the support of eigenfu…
▽ More
In this letter we announce rigorous results that elucidate the relation between metastable states and low-lying eigenvalues in Markov chains in a much more general setting and with considerable greater precision as was so far available. This includes a sharp uncertainty principle relating all low-lying eigenvalues to mean times of metastable transitions, a relation between the support of eigenfunctions and the attractor of a metastable state, and sharp estimates on the convergence of probability distribution of the metastable transition times to the exponential distribution.
△ Less
Submitted 21 July, 2000;
originally announced July 2000.
-
Sample path large deviations for a class of Markov chains related to disordered mean field models
Authors:
Anton Bovier,
Veronique Gayrard
Abstract:
We prove a large deviation principle on path space for a class of discrete time Markov processes whose state space is the intersection of a regular domain $Ł\subset \R^d$ with some lattice of spacing $\e$. Transitions from $x$ to $y$ are allowed if $\e^{-1}(x-y)\in \D$ for some fixed set of vectors $\D$. The transition probabilities $p_\e(t,x,y)$, which themselves depend on $\e$, are allowed to…
▽ More
We prove a large deviation principle on path space for a class of discrete time Markov processes whose state space is the intersection of a regular domain $Ł\subset \R^d$ with some lattice of spacing $\e$. Transitions from $x$ to $y$ are allowed if $\e^{-1}(x-y)\in \D$ for some fixed set of vectors $\D$. The transition probabilities $p_\e(t,x,y)$, which themselves depend on $\e$, are allowed to depend on the starting point $x$ and the time $t$ in a sufficiently regular way, except near the boundaries, where some singular behaviour is allowed. The rate function is identified as an action functional which is given as the integral of a Lagrange function. %of time dependent relativistic classical mechanics. Markov processes of this type arise in the study of mean field dynamics of disordered mean field models.
△ Less
Submitted 5 May, 1999;
originally announced May 1999.
-
Metastability in stochastic dynamics of disordered mean-field models
Authors:
A. Bovier,
M. Eckhoff,
V. Gayrard,
M. Klein
Abstract:
We study a class of Markov chains that describe reversible stochastic dynamics of a large class of disordered mean field models at low temperatures. Our main purpose is to give a precise relation between the metastable time scales in the problem to the properties of the rate functions of the corresponding Gibbs measures. We derive the analog of the Wentzell-Freidlin theory in this case, showing…
▽ More
We study a class of Markov chains that describe reversible stochastic dynamics of a large class of disordered mean field models at low temperatures. Our main purpose is to give a precise relation between the metastable time scales in the problem to the properties of the rate functions of the corresponding Gibbs measures. We derive the analog of the Wentzell-Freidlin theory in this case, showing that any transition can be decomposed, with probability exponentially close to one, into a deterministic sequence of ``admissible transitions''. For these admissible transitions we give upper and lower bounds on the expected transition times that differ only by a constant.
The distribution rescaled transition times are shown to converge to the exponential distribution. We exemplify our results in the context of the random field Curie-Weiss model.
△ Less
Submitted 20 May, 1999; v1 submitted 24 November, 1998;
originally announced November 1998.
-
Metastates in the Hopfield model in the replica symmetric regime
Authors:
Anton Bovier,
Veronique Gayrard
Abstract:
We study the finite dimensional marginals of the Gibbs measure in the Hopfield model at low temperature when the number of patterns, $M$, is proportional to the volume with a sufficiently small proportionality constant $\a>0$. It is shown that even when a single pattern is selected (by a magnetic field or by conditioning), the marginals do not converge almost surely, but only in law. The corresp…
▽ More
We study the finite dimensional marginals of the Gibbs measure in the Hopfield model at low temperature when the number of patterns, $M$, is proportional to the volume with a sufficiently small proportionality constant $\a>0$. It is shown that even when a single pattern is selected (by a magnetic field or by conditioning), the marginals do not converge almost surely, but only in law. The corresponding limiting law is constructed explicitly. We fit our result in the recently proposed language of ``metastates'' which we discuss in some length. As a byproduct, in a certain regime of the parameters $\a$ and $\b$ (the inverse temperature), we also give a simple proof of Talagrand's [T1] recent result that the replica symmetric solution found by Amit, Gutfreund, and Sompolinsky [AGS] can be rigorously justified.
△ Less
Submitted 9 June, 1997;
originally announced June 1997.
-
Hopfield models as generalized random mean field models
Authors:
Anton Bovier,
Veronique Gayrard
Abstract:
We give a comprehensive self-contained review on the rigorous analysis of the thermodynamics of a class of random spin systems of mean field type whose most prominent example is the Hopfield model. We focus on the low temperature phase and the analysis of the Gibbs measures with large deviation techniques. There is a very detailed and complete picture in the regime of ``small $\a$''; a particula…
▽ More
We give a comprehensive self-contained review on the rigorous analysis of the thermodynamics of a class of random spin systems of mean field type whose most prominent example is the Hopfield model. We focus on the low temperature phase and the analysis of the Gibbs measures with large deviation techniques. There is a very detailed and complete picture in the regime of ``small $\a$''; a particularly satisfactory result concerns a non-trivial regime of parameters in which we prove 1) the convergence of the local ``mean fields'' to gaussian random variables with constant variance and random mean; the random means are from site to site independent gaussians themselves; 2) ``propagation of chaos'', i.e. factorization of the extremal infinite volume Gibbs measures, and 3) the correctness of the ``replica symmetric solution'' of Amit, Gutfreund and Sompolinsky [AGS]. This last result was first proven by M. Talagrand [T4], using different techniques.
△ Less
Submitted 28 May, 1997; v1 submitted 15 July, 1996;
originally announced July 1996.
-
Distribution of overlap profiles in the one-dimensional Kac-Hopfield model
Authors:
Anton Bovier,
Veronique Gayrard,
Pierre Picco
Abstract:
We study a one-dimensional version of the Hopfield model with long, but finite range interactions below the critical temperature. In the thermodynamic limit we obtain large deviation estimates for the distribution of the ``local'' overlaps, the range of the interaction, $γ^{-1}$, being the large parameter. We show in particular that the local overlaps in a typical Gibbs configuration are constan…
▽ More
We study a one-dimensional version of the Hopfield model with long, but finite range interactions below the critical temperature. In the thermodynamic limit we obtain large deviation estimates for the distribution of the ``local'' overlaps, the range of the interaction, $γ^{-1}$, being the large parameter. We show in particular that the local overlaps in a typical Gibbs configuration are constant and equal to one of the mean-field equilibrium values on a scale $o(\g^{-2})$. We also give estimates on the size of typical ``jumps''. i.e. the regions where transitions from one equilibrium value to another take place. Contrary to the situation in the ferromagnetic Kac-model, the structure of the profiles is found to be governed by the quenched disorder rather than by entropy.
△ Less
Submitted 6 February, 1996;
originally announced February 1996.
-
The Retrieval Phase of the Hopfield Model: A Rigorous Analysis of the Overlap Distribution
Authors:
Anton Bovier,
Véronique Gayrard
Abstract:
Standard large deviation estimates or the use of the Hubbard-Stratonovich transformation reduce the analysis of the distribution of the overlap parameters essentially to that of an explicitly known random function $Φ_{N,\b}$ on $\R^M$. In this article we present a rather careful study of the structure of the minima of this random function related to the retrieval of the stored patterns. We denot…
▽ More
Standard large deviation estimates or the use of the Hubbard-Stratonovich transformation reduce the analysis of the distribution of the overlap parameters essentially to that of an explicitly known random function $Φ_{N,\b}$ on $\R^M$. In this article we present a rather careful study of the structure of the minima of this random function related to the retrieval of the stored patterns. We denote by $m^*(\b)$ the modulus of the spontaneous magnetization in the Curie-Weiss model and by $\a$ the ratio between the number of the stored patterns and the system size. We show that there exist strictly positive numbers $0<\g_a<\g_c$ such that 1) If $\sqrt\a\leq \g_a (m^*(\b))^2$, then the absolute minima of $Φ$ are located within small balls around the points $\pm m^*e^μ$, where $e^μ$ denotes the $μ$-th unit vector while 2) if $\sqrt\a\leq \g_c (m^*(\b))^2$ at least a local minimum surrounded by extensive energy barriers exists near these points. The random location of these minima is given within precise bounds. These are used to prove sharp estimates on the support of the Gibbs measures.
KEYWORDS: Hopfield model, neural networks, storage capacity, Gibbs measures, self-averaging, random matrices
△ Less
Submitted 25 July, 1995;
originally announced July 1995.
-
An almost sure large deviation principle for the Hopfield model
Authors:
Anton Bovier,
Véronique Gayrard
Abstract:
We prove a large deviation principle for the finite dimensional marginals of the Gibbs distribution of the macroscopic `overlap'-parameters in the Hopfield model in the case where the number of random patterns, $M$, as a function of the system size $N$ satisfies $\limsup M(N)/N=0$. In this case the rate function (or free energy as a function of the overlap parameters) is independent of the disor…
▽ More
We prove a large deviation principle for the finite dimensional marginals of the Gibbs distribution of the macroscopic `overlap'-parameters in the Hopfield model in the case where the number of random patterns, $M$, as a function of the system size $N$ satisfies $\limsup M(N)/N=0$. In this case the rate function (or free energy as a function of the overlap parameters) is independent of the disorder for almost all realization of the patterns and given by an explicit variational formula.
△ Less
Submitted 3 April, 1995;
originally announced April 1995.
