Showing 1–10 of 10 results for author: Arous, G B

Landscape complexity beyond invariance and the elastic manifold

Authors: Gérard Ben Arous, Paul Bourgade, Benjamin McKenna

Abstract: This paper characterizes the annealed, topological complexity (both of total critical points and of local minima) of the elastic manifold. This classical model of a disordered elastic system captures point configurations with self-interactions in a random medium. We establish the simple-vs.-glassy phase diagram in the model parameters, with these phases separated by a physical boundary known as th… ▽ More This paper characterizes the annealed, topological complexity (both of total critical points and of local minima) of the elastic manifold. This classical model of a disordered elastic system captures point configurations with self-interactions in a random medium. We establish the simple-vs.-glassy phase diagram in the model parameters, with these phases separated by a physical boundary known as the Larkin mass, confirming formulas of Fyodorov and Le Doussal. One essential, dynamical, step of the proof also applies to a general signal-to-noise model of soft spins in an anisotropic well, for which we prove a negative-second-moment threshold distinguishing positive from zero complexity. A universal near-critical behavior appears within this phase portrait, namely quadratic near-critical vanishing of the complexity of total critical points, and cubic near-critical vanishing of the complexity of local minima. These two models serve as a paradigm of complexity calculations for Gaussian landscapes exhibiting few distributional symmetries, i.e. beyond the invariant setting. The two main inputs for the proof are determinant asymptotics for non-invariant random matrices from our companion paper [Ben Arous, Bourgade, McKenna 2021], and the atypical convexity and integrability of the limiting variational problems. △ Less

Submitted 11 May, 2021; originally announced May 2021.

Comments: 37 pages, 2 figures

MSC Class: 60B20; 60G15; 82B44

Counting equilibria of large complex systems by instability index

Authors: Gérard Ben Arous, Yan V Fyodorov, Boris A Khoruzhenko

Abstract: We consider a nonlinear autonomous system of $N\gg 1$ degrees of freedom randomly coupled by both relaxational ('gradient') and non-relaxational ('solenoidal') random interactions. We show that with increased interaction strength such systems generically undergo an abrupt transition from a trivial phase portrait with a single stable equilibrium into a topologically non-trivial regime of 'absolute… ▽ More We consider a nonlinear autonomous system of $N\gg 1$ degrees of freedom randomly coupled by both relaxational ('gradient') and non-relaxational ('solenoidal') random interactions. We show that with increased interaction strength such systems generically undergo an abrupt transition from a trivial phase portrait with a single stable equilibrium into a topologically non-trivial regime of 'absolute instability' where equilibria are on average exponentially abundant, but typically all of them are unstable, unless the dynamics is purely gradient. When interactions increase even further the stable equilibria eventually become on average exponentially abundant unless the interaction is purely solenoidal. We further calculate the mean proportion of equilibria which have a fixed fraction of unstable directions. △ Less

Submitted 2 July, 2021; v1 submitted 3 August, 2020; originally announced August 2020.

Comments: Main paper - 7 pages, Supplementary Information - 20 pages. Revised version - minor changes, including adding short discussion of model assumptions

Journal ref: PNAS (2021) Vol. 118 No. 34 e2023719118

Landscape Complexity for the Empirical Risk of Generalized Linear Models

Authors: Antoine Maillard, Gérard Ben Arous, Giulio Biroli

Abstract: We present a method to obtain the average and the typical value of the number of critical points of the empirical risk landscape for generalized linear estimation problems and variants. This represents a substantial extension of previous applications of the Kac-Rice method since it allows to analyze the critical points of high dimensional non-Gaussian random functions. Under a technical hypothesis… ▽ More We present a method to obtain the average and the typical value of the number of critical points of the empirical risk landscape for generalized linear estimation problems and variants. This represents a substantial extension of previous applications of the Kac-Rice method since it allows to analyze the critical points of high dimensional non-Gaussian random functions. Under a technical hypothesis, we obtain a rigorous explicit variational formula for the annealed complexity, which is the logarithm of the average number of critical points at fixed value of the empirical risk. This result is simplified, and extended, using the non-rigorous Kac-Rice replicated method from theoretical physics. In this way we find an explicit variational formula for the quenched complexity, which is generally different from its annealed counterpart, and allows to obtain the number of critical points for typical instances up to exponential accuracy. △ Less

Submitted 18 January, 2023; v1 submitted 4 December, 2019; originally announced December 2019.

Comments: 18 pages and 18 pages appendix. Update to match the published version (v2). Corrections of remaining small typos (v3). Simplification of a technical argument in Appendix A (v4) and clarification of a technical hypothesis (v5)

Journal ref: Proceedings of The First Mathematical and Scientific Machine Learning Conference, PMLR 107:287-327, 2020

Complex energy landscapes in spiked-tensor and simple glassy models: ruggedness, arrangements of local minima and phase transitions

Authors: Valentina Ros, Gerard Ben Arous, Giulio Biroli, Chiara Cammarota

Abstract: We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the statistical properties of the corresponding landscapes and characterize the associate… ▽ More We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the statistical properties of the corresponding landscapes and characterize the associated geometrical phase transitions. In order to perform our study, we develop a framework based on the Kac-Rice method that allows to compute the complexity of the landscape, i.e. the logarithm of the typical number of stationary points and their Hessian. This approach generalizes the one used to compute rigorously the annealed complexity of mean-field glass models. We discuss its advantages with respect to previous frameworks, in particular the thermodynamical replica method which is shown to lead to partially incorrect predictions. △ Less

Submitted 24 April, 2018; v1 submitted 8 April, 2018; originally announced April 2018.

Comments: v2 with references added, typos corrected

Journal ref: Phys. Rev. X 9, 011003 (2019)

Comparing Dynamics: Deep Neural Networks versus Glassy Systems

Authors: M. Baity-Jesi, L. Sagun, M. Geiger, S. Spigler, G. Ben Arous, C. Cammarota, Y. LeCun, M. Wyart, G. Biroli

Abstract: We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are (1) the complexity of the loss landscape and of the dynamics within it, and (2) to what extent DNNs share similarities with glassy systems. Our findings, obtained for different architectures and datasets, suggest that dur… ▽ More We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems. The two main issues we address are (1) the complexity of the loss landscape and of the dynamics within it, and (2) to what extent DNNs share similarities with glassy systems. Our findings, obtained for different architectures and datasets, suggest that during the training process the dynamics slows down because of an increasingly large number of flat directions. At large times, when the loss is approaching zero, the system diffuses at the bottom of the landscape. Despite some similarities with the dynamics of mean-field glassy systems, in particular, the absence of barrier crossing, we find distinctive dynamical behaviors in the two cases, showing that the statistical properties of the corresponding loss and energy landscapes are different. In contrast, when the network is under-parametrized we observe a typical glassy behavior, thus suggesting the existence of different phases depending on whether the network is under-parametrized or over-parametrized. △ Less

Submitted 7 June, 2018; v1 submitted 19 March, 2018; originally announced March 2018.

Comments: 10 pages, 5 figures. Version accepted at ICML 2018

Journal ref: PMLR 80:324-333, 2018; Republication with DOI (cite this one): J. Stat. Mech. (2019) 124013

Universality of REM-like aging in mean field spin glasses

Authors: G. Ben Arous, A. Bovier, J. Cerny

Abstract: Aging has become the paradigm to describe dynamical behavior of glassy systems, and in particular spin glasses. Trap models have been introduced as simple caricatures of effective dynamics of such systems. In this Letter we show that in a wide class of mean field models and on a wide range of time scales, aging occurs precisely as predicted by the REM-like trap model of Bouchaud and Dean. This i… ▽ More Aging has become the paradigm to describe dynamical behavior of glassy systems, and in particular spin glasses. Trap models have been introduced as simple caricatures of effective dynamics of such systems. In this Letter we show that in a wide class of mean field models and on a wide range of time scales, aging occurs precisely as predicted by the REM-like trap model of Bouchaud and Dean. This is the first rigorous result about aging in mean field models except for the REM and the spherical model. △ Less

Submitted 12 December, 2007; originally announced December 2007.

Comments: 4 pages

Journal ref: J. Stat. Mech. (2008) L04003

Universality of the REM for dynamics of mean-field spin glasses

Authors: Gerard Ben Arous, Anton Bovier, Jiri Cerny

Abstract: We consider a version of a Glauber dynamics for a p-spin Sherrington--Kirkpatrick model of a spin glass that can be seen as a time change of simple random walk on the N-dimensional hypercube. We show that, for any p>2 and any inverse temperature β>0, there exist constants g>0, such that for all exponential time scales, $\exp(γN)$, with $γ< g$, the properly rescaled clock process (time-change pro… ▽ More We consider a version of a Glauber dynamics for a p-spin Sherrington--Kirkpatrick model of a spin glass that can be seen as a time change of simple random walk on the N-dimensional hypercube. We show that, for any p>2 and any inverse temperature β>0, there exist constants g>0, such that for all exponential time scales, $\exp(γN)$, with $γ< g$, the properly rescaled clock process (time-change process), converges to an α-stable subordinator where α=γ/β^2<1. Moreover, the dynamics exhibits aging at these time scales with time-time correlation function converging to the arcsine law of this α-stable subordinator. In other words, up to rescaling, on these time scales (that are shorter than the equilibration time of the system), the dynamics of p-spin models ages in the same way as the REM, and by extension Bouchaud's REM-like trap model, confirming the latter as a universal aging mechanism for a wide range of systems. The SK model (the case p=2) seems to belong to a different universality class. △ Less

Submitted 14 June, 2007; originally announced June 2007.

Comments: 30 pages, 1 fugre

MSC Class: 82C44; 60K35; 60G70

Cugliandolo-Kurchan equations for dynamics of Spin-Glasses

Authors: Gerard Ben Arous, Amir Dembo, Alice Guionnet

Abstract: We study the Langevin dynamics for the family of spherical $p$-spin disordered mean-field models and prove that in the limit of system size $N$ approaching infinity, the empirical state correlation and integrated response functions converge almost surely and uniformly in time, to the non-random unique strong solution of a pair of explicit non-linear integro-differential equations introduced by C… ▽ More We study the Langevin dynamics for the family of spherical $p$-spin disordered mean-field models and prove that in the limit of system size $N$ approaching infinity, the empirical state correlation and integrated response functions converge almost surely and uniformly in time, to the non-random unique strong solution of a pair of explicit non-linear integro-differential equations introduced by Cugliandolo and Kurchan. △ Less

Submitted 16 September, 2004; originally announced September 2004.

Comments: 31 pages

MSC Class: 82C44

Bouchaud's model exhibits two different aging regimes in dimension one

Authors: Gerard Ben Arous, Jiri Cerny

Abstract: Let E_i be a collection of i.i.d. exponential random variables. Bouchaud's model on Z is a Markov chain X(t) whose transition rates are given by w_{ij}=ν\exp(-β((1-a)E_i-aE_j)) if i, j are neighbors in Z. We study the behavior of two correlation functions: P[X(t_w+t)=X(t_w)] and P[X(t')=X(t_w) \forall t'\in[t_w,t_w+t]]. We prove the (sub)aging behavior of these functions when β>1 and a\in[0,1]. Let E_i be a collection of i.i.d. exponential random variables. Bouchaud's model on Z is a Markov chain X(t) whose transition rates are given by w_{ij}=ν\exp(-β((1-a)E_i-aE_j)) if i, j are neighbors in Z. We study the behavior of two correlation functions: P[X(t_w+t)=X(t_w)] and P[X(t')=X(t_w) \forall t'\in[t_w,t_w+t]]. We prove the (sub)aging behavior of these functions when β>1 and a\in[0,1]. △ Less

Submitted 20 May, 2005; v1 submitted 29 October, 2002; originally announced October 2002.

Comments: Published at http://dx.doi.org/10.1214/105051605000000124 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AAP-AAP078 MSC Class: 60K37; 82C44; 60G18 (Primary) 60F17. (Secondary)

Journal ref: Annals of Applied Probability 2005, Vol. 15, No. 2, 1161-1192

Aging in the random energy model

Authors: G. Ben Arous, A. Bovier, V. Gayrard

Abstract: In this letter we announce rigorous results on the phenomenon of aging in the Glauber dynamics of the random energy model and their relation to Bouchaud's 'REM-like' trap model. We show that, below the critical temperature, if we consider a time-scale that diverges with the system size in such a way that equilibrium is almost, but not quite reached on that scale, a suitably defined autocorrelati… ▽ More In this letter we announce rigorous results on the phenomenon of aging in the Glauber dynamics of the random energy model and their relation to Bouchaud's 'REM-like' trap model. We show that, below the critical temperature, if we consider a time-scale that diverges with the system size in such a way that equilibrium is almost, but not quite reached on that scale, a suitably defined autocorrelation function has the same asymptotic behaviour than its analog in the trap model. △ Less

Submitted 11 October, 2001; originally announced October 2001.

Comments: 4pp, PS