Showing 1–2 of 2 results for author: Granziol, D

Universal characteristics of deep neural network loss surfaces from random matrix theory

Authors: Nicholas P Baskerville, Jonathan P Keating, Francesco Mezzadri, Joseph Najnudel, Diego Granziol

Abstract: This paper considers several aspects of random matrix universality in deep neural networks. Motivated by recent experimental work, we use universal properties of random matrices related to local statistics to derive practical implications for deep neural networks based on a realistic model of their Hessians. In particular we derive universal aspects of outliers in the spectra of deep neural networ… ▽ More This paper considers several aspects of random matrix universality in deep neural networks. Motivated by recent experimental work, we use universal properties of random matrices related to local statistics to derive practical implications for deep neural networks based on a realistic model of their Hessians. In particular we derive universal aspects of outliers in the spectra of deep neural networks and demonstrate the important role of random matrix local laws in popular pre-conditioning gradient descent algorithms. We also present insights into deep neural network loss surfaces from quite general arguments based on tools from statistical physics and random matrix theory. △ Less

Submitted 20 June, 2022; v1 submitted 17 May, 2022; originally announced May 2022.

Comments: 42 pages

Journal ref: J. Phys A: Math. Theor. 55(49) 494002 (2022)

An information and field theoretic approach to the grand canonical ensemble

Authors: Diego Granziol, Stephen Roberts

Abstract: We present a novel derivation of the constraints required to obtain the underlying principles of statistical mechanics using a maximum entropy framework. We derive the mean value constraints by use of the central limit theorem and the scaling properties of Lagrange multipliers. We then arrive at the same result using a quantum free field theory and the Ward identities. The work provides a principl… ▽ More We present a novel derivation of the constraints required to obtain the underlying principles of statistical mechanics using a maximum entropy framework. We derive the mean value constraints by use of the central limit theorem and the scaling properties of Lagrange multipliers. We then arrive at the same result using a quantum free field theory and the Ward identities. The work provides a principled footing for maximum entropy methods in statistical physics, adding the body of work aligned to Jaynes's vision of statistical mechanics as a form of inference rather than a physical theory dependent on ergodicity, metric transitivity and equal a priori probabilities. We show that statistical independence, in the macroscopic limit, is the unifying concept that leads to all these derivations. △ Less

Submitted 29 March, 2017; originally announced March 2017.

Comments: 7 pages, 3 pages of Appendix