Showing 1–7 of 7 results for author: Marinucci, D

Fractal and Regular Geometry of Deep Neural Networks

Authors: Simmaco Di Lillo, Domenico Marinucci, Michele Salvi, Stefano Vigogna

Abstract: We study the geometric properties of random neural networks by investigating the boundary volumes of their excursion sets for different activation functions, as the depth increases. More specifically, we show that, for activations which are not very regular (e.g., the Heaviside step function), the boundary volumes exhibit fractal behavior, with their Hausdorff dimension monotonically increasing wi… ▽ More We study the geometric properties of random neural networks by investigating the boundary volumes of their excursion sets for different activation functions, as the depth increases. More specifically, we show that, for activations which are not very regular (e.g., the Heaviside step function), the boundary volumes exhibit fractal behavior, with their Hausdorff dimension monotonically increasing with the depth. On the other hand, for activations which are more regular (e.g., ReLU, logistic and $\tanh$), as the depth increases, the expected boundary volumes can either converge to zero, remain constant or diverge exponentially, depending on a single spectral parameter which can be easily computed. Our theoretical results are confirmed in some numerical experiments based on Monte Carlo simulations. △ Less

Submitted 8 April, 2025; originally announced April 2025.

MSC Class: 60G60; 62B10; 62M45; 68T07

Spectral complexity of deep neural networks

Authors: Simmaco Di Lillo, Domenico Marinucci, Michele Salvi, Stefano Vigogna

Abstract: It is well-known that randomly initialized, push-forward, fully-connected neural networks weakly converge to isotropic Gaussian processes, in the limit where the width of all layers goes to infinity. In this paper, we propose to use the angular power spectrum of the limiting field to characterize the complexity of the network architecture. In particular, we define sequences of random variables ass… ▽ More It is well-known that randomly initialized, push-forward, fully-connected neural networks weakly converge to isotropic Gaussian processes, in the limit where the width of all layers goes to infinity. In this paper, we propose to use the angular power spectrum of the limiting field to characterize the complexity of the network architecture. In particular, we define sequences of random variables associated with the angular power spectrum, and provide a full characterization of the network complexity in terms of the asymptotic distribution of these sequences as the depth diverges. On this basis, we classify neural networks as low-disorder, sparse, or high-disorder; we show how this classification highlights a number of distinct features for standard activation functions, and in particular, sparsity properties of ReLU networks. Our theoretical results are also validated by numerical simulations. △ Less

Submitted 19 May, 2025; v1 submitted 15 May, 2024; originally announced May 2024.

MSC Class: 68T07; 60G60; 33C55; 62M15

Quantitative CLTs in Deep Neural Networks

Authors: Stefano Favaro, Boris Hanin, Domenico Marinucci, Ivan Nourdin, Giovanni Peccati

Abstract: We study the distribution of a fully connected neural network with random Gaussian weights and biases in which the hidden layer widths are proportional to a large constant $n$. Under mild assumptions on the non-linearity, we obtain quantitative bounds on normal approximations valid at large but finite $n$ and any fixed network depth. Our theorems show both for the finite-dimensional distributions… ▽ More We study the distribution of a fully connected neural network with random Gaussian weights and biases in which the hidden layer widths are proportional to a large constant $n$. Under mild assumptions on the non-linearity, we obtain quantitative bounds on normal approximations valid at large but finite $n$ and any fixed network depth. Our theorems show both for the finite-dimensional distributions and the entire process, that the distance between a random fully connected network (and its derivatives) to the corresponding infinite width Gaussian process scales like $n^{-γ}$ for $γ>0$, with the exponent depending on the metric used to measure discrepancy. Our bounds are strictly stronger in terms of their dependence on network width than any previously available in the literature; in the one-dimensional case, we also prove that they are optimal, i.e., we establish matching lower bounds. △ Less

Submitted 17 June, 2024; v1 submitted 12 July, 2023; originally announced July 2023.

A Quantitative Functional Central Limit Theorem for Shallow Neural Networks

Authors: Valentina Cammarota, Domenico Marinucci, Michele Salvi, Stefano Vigogna

Abstract: We prove a Quantitative Functional Central Limit Theorem for one-hidden-layer neural networks with generic activation function. The rates of convergence that we establish depend heavily on the smoothness of the activation function, and they range from logarithmic in non-differentiable cases such as the Relu to $\sqrt{n}$ for very regular activations. Our main tools are functional versions of the S… ▽ More We prove a Quantitative Functional Central Limit Theorem for one-hidden-layer neural networks with generic activation function. The rates of convergence that we establish depend heavily on the smoothness of the activation function, and they range from logarithmic in non-differentiable cases such as the Relu to $\sqrt{n}$ for very regular activations. Our main tools are functional versions of the Stein-Malliavin approach; in particular, we exploit heavily a quantitative functional central limit theorem which has been recently established by Bourguin and Campese (2020). △ Less

Submitted 5 July, 2023; v1 submitted 29 June, 2023; originally announced June 2023.

MSC Class: 60F17; 68T07; 60G60

Multi-Scale CUSUM Tests for Time Dependent Spherical Random Fields

Authors: Alessia Caponera, Domenico Marinucci, Anna Vidotto

Abstract: This paper investigates the asymptotic behavior of structural break tests in the harmonic domain for time dependent spherical random fields. In particular, we prove a functional central limit theorem result for the fluctuations over time of the sample spherical harmonic coefficients, under the null of isotropy and stationarity; furthermore, we prove consistency of the corresponding CUSUM test, und… ▽ More This paper investigates the asymptotic behavior of structural break tests in the harmonic domain for time dependent spherical random fields. In particular, we prove a functional central limit theorem result for the fluctuations over time of the sample spherical harmonic coefficients, under the null of isotropy and stationarity; furthermore, we prove consistency of the corresponding CUSUM test, under a broad range of alternatives, including deterministic trend, abrupt change, and a nontrivial power alternative. Our results are then applied to NCEP data on global temperature: our estimates suggest that Climate Change does not simply affect global average temperatures, but also the nature of spatial fluctuations at different scales. △ Less

Submitted 30 July, 2024; v1 submitted 2 May, 2023; originally announced May 2023.

Statistical challenges in the analysis of Cosmic Microwave Background radiation

Authors: Paolo Cabella, Domenico Marinucci

Abstract: An enormous amount of observations on Cosmic Microwave Background radiation has been collected in the last decade, and much more data are expected in the near future from planned or operating satellite missions. These datasets are a goldmine of information for Cosmology and Theoretical Physics; their efficient exploitation posits several intriguing challenges from the statistical point of view.… ▽ More An enormous amount of observations on Cosmic Microwave Background radiation has been collected in the last decade, and much more data are expected in the near future from planned or operating satellite missions. These datasets are a goldmine of information for Cosmology and Theoretical Physics; their efficient exploitation posits several intriguing challenges from the statistical point of view. In this paper we review a number of open problems in CMB data analysis and we present applications to observations from the WMAP mission. △ Less

Submitted 15 May, 2009; v1 submitted 11 July, 2008; originally announced July 2008.

Comments: Published in at http://dx.doi.org/10.1214/08-AOAS190 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS190

Journal ref: Annals of Applied Statistics 2009, Vol. 3, No. 1, 61-95

On The Dependence Structure of Wavelet Coefficients for Spherical Random Fields

Authors: Xiaohong Lan, Domenico Marinucci

Abstract: We consider the correlation structure of the random coefficients for a wide class of wavelet systems on the sphere (Mexican needlets) which were recently introduced in the literature by Geller and Mayeli (2007). We provide necessary and sufficient conditions for these coefficients to be asymptotic uncorrelated in the real and in the frequency domain. Here, the asymptotic theory is developed in t… ▽ More We consider the correlation structure of the random coefficients for a wide class of wavelet systems on the sphere (Mexican needlets) which were recently introduced in the literature by Geller and Mayeli (2007). We provide necessary and sufficient conditions for these coefficients to be asymptotic uncorrelated in the real and in the frequency domain. Here, the asymptotic theory is developed in the high resolution sense. Statistical applications are also discussed, in particular with reference to the analysis of cosmological data. △ Less

Submitted 24 April, 2009; v1 submitted 27 May, 2008; originally announced May 2008.

Comments: Revised version for Stochastic Processes and their Applications

MSC Class: 60G60 (Primary); 62M40; 42C40; 42C10 (Secondary)

Journal ref: Stochastic Processes and their Applications, Vol. 119 (2009), pp. 3749-3766