Showing 1–7 of 7 results for author: Wirtz, T

Characteristics of Monte Carlo Dropout in Wide Neural Networks

Authors: Joachim Sicking, Maram Akila, Tim Wirtz, Sebastian Houben, Asja Fischer

Abstract: Monte Carlo (MC) dropout is one of the state-of-the-art approaches for uncertainty estimation in neural networks (NNs). It has been interpreted as approximately performing Bayesian inference. Based on previous work on the approximation of Gaussian processes by wide and deep neural networks with random weights, we study the limiting distribution of wide untrained NNs under dropout more rigorously a… ▽ More Monte Carlo (MC) dropout is one of the state-of-the-art approaches for uncertainty estimation in neural networks (NNs). It has been interpreted as approximately performing Bayesian inference. Based on previous work on the approximation of Gaussian processes by wide and deep neural networks with random weights, we study the limiting distribution of wide untrained NNs under dropout more rigorously and prove that they as well converge to Gaussian processes for fixed sets of weights and biases. We sketch an argument that this property might also hold for infinitely wide feed-forward networks that are trained with (full-batch) gradient descent. The theory is contrasted by an empirical analysis in which we find correlations and non-Gaussian behaviour for the pre-activations of finite width NNs. We therefore investigate how (strongly) correlated pre-activations can induce non-Gaussian behavior in NNs with strongly correlated weights. △ Less

Submitted 10 July, 2020; originally announced July 2020.

Comments: Accepted at the ICML 2020 workshop for Uncertainty and Robustness in Deep Learning

The Correlated Jacobi and the Correlated Cauchy-Lorentz ensembles

Authors: Tim Wirtz, Daniel Waltner, Mario Kieburg, Santosh Kumar

Abstract: We calculate the $k$-point generating function of the correlated Jacobi ensemble using supersymmetric methods. We use the result for complex matrices for $k=1$ to derive a closed-form expression for eigenvalue density. For real matrices we obtain the density in terms of a twofold integral that we evaluate numerically. For both expressions we find agreement when comparing with Monte Carlo simulatio… ▽ More We calculate the $k$-point generating function of the correlated Jacobi ensemble using supersymmetric methods. We use the result for complex matrices for $k=1$ to derive a closed-form expression for eigenvalue density. For real matrices we obtain the density in terms of a twofold integral that we evaluate numerically. For both expressions we find agreement when comparing with Monte Carlo simulations. Relations between these quantities for the Jacobi and the Cauchy-Lorentz ensemble are derived. △ Less

Submitted 4 May, 2015; originally announced May 2015.

Comments: 29 pages, 3 figures

Journal ref: J. Stat Phys 162 (2016) 495

Asymptotic Coincidence of the Statistics for Degenerate and Non-Degenerate Correlated Real Wishart Ensembles

Authors: Tim Wirtz, Mario Kieburg, Thomas Guhr

Abstract: The correlated Wishart model provides the standard benchmark when analyzing time series of any kind. Unfortunately, the real case, which is the most relevant one in applications, poses serious challenges for analytical calculations. Often these challenges are due to square root singularities which cannot be handled using common random matrix techniques. We present a new way to tackle this issue. U… ▽ More The correlated Wishart model provides the standard benchmark when analyzing time series of any kind. Unfortunately, the real case, which is the most relevant one in applications, poses serious challenges for analytical calculations. Often these challenges are due to square root singularities which cannot be handled using common random matrix techniques. We present a new way to tackle this issue. Using supersymmetry, we carry out an anlaytical study which we support by numerical simulations. For large but finite matrix dimensions, we show that statistical properties of the fully correlated real Wishart model generically approach those of a correlated real Wishart model with doubled matrix dimensions and doubly degenerate empirical eigenvalues. This holds for the local and global spectral statistics. With Monte Carlo simulations we show that this is even approximately true for small matrix dimensions. We explicitly investigate the $k$-point correlation function as well as the distribution of the largest eigenvalue for which we find a surprisingly compact formula in the doubly degenerate case. Moreover we show that on the local scale the $k$-point correlation function exhibits the sine and the Airy kernel in the bulk and at the soft edges, respectively. We also address the positions and the fluctuations of the possible outliers in the data. △ Less

Submitted 26 April, 2017; v1 submitted 13 April, 2015; originally announced April 2015.

Comments: Drastically extended version compared to the first version. New chapters on the analysis of the local and global spectral statistics including approximations on the position and standard deviation of outliers. New numerical simulations comparing the macroscopic level density and the distributions of the largest and the smallest eigenvalue. 24 pages, 5 figures

MSC Class: 05.45.Tp; 02.50.-r; 02.20.-a

Journal ref: J. Phys. A 50, 235203 (2017)

Eigenvalue Density of the Doubly Correlated Wishart Model: Exact Results

Authors: Daniel Waltner, Tim Wirtz, Thomas Guhr

Abstract: Data sets collected at different times and different observing points can possess correlations at different times $and$ at different positions. The doubly correlated Wishart model takes both into account. We calculate the eigenvalue density of the Wishart correlation matrices using supersymmetry. In the complex case we obtain a new closed form expression which we compare to previous results in the… ▽ More Data sets collected at different times and different observing points can possess correlations at different times $and$ at different positions. The doubly correlated Wishart model takes both into account. We calculate the eigenvalue density of the Wishart correlation matrices using supersymmetry. In the complex case we obtain a new closed form expression which we compare to previous results in the literature. In the more relevant and much more complicated real case we derive an expression for the density in terms of a fourfold integral. Finally, we calculate the density in the limit of large correlation matrices. △ Less

Submitted 9 December, 2014; originally announced December 2014.

Comments: 16 pages, 3 figures

Journal ref: J. Phys. A: Math. Theor. 48 (2015) 175204

Limiting Statistics of the Largest and Smallest Eigenvalues in the Correlated Wishart Model

Authors: Tim Wirtz, Mario Kieburg, Thomas Guhr

Abstract: The correlated Wishart model provides a standard tool for the analysis of correlations in a rich variety of systems. Although much is known for complex correlation matrices, the empirically much more important real case still poses substantial challenges. We put forward a new approach, which maps arbitrary statistical quantities, depending on invariants only, to invariant Hermitian matrix models.… ▽ More The correlated Wishart model provides a standard tool for the analysis of correlations in a rich variety of systems. Although much is known for complex correlation matrices, the empirically much more important real case still poses substantial challenges. We put forward a new approach, which maps arbitrary statistical quantities, depending on invariants only, to invariant Hermitian matrix models. For completeness we also include the quaternion case and deal with all three cases in a unified way. As an important application, we study the statistics of the largest eigenvalue and its limiting distributions in the correlated Wishart model, because they help to estimate the behavior of large complex systems. We show that even for fully correlated Wishart ensembles, the Tracy-Widom distribution can be the limiting distribution of the largest as well as the smallest eigenvalue, provided that a certain scaling of the empirical eigenvalues holds. △ Less

Submitted 23 December, 2014; v1 submitted 17 October, 2014; originally announced October 2014.

Comments: 6 pages, 2 figures, minor corrections

Journal ref: Europhysics Letters 109, 20005 (2015)

Distribution of the Smallest Eigenvalue in Complex and Real Correlated Wishart Ensembles

Authors: Tim Wirtz, Thomas Guhr

Abstract: For the correlated Gaussian Wishart ensemble we compute the distribution of the smallest eigenvalue and a related gap probability.We obtain exact results for the complex (β=2) and for the real case (β=1). For a particular set of empirical correlation matrices we find universality in the spectral density, for both real and complex ensembles and all kinds of rectangularity. We calculate the asymptot… ▽ More For the correlated Gaussian Wishart ensemble we compute the distribution of the smallest eigenvalue and a related gap probability.We obtain exact results for the complex (β=2) and for the real case (β=1). For a particular set of empirical correlation matrices we find universality in the spectral density, for both real and complex ensembles and all kinds of rectangularity. We calculate the asymptotic and universal results for the gap probability and the distribution of the smallest eigenvalue. We use the Supersymmetry method, in particular the generalized Hubbard-Stratonovich transformation and superbosonization. △ Less

Submitted 11 April, 2014; v1 submitted 9 October, 2013; originally announced October 2013.

Comments: 30 pages, 2 figures, improving pictures, correcting typos

Journal ref: J. Phys. A: Math. Theor. 47 (2014) 075004

Distribution of the Smallest Eigenvalue in the Correlated Wishart Model

Authors: Tim Wirtz, Thomas Guhr

Abstract: Wishart random matrix theory is of major importance for the analysis of correlated time series. The distribution of the smallest eigenvalue for Wishart correlation matrices is particularly interesting in many applications. In the complex and in the real case, we calculate it exactly for arbitrary empirical eigenvalues, i.e., for fully correlated Gaussian Wishart ensembles. To this end, we derive c… ▽ More Wishart random matrix theory is of major importance for the analysis of correlated time series. The distribution of the smallest eigenvalue for Wishart correlation matrices is particularly interesting in many applications. In the complex and in the real case, we calculate it exactly for arbitrary empirical eigenvalues, i.e., for fully correlated Gaussian Wishart ensembles. To this end, we derive certain dualities of matrix models in ordinary space. We thereby completely avoid the otherwise unsurmountable problem of computing a highly non-trivial group integral. Our results are compact and much easier to handle than previous ones. Furthermore, we obtain a new universality for the distribution of the smallest eigenvalue on the proper local scale. △ Less

Submitted 18 October, 2013; v1 submitted 20 June, 2013; originally announced June 2013.

Comments: 5 pages, 1 figure

Journal ref: Phys. Rev. Lett. 111, 094101 (2013)