-
DNN Approximation of Nonlinear Finite Element Equations
Authors:
Tuyen Tran,
Aidan Hamilton,
Maricela Best McKay,
Benjamin Quiring,
Panayot S. Vassilevski
Abstract:
We investigate the potential of applying (D)NN ((deep) neural networks) for approximating nonlinear mappings arising in the finite element discretization of nonlinear PDEs (partial differential equations). As an application, we apply the trained DNN to replace the coarse nonlinear operator thus avoiding the need to visit the fine level discretization in order to evaluate the actions of the true co…
▽ More
We investigate the potential of applying (D)NN ((deep) neural networks) for approximating nonlinear mappings arising in the finite element discretization of nonlinear PDEs (partial differential equations). As an application, we apply the trained DNN to replace the coarse nonlinear operator thus avoiding the need to visit the fine level discretization in order to evaluate the actions of the true coarse nonlinear operator. The feasibility of the studied approach is demonstrated in a two-level FAS (full approximation scheme) used to solve a nonlinear diffusion-reaction PDE.
△ Less
Submitted 12 November, 2019;
originally announced November 2019.
-
Asymptotics of eigenstructure of sample correlation matrices for high-dimensional spiked models
Authors:
David Morales-Jimenez,
Iain M. Johnstone,
Matthew R. McKay,
Jeha Yang
Abstract:
Sample correlation matrices are employed ubiquitously in statistics. However, quite surprisingly, little is known about their asymptotic spectral properties for high-dimensional data, particularly beyond the case of "null models" for which the data is assumed independent. Here, considering the popular class of spiked models, we apply random matrix theory to derive asymptotic first-order and distri…
▽ More
Sample correlation matrices are employed ubiquitously in statistics. However, quite surprisingly, little is known about their asymptotic spectral properties for high-dimensional data, particularly beyond the case of "null models" for which the data is assumed independent. Here, considering the popular class of spiked models, we apply random matrix theory to derive asymptotic first-order and distributional results for both the leading eigenvalues and eigenvectors of sample correlation matrices. These results are obtained under high-dimensional settings for which the number of samples n and variables p approach infinity, with p/n tending to a constant. To first order, the spectral properties of sample correlation matrices are seen to coincide with those of sample covariance matrices; however their asymptotic distributions can differ significantly, with fluctuations of both the sample eigenvalues and eigenvectors often being remarkably smaller than those of their sample covariance counterparts.
△ Less
Submitted 12 March, 2019; v1 submitted 24 October, 2018;
originally announced October 2018.
-
Large-dimensional behavior of regularized Maronna's M-estimators of covariance matrices
Authors:
Nicolas Auguin,
David Morales-Jimenez,
Matthew R. McKay,
Romain Couillet
Abstract:
Robust estimators of large covariance matrices are considered, comprising regularized (linear shrinkage) modifications of Maronna's classical M-estimators. These estimators provide robustness to outliers, while simultaneously being well-defined when the number of samples does not exceed the number of variables. By applying tools from random matrix theory, we characterize the asymptotic performance…
▽ More
Robust estimators of large covariance matrices are considered, comprising regularized (linear shrinkage) modifications of Maronna's classical M-estimators. These estimators provide robustness to outliers, while simultaneously being well-defined when the number of samples does not exceed the number of variables. By applying tools from random matrix theory, we characterize the asymptotic performance of such estimators when the numbers of samples and variables grow large together. In particular, our results show that, when outliers are absent, many estimators of the regularized-Maronna type share the same asymptotic performance, and for these estimators we present a data-driven method for choosing the asymptotically optimal regularization parameter with respect to a quadratic loss. Robustness in the presence of outliers is then studied: in the non-regularized case, a large-dimensional robustness metric is proposed, and explicitly computed for two particular types of estimators, exhibiting interesting differences depending on the underlying contamination model. The impact of outliers in regularized estimators is then studied, with interesting differences with respect to the non-regularized case, leading to new practical insights on the choice of particular estimators.
△ Less
Submitted 26 April, 2018;
originally announced April 2018.
-
Exact Statistical Characterization of $2\times2$ Gram Matrices with Arbitrary Variance Profile
Authors:
Nicolas Auguin,
David Morales-Jimenez,
Matthew McKay
Abstract:
This paper is concerned with the statistical properties of the Gram matrix $\mathbf{W}=\mathbf{H}\mathbf{H}^\dagger$, where $\mathbf{H}$ is a $2\times2$ complex central Gaussian matrix whose elements have arbitrary variances. With such arbitrary variance profile, this random matrix model fundamentally departs from classical Wishart models and presents a significant challenge as the classical analy…
▽ More
This paper is concerned with the statistical properties of the Gram matrix $\mathbf{W}=\mathbf{H}\mathbf{H}^\dagger$, where $\mathbf{H}$ is a $2\times2$ complex central Gaussian matrix whose elements have arbitrary variances. With such arbitrary variance profile, this random matrix model fundamentally departs from classical Wishart models and presents a significant challenge as the classical analytical toolbox no longer directly applies. We derive new exact expressions for the distribution of $\mathbf{W}$ and that of its eigenvalues by means of an explicit parameterization of the group of unitary matrices. Our results yield remarkably simple expressions, which are further leveraged to study the outage data rate of a dual-antenna communication system under different variance profiles.
△ Less
Submitted 12 April, 2017;
originally announced May 2017.
-
Large Dimensional Analysis of Robust M-Estimators of Covariance with Outliers
Authors:
David Morales-Jimenez,
Romain Couillet,
Matthew R. McKay
Abstract:
A large dimensional characterization of robust M-estimators of covariance (or scatter) is provided under the assumption that the dataset comprises independent (essentially Gaussian) legitimate samples as well as arbitrary deterministic samples, referred to as outliers. Building upon recent random matrix advances in the area of robust statistics, we specifically show that the so-called Maronna M-es…
▽ More
A large dimensional characterization of robust M-estimators of covariance (or scatter) is provided under the assumption that the dataset comprises independent (essentially Gaussian) legitimate samples as well as arbitrary deterministic samples, referred to as outliers. Building upon recent random matrix advances in the area of robust statistics, we specifically show that the so-called Maronna M-estimator of scatter asymptotically behaves similar to well-known random matrices when the population and sample sizes grow together to infinity. The introduction of outliers leads the robust estimator to behave asymptotically as the weighted sum of the sample outer products, with a constant weight for all legitimate samples and different weights for the outliers. A fine analysis of this structure reveals importantly that the propensity of the M-estimator to attenuate (or enhance) the impact of outliers is mostly dictated by the alignment of the outliers with the inverse population covariance matrix of the legitimate samples. Thus, robust M-estimators can bring substantial benefits over more simplistic estimators such as the per-sample normalized version of the sample covariance matrix, which is not capable of differentiating the outlying samples. The analysis shows that, within the class of Maronna's estimators of scatter, the Huber estimator is most favorable for rejecting outliers. On the contrary, estimators more similar to Tyler's scale invariant estimator (often preferred in the literature) run the risk of inadvertently enhancing some outliers.
△ Less
Submitted 4 March, 2015;
originally announced March 2015.
-
Hypergeometric Functions of Matrix Arguments and Linear Statistics of Multi-Spiked Hermitian Matrix Models
Authors:
Damien Passemier,
Matthew R. Mckay,
Yang Chen
Abstract:
This paper derives central limit theorems (CLTs) for general linear spectral statistics (LSS) of three important multi-spiked Hermitian random matrix ensembles. The first is the most common spiked scenario, proposed by Johnstone, which is a central Wishart ensemble with fixed-rank perturbation of the identity matrix, the second is a non-central Wishart ensemble with fixed-rank noncentrality parame…
▽ More
This paper derives central limit theorems (CLTs) for general linear spectral statistics (LSS) of three important multi-spiked Hermitian random matrix ensembles. The first is the most common spiked scenario, proposed by Johnstone, which is a central Wishart ensemble with fixed-rank perturbation of the identity matrix, the second is a non-central Wishart ensemble with fixed-rank noncentrality parameter, and the third is a similarly defined non-central $F$ ensemble. These CLT results generalize our recent work to account for multiple spikes, which is the most common scenario met in practice. The generalization is non-trivial, as it now requires dealing with hypergeometric functions of matrix arguments. To facilitate our analysis, for a broad class of such functions, we first generalize a recent result of Onatski to present new contour integral representations, which are particularly suitable for computing large-dimensional properties of spiked matrix ensembles. Armed with such representations, our CLT formulas are derived for each of the three spiked models of interest by employing the Coulomb fluid method from random matrix theory along with saddlepoint techniques. We find that for each matrix model, and for general LSS, the individual spikes contribute additively to yield a $O(1)$ correction term to the asymptotic mean of the linear statistic, which we specify explicitly, whilst having no effect on the leading order terms of the mean or variance.
△ Less
Submitted 4 June, 2014; v1 submitted 3 June, 2014;
originally announced June 2014.
-
Analysis and Design of Multiple-Antenna Cognitive Radios with Multiple Primary User Signals
Authors:
David Morales-Jimenez,
Raymond H. Y. Louie,
Matthew R. McKay,
Yang Chen
Abstract:
We consider multiple-antenna signal detection of primary user transmission signals by a secondary user receiver in cognitive radio networks. The optimal detector is analyzed for the scenario where the number of primary user signals is no less than the number of receive antennas at the secondary user. We first derive exact expressions for the moments of the generalized likelihood ratio test (GLRT)…
▽ More
We consider multiple-antenna signal detection of primary user transmission signals by a secondary user receiver in cognitive radio networks. The optimal detector is analyzed for the scenario where the number of primary user signals is no less than the number of receive antennas at the secondary user. We first derive exact expressions for the moments of the generalized likelihood ratio test (GLRT) statistic, yielding approximations for the false alarm and detection probabilities. We then show that the normalized GLRT statistic converges in distribution to a Gaussian random variable when the number of antennas and observations grow large at the same rate. Further, using results from large random matrix theory, we derive expressions to compute the detection probability without explicit knowledge of the channel, and then particularize these expressions for two scenarios of practical interest: 1) a single primary user sending spatially multiplexed signals, and 2) multiple spatially distributed primary users. Our analytical results are finally used to obtain simple design rules for the signal detection threshold.
△ Less
Submitted 23 April, 2015; v1 submitted 25 May, 2014;
originally announced May 2014.
-
Asymptotic Linear Spectral Statistics for Spiked Hermitian Random Matrix Models
Authors:
Damien Passemier,
Matthew R. Mckay,
Yang Chen
Abstract:
Using the Coulomb Fluid method, this paper derives central limit theorems (CLTs) for linear spectral statistics of three "spiked" Hermitian random matrix ensembles. These include Johnstone's spiked model (i.e., central Wishart with spiked correlation), non-central Wishart with rank-one non-centrality, and a related class of non-central $F$ matrices. For a generic linear statistic, we derive simple…
▽ More
Using the Coulomb Fluid method, this paper derives central limit theorems (CLTs) for linear spectral statistics of three "spiked" Hermitian random matrix ensembles. These include Johnstone's spiked model (i.e., central Wishart with spiked correlation), non-central Wishart with rank-one non-centrality, and a related class of non-central $F$ matrices. For a generic linear statistic, we derive simple and explicit CLT expressions as the matrix dimensions grow large. For all three ensembles under consideration, we find that the primary effect of the spike is to introduce an $O(1)$ correction term to the asymptotic mean of the linear spectral statistic, which we characterize with simple formulas. The utility of our proposed framework is demonstrated through application to three different linear statistics problems: the classical likelihood ratio test for a population covariance, the capacity analysis of multi-antenna wireless communication systems with a line-of-sight transmission path, and a classical multiple sample significance testing problem.
△ Less
Submitted 26 February, 2014;
originally announced February 2014.
-
Large Dimensional Analysis and Optimization of Robust Shrinkage Covariance Matrix Estimators
Authors:
Romain Couillet,
Matthew R. McKay
Abstract:
This article studies two regularized robust estimators of scatter matrices proposed (and proved to be well defined) in parallel in (Chen et al., 2011) and (Pascal et al., 2013), based on Tyler's robust M-estimator (Tyler, 1987) and on Ledoit and Wolf's shrinkage covariance matrix estimator (Ledoit and Wolf, 2004). These hybrid estimators have the advantage of conveying (i) robustness to outliers o…
▽ More
This article studies two regularized robust estimators of scatter matrices proposed (and proved to be well defined) in parallel in (Chen et al., 2011) and (Pascal et al., 2013), based on Tyler's robust M-estimator (Tyler, 1987) and on Ledoit and Wolf's shrinkage covariance matrix estimator (Ledoit and Wolf, 2004). These hybrid estimators have the advantage of conveying (i) robustness to outliers or impulsive samples and (ii) small sample size adequacy to the classical sample covariance matrix estimator. We consider here the case of i.i.d. elliptical zero mean samples in the regime where both sample and population sizes are large. We demonstrate that, under this setting, the estimators under study asymptotically behave similar to well-understood random matrix models. This characterization allows us to derive optimal shrinkage strategies to estimate the population scatter matrix, improving significantly upon the empirical shrinkage method proposed in (Chen et al., 2011).
△ Less
Submitted 18 January, 2015; v1 submitted 16 January, 2014;
originally announced January 2014.
-
Distributions of Demmel and Related Condition Numbers
Authors:
Prathapasinghe Dharmawansa,
Matthew McKay,
Yang Chen
Abstract:
Consider a random matrix $\mathbf{A}\in\mathbb{C}^{m\times n}$ ($m \geq n$) containing independent complex Gaussian entries with zero mean and unit variance, and let $0<λ_1\leq λ_{2}\leq ...\leq λ_n<\infty$ denote the eigenvalues of $\mathbf{A}^{*}\mathbf{A}$ where $(\cdot)^*$ represents conjugate-transpose. This paper investigates the distribution of the random variables…
▽ More
Consider a random matrix $\mathbf{A}\in\mathbb{C}^{m\times n}$ ($m \geq n$) containing independent complex Gaussian entries with zero mean and unit variance, and let $0<λ_1\leq λ_{2}\leq ...\leq λ_n<\infty$ denote the eigenvalues of $\mathbf{A}^{*}\mathbf{A}$ where $(\cdot)^*$ represents conjugate-transpose. This paper investigates the distribution of the random variables $\frac{\sum_{j=1}^n λ_j}{λ_k}$, for $k = 1$ and $k = 2$. These two variables are related to certain condition number metrics, including the so-called Demmel condition number, which have been shown to arise in a variety of applications. For both cases, we derive new exact expressions for the probability densities, and establish the asymptotic behavior as the matrix dimensions grow large. In particular, it is shown that as $n$ and $m$ tend to infinity with their difference fixed, both densities scale on the order of $n^3$. After suitable transformations, we establish exact expressions for the asymptotic densities, obtaining simple closed-form expressions in some cases. Our results generalize the work of Edelman on the Demmel condition number for the case $m = n$.
△ Less
Submitted 2 November, 2012;
originally announced November 2012.
-
Extreme Eigenvalue Distributions of Some Complex Correlated Non-Central Wishart and Gamma-Wishart Random Matrices
Authors:
Prathapasinghe Dharmawansa,
Matthew R. McKay
Abstract:
Let $\mathbf{W}$ be a correlated complex non-central Wishart matrix defined through $\mathbf{W}=\mathbf{X}^H\mathbf{X}$, where $\mathbf{X}$ is $n\times m \, (n\geq m)$ complex Gaussian with non-zero mean $\boldsymbolΥ$ and non-trivial covariance $\boldsymbolΣ$. We derive exact expressions for the cumulative distribution functions (c.d.f.s) of the extreme eigenvalues (i.e., maximum and minimum) of…
▽ More
Let $\mathbf{W}$ be a correlated complex non-central Wishart matrix defined through $\mathbf{W}=\mathbf{X}^H\mathbf{X}$, where $\mathbf{X}$ is $n\times m \, (n\geq m)$ complex Gaussian with non-zero mean $\boldsymbolΥ$ and non-trivial covariance $\boldsymbolΣ$. We derive exact expressions for the cumulative distribution functions (c.d.f.s) of the extreme eigenvalues (i.e., maximum and minimum) of $\mathbf{W}$ for some particular cases. These results are quite simple, involving rapidly converging infinite series, and apply for the practically important case where $\boldsymbolΥ$ has rank one. We also derive analogous results for a certain class of gamma-Wishart random matrices, for which $\boldsymbolΥ^H\boldsymbolΥ$ follows a matrix-variate gamma distribution. The eigenvalue distributions in this paper have various applications to wireless communication systems, and arise in other fields such as econometrics, statistical physics, and multivariate statistics.
△ Less
Submitted 5 January, 2011;
originally announced January 2011.
