-
Majorization-type cluster robust bounds for block filters and eigensolvers
Authors:
M. Zhou,
M. E. Argentati,
A. V. Knyazev,
K. Neymeyr
Abstract:
Convergence analysis of block iterative solvers for Hermitian eigenvalue problems and the closely related research on properties of matrix-based signal filters are challenging, and attract increasing attention due to their recent applications in spectral data clustering and graph-based signal processing. We combine majorization-based techniques pioneered for investigating the Rayleigh-Ritz method…
▽ More
Convergence analysis of block iterative solvers for Hermitian eigenvalue problems and the closely related research on properties of matrix-based signal filters are challenging, and attract increasing attention due to their recent applications in spectral data clustering and graph-based signal processing. We combine majorization-based techniques pioneered for investigating the Rayleigh-Ritz method in [SIAM J. Matrix Anal. Appl., 31 (2010), pp. 1521-1537] with tools of classical analysis of the block power method by Rutishauser [Numer. Math., 13 (1969), pp. 4-13] to derive convergence rate bounds of an abstract block iteration, wherein tuples of tangents of principal angles or relative errors of Ritz values are bounded using majorization in terms of arranged partial sums and tuples of convergence factors. Our novel bounds are robust in presence of clusters of eigenvalues, improve some previous results, and are applicable to most known block iterative solvers and matrix-based filters, e.g., to block power, Chebyshev, and Lanczos methods combined with shift-and-invert approaches and polynomial filtering.
△ Less
Submitted 12 January, 2022;
originally announced January 2022.
-
Convergence theory for preconditioned eigenvalue solvers in a nutshell
Authors:
Merico E. Argentati,
Andrew V. Knyazev,
Klaus Neymeyr,
Evgueni E. Ovtchinnikov,
Ming Zhou
Abstract:
Preconditioned iterative methods for numerical solution of large matrix eigenvalue problems are increasingly gaining importance in various application areas, ranging from material sciences to data mining. Some of them, e.g., those using multilevel preconditioning for elliptic differential operators or graph Laplacian eigenvalue problems, exhibit almost optimal complexity in practice, i.e., their c…
▽ More
Preconditioned iterative methods for numerical solution of large matrix eigenvalue problems are increasingly gaining importance in various application areas, ranging from material sciences to data mining. Some of them, e.g., those using multilevel preconditioning for elliptic differential operators or graph Laplacian eigenvalue problems, exhibit almost optimal complexity in practice, i.e., their computational costs to calculate a fixed number of eigenvalues and eigenvectors grow linearly with the matrix problem size. Theoretical justification of their optimality requires convergence rate bounds that do not deteriorate with the increase of the problem size. Such bounds were pioneered by E. D'yakonov over three decades ago, but to date only a handful have been derived, mostly for symmetric eigenvalue problems. Just a few of known bounds are sharp. One of them is proved in [doi:10.1016/S0024-3795(01)00461-X] for the simplest preconditioned eigensolver with a fixed step size. The original proof has been greatly simplified and shortened in [doi:10.1137/080727567] by using a gradient flow integration approach. In the present work, we give an even more succinct proof, using novel ideas based on Karush-Kuhn-Tucker theory and nonlinear programming.
△ Less
Submitted 6 November, 2015; v1 submitted 16 December, 2014;
originally announced December 2014.
-
Bounds for the Rayleigh quotient and the spectrum of self-adjoint operators
Authors:
Peizhen Zhu,
Merico E. Argentati,
Andrew V. Knyazev
Abstract:
The absolute change in the Rayleigh quotient (RQ) is bounded in this paper in terms of the norm of the residual and the change in the vector. If $x$ is an eigenvector of a self-adjoint bounded operator $A$ in a Hilbert space, then the RQ of the vector $x$, denoted by $ρ(x)$, is an exact eigenvalue of $A$. In this case, the absolute change of the RQ $|ρ(x)-ρ(y)|$ becomes the absolute error in an ei…
▽ More
The absolute change in the Rayleigh quotient (RQ) is bounded in this paper in terms of the norm of the residual and the change in the vector. If $x$ is an eigenvector of a self-adjoint bounded operator $A$ in a Hilbert space, then the RQ of the vector $x$, denoted by $ρ(x)$, is an exact eigenvalue of $A$. In this case, the absolute change of the RQ $|ρ(x)-ρ(y)|$ becomes the absolute error in an eigenvalue $ρ(x)$ of $A$ approximated by the RQ $ρ(y)$ on a given vector $y.$ There are three traditional kinds of bounds of the eigenvalue error: a priori bounds via the angle between vectors $x$ and $y$; a posteriori bounds via the norm of the residual $Ay-ρ(y)y$ of vector $y$; mixed type bounds using both the angle and the norm of the residual. We propose a unifying approach to prove known bounds of the spectrum, analyze their sharpness, and derive new sharper bounds. The proof approach is based on novel RQ vector perturbation identities.
△ Less
Submitted 29 December, 2012; v1 submitted 13 July, 2012;
originally announced July 2012.
-
Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) in hypre and PETSc
Authors:
A. V. Knyazev,
M. E. Argentati,
I. Lashuk,
E. E. Ovtchinnikov
Abstract:
We describe our software package Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) publicly released recently. BLOPEX is available as a stand-alone serial library, as an external package to PETSc (``Portable, Extensible Toolkit for Scientific Computation'', a general purpose suite of tools for the scalable solution of partial differential equations and related problems developed b…
▽ More
We describe our software package Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) publicly released recently. BLOPEX is available as a stand-alone serial library, as an external package to PETSc (``Portable, Extensible Toolkit for Scientific Computation'', a general purpose suite of tools for the scalable solution of partial differential equations and related problems developed by Argonne National Laboratory), and is also built into {\it hypre} (``High Performance Preconditioners'', scalable linear solvers package developed by Lawrence Livermore National Laboratory). The present BLOPEX release includes only one solver--the Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) method for symmetric eigenvalue problems. {\it hypre} provides users with advanced high-quality parallel preconditioners for linear systems, in particular, with domain decomposition and multigrid preconditioners. With BLOPEX, the same preconditioners can now be efficiently used for symmetric eigenvalue problems. PETSc facilitates the integration of independently developed application modules with strict attention to component interoperability, and makes BLOPEX extremely easy to compile and use with preconditioners that are available via PETSc. We present the LOBPCG algorithm in BLOPEX for {\it hypre} and PETSc. We demonstrate numerically the scalability of BLOPEX by testing it on a number of distributed and shared memory parallel systems, including a Beowulf system, SUN Fire 880, an AMD dual-core Opteron workstation, and IBM BlueGene/L supercomputer, using PETSc domain decomposition and {\it hypre} multigrid preconditioning. We test BLOPEX on a model problem, the standard 7-point finite-difference approximation of the 3-D Laplacian, with the problem size in the range $10^5-10^8$.
△ Less
Submitted 17 May, 2007;
originally announced May 2007.
-
Angles Between Infinite Dimensional Subspaces with Applications to the Rayleigh-Ritz and Alternating Projectors Methods
Authors:
Andrew Knyazev,
Abram Jujunashvili,
Merico Argentati
Abstract:
We define angles from-to and between infinite dimensional subspaces of a Hilbert space, inspired by the work of E. J. Hannan, 1961/1962 for general canonical correlations of stochastic processes. The spectral theory of selfadjoint operators is used to investigate the properties of the angles, e.g., to establish connections between the angles corresponding to orthogonal complements. The classical g…
▽ More
We define angles from-to and between infinite dimensional subspaces of a Hilbert space, inspired by the work of E. J. Hannan, 1961/1962 for general canonical correlations of stochastic processes. The spectral theory of selfadjoint operators is used to investigate the properties of the angles, e.g., to establish connections between the angles corresponding to orthogonal complements. The classical gaps and angles of Dixmier and Friedrichs are characterized in terms of the angles. We introduce principal invariant subspaces and prove that they are connected by an isometry that appears in the polar decomposition of the product of corresponding orthogonal projectors. Point angles are defined by analogy with the point operator spectrum. We bound the Hausdorff distance between the sets of the squared cosines of the angles corresponding to the original subspaces and their perturbations. We show that the squared cosines of the angles from one subspace to another can be interpreted as Ritz values in the Rayleigh-Ritz method, where the former subspace serves as a trial subspace and the orthogonal projector of the latter subspace serves as an operator in the Rayleigh-Ritz method. The Hausdorff distance between the Ritz values, corresponding to different trial subspaces, is shown to be bounded by a constant times the gap between the trial subspaces. We prove a similar eigenvalue perturbation bound that involves the gap squared. Finally, we consider the classical alternating projectors method and propose its ultimate acceleration, using the conjugate gradient approach. The corresponding convergence rate estimate is obtained in terms of the angles. We illustrate a possible acceleration for the domain decomposition method with a small overlap for the 1D diffusion equation.
△ Less
Submitted 1 June, 2010; v1 submitted 7 May, 2007;
originally announced May 2007.
-
Rayleigh-Ritz majorization error bounds with applications to FEM
Authors:
Andrew V. Knyazev,
Merico E. Argentati
Abstract:
The Rayleigh-Ritz (RR) method finds the stationary values, called Ritz values, of the Rayleigh quotient on a given trial subspace as approximations to eigenvalues of a Hermitian operator $A$. If the trial subspace is $A$-invariant, the Ritz values are exactly some of the eigenvalues of $A$. Given two subspaces $\X$ and $\Y$ of the same finite dimension, such that $\X$ is $A$-invariant, the absol…
▽ More
The Rayleigh-Ritz (RR) method finds the stationary values, called Ritz values, of the Rayleigh quotient on a given trial subspace as approximations to eigenvalues of a Hermitian operator $A$. If the trial subspace is $A$-invariant, the Ritz values are exactly some of the eigenvalues of $A$. Given two subspaces $\X$ and $\Y$ of the same finite dimension, such that $\X$ is $A$-invariant, the absolute changes in the Ritz values of $A$ with respect to $\X$ compared to the Ritz values with respect to $\Y$ represent the RR absolute eigenvalue approximation error. Our first main result is a sharp majorization-type RR error bound in terms of the principal angles between $\X$ and $\Y$ for an arbitrary $A$-invariant $\X$, which was a conjecture in [SIAM J. Matrix Anal. Appl., 30 (2008), pp. 548-559]. Second, we prove a novel type of RR error bound that deals with the products of the errors, rather than the sums. Third, we establish majorization bounds for the relative errors. We extend our bounds to the case $\dim\X\leq\dim\Y<\infty$ in Hilbert spaces and apply them in the context of the finite element method.
△ Less
Submitted 5 October, 2009; v1 submitted 26 January, 2007;
originally announced January 2007.
-
Bounds on changes in Ritz values for a perturbed invariant subspace of a Hermitian matrix
Authors:
M. E. Argentati,
A. V. Knyazev,
C. C. Paige,
I. Panayotov
Abstract:
The Rayleigh-Ritz method is widely used for eigenvalue approximation. Given a matrix $X$ with columns that form an orthonormal basis for a subspace $\X$, and a Hermitian matrix $A$, the eigenvalues of $X^HAX$ are called Ritz values of $A$ with respect to $\X$. If the subspace $\X$ is $A$-invariant then the Ritz values are some of the eigenvalues of $A$. If the $A$-invariant subspace $\X$ is pert…
▽ More
The Rayleigh-Ritz method is widely used for eigenvalue approximation. Given a matrix $X$ with columns that form an orthonormal basis for a subspace $\X$, and a Hermitian matrix $A$, the eigenvalues of $X^HAX$ are called Ritz values of $A$ with respect to $\X$. If the subspace $\X$ is $A$-invariant then the Ritz values are some of the eigenvalues of $A$. If the $A$-invariant subspace $\X$ is perturbed to give rise to another subspace $\Y$, then the vector of absolute values of changes in Ritz values of $A$ represents the absolute eigenvalue approximation error using $\Y$. We bound the error in terms of principal angles between $\X$ and $\Y$. We capitalize on ideas from a recent paper [DOI: 10.1137/060649070] by A. Knyazev and M. Argentati, where the vector of absolute values of differences between Ritz values for subspaces $\X$ and $\Y$ was weakly (sub-)majorized by a constant times the sine of the vector of principal angles between $\X$ and $\Y$, the constant being the spread of the spectrum of $A$. In that result no assumption was made on either subspace being $A$-invariant. It was conjectured there that if one of the trial subspaces is $A$-invariant then an analogous weak majorization bound should only involve terms of the order of sine squared. Here we confirm this conjecture. Specifically we prove that the absolute eigenvalue error is weakly majorized by a constant times the sine squared of the vector of principal angles between the subspaces $\X$ and $\Y$, where the constant is proportional to the spread of the spectrum of $A$. For many practical cases we show that the proportionality factor is simply one, and that this bound is sharp. For the general case we can only prove the result with a slightly larger constant, which we believe is artificial.
△ Less
Submitted 3 February, 2008; v1 submitted 16 October, 2006;
originally announced October 2006.
-
Majorization for Changes in Angles Between Subspaces, Ritz Values, and Graph Laplacian Spectra
Authors:
A. V. Knyazev,
M. E. Argentati
Abstract:
Many inequality relations between real vector quantities can be succinctly expressed as "weak (sub)majorization" relations. We explain these ideas and apply them in several areas: angles between subspaces, Ritz values, and graph Laplacian spectra, which we show are all surprisingly related... An application of our Ritz values weak majorization result for Laplacian graph spectra comparison is sug…
▽ More
Many inequality relations between real vector quantities can be succinctly expressed as "weak (sub)majorization" relations. We explain these ideas and apply them in several areas: angles between subspaces, Ritz values, and graph Laplacian spectra, which we show are all surprisingly related... An application of our Ritz values weak majorization result for Laplacian graph spectra comparison is suggested, based on the possibility to interpret eigenvalues of the edge Laplacian of a given graph as Ritz values of the edge Laplacian of the complete graph. We prove that $ \sum_k |\lambda1_k - \lambda2_k| \leq n l,$ where $\lambda1_k$ and $\lambda2_k$ are all ordered elements of the Laplacian spectra of two graphs with the same $n$ vertices and with $l$ equal to the number of differing edges.
△ Less
Submitted 20 March, 2006; v1 submitted 29 August, 2005;
originally announced August 2005.
