-
Hybrid CG-Tikhonov is a filtration of the CG Lanczos vectors
Authors:
Daniel Gerth,
Kirk M. Soodhalter
Abstract:
We consider iterative methods for solving linear ill-posed problems with compact operator and right-hand side only available via noise-polluted measurements. Conjugate gradients (\cg) applied to the normal equations with an appropriate stopping rule and \cg applied to the system solving for a Tikhonov-regularized solution (\cgt) $(A^\ast A + c I_{\mathcal{X}}) x^{(δ,c)} = A^\ast y^δ$ are closely r…
▽ More
We consider iterative methods for solving linear ill-posed problems with compact operator and right-hand side only available via noise-polluted measurements. Conjugate gradients (\cg) applied to the normal equations with an appropriate stopping rule and \cg applied to the system solving for a Tikhonov-regularized solution (\cgt) $(A^\ast A + c I_{\mathcal{X}}) x^{(δ,c)} = A^\ast y^δ$ are closely related regularization methods that build iterates from the same family of Krylov subspaces.
In this work, we show that the \cgt iterate can be expressed as $x^{(δ,c)}_m = \sum_{i=1}^{m} γ^{(m)}_i(c) z_i^{(m)}v_i$, where $\braces{γ_i^{(m)}(c)}_{i=1}^m$ are functions of the Tikhonov parameter and $x^{(δ)}_m = \sum_{i=1}^{m} z_i^{(m)}v_i$ is the $m$-th \cg iterate. We call these functions \emph{Lanczos filters}, and they can be shown to have decay properties as $c\rightarrow\infty$ with the speed of decay increasing with $i$. This has the effect of filtering out the contribution of the later terms of the \cg iterate. The filters can be constructed using quantities defined via recursions at each iteration.
We demonstrate with numerical experiments that good parameter choices correspond to appropriate damping of the Lanczos vectors. The filtration approach also provides a platform for further development of parameter choice rules, and similar representations may hold for other hybrid iterative schemes.
△ Less
Submitted 3 July, 2025; v1 submitted 29 May, 2025;
originally announced May 2025.
-
A Quantum-Centric Super-Krylov Diagonalization Method
Authors:
Adam Byrne,
William Kirby,
Kirk M. Soodhalter,
Sergiy Zhuk
Abstract:
The problem of estimating the ground-state energy of a quantum system is ubiquitous in chemistry and condensed matter physics. Krylov quantum diagonalization (KQD) methods have emerged as a promising approach for this task, although many existing methods rely on subroutines - particularly the Hadamard test - that are challenging to implement on near-term quantum computers. We present a KQD method…
▽ More
The problem of estimating the ground-state energy of a quantum system is ubiquitous in chemistry and condensed matter physics. Krylov quantum diagonalization (KQD) methods have emerged as a promising approach for this task, although many existing methods rely on subroutines - particularly the Hadamard test - that are challenging to implement on near-term quantum computers. We present a KQD method that uses only real-time evolutions and recovery probabilities, making it very well adapted for existing quantum hardware. Additionally, we propose a classical post-processing derivative estimation algorithm. Under assumptions on the spectrum of the Hamiltonian, we prove that our algorithm converges exponentially quickly to the ground-state energy. Finally, we provide classical numerical simulations for the transverse-field Ising model on 100 qubits.
△ Less
Submitted 14 May, 2025; v1 submitted 23 December, 2024;
originally announced December 2024.
-
GMRES with randomized sketching and deflated restarting
Authors:
Liam Burke,
Stefan Güttel,
Kirk M. Soodhalter
Abstract:
We present a new Krylov subspace recycling method for solving a linear system of equations, or a sequence of slowly changing linear systems. Our approach is to reduce the computational overhead of recycling techniques while still benefiting from the acceleration afforded by such techniques. As such, this method augments an unprojected Krylov subspace. Furthermore, it combines randomized sketching…
▽ More
We present a new Krylov subspace recycling method for solving a linear system of equations, or a sequence of slowly changing linear systems. Our approach is to reduce the computational overhead of recycling techniques while still benefiting from the acceleration afforded by such techniques. As such, this method augments an unprojected Krylov subspace. Furthermore, it combines randomized sketching and deflated restarting in a way that avoids orthogononalizing a full Krylov basis. We call this new method GMRES-SDR (sketched deflated restarting). With this new method, we provide new theory, which initially characterizes unaugmented sketched GMRES as a projection method for which the projectors involve the sketching operator. We demonstrate that sketched GMRES and its sibling method sketched FOM are an MR/OR pairing, just like GMRES and FOM. We furthermore obtain residual convergence estimates. Building on this, we characterize GMRES-SDR also in terms of sketching-based projectors. Compression of the augmented Krylov subspace for recycling is performed using a sketched version of harmonic Ritz vectors. We present results of numerical experiments demonstrating the effectiveness of GMRES-SDR over competitor methods such as GMRES-DR and GCRO-DR.
△ Less
Submitted 27 September, 2024; v1 submitted 23 November, 2023;
originally announced November 2023.
-
Chebyshev HOPGD with sparse grid sampling for parameterized linear systems
Authors:
Siobhán Correnty,
Melina A. Freitag,
Kirk M. Soodhalter
Abstract:
We consider approximating solutions to parameterized linear systems of the form $A(μ_1,μ_2) x(μ_1,μ_2) = b$. Here the matrix $A(μ_1,μ_2) \in \mathbb{R}^{n \times n}$ is nonsingular, large, and sparse and depends nonlinearly on the parameters. Specifically, the system arises from a discretization of a partial differential equation and $x(μ_1,μ_2) \in \mathbb{R}^n$, $b \in \mathbb{R}^n$. The treatme…
▽ More
We consider approximating solutions to parameterized linear systems of the form $A(μ_1,μ_2) x(μ_1,μ_2) = b$. Here the matrix $A(μ_1,μ_2) \in \mathbb{R}^{n \times n}$ is nonsingular, large, and sparse and depends nonlinearly on the parameters. Specifically, the system arises from a discretization of a partial differential equation and $x(μ_1,μ_2) \in \mathbb{R}^n$, $b \in \mathbb{R}^n$. The treatment of linear systems with nonlinear dependence on a single parameter has been well-studied, and robust methods combining companion linearization, Krylov subspace methods, and Chebyshev interpolation have enabled fast solution for multiple parameter values at the cost of a single iteration.
Solution of systems depending nonlinearly on multiple parameters is more challenging. This work overcomes those additional challenges by combining companion linearization, the Krylov subspace method preconditioned bi-conjugate gradient (BiCG), and a decomposition of a tensor matrix of precomputed solutions, called snapshots. This produces a reduced order model of $x(μ_1,μ_2)$, and this model can be evaluated inexpensively for many values of the parameters. An interpolation of the model is used to produce approximations on the entire parameter space. In addition this method can be used to solve a parameter estimation problem.
This approach allows us to achieve similar computational savings as for the one-parameter case; we can solve for many parameter pairs at the cost of many fewer applications of an efficient iterative method. The technique is presented for dependence on two parameters, but the strategy can be extended to more parameters using the same approach. Numerical examples of a parameterized Helmholtz equation show the competitiveness of our approach.
△ Less
Submitted 25 February, 2025; v1 submitted 25 September, 2023;
originally announced September 2023.
-
The effect of approximate coarsest-level solves on the convergence of multigrid V-cycle methods
Authors:
Petr Vacek,
Erin Carson,
Kirk M. Soodhalter
Abstract:
The multigrid V-cycle method is a popular method for solving systems of linear equations. It computes an approximate solution by using smoothing on fine levels and solving a system of linear equations on the coarsest level. Solving on the coarsest level depends on the size and difficulty of the problem. If the size permits, it is typical to use a direct method based on LU or Cholesky decomposition…
▽ More
The multigrid V-cycle method is a popular method for solving systems of linear equations. It computes an approximate solution by using smoothing on fine levels and solving a system of linear equations on the coarsest level. Solving on the coarsest level depends on the size and difficulty of the problem. If the size permits, it is typical to use a direct method based on LU or Cholesky decomposition. In settings with large coarsest-level problems, approximate solvers such as iterative Krylov subspace methods, or direct methods based on low-rank approximation, are often used. The accuracy of the coarsest-level solver is typically determined based on the experience of the users with the concrete problems and methods.
In this paper we present an approach to analyzing the effects of approximate coarsest-level solves on the convergence of the V-cycle method for symmetric positive definite problems. Using these results, we derive coarsest-level stopping criterion through which we may control the difference between the approximation computed by a V-cycle method with approximate coarsest-level solver and the approximation which would be computed if the coarsest-level problems were solved exactly. The coarsest-level stopping criterion may thus be set up such that the V-cycle method converges to a chosen finest-level accuracy in (nearly) the same number of V-cycle iterations as the V-cycle method with exact coarsest-level solver. We also utilize the theoretical results to discuss how the convergence of the V-cycle method may be affected by the choice of a tolerance in a coarsest-level stopping criterion based on the relative residual norm.
△ Less
Submitted 7 May, 2024; v1 submitted 9 June, 2023;
originally announced June 2023.
-
Krylov Subspace Recycling For Matrix Functions
Authors:
Liam Burke,
Andreas Frommer,
Gustavo Ramirez-Hidalgo,
Kirk M. Soodhalter
Abstract:
We derive an augmented Krylov subspace method with subspace recycling for computing a sequence of matrix function applications on a set of vectors. The matrix is either fixed or changes as the sequence progresses. We assume consecutive matrices are closely related, but make no assumptions on the relationship between the vectors. We present three versions of the method with different practical impl…
▽ More
We derive an augmented Krylov subspace method with subspace recycling for computing a sequence of matrix function applications on a set of vectors. The matrix is either fixed or changes as the sequence progresses. We assume consecutive matrices are closely related, but make no assumptions on the relationship between the vectors. We present three versions of the method with different practical implementations. We demonstrate the effectiveness of the method using a range of numerical experiments with a selection of functions and matrices. We primarily focus our attention on the sign function arising in the overlap formalism of lattice QCD.
△ Less
Submitted 28 September, 2022;
originally announced September 2022.
-
Augmented unprojected Krylov subspace methods
Authors:
Liam Burke,
Kirk M. Soodhalter
Abstract:
Augmented Krylov subspace methods aid in accelerating the convergence of a standard Krylov subspace method by including additional vectors in the search space. A residual projection framework based on residual (Petrov-) Galerkin constraints was presented in [Gaul et al. SIAM J. Matrix Anal. Appl 2013], and later generalised in a recent survey on subspace recycling iterative methods [Soodhalter et…
▽ More
Augmented Krylov subspace methods aid in accelerating the convergence of a standard Krylov subspace method by including additional vectors in the search space. A residual projection framework based on residual (Petrov-) Galerkin constraints was presented in [Gaul et al. SIAM J. Matrix Anal. Appl 2013], and later generalised in a recent survey on subspace recycling iterative methods [Soodhalter et al. GAMM-Mitt. 2020]. The framework describes augmented Krylov subspace methods in terms of applying a standard Krylov subspace method to an appropriately projected problem.
In this work we show that the projected problem has an equivalent unprojected formulation, and that viewing the framework in this way provides a similar description for the class of unprojected augmented Krylov subspace methods. We derive the first unprojected augmented Full Orthogonalization Method (FOM), and demonstrate its effectiveness as a recycling method. We then show how the R$^{3}$GMRES algorithm fits within the framework. We show that unprojected augmented short recurrence methods fit within the framework, but can only be implemented in practice under certain conditions on the augmentation subspace. We demonstrate this using the Augmented Conjugate Gradient (AugCG) algorithm as an example.
△ Less
Submitted 18 May, 2023; v1 submitted 24 June, 2022;
originally announced June 2022.
-
Preconditioned infinite GMRES for parameterized linear systems
Authors:
Siobhán Correnty,
Elias Jarlebring,
Kirk M. Soodhalter
Abstract:
We are interested in obtaining approximate solutions to parameterized linear systems of the form $A(μ) x(μ) = b$ for many values of the parameter $μ$. Here $A(μ)$ is large, sparse, and nonsingular, with a nonlinear analytic dependence on $μ$. Our approach is based on a companion linearization for parameterized linear systems. The companion matrix is similar to the operator in the infinite Arnoldi…
▽ More
We are interested in obtaining approximate solutions to parameterized linear systems of the form $A(μ) x(μ) = b$ for many values of the parameter $μ$. Here $A(μ)$ is large, sparse, and nonsingular, with a nonlinear analytic dependence on $μ$. Our approach is based on a companion linearization for parameterized linear systems. The companion matrix is similar to the operator in the infinite Arnoldi method, and we use this to adapt the flexible GMRES setting. In this way, our method returns a function $\tilde{x}(μ)$ which is cheap to evaluate for different $μ$, and the preconditioner is applied only approximately. This novel approach leads to increased freedom to carry out the action of the operation inexactly, which provides performance improvement over the method infinite GMRES, without a loss of accuracy in general. We show that the error of our method is estimated based on the magnitude of the parameter $μ$, the inexactness of the preconditioning, and the spectrum of the linear companion matrix. Numerical examples from a finite element discretization of a Helmholtz equation with a parameterized material coefficient illustrate the competitiveness of our approach. The simulations are reproducible and publicly available online.
△ Less
Submitted 10 June, 2022;
originally announced June 2022.
-
A projected Nesterov-Kaczmarz approach to stellar population-kinematic distribution reconstruction in Extragalactic Archaeology
Authors:
Fabian Hinterer,
Simon Hubmer,
Prashin Jethwa,
Kirk M. Soodhalter,
Glenn van de Ven,
Ronny Ramlau
Abstract:
In this paper, we consider the problem of reconstructing a galaxy's stellar population-kinematic distribution function from optical integral field unit measurements. These quantities are connected via a high-dimensional integral equation. To solve this problem, we propose a projected Nesterov-Kaczmarz reconstruction (PNKR) method, which efficiently leverages the problem structure and incorporates…
▽ More
In this paper, we consider the problem of reconstructing a galaxy's stellar population-kinematic distribution function from optical integral field unit measurements. These quantities are connected via a high-dimensional integral equation. To solve this problem, we propose a projected Nesterov-Kaczmarz reconstruction (PNKR) method, which efficiently leverages the problem structure and incorporates physical prior information such as smoothness and non-negativity constraints. To test the performance of our reconstruction approach, we apply it to a dataset simulated from a known ground truth density, and validate it by comparing our recoveries to those obtained by the widely used pPXF software.
△ Less
Submitted 2 November, 2022; v1 submitted 8 June, 2022;
originally announced June 2022.
-
Fast solution of Sylvester-structured systems for spatial source separation of the Cosmic Microwave Background
Authors:
Dung Pham,
Kirk M. Soodhalter,
Simon Wilson
Abstract:
Implementation of many statistical methods for large, multivariate data sets requires one to solve a linear system that, depending on the method, is of the dimension of the number of observations or each individual data vector. This is often the limiting factor in scaling the method with data size and complexity. In this paper we illustrate the use of Krylov subspace methods to address this issue…
▽ More
Implementation of many statistical methods for large, multivariate data sets requires one to solve a linear system that, depending on the method, is of the dimension of the number of observations or each individual data vector. This is often the limiting factor in scaling the method with data size and complexity. In this paper we illustrate the use of Krylov subspace methods to address this issue in a statistical solution to a source separation problem in cosmology where the data size is prohibitively large for direct solution of the required system. Two distinct approaches, adapted from techniques in the literature, are described: one that uses the method of conjugate gradients directly to the Kronecker-structured problem and another that reformulates the system as a Sylvester matrix equation. We show that both approaches produce an accurate solution within an acceptable computation time and with practical memory requirements for the data size that is currently available.
△ Less
Submitted 26 September, 2024; v1 submitted 17 April, 2022;
originally announced April 2022.
-
A note on augmented unprojected Krylov subspace methods
Authors:
Kirk M. Soodhalter
Abstract:
Subspace recycling iterative methods and other subspace augmentation schemes are a successful extension to Krylov subspace methods in which a Krylov subspace is augmented with a fixed subspace spanned by vectors deemed to be helpful in accelerating convergence or conveying knowledge of the solution. Recently, a survey was published, in which a framework describing the vast majority of such methods…
▽ More
Subspace recycling iterative methods and other subspace augmentation schemes are a successful extension to Krylov subspace methods in which a Krylov subspace is augmented with a fixed subspace spanned by vectors deemed to be helpful in accelerating convergence or conveying knowledge of the solution. Recently, a survey was published, in which a framework describing the vast majority of such methods was proposed [Soodhalter et al, GAMM-Mitt. 2020]. In many of these methods, the Krylov subspace is one generated by the system matrix composed with a projector that depends on the augmentation space. However, it is not a requirement that a projected Krylov subspace be used. There are augmentation methods built on using Krylov subspaces generated by the original system matrix, and these methods also fit into the general framework.
In this note, we observe that one gains implementation benefits by considering such augmentation methods with unprojected Krylov subspaces in the general framework. We demonstrate this by applying the idea to the R$^3$GMRES method proposed in [Dong et al. ETNA 2014] to obtain a simplified implementation and to connect that algorithm to early augmentation schemes based on flexible preconditioning [Saad. SIMAX 1997].
△ Less
Submitted 10 February, 2022; v1 submitted 18 June, 2021;
originally announced June 2021.
-
Subspace Recycling-based Regularization Methods
Authors:
Ronny Ramlau,
Kirk M. Soodhalter,
Victoria Hutterer
Abstract:
Subspace recycling techniques have been used quite successfully for the acceleration of iterative methods for solving large-scale linear systems. These methods often work by augmenting a solution subspace generated iteratively by a known algorithm with a fixed subspace of vectors which are ``useful'' for solving the problem. Often, this has the effect of inducing a projected version of the origina…
▽ More
Subspace recycling techniques have been used quite successfully for the acceleration of iterative methods for solving large-scale linear systems. These methods often work by augmenting a solution subspace generated iteratively by a known algorithm with a fixed subspace of vectors which are ``useful'' for solving the problem. Often, this has the effect of inducing a projected version of the original linear system to which the known iterative method is then applied, and this projection can act as a deflation preconditioner, accelerating convergence. Most often, these methods have been applied for the solution of well-posed problems. However, they have also begun to be considered for the solution of ill-posed problems.
In this paper, we consider subspace augmentation-type iterative schemes applied to linear ill-posed problems in a continuous Hilbert space setting, based on a recently developed framework describing these methods. We show that under suitable assumptions, a recycling method satisfies the formal definition of a regularization, as long as the underlying scheme is itself a regularization. We then develop an augmented subspace version of the gradient descent method and demonstrate its effectiveness, both on an academic Gaussian blur model and on problems arising from the adaptive optics community for the resolution of large sky images by ground-based extremely large telescopes.
△ Less
Submitted 17 May, 2021; v1 submitted 10 November, 2020;
originally announced November 2020.
-
A survey of subspace recycling iterative methods
Authors:
Kirk M. Soodhalter,
Eric de Sturler,
Misha Kilmer
Abstract:
This survey concerns subspace recycling methods, a popular class of iterative methods that enable effective reuse of subspace information in order to speed up convergence and find good initial guesses over a sequence of linear systems with slowly changing coefficient matrices, multiple right-hand sides, or both. The subspace information that is recycled is usually generated during the run of an it…
▽ More
This survey concerns subspace recycling methods, a popular class of iterative methods that enable effective reuse of subspace information in order to speed up convergence and find good initial guesses over a sequence of linear systems with slowly changing coefficient matrices, multiple right-hand sides, or both. The subspace information that is recycled is usually generated during the run of an iterative method (usually a Krylov subspace method) on one or more of the systems. Following introduction of definitions and notation, we examine the history of early augmentation schemes along with deflation preconditioning schemes and their influence on the development of recycling methods. We then discuss a general residual constraint framework through which many augmented Krylov and recycling methods can both be viewed. We review several augmented and recycling methods within this framework. We then discuss some known effective strategies for choosing subspaces to recycle before taking the reader through more recent developments that have generalized recycling for (sequences of) shifted linear systems, some of them with multiple right-hand sides in mind. We round out our survey with a brief review of application areas that have seen benefit from subspace recycling methods.
△ Less
Submitted 29 July, 2020; v1 submitted 28 January, 2020;
originally announced January 2020.
-
Admissible and attainable convergence behavior of block Arnoldi and GMRES
Authors:
Marie Kubínová,
Kirk M. Soodhalter
Abstract:
It is well-established that any non-increasing convergence curve is possible for GMRES and a family of pairs $(A,b)$ can be constructed for which GMRES exhibits a given convergence curve with $A$ having arbitrary spectrum. No analog of this result has been established for block GMRES, wherein multiple right-hand sides are considered. By reframing the problem as a single linear system over a ring o…
▽ More
It is well-established that any non-increasing convergence curve is possible for GMRES and a family of pairs $(A,b)$ can be constructed for which GMRES exhibits a given convergence curve with $A$ having arbitrary spectrum. No analog of this result has been established for block GMRES, wherein multiple right-hand sides are considered. By reframing the problem as a single linear system over a ring of square matrices, we develop convergence results for block Arnoldi and block GMRES. In particular, we show what convergence behavior is admissible for block GMRES and how the matrices and right-hand sides producing any admissible behavior can be constructed. Moreover, we show that the convergence of the block Arnoldi method for eigenvalue approximation can be almost fully independent of the convergence of block GMRES for the same coefficient matrix and the same starting vectors.
△ Less
Submitted 3 March, 2020; v1 submitted 8 July, 2019;
originally announced July 2019.
-
Error Analysis and Improving the Accuracy of Winograd Convolution for Deep Neural Networks
Authors:
Barbara Barabasz,
Andrew Anderson,
Kirk M. Soodhalter,
David Gregg
Abstract:
Popular deep neural networks (DNNs) spend the majority of their execution time computing convolutions. The Winograd family of algorithms can greatly reduce the number of arithmetic operations required and is present in many DNN software frameworks. However, the performance gain is at the expense of a reduction in floating point (FP) numerical accuracy. In this paper, we analyse the worst case FP e…
▽ More
Popular deep neural networks (DNNs) spend the majority of their execution time computing convolutions. The Winograd family of algorithms can greatly reduce the number of arithmetic operations required and is present in many DNN software frameworks. However, the performance gain is at the expense of a reduction in floating point (FP) numerical accuracy. In this paper, we analyse the worst case FP error and prove the estimation of norm and conditioning of the algorithm. We show that the bound grows exponentially with the size of the convolution, but the error bound of the \textit{modified} algorithm is smaller than the original one. We propose several methods for reducing FP error. We propose a canonical evaluation ordering based on Huffman coding that reduces summation error. We study the selection of sampling "points" experimentally and find empirically good points for the most important sizes. We identify the main factors associated with good points. In addition, we explore other methods to reduce FP error, including mixed-precision convolution, and pairwise summation across DNN channels. Using our methods we can significantly reduce FP error for a given block size, which allows larger block sizes and reduced computation.
△ Less
Submitted 1 May, 2019; v1 submitted 29 March, 2018;
originally announced March 2018.
-
A block Recycled GMRES method with investigations into aspects of solver performance
Authors:
Michael L. Parks,
Kirk M. Soodhalter,
Daniel B. Szyld
Abstract:
We propose a block Krylov subspace version of the GCRO-DR method proposed in [Parks et al.; SISC 2005], which is an iterative method allowing for the efficient minimization of the the residual over an augmented Krylov subspace. We offer a clean derivation of our proposed method and discuss methods of selecting recycling subspaces at restart as well as implementation decisions in the context of hig…
▽ More
We propose a block Krylov subspace version of the GCRO-DR method proposed in [Parks et al.; SISC 2005], which is an iterative method allowing for the efficient minimization of the the residual over an augmented Krylov subspace. We offer a clean derivation of our proposed method and discuss methods of selecting recycling subspaces at restart as well as implementation decisions in the context of high-performance computing. Numerical experiments are split into those demonstrating convergence properties and those demonstrating the data movement and cache efficiencies of the dominant operations of the method, measured using processor monitoring code from Intel.
△ Less
Submitted 19 June, 2025; v1 submitted 6 April, 2016;
originally announced April 2016.
-
A modified implementation of MINRES to monitor residual subvector norms for block systems
Authors:
Roland Herzog,
Kirk M. Soodhalter
Abstract:
Saddle-point systems, i.e., structured linear systems with symmetric matrices are considered. A modified implementation of (preconditioned) MINRES is derived which allows to monitor the norms of the subvectors individually. Compared to the implementation from the textbook of [Elman, Sylvester and Wathen, Oxford University Press, 2014], our method requires one extra vector of storage and no additio…
▽ More
Saddle-point systems, i.e., structured linear systems with symmetric matrices are considered. A modified implementation of (preconditioned) MINRES is derived which allows to monitor the norms of the subvectors individually. Compared to the implementation from the textbook of [Elman, Sylvester and Wathen, Oxford University Press, 2014], our method requires one extra vector of storage and no additional applications of the preconditioner. Numerical experiments are included.
△ Less
Submitted 7 September, 2016; v1 submitted 14 March, 2016;
originally announced March 2016.
-
Block Krylov subspace recycling for shifted systems with unrelated right-hand sides
Authors:
Kirk M. Soodhalter
Abstract:
Many Krylov subspace methods for shifted linear systems take advantage of the invariance of the Krylov subspace under a shift of the matrix. However, exploiting this fact in the non-Hermitian case introduces restrictions; e.g., initial residuals must be collinear and this collinearity must be maintained at restart. Thus we cannot simultaneously solve shifted systems with unrelated right-hand sides…
▽ More
Many Krylov subspace methods for shifted linear systems take advantage of the invariance of the Krylov subspace under a shift of the matrix. However, exploiting this fact in the non-Hermitian case introduces restrictions; e.g., initial residuals must be collinear and this collinearity must be maintained at restart. Thus we cannot simultaneously solve shifted systems with unrelated right-hand sides using this strategy, and all shifted residuals cannot be simultaneously minimized over a Krylov subspace such that collinearity is maintained. It has been shown that this renders them generally incompatible with techniques of subspace recycling [Soodhalter et al. APNUM '14].
This problem, however, can be overcome. By interpreting a family of shifted systems as one Sylvester equation, we can take advantage of the known "shift invariance" of the Krylov subspace generated by the Sylvester operator. Thus we can simultaneously solve all systems over one block Krylov subspace using FOM or GMRES type methods, even when they have unrelated right-hand sides. Because residual collinearity is no longer a requirement at restart, these methods are fully compatible with subspace recycling techniques. Furthermore, we realize the benefits of block sparse matrix operations which arise in the context of high-performance computing applications.
In this paper, we discuss exploiting this Sylvester equation point of view which has yielded methods for shifted systems which are compatible with unrelated right-hand sides. From this, we propose a recycled GMRES method for simultaneous solution of shifted systems.Numerical experiments demonstrate the effectiveness of the methods.
△ Less
Submitted 26 October, 2015; v1 submitted 1 December, 2014;
originally announced December 2014.
-
Stagnation of block GMRES and its relationship to block FOM
Authors:
Kirk M. Soodhalter
Abstract:
We analyze the the convergence behavior of block GMRES and characterize the phenomenon of stagnation which is then related to the behavior of the block FOM method. We generalize the block FOM method to generate well-defined approximations in the case that block FOM would normally break down, and these generalized solutions are used in our analysis. This behavior is also related to the principal an…
▽ More
We analyze the the convergence behavior of block GMRES and characterize the phenomenon of stagnation which is then related to the behavior of the block FOM method. We generalize the block FOM method to generate well-defined approximations in the case that block FOM would normally break down, and these generalized solutions are used in our analysis. This behavior is also related to the principal angles between the column-space of the previous block GMRES residual and the current minimum residual constraint space. At iteration $j$, it is shown that the proper generalization of GMRES stagnation to the block setting relates to the columnspace of the $j$th block Arnoldi vector. Our analysis covers both the cases of normal iterations as well as block Arnoldi breakdown wherein dependent basis vectors are replaced with random ones. Numerical examples are given to illustrate what we have proven, including a small application problem to demonstrate the validity of the analysis in a less pathological case.
△ Less
Submitted 11 July, 2016; v1 submitted 28 November, 2014;
originally announced November 2014.
-
Two recursive GMRES-type methods for shifted linear systems with general preconditioning
Authors:
Kirk M. Soodhalter
Abstract:
We present two minimum residual methods for solving sequences of shifted linear systems, the right-preconditioned shifted GMRES and shifted recycled GMRES algorithms which use a seed projection strategy often employed to solve multiple related problems. These methods are compatible with general preconditioning of all systems, and when restricted to right preconditioning, require no extra applicati…
▽ More
We present two minimum residual methods for solving sequences of shifted linear systems, the right-preconditioned shifted GMRES and shifted recycled GMRES algorithms which use a seed projection strategy often employed to solve multiple related problems. These methods are compatible with general preconditioning of all systems, and when restricted to right preconditioning, require no extra applications of the operator or preconditioner. These seed projection methods perform a minimum residual iteration for the base system while improving the approximations for the shifted systems at little additional cost. The iteration continues until the base system approximation is of satisfactory quality. The method is then recursively called for the remaining unconverged systems. We present both methods inside of a general framework which allows these techniques to be extended to the setting of flexible preconditioning and inexact Krylov methods. We present some analysis of such methods and numerical experiments demonstrating the effectiveness of the algorithms we have derived.
△ Less
Submitted 3 February, 2016; v1 submitted 18 March, 2014;
originally announced March 2014.
-
Krylov Subspace Recycling for Sequences of Shifted Linear Systems
Authors:
Kirk M. Soodhalter,
Daniel B. Szyld,
Fei Xue
Abstract:
We study the use of Krylov subspace recycling for the solution of a sequence of slowly-changing families of linear systems, where each family consists of shifted linear systems that differ in the coefficient matrix only by multiples of the identity. Our aim is to explore the simultaneous solution of each family of shifted systems within the framework of subspace recycling, using one augmented subs…
▽ More
We study the use of Krylov subspace recycling for the solution of a sequence of slowly-changing families of linear systems, where each family consists of shifted linear systems that differ in the coefficient matrix only by multiples of the identity. Our aim is to explore the simultaneous solution of each family of shifted systems within the framework of subspace recycling, using one augmented subspace to extract candidate solutions for all the shifted systems. The ideal method would use the same augmented subspace for all systems and have fixed storage requirements, independent of the number of shifted systems per family. We show that a method satisfying both requirements cannot exist in this framework.
As an alternative, we introduce two schemes. One constructs a separate deflation space for each shifted system but solves each family of shifted systems simultaneously. The other builds only one recycled subspace and constructs approximate corrections to the solutions of the shifted systems at each cycle of the iterative linear solver while only minimizing the base system residual. At convergence of the base system solution, we apply the method recursively to the remaining unconverged systems. We present numerical examples involving systems arising in lattice quantum chromodynamics.
△ Less
Submitted 29 August, 2013; v1 submitted 12 January, 2013;
originally announced January 2013.
-
A block MINRES algorithm based on the banded Lanczos method
Authors:
Kirk M. Soodhalter
Abstract:
We develop a block minimum residual (MINRES) algorithm for symmetric indefinite matrices. This version is built upon the band Lanczos method that generates one basis vector of the block Krylov subspace per iteration rather than a whole block as in the block Lanczos process. However, we modify the method such that the most expensive operations are still performed in a block fashion. The benefit of…
▽ More
We develop a block minimum residual (MINRES) algorithm for symmetric indefinite matrices. This version is built upon the band Lanczos method that generates one basis vector of the block Krylov subspace per iteration rather than a whole block as in the block Lanczos process. However, we modify the method such that the most expensive operations are still performed in a block fashion. The benefit of using the band Lanczos method is that one can detect breakdowns from scalar values arising in the computation, allowing for a handling of breakdown which is straightforward to implement.
We derive a progressive formulation of the MINRES method based on the band Lanczos process and give some implementation details. Specifically, a simple reordering of the steps allows us to perform many of the operations at the block level in order to take advantage of communication efficiencies offered by the block Lanczos process. This is an important concern in the context of next-generation super computing applications.
We also present a technique allowing us to maintain the block size by replacing dependent Lanczos vectors with pregenerated random vectors whose orthogonality against all Lanczos vectors is maintained. Numerical results illustrate the performance on some sample problems. We present experiments that show how the relationship between right-hand sides can effect the performance of the method.
△ Less
Submitted 13 May, 2014; v1 submitted 10 January, 2013;
originally announced January 2013.
