-
An Iterative Block Matrix Inversion (IBMI) Algorithm for Symmetric Positive Definite Matrices with Applications to Covariance Matrices
Authors:
Ann Paterson,
Jennifer Pestana,
Victorita Dolean Maini
Abstract:
Obtaining the inverse of a large symmetric positive definite matrix $\mathcal{A}\in\mathbb{R}^{p\times p}$ is a continual challenge across many mathematical disciplines. The computational complexity associated with direct methods can be prohibitively expensive, making it infeasible to compute the inverse. In this paper, we present a novel iterative algorithm (IBMI), which is designed to approximat…
▽ More
Obtaining the inverse of a large symmetric positive definite matrix $\mathcal{A}\in\mathbb{R}^{p\times p}$ is a continual challenge across many mathematical disciplines. The computational complexity associated with direct methods can be prohibitively expensive, making it infeasible to compute the inverse. In this paper, we present a novel iterative algorithm (IBMI), which is designed to approximate the inverse of a large, dense, symmetric positive definite matrix. The matrix is first partitioned into blocks, and an iterative process using block matrix inversion is repeated until the matrix approximation reaches a satisfactory level of accuracy. We demonstrate that the two-block, non-overlapping approach converges for any positive definite matrix, while numerical results provide strong evidence that the multi-block, overlapping approach also converges for such matrices.
△ Less
Submitted 10 February, 2025;
originally announced February 2025.
-
ELM-FBPINN: efficient finite-basis physics-informed neural networks
Authors:
Samuel Anderson,
Victorita Dolean,
Ben Moseley,
Jennifer Pestana
Abstract:
Physics Informed Neural Networks (PINNs) offer several advantages when compared to traditional numerical methods for solving PDEs, such as being a mesh-free approach and being easily extendable to solving inverse problems. One promising approach for allowing PINNs to scale to multi-scale problems is to combine them with domain decomposition; for example, finite basis physics-informed neural networ…
▽ More
Physics Informed Neural Networks (PINNs) offer several advantages when compared to traditional numerical methods for solving PDEs, such as being a mesh-free approach and being easily extendable to solving inverse problems. One promising approach for allowing PINNs to scale to multi-scale problems is to combine them with domain decomposition; for example, finite basis physics-informed neural networks (FBPINNs) replace the global PINN network with many localised networks which are summed together to approximate the solution. In this work, we significantly accelerate the training of FBPINNs by linearising their underlying optimisation problem. We achieve this by employing extreme learning machines (ELMs) as their subdomain networks and showing that this turns the FBPINN optimisation problem into one of solving a linear system or least-squares problem. We test our workflow in a preliminary fashion by using it to solve an illustrative 1D problem.
△ Less
Submitted 3 September, 2024;
originally announced September 2024.
-
Fast solution of incompressible flow problems with two-level pressure approximation
Authors:
Jennifer Pestana,
David J. Silvester
Abstract:
This paper develops efficient preconditioned iterative solvers for incompressible flow problems discretised by an enriched Taylor-Hood mixed approximation, in which the usual pressure space is augmented by a piecewise constant pressure to ensure local mass conservation. This enrichment process causes over-specification of the pressure when the pressure space is defined by the union of standard Tay…
▽ More
This paper develops efficient preconditioned iterative solvers for incompressible flow problems discretised by an enriched Taylor-Hood mixed approximation, in which the usual pressure space is augmented by a piecewise constant pressure to ensure local mass conservation. This enrichment process causes over-specification of the pressure when the pressure space is defined by the union of standard Taylor-Hood basis functions and piecewise constant pressure basis functions, which complicates the design and implementation of efficient solvers for the resulting linear systems. We first describe the impact of this choice of pressure space specification on the matrices involved. Next, we show how to recover effective solvers for Stokes problems, with preconditioners based on the singular pressure mass matrix, and for Oseen systems arising from linearised Navier-Stokes equations, by using a two-stage pressure convection-diffusion strategy. The codes used to generate the numerical results are available online.
△ Less
Submitted 28 May, 2024; v1 submitted 17 March, 2023;
originally announced March 2023.
-
The asymptotic spectrum of flipped multilevel Toeplitz matrices and of certain preconditionings
Authors:
M. Mazza,
J. Pestana
Abstract:
In this work, we perform a spectral analysis of flipped multilevel Toeplitz sequences, i.e., we study the asymptotic spectral behaviour of $\{Y_{\boldsymbol{n}}T_{\boldsymbol{n}}(f)\}_{\boldsymbol{n}}$, where $T_{\boldsymbol{n}}(f)$ is a real, square multilevel Toeplitz matrix generated by a function $f\in L^1([-π,π]^d)$ and $Y_{\boldsymbol{n}}$ is the exchange matrix, which has $1$s on the main a…
▽ More
In this work, we perform a spectral analysis of flipped multilevel Toeplitz sequences, i.e., we study the asymptotic spectral behaviour of $\{Y_{\boldsymbol{n}}T_{\boldsymbol{n}}(f)\}_{\boldsymbol{n}}$, where $T_{\boldsymbol{n}}(f)$ is a real, square multilevel Toeplitz matrix generated by a function $f\in L^1([-π,π]^d)$ and $Y_{\boldsymbol{n}}$ is the exchange matrix, which has $1$s on the main anti-diagonal. In line with what we have shown for unilevel flipped Toeplitz matrix sequences, the asymptotic spectrum is determined by a $2\times 2$ matrix-valued function whose eigenvalues are $\pm |f|$. Furthermore, we characterize the eigenvalue distribution of certain preconditioned flipped multilevel Toeplitz sequences with an analysis that covers both multilevel Toeplitz and circulant preconditioners. Finally, all our findings are illustrated by several numerical experiments.
△ Less
Submitted 16 November, 2020;
originally announced November 2020.
-
Analysis of parallel Schwarz algorithms for time-harmonic problems using block Toeplitz matrices
Authors:
Niall Bootland,
Victorita Dolean,
Alexandros Kyriakis,
Jennifer Pestana
Abstract:
In this work we study the convergence properties of the one-level parallel Schwarz method with Robin transmission conditions applied to the one-dimensional and two-dimensional Helmholtz and Maxwell's equations. One-level methods are not scalable in general. However, it has recently been proven that when impedance transmission conditions are used in the case of the algorithm applied to the equation…
▽ More
In this work we study the convergence properties of the one-level parallel Schwarz method with Robin transmission conditions applied to the one-dimensional and two-dimensional Helmholtz and Maxwell's equations. One-level methods are not scalable in general. However, it has recently been proven that when impedance transmission conditions are used in the case of the algorithm applied to the equations with absorption, under certain assumptions, scalability can be achieved and no coarse space is required. We show here that this result is also true for the iterative version of the method at the continuous level for strip-wise decompositions into subdomains that can typically be encountered when solving wave-guide problems. The convergence proof relies on the particular block Toeplitz structure of the global iteration matrix. Although non-Hermitian, we prove that its limiting spectrum has a near identical form to that of a Hermitian matrix of the same structure. We illustrate our results with numerical experiments.
△ Less
Submitted 31 August, 2021; v1 submitted 15 June, 2020;
originally announced June 2020.
-
Preconditioners for symmetrized Toeplitz and multilevel Toeplitz matrices
Authors:
Jennifer Pestana
Abstract:
When solving linear systems with nonsymmetric Toeplitz or multilevel Toeplitz matrices using Krylov subspace methods, the coefficient matrix may be symmetrized. The preconditioned MINRES method can then be applied to this symmetrized system, which allows rigorous upper bounds on the number of MINRES iterations to be obtained. However, effective preconditioners for symmetrized (multilevel) Toeplitz…
▽ More
When solving linear systems with nonsymmetric Toeplitz or multilevel Toeplitz matrices using Krylov subspace methods, the coefficient matrix may be symmetrized. The preconditioned MINRES method can then be applied to this symmetrized system, which allows rigorous upper bounds on the number of MINRES iterations to be obtained. However, effective preconditioners for symmetrized (multilevel) Toeplitz matrices are lacking. Here, we propose novel ideal preconditioners, and investigate the spectra of the preconditioned matrices. We show how these preconditioners can be approximated and demonstrate their effectiveness via numerical experiments.
△ Less
Submitted 12 April, 2019; v1 submitted 6 December, 2018;
originally announced December 2018.
-
GMRES convergence bounds for eigenvalue problems
Authors:
Melina Freitag,
Patrick Kürschner,
Jennifer Pestana
Abstract:
The convergence of GMRES for solving linear systems can be influenced heavily by the structure of the right hand side. Within the solution of eigenvalue problems via inverse iteration or subspace iteration, the right hand side is generally related to an approximate invariant subspace of the linear system. We give detailed and new bounds on (block) GMRES that take the special behavior of the right…
▽ More
The convergence of GMRES for solving linear systems can be influenced heavily by the structure of the right hand side. Within the solution of eigenvalue problems via inverse iteration or subspace iteration, the right hand side is generally related to an approximate invariant subspace of the linear system. We give detailed and new bounds on (block) GMRES that take the special behavior of the right hand side into account and explain the initial sharp decrease of the GMRES residual. The bounds give rise to adapted preconditioners applied to the eigenvalue problems, e.g. tuned and polynomial preconditioners. The numerical results show that the new (block) GMRES bounds are much sharper than conventional bounds and that preconditioned subspace iteration with either a tuned or polynomial preconditioner should be used in practice.
△ Less
Submitted 30 May, 2017; v1 submitted 20 September, 2016;
originally announced September 2016.
-
Fast Multipole Preconditioners for Sparse Matrices Arising from Elliptic Equations
Authors:
Huda Ibeid,
Rio Yokota,
Jennifer Pestana,
David Keyes
Abstract:
Among optimal hierarchical algorithms for the computational solution of elliptic problems, the Fast Multipole Method (FMM) stands out for its adaptability to emerging architectures, having high arithmetic intensity, tunable accuracy, and relaxable global synchronization requirements. We demonstrate that, beyond its traditional use as a solver in problems for which explicit free-space kernel repres…
▽ More
Among optimal hierarchical algorithms for the computational solution of elliptic problems, the Fast Multipole Method (FMM) stands out for its adaptability to emerging architectures, having high arithmetic intensity, tunable accuracy, and relaxable global synchronization requirements. We demonstrate that, beyond its traditional use as a solver in problems for which explicit free-space kernel representations are available, the FMM has applicability as a preconditioner in finite domain elliptic boundary value problems, by equipping it with boundary integral capability for satisfying conditions at finite boundaries and by wrapping it in a Krylov method for extensibility to more general operators. Here, we do not discuss the well developed applications of FMM to implement matrix-vector multiplications within Krylov solvers of boundary element methods. Instead, we propose using FMM for the volume-to-volume contribution of inhomogeneous Poisson-like problems, where the boundary integral is a small part of the overall computation. Our method may be used to precondition sparse matrices arising from finite difference/element discretizations, and can handle a broader range of scientific applications. Compared with multigrid methods, it is capable of comparable algebraic convergence rates down to the truncation error of the discretized PDE, and it offers potentially superior multicore and distributed memory scalability properties on commodity architecture supercomputers. Compared with other methods exploiting the low rank character of off-diagonal blocks of the dense resolvent operator, FMM-preconditioned Krylov iteration may reduce the amount of communication because it is matrix-free and exploits the tree structure of FMM. We describe our tests in reproducible detail with freely available codes and outline directions for further extensibility.
△ Less
Submitted 19 January, 2016; v1 submitted 15 August, 2013;
originally announced August 2013.
