-
Generalizing Reduced Rank Extrapolation to Low-Rank Matrix Sequences
Authors:
Pascal den Boef,
Patrick Kürschner,
Xiaobo Liu,
Jos Maubach,
Jens Saak,
Wil Schilders,
Jonas Schulze,
Nathan van de Wouw
Abstract:
Reduced rank extrapolation (RRE) is an acceleration method typically used to accelerate the iterative solution of nonlinear systems of equations using a fixed-point process. In this context, the iterates are vectors generated from a fixed-point mapping function. However, when considering the iterative solution of large-scale matrix equations, the iterates are low-rank matrices generated from a fix…
▽ More
Reduced rank extrapolation (RRE) is an acceleration method typically used to accelerate the iterative solution of nonlinear systems of equations using a fixed-point process. In this context, the iterates are vectors generated from a fixed-point mapping function. However, when considering the iterative solution of large-scale matrix equations, the iterates are low-rank matrices generated from a fixed-point process for which, generally, the mapping function changes in each iteration. To enable acceleration of the iterative solution for these problems, we propose two novel generalizations of RRE. First, we show how to effectively compute RRE for sequences of low-rank matrices. Second, we derive a formulation of RRE that is suitable for fixed-point processes for which the mapping function changes each iteration. We demonstrate the potential of the methods on several numerical examples involving the iterative solution of large-scale Lyapunov and Riccati matrix equations.
△ Less
Submitted 3 March, 2025; v1 submitted 13 February, 2025;
originally announced February 2025.
-
Inexact linear solves in the low-rank ADI iteration for large Sylvester equations
Authors:
Patrick Kürschner
Abstract:
We consider the low-rank alternating directions implicit (ADI) iteration for approximately solving large-scale algebraic Sylvester equations. Inside every iteration step of this iterative process a pair of linear systems of equations has to be solved. We investigate the situation when those inner linear systems are solved inexactly by an iterative methods such as, for example, preconditioned Krylo…
▽ More
We consider the low-rank alternating directions implicit (ADI) iteration for approximately solving large-scale algebraic Sylvester equations. Inside every iteration step of this iterative process a pair of linear systems of equations has to be solved. We investigate the situation when those inner linear systems are solved inexactly by an iterative methods such as, for example, preconditioned Krylov subspace methods. The main contribution of this work are thresholds for the required accuracies regarding the inner linear systems which dictate when the employed inner Krylov subspace methods can be safely terminated. The goal is to save computational effort by solving the inner linear system as inaccurate as possible without endangering the functionality of the low-rank Sylvester-ADI method. Ideally, the inexact ADI method mimics the convergence behaviour of the more expensive exact ADI method, where the linear systems are solved directly. Alongside the theoretical results, also strategies for an actual practical implementation of the stopping criteria are developed. Numerical experiments confirm the effectiveness of the proposed strategies.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
On the convergence of Krylov methods with low-rank truncations
Authors:
Davide Palitta,
Patrick Kürschner
Abstract:
Low-rank Krylov methods are one of the few options available in the literature to address the numerical solution of large-scale general linear matrix equations. These routines amount to well-known Krylov schemes that have been equipped with a couple of low-rank truncations to maintain a feasible storage demand in the overall solution procedure. However, such truncations may affect the convergence…
▽ More
Low-rank Krylov methods are one of the few options available in the literature to address the numerical solution of large-scale general linear matrix equations. These routines amount to well-known Krylov schemes that have been equipped with a couple of low-rank truncations to maintain a feasible storage demand in the overall solution procedure. However, such truncations may affect the convergence properties of the adopted Krylov method. In this paper we show how the truncation steps have to be performed in order to maintain the convergence of the Krylov routine. Several numerical experiments validate our theoretical findings.
△ Less
Submitted 25 January, 2020; v1 submitted 3 September, 2019;
originally announced September 2019.
-
Low-rank updates and divide-and-conquer methods for quadratic matrix equations
Authors:
Daniel Kressner,
Patrick Kürschner,
Stefano Massei
Abstract:
In this work, we consider two types of large-scale quadratic matrix equations: Continuous-time algebraic Riccati equations, which play a central role in optimal and robust control, and unilateral quadratic matrix equations, which arise from stochastic processes on 2D lattices and vibrating systems. We propose a simple and fast way to update the solution to such matrix equations under low-rank modi…
▽ More
In this work, we consider two types of large-scale quadratic matrix equations: Continuous-time algebraic Riccati equations, which play a central role in optimal and robust control, and unilateral quadratic matrix equations, which arise from stochastic processes on 2D lattices and vibrating systems. We propose a simple and fast way to update the solution to such matrix equations under low-rank modifications of the coefficients. Based on this procedure, we develop a divide-and-conquer method for quadratic matrix equations with coefficients that feature a specific type of hierarchical low-rank structure, which includes banded matrices. This generalizes earlier work on linear matrix equations. Numerical experiments indicate the advantages of our newly proposed method versus iterative schemes combined with hierarchical low-rank arithmetic.
△ Less
Submitted 6 March, 2019;
originally announced March 2019.
-
Numerical computation and new output bounds for time-limited balanced truncation of discrete-time systems
Authors:
Igor Pontes Duff,
Patrick Kürschner
Abstract:
In this paper, balancing based model order reduction (MOR) for large-scale linear discrete-time time-invariant systems in prescribed finite time intervals is studied. The first main topic is the development of error bounds regarding the approximated output vector within the time limits. The influence of different components in the established bounds will be highlighted. After that, the second part…
▽ More
In this paper, balancing based model order reduction (MOR) for large-scale linear discrete-time time-invariant systems in prescribed finite time intervals is studied. The first main topic is the development of error bounds regarding the approximated output vector within the time limits. The influence of different components in the established bounds will be highlighted. After that, the second part of the article proposes strategies that enable an efficient numerical execution of time-limited balanced truncation for large-scale systems. Numerical experiments illustrate the performance of the proposed techniques.
△ Less
Submitted 5 February, 2019;
originally announced February 2019.
-
Residual minimizing shift parameters for the low-rank ADI iteration
Authors:
Patrick Kürschner
Abstract:
The low-rank alternating directions implicit (LR-ADI) iteration is a frequently employed method for efficiently computing low-rank approximate solutions of large-scale Lyapunov equations. In order to achieve a rapid error reduction, the iteration requires shift parameters whose selection and generation is often a difficult task, especially for nonsymmetric coefficients in the Lyapunov equation. Th…
▽ More
The low-rank alternating directions implicit (LR-ADI) iteration is a frequently employed method for efficiently computing low-rank approximate solutions of large-scale Lyapunov equations. In order to achieve a rapid error reduction, the iteration requires shift parameters whose selection and generation is often a difficult task, especially for nonsymmetric coefficients in the Lyapunov equation. This article represents a follow up of Benner et al. [ETNA, 43 (2014-2015), pp. 142-162] and investigates self-generating shift parameters based on a minimization principle for the Lyapunov residual norm. Since the involved objective functions are too expensive to evaluate and, hence, intractable, compressed objective functions are introduced which are efficiently constructed from the available data generated by the LR-ADI iteration. Several numerical experiments indicate that these residual minimizing shifts using approximated objective functions outperform existing precomputed and dynamic shift parameter selection techniques, although their generation is more involved.
△ Less
Submitted 13 November, 2018;
originally announced November 2018.
-
A numerical comparison of solvers for large-scale, continuous-time algebraic Riccati equations and LQR problems
Authors:
Peter Benner,
Zvonimir Bujanović,
Patrick Kürschner,
Jens Saak
Abstract:
In this paper, we discuss numerical methods for solving large-scale continuous-time algebraic Riccati equations. These methods have been the focus of intensive research in recent years, and significant progress has been made in both the theoretical understanding and efficient implementation of various competing algorithms. There are several goals of this manuscript: first, to gather in one place a…
▽ More
In this paper, we discuss numerical methods for solving large-scale continuous-time algebraic Riccati equations. These methods have been the focus of intensive research in recent years, and significant progress has been made in both the theoretical understanding and efficient implementation of various competing algorithms. There are several goals of this manuscript: first, to gather in one place an overview of different approaches for solving large-scale Riccati equations, and to point to the recent advances in each of them. Second, to analyze and compare the main computational ingredients of these algorithms, to detect their strong points and their potential bottlenecks. And finally, to compare the effective implementations of all methods on a set of relevant benchmark examples, giving an indication of their relative performance.
△ Less
Submitted 10 April, 2020; v1 submitted 2 November, 2018;
originally announced November 2018.
-
Inexact methods for the low rank solution to large scale Lyapunov equations
Authors:
Patrick Kürschner,
Melina A. Freitag
Abstract:
The rational Krylov subspace method (RKSM) and the low-rank alternating directions implicit (LR-ADI) iteration are established numerical tools for computing low-rank solution factors of large-scale Lyapunov equations. In order to generate the basis vectors for the RKSM, or extend the low-rank factors within the LR-ADI method the repeated solution to a shifted linear system is necessary. For very l…
▽ More
The rational Krylov subspace method (RKSM) and the low-rank alternating directions implicit (LR-ADI) iteration are established numerical tools for computing low-rank solution factors of large-scale Lyapunov equations. In order to generate the basis vectors for the RKSM, or extend the low-rank factors within the LR-ADI method the repeated solution to a shifted linear system is necessary. For very large systems this solve is usually implemented using iterative methods, leading to inexact solves within this inner iteration. We derive theory for a relaxation strategy within these inexact solves, both for the RKSM and the LR-ADI method. Practical choices for relaxing the solution tolerance within the inner linear system are then provided. The theory is supported by several numerical examples.
△ Less
Submitted 6 May, 2019; v1 submitted 18 September, 2018;
originally announced September 2018.
-
Greedy low-rank algorithm for spatial connectome regression
Authors:
Patrick Kürschner,
Sergey Dolgov,
Kameron Decker Harris,
Peter Benner
Abstract:
Recovering brain connectivity from tract tracing data is an important computational problem in the neurosciences. Mesoscopic connectome reconstruction was previously formulated as a structured matrix regression problem (Harris et al., 2016), but existing techniques do not scale to the whole-brain setting. The corresponding matrix equation is challenging to solve due to large scale, ill-conditionin…
▽ More
Recovering brain connectivity from tract tracing data is an important computational problem in the neurosciences. Mesoscopic connectome reconstruction was previously formulated as a structured matrix regression problem (Harris et al., 2016), but existing techniques do not scale to the whole-brain setting. The corresponding matrix equation is challenging to solve due to large scale, ill-conditioning, and a general form that lacks a convergent splitting. We propose a greedy low-rank algorithm for connectome reconstruction problem in very high dimensions. The algorithm approximates the solution by a sequence of rank-one updates which exploit the sparse and positive definite problem structure. This algorithm was described previously (Kressner and Sirković, 2015) but never implemented for this connectome problem, leading to a number of challenges. We have had to design judicious stopping criteria and employ efficient solvers for the three main sub-problems of the algorithm, including an efficient GPU implementation that alleviates the main bottleneck for large datasets. The performance of the method is evaluated on three examples: an artificial "toy" dataset and two whole-cortex instances using data from the Allen Mouse Brain Connectivity Atlas. We find that the method is significantly faster than previous methods and that moderate ranks offer good approximation. This speedup allows for the estimation of increasingly large-scale connectomes across taxa as these data become available from tracing experiments. The data and code are available online.
△ Less
Submitted 1 November, 2019; v1 submitted 16 August, 2018;
originally announced August 2018.
-
An $\mathcal H_2$-Type Error Bound for Time-Limited Balanced Truncation
Authors:
Martin Redmann,
Patrick Kürschner
Abstract:
When solving partial differential equations numerically, usually a high order spatial discretization is needed. Model order reduction (MOR) techniques are often used to reduce the order of spatially-discretized systems and hence reduce computational complexity. A particular MOR technique to obtain a reduced order model (ROM) is balanced truncation (BT). However, if one aims at finding a good ROM o…
▽ More
When solving partial differential equations numerically, usually a high order spatial discretization is needed. Model order reduction (MOR) techniques are often used to reduce the order of spatially-discretized systems and hence reduce computational complexity. A particular MOR technique to obtain a reduced order model (ROM) is balanced truncation (BT). However, if one aims at finding a good ROM on a certain finite time interval only, time-limited BT (TLBT) can be a more accurate alternative. So far, no error bound on TLBT has been proved. In this paper, we close this gap in the theory by providing an $\mathcal H_2$ error bound for TLBT with two different representations. The performance of the error bound is then shown in several numerical experiments.
△ Less
Submitted 20 October, 2017;
originally announced October 2017.
-
Balanced truncation model order reduction in limited time intervals for large systems
Authors:
Patrick Kürschner
Abstract:
In this article we investigate model order reduction of large-scale systems using time-limited balanced truncation, which restricts the well known balanced truncation framework to prescribed finite time intervals. The main emphasis is on the efficient numerical realization of this model reduction approach in case of large system dimensions. We discuss numerical methods to deal with the resulting m…
▽ More
In this article we investigate model order reduction of large-scale systems using time-limited balanced truncation, which restricts the well known balanced truncation framework to prescribed finite time intervals. The main emphasis is on the efficient numerical realization of this model reduction approach in case of large system dimensions. We discuss numerical methods to deal with the resulting matrix exponential functions and Lyapunov equations which are solved for low-rank approximations. Our main tool for this purpose are rational Krylov subspace methods. We also discuss the eigenvalue decay and numerical rank of the solutions of the Lyapunov equations. These results, and also numerical experiments, will show that depending on the final time horizon, the numerical rank of the Lyapunov solutions in time-limited balanced truncation can be smaller compared to standard balanced truncation. In numerical experiments we test the approaches for computing low-rank factors of the involved Lyapunov solutions and illustrate that time-limited balanced truncation can generate reduced order models having a higher accuracy in the considered time region.
△ Less
Submitted 4 January, 2018; v1 submitted 10 July, 2017;
originally announced July 2017.
-
Combined Error Estimates for Local Fluctuations of SPDEs
Authors:
Christian Kuehn,
Patrick Kuerschner
Abstract:
In this work, we study the numerical approximation of local fluctuations of certain classes of parabolic stochastic partial differential equations (SPDEs). Our focus is on effects for small spatially-correlated noise on a time scale before large deviation effects have occurred. In particular, we are interested in the local directions of the noise described by a covariance operator. We introduce a…
▽ More
In this work, we study the numerical approximation of local fluctuations of certain classes of parabolic stochastic partial differential equations (SPDEs). Our focus is on effects for small spatially-correlated noise on a time scale before large deviation effects have occurred. In particular, we are interested in the local directions of the noise described by a covariance operator. We introduce a new strategy and prove a Combined ERror EStimate (CERES) for the five main errors: the spatial discretization error, the local linearization error, the noise truncation error, the local relaxation error to steady state, and the approximation error via an iterative low-rank matrix algorithm. In summary, we obtain one CERES describing, apart from modelling of the original equations and standard round-off, all sources of error for a local fluctuation analysis of an SPDE in one estimate. To prove our results, we rely on a combination of methods from optimal Galerkin approximation of SPDEs, covariance moment estimates, analytical techniques for Lyapunov equations, iterative numerical schemes for low-rank solution of Lyapunov equations, and working with related spectral norms for different classes of operators.
△ Less
Submitted 20 February, 2019; v1 submitted 14 November, 2016;
originally announced November 2016.
-
GMRES convergence bounds for eigenvalue problems
Authors:
Melina Freitag,
Patrick Kürschner,
Jennifer Pestana
Abstract:
The convergence of GMRES for solving linear systems can be influenced heavily by the structure of the right hand side. Within the solution of eigenvalue problems via inverse iteration or subspace iteration, the right hand side is generally related to an approximate invariant subspace of the linear system. We give detailed and new bounds on (block) GMRES that take the special behavior of the right…
▽ More
The convergence of GMRES for solving linear systems can be influenced heavily by the structure of the right hand side. Within the solution of eigenvalue problems via inverse iteration or subspace iteration, the right hand side is generally related to an approximate invariant subspace of the linear system. We give detailed and new bounds on (block) GMRES that take the special behavior of the right hand side into account and explain the initial sharp decrease of the GMRES residual. The bounds give rise to adapted preconditioners applied to the eigenvalue problems, e.g. tuned and polynomial preconditioners. The numerical results show that the new (block) GMRES bounds are much sharper than conventional bounds and that preconditioned subspace iteration with either a tuned or polynomial preconditioner should be used in practice.
△ Less
Submitted 30 May, 2017; v1 submitted 20 September, 2016;
originally announced September 2016.
-
RADI: A low-rank ADI-type algorithm for large scale algebraic Riccati equations
Authors:
Peter Benner,
Zvonimir Bujanović,
Patrick Kürschner,
Jens Saak
Abstract:
This paper introduces a new algorithm for solving large-scale continuous-time algebraic Riccati equations (CARE). The advantage of the new algorithm is in its immediate and efficient low-rank formulation, which is a generalization of the Cholesky-factored variant of the Lyapunov ADI method. We discuss important implementation aspects of the algorithm, such as reducing the use of complex arithmetic…
▽ More
This paper introduces a new algorithm for solving large-scale continuous-time algebraic Riccati equations (CARE). The advantage of the new algorithm is in its immediate and efficient low-rank formulation, which is a generalization of the Cholesky-factored variant of the Lyapunov ADI method. We discuss important implementation aspects of the algorithm, such as reducing the use of complex arithmetic and shift selection strategies. We show that there is a very tight relation between the new algorithm and three other algorithms for CARE previously known in the literature -- all of these seemingly different methods in fact produce exactly the same iterates when used with the same parameters: they are algorithmically different descriptions of the same approximation sequence to the Riccati solution.
△ Less
Submitted 9 October, 2016; v1 submitted 30 September, 2015;
originally announced October 2015.
