-
A general error analysis for randomized low-rank approximation with application to data assimilation
Authors:
Alexandre Scotto Di Perrotolo,
Youssef Diouane,
Selime Gürol,
Xavier Vasseur
Abstract:
Randomized algorithms have proven to perform well on a large class of numerical linear algebra problems. Their theoretical analysis is critical to provide guarantees on their behaviour, and in this sense, the stochastic analysis of the randomized low-rank approximation error plays a central role. Indeed, several randomized methods for the approximation of dominant eigen- or singular modes can be r…
▽ More
Randomized algorithms have proven to perform well on a large class of numerical linear algebra problems. Their theoretical analysis is critical to provide guarantees on their behaviour, and in this sense, the stochastic analysis of the randomized low-rank approximation error plays a central role. Indeed, several randomized methods for the approximation of dominant eigen- or singular modes can be rewritten as low-rank approximation methods. However, despite the large variety of algorithms, the existing theoretical frameworks for their analysis rely on a specific structure for the covariance matrix that is not adapted to all the algorithms. We propose a general framework for the stochastic analysis of the low-rank approximation error in Frobenius norm for centered and non-standard Gaussian matrices. Under minimal assumptions on the covariance matrix, we derive accurate bounds both in expectation and probability. Our bounds have clear interpretations that enable us to derive properties and motivate practical choices for the covariance matrix resulting in efficient low-rank approximation algorithms. The most commonly used bounds in the literature have been demonstrated as a specific instance of the bounds proposed here, with the additional contribution of being tighter. Numerical experiments related to data assimilation further illustrate that exploiting the problem structure to select the covariance matrix improves the performance as suggested by our bounds.
△ Less
Submitted 8 May, 2024;
originally announced May 2024.
-
A general error analysis for randomized low-rank approximation methods
Authors:
Youssef Diouane,
Selime Gürol,
Alexandre Scotto Di Perrotolo,
Xavier Vasseur
Abstract:
We propose a general error analysis related to the low-rank approximation of a given real matrix in both the spectral and Frobenius norms. First, we derive deterministic error bounds that hold with some minimal assumptions. Second, we derive error bounds in expectation in the non-standard Gaussian case, assuming a non-trivial mean and a general covariance matrix for the random matrix variable. The…
▽ More
We propose a general error analysis related to the low-rank approximation of a given real matrix in both the spectral and Frobenius norms. First, we derive deterministic error bounds that hold with some minimal assumptions. Second, we derive error bounds in expectation in the non-standard Gaussian case, assuming a non-trivial mean and a general covariance matrix for the random matrix variable. The proposed analysis generalizes and improves the error bounds for spectral and Frobenius norms proposed by Halko, Martinsson and Tropp. Third, we consider the Randomized Singular Value Decomposition and specialize our error bounds in expectation in this setting. Numerical experiments on an instructional synthetic test case demonstrate the tightness of the new error bounds.
△ Less
Submitted 20 June, 2022; v1 submitted 17 June, 2022;
originally announced June 2022.
-
Numerical approximation of port-Hamiltonian systems for hyperbolic or parabolic PDEs with boundary control
Authors:
Andrea Brugnoli,
Ghislain Haine,
Anass Serhani,
Xavier Vasseur
Abstract:
We consider the design of structure-preserving discretization methods for the solution of systems of boundary controlled Partial Differential Equations (PDEs) thanks to the port-Hamiltonian formalism. We first provide a novel general structure of infinite-dimensional port-Hamiltonian systems (pHs) for which the Partitioned Finite Element Method (PFEM) straightforwardly applies. The proposed strate…
▽ More
We consider the design of structure-preserving discretization methods for the solution of systems of boundary controlled Partial Differential Equations (PDEs) thanks to the port-Hamiltonian formalism. We first provide a novel general structure of infinite-dimensional port-Hamiltonian systems (pHs) for which the Partitioned Finite Element Method (PFEM) straightforwardly applies. The proposed strategy is applied to abstract multidimensional linear hyperbolic and parabolic systems of PDEs. Then we show that instructional model problems based on the wave equation, Mindlin equation and heat equation fit within this unified framework. Secondly we introduce the ongoing project SCRIMP (Simulation and ContRol of Interactions in Multi-Physics) developed for the numerical simulation of infinite-dimensional pHs. SCRIMP notably relies on the FEniCS open-source computing platform for the finite element spatial discretization. Finally, we illustrate how to solve the considered model problems within this framework by carefully explaining the methodology. As additional support, companion interactive Jupyter notebooks are available.
△ Less
Submitted 29 September, 2020; v1 submitted 16 July, 2020;
originally announced July 2020.
-
On the iterative solution of systems of the form $A^T A x=A^Tb+c$
Authors:
Henri Calandra,
Serge Gratton,
Elisa Riccietti,
Xavier Vasseur
Abstract:
Given a full column rank matrix $A \in \mathbb{R}^{m\times n}$ ($m\geq n$), we consider a special class of linear systems of the form $A^\top Ax=A^\top b+c$ with $x, c \in \mathbb{R}^{n}$ and $b \in \mathbb{R}^{m}$. The occurrence of $c$ in the right-hand side of the equation prevents the direct application of standard methods for least squares problems. Hence, we investigate alternative solution…
▽ More
Given a full column rank matrix $A \in \mathbb{R}^{m\times n}$ ($m\geq n$), we consider a special class of linear systems of the form $A^\top Ax=A^\top b+c$ with $x, c \in \mathbb{R}^{n}$ and $b \in \mathbb{R}^{m}$. The occurrence of $c$ in the right-hand side of the equation prevents the direct application of standard methods for least squares problems. Hence, we investigate alternative solution methods that, as in the case of normal equations, take advantage of the peculiar structure of the system to avoid unstable computations, such as forming $A^\top A$ explicitly. We propose two iterative methods that are based on specific reformulations of the problem and we provide explicit closed formulas for the structured condition number related to each problem. These formula allow us to compute a more accurate estimate of the forward error than the standard one used for generic linear systems, that does not take into account the structure of the perturbations. The relevance of our estimates is shown on a set of synthetic test problems. Numerical experiments highlight both the increased robustness and accuracy of the proposed methods compared to the standard conjugate gradient method. It is also found that the new methods can compare to standard direct methods in terms of solution accuracy.
△ Less
Submitted 31 October, 2019;
originally announced November 2019.
-
On high-order multilevel optimization strategies
Authors:
Henri Calandra,
Serge Gratton,
Elisa Riccietti,
Xavier Vasseur
Abstract:
We propose a new family of multilevel methods for unconstrained minimization. The resulting strategies are multilevel extensions of high-order optimization methods based on q-order Taylor models (with q >= 1) that have been recently proposed in the literature. The use of high-order models, while decreasing the worst-case complexity bound, makes these methods computationally more expensive. Hence,…
▽ More
We propose a new family of multilevel methods for unconstrained minimization. The resulting strategies are multilevel extensions of high-order optimization methods based on q-order Taylor models (with q >= 1) that have been recently proposed in the literature. The use of high-order models, while decreasing the worst-case complexity bound, makes these methods computationally more expensive. Hence, to counteract this effect, we propose a multilevel strategy that exploits a hierarchy of problems of decreasing dimension, still approximating the original one, to reduce the global cost of the step computation. A theoretical analysis of the family of methods is proposed. Specifically, local and global convergence results are proved and a complexity bound to reach first order stationary points is also derived. A multilevel version of the well known adaptive method based on cubic regularization (ARC, corresponding to q = 2 in our setting) has been implemented. Numerical experiments clearly highlight the relevance of the new multilevel approach leading to considerable computational savings in terms of floating point operations compared to the classical one-level strategy.
△ Less
Submitted 9 April, 2019;
originally announced April 2019.
-
On the approximation of the solution of partial differential equations by artificial neural networks trained by a multilevel Levenberg-Marquardt method
Authors:
Henri Calandra,
Serge Gratton,
Elisa Riccietti,
Xavier Vasseur
Abstract:
This paper is concerned with the approximation of the solution of partial differential equations by means of artificial neural networks. Here a feedforward neural network is used to approximate the solution of the partial differential equation. The learning problem is formulated as a least squares problem, choosing the residual of the partial differential equation as a loss function, whereas a mul…
▽ More
This paper is concerned with the approximation of the solution of partial differential equations by means of artificial neural networks. Here a feedforward neural network is used to approximate the solution of the partial differential equation. The learning problem is formulated as a least squares problem, choosing the residual of the partial differential equation as a loss function, whereas a multilevel Levenberg-Marquardt method is employed as a training method. This setting allows us to get further insight into the potential of multilevel methods. Indeed, when the least squares problem arises from the training of artificial neural networks, the variables subject to optimization are not related by any geometrical constraints and the standard interpolation and restriction operators cannot be employed any longer. A heuristic, inspired by algebraic multigrid methods, is then proposed to construct the multilevel transfer operators. Numerical experiments show encouraging results related to the efficiency of the new multilevel optimization method for the training of artificial neural networks, compared to the standard corresponding one-level procedure.
△ Less
Submitted 9 April, 2019;
originally announced April 2019.
-
Deflation and augmentation techniques in Krylov subspace methods for the solution of linear systems
Authors:
Olivier Coulaud,
Luc Giraud,
Pierre Ramet,
Xavier Vasseur
Abstract:
In this paper we present deflation and augmentation techniques that have been designed to accelerate the convergence of Krylov subspace methods for the solution of linear systems of equations. We review numerical approaches both for linear systems with a non-Hermitian coefficient matrix, mainly within the Arnoldi framework, and for Hermitian positive definite problems with the conjugate gradient m…
▽ More
In this paper we present deflation and augmentation techniques that have been designed to accelerate the convergence of Krylov subspace methods for the solution of linear systems of equations. We review numerical approaches both for linear systems with a non-Hermitian coefficient matrix, mainly within the Arnoldi framework, and for Hermitian positive definite problems with the conjugate gradient method.
△ Less
Submitted 21 March, 2013;
originally announced March 2013.
