-
On the Convergence of CROP-Anderson Acceleration Method
Authors:
Ning Wan,
Agnieszka Międlar
Abstract:
Anderson Acceleration is a well-established method that allows to speed up or encourage convergence of fixed-point iterations. It has been successfully used in a variety of applications, in particular within the Self-Consistent Field (SCF) iteration method for quantum chemistry and physics computations. In recent years, the Conjugate Residual with OPtimal trial vectors (CROP) algorithm was introdu…
▽ More
Anderson Acceleration is a well-established method that allows to speed up or encourage convergence of fixed-point iterations. It has been successfully used in a variety of applications, in particular within the Self-Consistent Field (SCF) iteration method for quantum chemistry and physics computations. In recent years, the Conjugate Residual with OPtimal trial vectors (CROP) algorithm was introduced and shown to have a better performance than the classical Anderson Acceleration with less storage needed. This paper aims to delve into the intricate connections between the classical Anderson Acceleration method and the CROP algorithm. Our objectives include a comprehensive study of their convergence properties, explaining the underlying relationships, and substantiating our findings through some numerical examples. Through this exploration, we contribute valuable insights that can enhance the understanding and application of acceleration methods in practical computations, as well as the developments of new and more efficient acceleration schemes.
△ Less
Submitted 4 October, 2024;
originally announced October 2024.
-
Randomized low-rank approximations beyond Gaussian random matrices
Authors:
Arvind K. Saibaba,
Agnieszka Międlar
Abstract:
This paper expands the analysis of randomized low-rank approximation beyond the Gaussian distribution to four classes of random matrices: (1) independent sub-Gaussian entries, (2) independent sub-Gaussian columns, (3) independent bounded columns, and (4) independent columns with bounded second moment. Using a novel interpretation of the low-rank approximation error involving sample covariance matr…
▽ More
This paper expands the analysis of randomized low-rank approximation beyond the Gaussian distribution to four classes of random matrices: (1) independent sub-Gaussian entries, (2) independent sub-Gaussian columns, (3) independent bounded columns, and (4) independent columns with bounded second moment. Using a novel interpretation of the low-rank approximation error involving sample covariance matrices, we provide insight into the requirements of a \textit{good random matrix} for the purpose of randomized low-rank approximation. Although our bounds involve unspecified absolute constants (a consequence of the underlying non-asymptotic theory of random matrices), they allow for qualitative comparisons across distributions. The analysis offers some details on the minimal number of samples (the number of columns $\ell$ of the random matrix $\boldsymbolΩ$) and the error in the resulting low-rank approximation. We illustrate our analysis in the context of the randomized subspace iteration method as a representative algorithm for low-rank approximation, however, all the results are broadly applicable to other low-rank approximation techniques. We conclude our discussion with numerical examples using both synthetic and real-world test matrices.
△ Less
Submitted 10 August, 2023;
originally announced August 2023.
-
A \texttt{Matlab} Toolbox for the Regularization of Descriptor Systems Arising from Generalized Realization Procedures
Authors:
A. Binder,
V. Mehrmann,
A. Miedlar,
P. Schulze
Abstract:
In this report we introduce a \texttt{Matlab} toolbox for the regularization of descriptor systems. We apply it, in particular, for systems resulting from the generalized realization procedure of \cite{MayA07}, which generates, via rational interpolation techniques, a linear \emph{descriptor system} from interpolation data. The resulting system needs to be regularized to make it feasible for the u…
▽ More
In this report we introduce a \texttt{Matlab} toolbox for the regularization of descriptor systems. We apply it, in particular, for systems resulting from the generalized realization procedure of \cite{MayA07}, which generates, via rational interpolation techniques, a linear \emph{descriptor system} from interpolation data. The resulting system needs to be regularized to make it feasible for the use in simulation, optimization, and control. This process is called \emph{regularization}
△ Less
Submitted 5 December, 2022;
originally announced December 2022.
-
Modifying the Asynchronous Jacobi Method for Data Corruption Resilience
Authors:
Christopher J. Vogl,
Zachary Atkins,
Alyson Fox,
Agnieszka Miedlar,
Colin Ponce
Abstract:
Moving scientific computation from high-performance computing (HPC) and cloud computing (CC) environments to devices on the edge, i.e., physically near instruments of interest, has received tremendous interest in recent years. Such edge computing environments can operate on data in-situ, offering enticing benefits over data aggregation to HPC and CC facilities that include avoiding costs of transm…
▽ More
Moving scientific computation from high-performance computing (HPC) and cloud computing (CC) environments to devices on the edge, i.e., physically near instruments of interest, has received tremendous interest in recent years. Such edge computing environments can operate on data in-situ, offering enticing benefits over data aggregation to HPC and CC facilities that include avoiding costs of transmission, increased data privacy, and real-time data analysis. Because of the inherent unreliability of edge computing environments, new fault tolerant approaches must be developed before the benefits of edge computing can be realized. Motivated by algorithm-based fault tolerance, a variant of the asynchronous Jacobi (ASJ) method is developed that achieves resilience to data corruption by rejecting solution approximations from neighbor devices according to a bound derived from convergence theory. Numerical results on a two-dimensional Poisson problem show the new rejection criterion, along with a novel approximation to the shortest path length on which the criterion depends, restores convergence for the ASJ variant in the presence of certain types data corruption. Numerical results are obtained for when the singular values in the analytic bound are approximated. A linear system with a more dense sparsity pattern is also explored. All results indicate that successful resilience to data corruption depends on whether the bound tightens fast enough to reject corrupted data before the iteration evolution deviates significantly from that predicted by the convergence theory defining the bound. This observation generalizes to future work on algorithm-based fault tolerance for other asynchronous algorithms, including upcoming approaches that leverage Krylov subspaces.
△ Less
Submitted 25 April, 2024; v1 submitted 16 June, 2022;
originally announced June 2022.
-
Randomized algorithms for rounding in the Tensor-Train format
Authors:
Hussam Al Daas,
Grey Ballard,
Paul Cazeaux,
Eric Hallman,
Agnieszka Miedlar,
Mirjeta Pasha,
Tim W. Reid,
Arvind K. Saibaba
Abstract:
The Tensor-Train (TT) format is a highly compact low-rank representation for high-dimensional tensors. TT is particularly useful when representing approximations to the solutions of certain types of parametrized partial differential equations. For many of these problems, computing the solution explicitly would require an infeasible amount of memory and computational time. While the TT format makes…
▽ More
The Tensor-Train (TT) format is a highly compact low-rank representation for high-dimensional tensors. TT is particularly useful when representing approximations to the solutions of certain types of parametrized partial differential equations. For many of these problems, computing the solution explicitly would require an infeasible amount of memory and computational time. While the TT format makes these problems tractable, iterative techniques for solving the PDEs must be adapted to perform arithmetic while maintaining the implicit structure. The fundamental operation used to maintain feasible memory and computational time is called rounding, which truncates the internal ranks of a tensor already in TT format. We propose several randomized algorithms for this task that are generalizations of randomized low-rank matrix approximation algorithms and provide significant reduction in computation compared to deterministic TT-rounding algorithms. Randomization is particularly effective in the case of rounding a sum of TT-tensors (where we observe 20x speedup), which is the bottleneck computation in the adaptation of GMRES to vectors in TT format. We present the randomized algorithms and compare their empirical accuracy and computational time with deterministic alternatives.
△ Less
Submitted 8 October, 2021;
originally announced October 2021.
-
Distribution System Voltage Prediction from Smart Inverters using Decentralized Regression
Authors:
Zachary R. Atkins,
Christopher J. Vogl,
Achintya Madduri,
Nan Duan,
Agnieszka K. Miedlar,
Daniel Merl
Abstract:
As photovoltaic (PV) penetration continues to rise and smart inverter functionality continues to expand, smart inverters and other distributed energy resources (DERs) will play increasingly important roles in distribution system power management and security. In this paper, it is demonstrated that a constellation of smart inverters in a simulated distribution circuit can enable precise voltage pre…
▽ More
As photovoltaic (PV) penetration continues to rise and smart inverter functionality continues to expand, smart inverters and other distributed energy resources (DERs) will play increasingly important roles in distribution system power management and security. In this paper, it is demonstrated that a constellation of smart inverters in a simulated distribution circuit can enable precise voltage predictions using an asynchronous and decentralized prediction algorithm. Using simulated data and a constellation of 15 inverters in a ring communication topology, the COLA algorithm is shown to accomplish the learning task required for voltage magnitude prediction with far less communication overhead than fully connected P2P learning protocols. Additionally, a dynamic stopping criterion is proposed that does not require a regularizer like the original COLA stopping criterion.
△ Less
Submitted 12 January, 2021;
originally announced January 2021.
-
A rational approximation method for solving acoustic nonlinear eigenvalue problems
Authors:
Mohamed El-Guide,
Agnieszka Miedlar,
Yousef Saad
Abstract:
We present two approximation methods for computing eigenfrequencies and eigenmodes of large-scale nonlinear eigenvalue problems resulting from boundary element method (BEM) solutions of some types of acoustic eigenvalue problems in three-dimensional space. The main idea of the first method is to approximate the resulting boundary element matrix within a contour in the complex plane by a high accur…
▽ More
We present two approximation methods for computing eigenfrequencies and eigenmodes of large-scale nonlinear eigenvalue problems resulting from boundary element method (BEM) solutions of some types of acoustic eigenvalue problems in three-dimensional space. The main idea of the first method is to approximate the resulting boundary element matrix within a contour in the complex plane by a high accuracy rational approximation using the Cauchy integral formula. The second method is based on the Chebyshev interpolation within real intervals. A Rayleigh-Ritz procedure, which is suitable for parallelization is developed for both the Cauchy and the Chebyshev approximation methods when dealing with large-scale practical applications. The performance of the proposed methods is illustrated with a variety of benchmark examples and large-scale industrial applications with degrees of freedom varying from several hundred up to around two million.
△ Less
Submitted 7 June, 2019;
originally announced June 2019.
-
A rational approximation method for the nonlinear eigenvalue problem
Authors:
Yousef Saad,
Mohamed El-Guide,
Agnieszka Międlar
Abstract:
This paper presents a method for computing eigenvalues and eigenvectors for some types of nonlinear eigenvalue problems. The main idea is to approximate the functions involved in the eigenvalue problem by rational functions and then apply a form of linearization. Eigenpairs of the expanded form of this linearization are not extracted directly. Instead, its structure is exploited to develop a schem…
▽ More
This paper presents a method for computing eigenvalues and eigenvectors for some types of nonlinear eigenvalue problems. The main idea is to approximate the functions involved in the eigenvalue problem by rational functions and then apply a form of linearization. Eigenpairs of the expanded form of this linearization are not extracted directly. Instead, its structure is exploited to develop a scheme that allows to extract all eigenvalues in a certain region of the complex plane by solving an eigenvalue problem of much smaller dimension. Because of its simple implementation and the ability to work efficiently in large dimensions, the presented method is appealing when solving challenging engineering problems. A few theoretical results are established to explain why the new approach works and numerical experiments are presented to validate the proposed algorithm.
△ Less
Submitted 9 June, 2020; v1 submitted 4 January, 2019;
originally announced January 2019.
-
FEAST Eigensolver for Nonlinear Eigenvalue Problems
Authors:
Brendan Gavin,
Agnieszka Międlar,
Eric Polizzi
Abstract:
The linear FEAST algorithm is a method for solving linear eigenvalue problems. It uses complex contour integration to calculate the eigenvectors whose eigenvalues that are located inside some user-defined region in the complex plane. This makes it possible to parallelize the process of solving eigenvalue problems by simply dividing the complex plane into a collection of disjoint regions and calcul…
▽ More
The linear FEAST algorithm is a method for solving linear eigenvalue problems. It uses complex contour integration to calculate the eigenvectors whose eigenvalues that are located inside some user-defined region in the complex plane. This makes it possible to parallelize the process of solving eigenvalue problems by simply dividing the complex plane into a collection of disjoint regions and calculating the eigenpairs in each region independently of the eigenpairs in the other regions. In this paper we present a generalization of the linear FEAST algorithm that can be used to solve nonlinear eigenvalue problems. Like its linear progenitor, the nonlinear FEAST algorithm can be used to solve nonlinear eigenvalue problems for the eigenpairs whose eigenvalues lie in a user-defined region in the complex plane, thereby allowing for the calculation of large numbers of eigenpairs in parallel. We describe the nonlinear FEAST algorithm, and use several physically-motivated examples to demonstrate its properties.
△ Less
Submitted 29 January, 2018;
originally announced January 2018.
