-
Accelerating a restarted Krylov method for matrix functions with randomization
Authors:
Nicolas L. Guidotti,
Per-Gunnar Martinsson,
Juan A. Acebrón,
José Monteiro
Abstract:
Many scientific applications require the evaluation of the action of the matrix function over a vector and the most common methods for this task are those based on the Krylov subspace. Since the orthogonalization cost and memory requirement can quickly become overwhelming as the basis grows, the Krylov method is often restarted after a few iterations. This paper proposes a new acceleration techniq…
▽ More
Many scientific applications require the evaluation of the action of the matrix function over a vector and the most common methods for this task are those based on the Krylov subspace. Since the orthogonalization cost and memory requirement can quickly become overwhelming as the basis grows, the Krylov method is often restarted after a few iterations. This paper proposes a new acceleration technique for restarted Krylov methods based on randomization. The numerical experiments show that the randomized method greatly outperforms the classical approach with the same level of accuracy. In fact, randomization can actually improve the convergence rate of restarted methods in some cases. The paper also compares the performance and stability of the randomized methods proposed so far for solving very large finite element problems, complementing the numerical analyses from previous studies.
△ Less
Submitted 10 April, 2025; v1 submitted 28 March, 2025;
originally announced March 2025.
-
Sensitivity analysis of fractional linear systems based on random walks with negligible memory usage
Authors:
Andrés Centeno,
Juan A. Acebrón,
José Monteiro
Abstract:
A random walk-based method is proposed to efficiently compute the solution of a large class of fractional in time linear systems of differential equations (linear F-ODE systems), along with the derivatives with respect to the system parameters. Such a method is unbiased and unconditionally stable, and can therefore be used to provide an unbiased estimation of individual entries of the solution, or…
▽ More
A random walk-based method is proposed to efficiently compute the solution of a large class of fractional in time linear systems of differential equations (linear F-ODE systems), along with the derivatives with respect to the system parameters. Such a method is unbiased and unconditionally stable, and can therefore be used to provide an unbiased estimation of individual entries of the solution, or the full solution. By using stochastic differentiation techniques, it can be used as well to provide unbiased estimators of the sensitivities of the solution with respect to the problem parameters without any additional computational cost. The time complexity of the algorithm is discussed here, along with suitable variance bounds, which prove in practice the convergence of the algorithm. Finally, several test cases were run to assess the validity of the algorithm.
△ Less
Submitted 8 August, 2024;
originally announced August 2024.
-
A new parallel solver suited for arbitrary semilinear parabolic partial differential equations based on generalized random trees
Authors:
Juan A. Acebron,
Angel Rodriguez-Rozas
Abstract:
A probabilistic representation for initial value semilinear parabolic problems based on generalized random trees has been derived. Two different strategies have been proposed, both requiring generating suitable random trees combined with a Pade approximant for approximating accurately a given divergent series. Such series are obtained by summing the partial contribution to the solution coming from…
▽ More
A probabilistic representation for initial value semilinear parabolic problems based on generalized random trees has been derived. Two different strategies have been proposed, both requiring generating suitable random trees combined with a Pade approximant for approximating accurately a given divergent series. Such series are obtained by summing the partial contribution to the solution coming from trees with arbitrary number of branches. The new representation greatly expands the class of problems amenable to be solved probabilistically, and was used successfully to develop a generalized probabilistic domain decomposition method. Such a method has been shown to be suited for massively parallel computers, enjoying full scalability and fault tolerance. Finally, a few numerical examples are given to illustrate the remarkable performance of the algorithm, comparing the results with those obtained with a classical method.
△ Less
Submitted 9 February, 2024;
originally announced February 2024.
-
Parallelization Strategies for the Randomized Kaczmarz Algorithm on Large-Scale Dense Systems
Authors:
Inês Ferreira,
Juan A. Acebrón,
José Monteiro
Abstract:
The Kaczmarz algorithm is an iterative technique designed to solve consistent linear systems of equations. It falls within the category of row-action methods, focusing on handling one equation per iteration. This characteristic makes it especially useful in solving very large systems. The recent introduction of a randomized version, the Randomized Kaczmarz method, renewed interest in the algorithm…
▽ More
The Kaczmarz algorithm is an iterative technique designed to solve consistent linear systems of equations. It falls within the category of row-action methods, focusing on handling one equation per iteration. This characteristic makes it especially useful in solving very large systems. The recent introduction of a randomized version, the Randomized Kaczmarz method, renewed interest in the algorithm, leading to the development of numerous variations. Subsequently, parallel implementations for both the original and Randomized Kaczmarz method have since then been proposed. However, previous work has addressed sparse linear systems, whereas we focus on solving dense systems. In this paper, we explore in detail approaches to parallelizing the Kaczmarz method for both shared and distributed memory for large dense systems. In particular, we implemented the Randomized Kaczmarz with Averaging (RKA) method that, for inconsistent systems, unlike the standard Randomized Kaczmarz algorithm, reduces the final error of the solution. While efficient parallelization of this algorithm is not achievable, we introduce a block version of the averaging method that can outperform the RKA method.
△ Less
Submitted 30 January, 2024;
originally announced January 2024.
-
Survey of a Class of Iterative Row-Action Methods: The Kaczmarz Method
Authors:
Inês A. Ferreira,
Juan A. Acebrón,
José Monteiro
Abstract:
The Kaczmarz algorithm is an iterative method that solves linear systems of equations. It stands out among iterative algorithms when dealing with large systems for two reasons. First, at each iteration, the Kaczmarz algorithm uses a single equation, resulting in minimal computational work per iteration. Second, solving the entire system may only require the use of a small subset of the equations.…
▽ More
The Kaczmarz algorithm is an iterative method that solves linear systems of equations. It stands out among iterative algorithms when dealing with large systems for two reasons. First, at each iteration, the Kaczmarz algorithm uses a single equation, resulting in minimal computational work per iteration. Second, solving the entire system may only require the use of a small subset of the equations. These characteristics have attracted significant attention to the Kaczmarz algorithm. Researchers have observed that randomly choosing equations can improve the convergence rate of the algorithm. This insight led to the development of the Randomized Kaczmarz algorithm and, subsequently, several other variations emerged. In this paper, we extensively analyze the native Kaczmarz algorithm and many of its variations using large-scale dense random systems as benchmarks. Through our investigation, we have verified that, for consistent systems, various row sampling schemes can outperform both the original and Randomized Kaczmarz method. Specifically, sampling without replacement and using quasirandom numbers are the fastest techniques. However, for inconsistent systems, the Conjugate Gradient method for Least-Squares problems overcomes all variations of the Kaczmarz method for these types of systems.
△ Less
Submitted 9 April, 2024; v1 submitted 5 January, 2024;
originally announced January 2024.
-
A hybrid probabilistic domain decomposition algorithm suited for very large-scale elliptic PDEs
Authors:
Francisco Bernal,
Jorge Morón-Vidal,
Juan A. Acebrón
Abstract:
State of the art domain decomposition algorithms for large-scale boundary value problems (with $M\gg 1$ degrees of freedom) suffer from bounded strong scalability because they involve the synchronisation and communication of workers inherent to iterative linear algebra. Here, we introduce PDDSparse, a different approach to scientific supercomputing which relies on a "Feynman-Kac formula for domain…
▽ More
State of the art domain decomposition algorithms for large-scale boundary value problems (with $M\gg 1$ degrees of freedom) suffer from bounded strong scalability because they involve the synchronisation and communication of workers inherent to iterative linear algebra. Here, we introduce PDDSparse, a different approach to scientific supercomputing which relies on a "Feynman-Kac formula for domain decomposition". Concretely, the interfacial values (only) are determined by a stochastic, highly sparse linear system $G(ω){\vec u}={\vec b}(ω)$ of size ${\cal O}(\sqrt{M})$, whose coefficients are constructed with Monte Carlo simulations-hence embarrassingly in parallel. In addition to a wider scope for strong scalability in the deep supercomputing regime, PDDSparse has built-in fault tolerance and is ideally suited for GPUs. A proof of concept example with up to 1536 cores is discussed in detail.
△ Less
Submitted 13 January, 2023;
originally announced January 2023.
-
A Monte Carlo method for computing the action of a matrix exponential on a vector
Authors:
Juan A. Acebron
Abstract:
A Monte Carlo method for computing the action of a matrix exponential for a certain class of matrices on a vector is proposed. The method is based on generating random paths, which evolve through the indices of the matrix, governed by a given continuous-time Markov chain. The vector solution is computed probabilistically by averaging over a suitable multiplicative functional. This representation e…
▽ More
A Monte Carlo method for computing the action of a matrix exponential for a certain class of matrices on a vector is proposed. The method is based on generating random paths, which evolve through the indices of the matrix, governed by a given continuous-time Markov chain. The vector solution is computed probabilistically by averaging over a suitable multiplicative functional. This representation extends the existing linear algebra Monte Carlo-based methods, and was used in practice to develop an efficient algorithm capable of computing both, a single entry or the full vector solution. Finally, several relevant benchmarks were executed to assess the performance of the algorithm. A comparison with the results obtained with a Krylov-based method shows the remarkable performance of the algorithm for solving large-scale problems.
△ Less
Submitted 18 June, 2019; v1 submitted 29 April, 2019;
originally announced April 2019.
-
A highly parallel algorithm for computing the action of a matrix exponential on a vector based on a multilevel Monte Carlo method
Authors:
Juan A. Acebron,
Jose R. Herrero,
Jose Monteiro
Abstract:
A novel algorithm for computing the action of a matrix exponential over a vector is proposed. The algorithm is based on a multilevel Monte Carlo method, and the vector solution is computed probabilistically generating suitable random paths which evolve through the indices of the matrix according to a suitable probability law. The computational complexity is proved in this paper to be significantly…
▽ More
A novel algorithm for computing the action of a matrix exponential over a vector is proposed. The algorithm is based on a multilevel Monte Carlo method, and the vector solution is computed probabilistically generating suitable random paths which evolve through the indices of the matrix according to a suitable probability law. The computational complexity is proved in this paper to be significantly better than the classical Monte Carlo method, which allows the computation of much more accurate solutions. Furthermore, the positive features of the algorithm in terms of parallelism were exploited in practice to develop a highly scalable implementation capable of solving some test problems very efficiently using high performance supercomputers equipped with a large number of cores. For the specific case of shared memory architectures the performance of the algorithm was compared with the results obtained using an available Krylov-based algorithm, outperforming the latter in all benchmarks analyzed so far.
△ Less
Submitted 4 July, 2019; v1 submitted 29 April, 2019;
originally announced April 2019.
-
A Multigrid-like Algorithm for Probabilistic Domain Decomposition
Authors:
Francisco Bernal,
Juan A. Acebrón
Abstract:
We present an iterative scheme, reminiscent of the Multigrid method, to solve large boundary value problems with Probabilistic Domain Decomposition (PDD). In it, increasingly accurate approximations to the solution are used as control variates in order to reduce the Monte Carlo error of the following iterates--resulting in an overall acceleration of PDD for a given error tolerance. The key ingredi…
▽ More
We present an iterative scheme, reminiscent of the Multigrid method, to solve large boundary value problems with Probabilistic Domain Decomposition (PDD). In it, increasingly accurate approximations to the solution are used as control variates in order to reduce the Monte Carlo error of the following iterates--resulting in an overall acceleration of PDD for a given error tolerance. The key ingredient of the proposed algorithm is the ability to approximately predict the speedup with little computational overhead and in parallel. Besides, the theoretical framework allows to explore other aspects of PDD, such as stability. One numerical example is worked out, yielding an improvement of between one and two orders of magnitude over the previous version of PDD.
△ Less
Submitted 5 January, 2017; v1 submitted 9 December, 2015;
originally announced December 2015.
-
A Comparison of Higher-Order Weak Numerical Schemes for Stopped Stochastic Differential Equations
Authors:
Francisco Bernal,
Juan A. Acebrón
Abstract:
We review, implement, and compare numerical integration schemes for spatially bounded diffusions stopped at the boundary which possess a convergence rate of the discretization error with respect to the timestep $h$ higher than ${\cal O}(\sqrt{h})$. We address specific implementation issues of the most general-purpose of such schemes. They have been coded into a single Matlab program and compared,…
▽ More
We review, implement, and compare numerical integration schemes for spatially bounded diffusions stopped at the boundary which possess a convergence rate of the discretization error with respect to the timestep $h$ higher than ${\cal O}(\sqrt{h})$. We address specific implementation issues of the most general-purpose of such schemes. They have been coded into a single Matlab program and compared, according to their accuracy and computational cost, on a wide range of problems in up to ${\mathbb R}^{48}$. The paper is self-contained and the code will be made freely downloadable.
△ Less
Submitted 28 February, 2016; v1 submitted 23 November, 2015;
originally announced November 2015.
