Showing 1–8 of 8 results for author: Fasi, M

Analysis of Floating-Point Matrix Multiplication Computed via Integer Arithmetic

Authors: Ahmad Abdelfattah, Jack Dongarra, Massimiliano Fasi, Mantas Mikaitis, Françoise Tisseur

Abstract: Ootomo, Ozaki, and Yokota [Int. J. High Perform. Comput. Appl., 38 (2024), p. 297-313] have proposed a strategy to recast a floating-point matrix multiplication in terms of integer matrix products. The factors A and B are split into integer slices, the product of these slices is computed exactly, and AB is approximated by accumulating these integer products in floating-point arithmetic. This techn… ▽ More Ootomo, Ozaki, and Yokota [Int. J. High Perform. Comput. Appl., 38 (2024), p. 297-313] have proposed a strategy to recast a floating-point matrix multiplication in terms of integer matrix products. The factors A and B are split into integer slices, the product of these slices is computed exactly, and AB is approximated by accumulating these integer products in floating-point arithmetic. This technique is particularly well suited to mixed-precision matrix multiply-accumulate units with integer support, such as the NVIDIA tensor cores or the AMD matrix cores. The number of slices allows for performance-accuracy tradeoffs: more slices yield better accuracy but require more multiplications, which in turn reduce performance. We propose an inexpensive way to estimate the minimum number of multiplications needed to achieve a prescribed level of accuracy. Our error analysis shows that the algorithm may become inaccurate (or inefficient) if rows of A or columns of B are badly scaled. We perform a range of numerical experiments, both in simulation and on the latest NVIDIA GPUs, that confirm the analysis and illustrate strengths and weaknesses of the algorithm. △ Less

Submitted 12 June, 2025; originally announced June 2025.

MSC Class: 65F99; 65G50; 65Y10

TypedMatrices.jl: An Extensible and Type-Based Matrix Collection for Julia

Authors: Anzhi Zhang, Massimiliano Fasi

Abstract: TypedMatrices.jl is a Julia package to organize test matrices. By default, the package comes with a number of built-in matrices and interfaces to help users select test cases based on their properties. The package is designed to be extensible, allowing users to define their own matrix types. We discuss the design and implementation of the package and demonstrate its usage with a number of examples… ▽ More TypedMatrices.jl is a Julia package to organize test matrices. By default, the package comes with a number of built-in matrices and interfaces to help users select test cases based on their properties. The package is designed to be extensible, allowing users to define their own matrix types. We discuss the design and implementation of the package and demonstrate its usage with a number of examples. △ Less

Submitted 14 March, 2025; originally announced March 2025.

Mixed-precision algorithms for solving the Sylvester matrix equation

Authors: Andrii Dmytryshyn, Massimiliano Fasi, Nicholas J. Higham, Xiaobo Liu

Abstract: We consider the solution of the general Sylvester equation $AX+XB=C$ in mixed precision. First, we investigate the use of GMRES-based iterative refinement (GMRES-IR) to solve the equation using implicitly its Kronecker product form: we propose an efficient scheme to use the Schur factors of the coefficient matrices as preconditioners, but we demonstrate that this approach is not suitable in the ca… ▽ More We consider the solution of the general Sylvester equation $AX+XB=C$ in mixed precision. First, we investigate the use of GMRES-based iterative refinement (GMRES-IR) to solve the equation using implicitly its Kronecker product form: we propose an efficient scheme to use the Schur factors of the coefficient matrices as preconditioners, but we demonstrate that this approach is not suitable in the case of the Sylvester equation. By revisiting a stationary iteration for linear systems, we therefore derive a new iterative refinement scheme for the quasi-triangular Sylvester equation, and our rounding error analysis provides sufficient conditions for convergence and a bound on the attainable relative residual. We leverage this iterative scheme to solve the general Sylvester equation in mixed precision. The new algorithms compute the Schur decomposition of the matrix coefficients in low precision, use the low-precision Schur factors to obtain an approximate solution to the quasi-triangular equation, and iteratively refine it to obtain a working-precision solution to the quasi-triangular equation. However, being only orthonormal to low precision, the unitary Schur factors of $A$ and $B$ cannot be used to recover the solution to the original equation. We propose two effective approaches to address this issue: one is based on re-orthonormalization in the working precision, and the other on explicit inversion of the almost-unitary factors. We test these mixed-precision algorithms on various Sylvester and Lyapunov equations from the literature. Our numerical experiments show that, for both classes of equations, the new algorithms are at least as accurate as existing ones. Our cost analysis, on the other hand, suggests that they would typically be faster than mono-precision alternatives if implemented on hardware that natively supports low precision. △ Less

Submitted 5 March, 2025; originally announced March 2025.

MSC Class: 65F45; 15A24; 65F10; 65G50; 65F25

Probabilistic error analysis of limited-precision stochastic rounding

Authors: El-Mehdi El Arar, Massimiliano Fasi, Silviu-Ioan Filip, Mantas Mikaitis

Abstract: Classical probabilistic rounding error analysis is particularly well suited to stochastic rounding (SR), and it yields strong results when dealing with floating-point algorithms that rely heavily on summation. For many numerical linear algebra algorithms, one can prove probabilistic error bounds that grow as O(nu), where n is the problem size and u is the unit roundoff. These probabilistic bounds… ▽ More Classical probabilistic rounding error analysis is particularly well suited to stochastic rounding (SR), and it yields strong results when dealing with floating-point algorithms that rely heavily on summation. For many numerical linear algebra algorithms, one can prove probabilistic error bounds that grow as O(nu), where n is the problem size and u is the unit roundoff. These probabilistic bounds are asymptotically tighter than the worst-case ones, which grow as O(nu). For certain classes of algorithms, SR has been shown to be unbiased. However, all these results were derived under the assumption that SR is implemented exactly, which typically requires a number of random bits that is too large to be suitable for practical implementations. We investigate the effect of the number of random bits on the probabilistic rounding error analysis of SR. To this end, we introduce a new rounding mode, limited-precision SR. By taking into account the number r of random bits used, this new rounding mode matches hardware implementations accurately, unlike the ideal SR operator generally used in the literature. We show that this new rounding mode is biased and that the bias is a function of r. As r approaches infinity, however, the bias disappears, and limited-precision SR converges to the ideal, unbiased SR operator. We develop a novel model for probabilistic error analysis of algorithms employing SR. Several numerical examples corroborate our theoretical findings. △ Less

Submitted 25 February, 2025; v1 submitted 6 August, 2024; originally announced August 2024.

Challenges in computing matrix functions

Authors: Massimiliano Fasi, Stéphane Gaudreault, Kathryn Lund, Marcel Schweitzer

Abstract: This manuscript summarizes the outcome of the focus groups at "The f(A)bulous workshop on matrix functions and exponential integrators", held at the Max Planck Institute for Dynamics of Complex Technical Systems in Magdeburg, Germany, on 25-27 September 2023. There were three focus groups in total, each with a different theme: knowledge transfer, high-performance and energy-aware computing, and be… ▽ More This manuscript summarizes the outcome of the focus groups at "The f(A)bulous workshop on matrix functions and exponential integrators", held at the Max Planck Institute for Dynamics of Complex Technical Systems in Magdeburg, Germany, on 25-27 September 2023. There were three focus groups in total, each with a different theme: knowledge transfer, high-performance and energy-aware computing, and benchmarking. We collect insights, open issues, and perspectives from each focus group, as well as from general discussions throughout the workshop. Our primary aim is to highlight ripe research directions and continue to build on the momentum from a lively meeting. △ Less

Submitted 29 January, 2024; originally announced January 2024.

MSC Class: 15A16; 15-11; 65F60; 65Y05; 65Y20; 65-11

Simulations of quantum dynamics with fermionic phase-space representations using numerical matrix factorizations as stochastic gauges

Authors: F Rousse, M Fasi, A Dmytryshyn, M Gulliksson, M Ogren

Abstract: The Gaussian phase-space representation can be used to implement quantum dynamics for fermionic particles numerically. To improve numerical results, we explore the use of dynamical diffusion gauges in such implementations. This is achieved by benchmarking quantum dynamics of few-body systems against independent exact solutions. A diffusion gauge is implemented here as a so-called noise-matrix, whi… ▽ More The Gaussian phase-space representation can be used to implement quantum dynamics for fermionic particles numerically. To improve numerical results, we explore the use of dynamical diffusion gauges in such implementations. This is achieved by benchmarking quantum dynamics of few-body systems against independent exact solutions. A diffusion gauge is implemented here as a so-called noise-matrix, which satisfies a matrix equation defined by the corresponding Fokker--Planck equation of the phase-space representation. For the physical systems with fermionic particles considered here, the numerical evaluation of the new diffusion gauges allows us to double the practical simulation time, compared with hitherto known analytic noise-matrices. This development may have far reaching consequences for future quantum dynamical simulations of many-body systems. △ Less

Submitted 11 April, 2023; originally announced April 2023.

Journal ref: J. Phys. A: Math. Theor. 57 (2024) 015303

Computational graphs for matrix functions

Authors: Elias Jarlebring, Massimiliano Fasi, Emil Ringh

Abstract: Many numerical methods for evaluating matrix functions can be naturally viewed as computational graphs. Rephrasing these methods as directed acyclic graphs (DAGs) is a particularly effective approach to study existing techniques, improve them, and eventually derive new ones. The accuracy of these matrix techniques can be characterized by the accuracy of their scalar counterparts, thus designing al… ▽ More Many numerical methods for evaluating matrix functions can be naturally viewed as computational graphs. Rephrasing these methods as directed acyclic graphs (DAGs) is a particularly effective approach to study existing techniques, improve them, and eventually derive new ones. The accuracy of these matrix techniques can be characterized by the accuracy of their scalar counterparts, thus designing algorithms for matrix functions can be regarded as a scalar-valued optimization problem. The derivatives needed during the optimization can be calculated automatically by exploiting the structure of the DAG, in a fashion analogous to backpropagation. This paper describes GraphMatFun.jl, a Julia package that offers the means to generate and manipulate computational graphs, optimize their coefficients, and generate Julia, MATLAB, and C code to evaluate them efficiently at a matrix argument. The software also provides tools to estimate the accuracy of a graph-based algorithm and thus obtain numerically reliable methods. For the exponential, for example, using a particular form (degree-optimal) of polynomials produces implementations that in many cases are cheaper, in terms of computational cost, than the Padé-based techniques typically used in mathematical software. The optimized graphs and the corresponding generated code are available online. △ Less

Submitted 19 August, 2021; v1 submitted 26 July, 2021; originally announced July 2021.

Report number: MIMS EPrint: 2021.12 MSC Class: 65F60 (Primary) 15A16 (Secondary)

Sampling the eigenvalues of random orthogonal and unitary matrices

Authors: Massimiliano Fasi, Leonardo Robol

Abstract: We develop an efficient algorithm for sampling the eigenvalues of random matrices distributed according to the Haar measure over the orthogonal or unitary group. Our technique samples directly a factorization of the Hessenberg form of such matrices, and then computes their eigenvalues with a tailored core-chasing algorithm. This approach requires a number of floating-point operations that is quadr… ▽ More We develop an efficient algorithm for sampling the eigenvalues of random matrices distributed according to the Haar measure over the orthogonal or unitary group. Our technique samples directly a factorization of the Hessenberg form of such matrices, and then computes their eigenvalues with a tailored core-chasing algorithm. This approach requires a number of floating-point operations that is quadratic in the order of the matrix being sampled, and can be adapted to other matrix groups. In particular, we explain how it can be used to sample the Haar measure over the special orthogonal and unitary groups and the conditional probability distribution obtained by requiring the determinant of the sampled matrix be a given complex number on the complex unit circle. △ Less

Submitted 24 February, 2021; v1 submitted 24 September, 2020; originally announced September 2020.

MSC Class: 15B10; 15B52; 65F15