-
Fast and Practical Quantum-Inspired Classical Algorithms for Solving Linear Systems
Authors:
Qian Zuo,
Tongyang Li
Abstract:
We propose fast and practical quantum-inspired classical algorithms for solving linear systems. Specifically, given sampling and query access to a matrix $A\in\mathbb{R}^{m\times n}$ and a vector $b\in\mathbb{R}^m$, we propose classical algorithms that produce a data structure for the solution $x\in\mathbb{R}^{n}$ of the linear system $Ax=b$ with the ability to sample and query its entries. The re…
▽ More
We propose fast and practical quantum-inspired classical algorithms for solving linear systems. Specifically, given sampling and query access to a matrix $A\in\mathbb{R}^{m\times n}$ and a vector $b\in\mathbb{R}^m$, we propose classical algorithms that produce a data structure for the solution $x\in\mathbb{R}^{n}$ of the linear system $Ax=b$ with the ability to sample and query its entries. The resulting $x$ satisfies $\|x-A^{+}b\|\leqε\|A^{+}b\|$, where $\|\cdot\|$ is the spectral norm and $A^+$ is the Moore-Penrose inverse of $A$. Our algorithm has time complexity $\widetilde{O}(κ_F^4/κε^2)$ in the general case, where $κ_{F} =\|A\|_F\|A^+\|$ and $κ=\|A\|\|A^+\|$ are condition numbers. Compared to the prior state-of-the-art result [Shao and Montanaro, arXiv:2103.10309v2], our algorithm achieves a polynomial speedup in condition numbers. When $A$ is $s$-sparse, our algorithm has complexity $\widetilde{O}(s κ\log(1/ε))$, matching the quantum lower bound for solving linear systems in $κ$ and $1/ε$ up to poly-logarithmic factors [Harrow and Kothari]. When $A$ is $s$-sparse and symmetric positive-definite, our algorithm has complexity $\widetilde{O}(s\sqrtκ\log(1/ε))$.
Technically, our main contribution is the application of the heavy ball momentum method to quantum-inspired classical algorithms for solving linear systems, where we propose two new methods with speedups: quantum-inspired Kaczmarz method with momentum and quantum-inspired coordinate descent method with momentum. Their analysis exploits careful decomposition of the momentum transition matrix and the application of novel spectral norm concentration bounds for independent random matrices. Finally, we also conduct numerical experiments for our algorithms on both synthetic and real-world datasets, and the experimental results support our theoretical claims.
△ Less
Submitted 30 November, 2023; v1 submitted 13 July, 2023;
originally announced July 2023.
-
On the Polyak momentum variants of the greedy deterministic single and multiple row-action methods
Authors:
Nian-Ci Wu,
Qian Zuo,
Yatian Wang
Abstract:
For solving a consistent system of linear equations, the classical row-action (also known as Kaczmarz) method is a simple while really effective iteration solver. Based on the greedy index selection strategy and Polyak's heavy-ball momentum acceleration technique, we propose two deterministic row-action methods and establish the corresponding convergence theory. We show that our algorithm can line…
▽ More
For solving a consistent system of linear equations, the classical row-action (also known as Kaczmarz) method is a simple while really effective iteration solver. Based on the greedy index selection strategy and Polyak's heavy-ball momentum acceleration technique, we propose two deterministic row-action methods and establish the corresponding convergence theory. We show that our algorithm can linearly converge to a least-squares solution with minimum Euclidean norm. Several numerical studies have been presented to corroborate our theoretical findings. Real-world applications, such as data fitting in computer-aided geometry design, are also presented for illustrative purposes.
△ Less
Submitted 12 December, 2022;
originally announced December 2022.
-
On the extended randomized multiple row method for solving linear least-squares problems
Authors:
Nian-Ci Wu,
Chengzhi Liu,
Yatian Wang,
Qian Zuo
Abstract:
The randomized row method is a popular representative of the iterative algorithm because of its efficiency in solving the overdetermined and consistent systems of linear equations. In this paper, we present an extended randomized multiple row method to solve a given overdetermined and inconsistent linear system and analyze its computational complexities at each iteration. We prove that the propose…
▽ More
The randomized row method is a popular representative of the iterative algorithm because of its efficiency in solving the overdetermined and consistent systems of linear equations. In this paper, we present an extended randomized multiple row method to solve a given overdetermined and inconsistent linear system and analyze its computational complexities at each iteration. We prove that the proposed method can linearly converge in the mean square to the least-squares solution with a minimum Euclidean norm. Several numerical studies are presented to corroborate our theoretical findings. The real-world applications, such as image reconstruction and large noisy data fitting in computer-aided geometric design, are also presented for illustration purposes.
△ Less
Submitted 7 October, 2022;
originally announced October 2022.
-
Quantum-inspired algorithm for truncated total least squares solution
Authors:
Qian Zuo,
Yimin Wei,
Hua Xiang
Abstract:
Total least squares (TLS) methods have been widely used in data fitting. Compared with the least squares method, for TLS problem we takes into account not only the observation errors, but also the errors in the measurement matrix. This is more realistic in practical applications. For the large-scale discrete ill-posed problem $Ax \approx b$, we introduce the quantum-inspired techniques to approxim…
▽ More
Total least squares (TLS) methods have been widely used in data fitting. Compared with the least squares method, for TLS problem we takes into account not only the observation errors, but also the errors in the measurement matrix. This is more realistic in practical applications. For the large-scale discrete ill-posed problem $Ax \approx b$, we introduce the quantum-inspired techniques to approximate the truncated total least squares (TTLS) solution. We analyze the accuracy of the quantum-inspired truncated total least squares algorithm and perform numerical experiments to demonstrate the efficiency of our method.
△ Less
Submitted 1 May, 2022;
originally announced May 2022.
-
On the relaxed greedy deterministic row and column iterative methods
Authors:
Nian-Ci Wu,
Ling-Xia Cui,
Qian Zuo
Abstract:
For solving the large-scale linear system by iteration methods, we utilize the Petrov-Galerkin conditions and relaxed greedy index selection technique and provide two relaxed greedy deterministic row (RGDR) and column (RGDC) iterative methods, in which one special case of RGDR reduces to the fast deterministic block Kaczmarz method proposed in Chen and Huang (Numer. Algor., 89: 1007-1029, 2021). O…
▽ More
For solving the large-scale linear system by iteration methods, we utilize the Petrov-Galerkin conditions and relaxed greedy index selection technique and provide two relaxed greedy deterministic row (RGDR) and column (RGDC) iterative methods, in which one special case of RGDR reduces to the fast deterministic block Kaczmarz method proposed in Chen and Huang (Numer. Algor., 89: 1007-1029, 2021). Our convergence analyses reveal that the resulting algorithms all have the linear convergence rates, which are bounded by the explicit expressions. Numerical examples show that the proposed algorithms are more effective than the relaxed greedy randomized row and column iterative methods.
△ Less
Submitted 28 March, 2022;
originally announced March 2022.
-
A quantum-inspired algorithm for approximating statistical leverage scores
Authors:
Qian Zuo,
Hua Xiang
Abstract:
Suppose a matrix $A \in \mathbb{R}^{m \times n}$ of rank $r$ with singular value decomposition $A = U_{A}Σ_{A} V_{A}^{T}$, where $U_{A} \in \mathbb{R}^{m \times r}$, $V_{A} \in \mathbb{R}^{n \times r}$ are orthonormal and $Σ_{A} \in \mathbb{R}^{r \times r}$ is a diagonal matrix. The statistical leverage scores of a matrix $A$ are the squared row-norms defined by $\ell_{i} = \|(U_{A})_{i,:}\|_2^2$,…
▽ More
Suppose a matrix $A \in \mathbb{R}^{m \times n}$ of rank $r$ with singular value decomposition $A = U_{A}Σ_{A} V_{A}^{T}$, where $U_{A} \in \mathbb{R}^{m \times r}$, $V_{A} \in \mathbb{R}^{n \times r}$ are orthonormal and $Σ_{A} \in \mathbb{R}^{r \times r}$ is a diagonal matrix. The statistical leverage scores of a matrix $A$ are the squared row-norms defined by $\ell_{i} = \|(U_{A})_{i,:}\|_2^2$, where $i \in [m]$, and the matrix coherence is the largest statistical leverage score. These quantities play an important role in machine learning algorithms such as matrix completion and Nyström-based low rank matrix approximation as well as large-scale statistical data analysis applications, whose usual algorithm complexity is polynomial in the dimension of the matrix $A$. As an alternative to the conventional approach, and inspired by recent development on dequantization techniques, we propose a quantum-inspired algorithm for approximating the statistical leverage scores. We then analyze the accuracy of the algorithm and perform numerical experiments to illustrate the feasibility of our algorithm. Theoretical analysis shows that our novel algorithm takes time polynomial in an integer $k$, condition number $κ$ and logarithm of the matrix size.
△ Less
Submitted 1 April, 2022; v1 submitted 17 November, 2021;
originally announced November 2021.
