-
Randomized Rank-Revealing QLP for Low-Rank Matrix Decomposition
Authors:
Maboud F. Kaloorazi,
Kai Liu,
Jie Chen,
Rodrigo C. de Lamare,
Susanto Rahardja
Abstract:
The pivoted QLP decomposition is computed through two consecutive pivoted QR decompositions, and provides an approximation to the singular value decomposition. This work is concerned with a partial QLP decomposition of low-rank matrices computed through randomization, termed Randomized Unpivoted QLP (RU-QLP). Like pivoted QLP, RU-QLP is rank-revealing and yet it utilizes random column sampling and…
▽ More
The pivoted QLP decomposition is computed through two consecutive pivoted QR decompositions, and provides an approximation to the singular value decomposition. This work is concerned with a partial QLP decomposition of low-rank matrices computed through randomization, termed Randomized Unpivoted QLP (RU-QLP). Like pivoted QLP, RU-QLP is rank-revealing and yet it utilizes random column sampling and the unpivoted QR decomposition. The latter modifications allow RU-QLP to be highly parallelizable on modern computational platforms. We provide an analysis for RU-QLP, deriving bounds in spectral and Frobenius norms on: i) the rank-revealing property; ii) principal angles between approximate subspaces and exact singular subspaces and vectors; and iii) low-rank approximation errors. Effectiveness of the bounds is illustrated through numerical tests. We further use a modern, multicore machine equipped with a GPU to demonstrate the efficiency of RU-QLP. Our results show that compared to the randomized SVD, RU-QLP achieves a speedup of up to 7.1 times on the CPU and up to 2.3 times with the GPU.
△ Less
Submitted 4 July, 2023; v1 submitted 26 September, 2022;
originally announced September 2022.
-
An Efficient Randomized QLP Algorithm for Approximating the Singular Value Decomposition
Authors:
M. F. Kaloorazi,
K. Liu,
J. Chen,
R. C. de Lamare
Abstract:
In this paper, we introduce a randomized QLP decomposition called Rand-QLP. Operating on a matrix $\bf A$, Rand-QLP gives ${\bf A}={\bf QLP}^T$, where $\bf Q$ and $\bf P$ are orthonormal, and $\bf L$ is lower-triangular. Under the assumption that the rank of the input matrix is $k$, we derive several error bounds for Rand-QLP: bounds for the first $k$ approximate singular values and for the traili…
▽ More
In this paper, we introduce a randomized QLP decomposition called Rand-QLP. Operating on a matrix $\bf A$, Rand-QLP gives ${\bf A}={\bf QLP}^T$, where $\bf Q$ and $\bf P$ are orthonormal, and $\bf L$ is lower-triangular. Under the assumption that the rank of the input matrix is $k$, we derive several error bounds for Rand-QLP: bounds for the first $k$ approximate singular values and for the trailing block of the middle factor $\bf L$, which show that the decomposition is rank-revealing; bounds for the distance between approximate subspaces and the exact ones for all four fundamental subspaces of a given matrix; and bounds for the errors of low-rank approximations constructed by the columns of $\bf Q$ and $\bf P$. Rand-QLP is able to effectively leverage modern computational architectures, due to the utilization of random sampling and the unpivoted QR decomposition, thus addressing a serious bottleneck associated with classical algorithms such as the singular value decomposition (SVD), column-pivoted QR (CPQR) and most recent matrix decomposition algorithms. To assess the performance behavior of different algorithms, we use an Intel Xeon Gold 6240 CPU running at 2.6 GHz with a NVIDIA GeForce RTX 2080Ti GPU. In comparison to CPQR and the SVD, Rand-QLP respectively achieves a speedup of up to 5 times and 6.6 times on the CPU and up to 3.8 times and 4.4 times with the hybrid GPU architecture. In terms of quality of approximation, our results on synthetic and real data show that the approximations by Rand-QLP are comparable to those of pivoted QLP and the optimal SVD, and in most cases are considerably better than those of CPQR.
△ Less
Submitted 28 January, 2023; v1 submitted 3 October, 2021;
originally announced October 2021.
-
Study of Compressed Randomized UTV Decompositions for Low-Rank Matrix Approximations in Data Science
Authors:
M. Kaloorazi,
R. C. de Lamare
Abstract:
In this work, a novel rank-revealing matrix decomposition algorithm termed Compressed Randomized UTV (CoR-UTV) decomposition along with a CoR-UTV variant aided by the power method technique is proposed. CoR-UTV computes an approximation to a low-rank input matrix by making use of random sampling schemes. Given a large and dense matrix of size $m\times n$ with numerical rank $k$, where…
▽ More
In this work, a novel rank-revealing matrix decomposition algorithm termed Compressed Randomized UTV (CoR-UTV) decomposition along with a CoR-UTV variant aided by the power method technique is proposed. CoR-UTV computes an approximation to a low-rank input matrix by making use of random sampling schemes. Given a large and dense matrix of size $m\times n$ with numerical rank $k$, where $k \ll \text{min} \{m,n\}$, CoR-UTV requires a few passes over the data, and runs in $O(mnk)$ floating-point operations. Furthermore, CoR-UTV can exploit modern computational platforms and can be optimized for maximum efficiency. CoR-UTV is also applied for solving robust principal component analysis problems. Simulations show that CoR-UTV outperform existing approaches.
△ Less
Submitted 7 June, 2019;
originally announced June 2019.
-
Direction Finding Based on Multi-Step Knowledge-Aided Iterative Conjugate Gradient Algorithms
Authors:
S. Pinto,
R. C. de Lamare
Abstract:
In this work, we present direction-of-arrival (DoA) estimation algorithms based on the Krylov subspace that effectively exploit prior knowledge of the signals that impinge on a sensor array. The proposed multi-step knowledge-aided iterative conjugate gradient (CG) (MS-KAI-CG) algorithms perform subtraction of the unwanted terms found in the estimated covariance matrix of the sensor data. Furthermo…
▽ More
In this work, we present direction-of-arrival (DoA) estimation algorithms based on the Krylov subspace that effectively exploit prior knowledge of the signals that impinge on a sensor array. The proposed multi-step knowledge-aided iterative conjugate gradient (CG) (MS-KAI-CG) algorithms perform subtraction of the unwanted terms found in the estimated covariance matrix of the sensor data. Furthermore, we develop a version of MS-KAI-CG equipped with forward-backward averaging, called MS-KAI-CG-FB, which is appropriate for scenarios with correlated signals. Unlike current knowledge-aided methods, which take advantage of known DoAs to enhance the estimation of the covariance matrix of the input data, the MS-KAI-CG algorithms take advantage of the knowledge of the structure of the forward-backward smoothed covariance matrix and its disturbance terms. Simulations with both uncorrelated and correlated signals show that the MS-KAI-CG algorithms outperform existing techniques.
△ Less
Submitted 15 December, 2018;
originally announced December 2018.
-
Randomized Rank-Revealing UZV Decomposition for Low-Rank Approximation of Matrices
Authors:
Maboud F. Kaloorazi,
Rodrigo C. de Lamare
Abstract:
Low-rank matrix approximation plays an increasingly important role in signal and image processing applications. This paper presents a new rank-revealing decomposition method called randomized rank-revealing UZV decomposition (RRR-UZVD). RRR-UZVD is powered by randomization to approximate a low-rank input matrix. Given a large and dense matrix ${\bf A} \in \mathbb R^{m \times n}$ whose numerical ra…
▽ More
Low-rank matrix approximation plays an increasingly important role in signal and image processing applications. This paper presents a new rank-revealing decomposition method called randomized rank-revealing UZV decomposition (RRR-UZVD). RRR-UZVD is powered by randomization to approximate a low-rank input matrix. Given a large and dense matrix ${\bf A} \in \mathbb R^{m \times n}$ whose numerical rank is $k$, where $k$ is much smaller than $m$ and $n$, RRR-UZVD constructs an approximation $\hat{\bf A}$ such as $\hat{\bf A}={\bf UZV}^T$, where ${\bf U}$ and ${\bf V}$ have orthonormal columns, the leading-diagonal block of ${\bf Z}$ reveals the rank of $\bf A$, and its off-diagonal blocks have small $\ell_2$-norms. RRR-UZVD is simple, accurate, and only requires a few passes through $\bf A$ with an arithmetic cost of $O(mnk)$ floating-point operations. To demonstrate the effectiveness of the proposed method, we conduct experiments on synthetic data, as well as real data in applications of image reconstruction and robust principal component analysis.
△ Less
Submitted 20 November, 2018;
originally announced November 2018.
-
Compressed Randomized UTV Decompositions for Low-Rank Approximations and Big Data Applications
Authors:
M. Kaloorazi,
R. C. de Lamare
Abstract:
Low-rank matrix approximations play a fundamental role in numerical linear algebra and signal processing applications. This paper introduces a novel rank-revealing matrix decomposition algorithm termed Compressed Randomized UTV (CoR-UTV) decomposition along with a CoR-UTV variant aided by the power method technique. CoR-UTV is primarily developed to compute an approximation to a low-rank input mat…
▽ More
Low-rank matrix approximations play a fundamental role in numerical linear algebra and signal processing applications. This paper introduces a novel rank-revealing matrix decomposition algorithm termed Compressed Randomized UTV (CoR-UTV) decomposition along with a CoR-UTV variant aided by the power method technique. CoR-UTV is primarily developed to compute an approximation to a low-rank input matrix by making use of random sampling schemes. Given a large and dense matrix of size $m\times n$ with numerical rank $k$, where $k \ll \text{min} \{m,n\}$, CoR-UTV requires a few passes over the data, and runs in $O(mnk)$ floating-point operations. Furthermore, CoR-UTV can exploit modern computational platforms and, consequently, can be optimized for maximum efficiency. CoR-UTV is simple and accurate, and outperforms reported alternative methods in terms of efficiency and accuracy. Simulations with synthetic data as well as real data in image reconstruction and robust principal component analysis applications support our claims.
△ Less
Submitted 15 October, 2018;
originally announced October 2018.
-
Subspace-Orbit Randomized Decomposition for Low-rank Matrix Approximation
Authors:
Maboud F. Kaloorazi,
Rodrigo C. de Lamare
Abstract:
An efficient, accurate and reliable approximation of a matrix by one of lower rank is a fundamental task in numerical linear algebra and signal processing applications. In this paper, we introduce a new matrix decomposition approach termed Subspace-Orbit Randomized singular value decomposition (SOR-SVD), which makes use of random sampling techniques to give an approximation to a low-rank matrix. G…
▽ More
An efficient, accurate and reliable approximation of a matrix by one of lower rank is a fundamental task in numerical linear algebra and signal processing applications. In this paper, we introduce a new matrix decomposition approach termed Subspace-Orbit Randomized singular value decomposition (SOR-SVD), which makes use of random sampling techniques to give an approximation to a low-rank matrix. Given a large and dense data matrix of size $m\times n$ with numerical rank $k$, where $k \ll \text{min} \{m,n\}$, the algorithm requires a few passes through data, and can be computed in $O(mnk)$ floating-point operations. Moreover, the SOR-SVD algorithm can utilize advanced computer architectures, and, as a result, it can be optimized for maximum efficiency. The SOR-SVD algorithm is simple, accurate, and provably correct, and outperforms previously reported techniques in terms of accuracy and efficiency. Our numerical experiments support these claims.
△ Less
Submitted 2 April, 2018;
originally announced April 2018.
-
Report on Two-Step Knowledge-Aided Iterative ESPRIT Algorithm
Authors:
R. C. de Lamare
Abstract:
In this work, we propose a subspace-based algorithm for direction-of-arrival (DOA) estimation, referred to as two-step knowledge-aided iterative estimation of signal parameters via rotational invariance techniques (ESPRIT) method (Two-Step KAI-ESPRIT), which achieves more accurate estimates than those of prior art. The proposed Two-Step KAI-ESPRIT improves the estimation of the covariance matrix o…
▽ More
In this work, we propose a subspace-based algorithm for direction-of-arrival (DOA) estimation, referred to as two-step knowledge-aided iterative estimation of signal parameters via rotational invariance techniques (ESPRIT) method (Two-Step KAI-ESPRIT), which achieves more accurate estimates than those of prior art. The proposed Two-Step KAI-ESPRIT improves the estimation of the covariance matrix of the input data by incorporating prior knowledge of signals and by exploiting knowledge of the structure of the covariance matrix and its perturbation terms. Simulation results illustrate the improvement achieved by the proposed method.
△ Less
Submitted 20 March, 2017;
originally announced March 2017.
