-
Scientific Applications Leveraging Randomized Linear Algebra
Authors:
Vivak Patel,
D. Adrian Maldonado,
Maksim Melnichenko,
Nathaniel Pritchard,
Vishwas Rao,
Elizaveta Rebrova,
Sriram Sankararaman
Abstract:
This report showcases the role of, and future directions for, the field of Randomized Numerical Linear Algebra (RNLA) in a selection of scientific applications. These applications span the domains of imaging, genomics and time-varying systems, and are thematically connected by needing to perform linear algebra routines on large-scale matrices (with up to quantillions of entries). At such scales, t…
▽ More
This report showcases the role of, and future directions for, the field of Randomized Numerical Linear Algebra (RNLA) in a selection of scientific applications. These applications span the domains of imaging, genomics and time-varying systems, and are thematically connected by needing to perform linear algebra routines on large-scale matrices (with up to quantillions of entries). At such scales, the linear algebra routines face typical bottlenecks: memory constraints, data access latencies, and substantial floating-point operation costs. RNLA routines are discussed at a high level to demonstrate how RNLA is able to solve the challenges faced by traditional linear algebra routines, and, consequently, address the computational problem posed in the underlying application. For each application, RNLA's open challenges and possible future directions are also presented, which broadly fall into the categories: creating structure-aware RNLA algorithms; co-designing RNLA algorithms with hardware and mixed-precision considerations; and advancing modular, composable software infrastructure. Ultimately, this report serves two purposes: it invites domain scientists to engage with RNLA; and it offers a guide for future RNLA research grounded in real applications.
△ Less
Submitted 19 June, 2025;
originally announced June 2025.
-
The statistical spread of transmission outages on a fast protection time scale based on utility data
Authors:
Ian Dobson,
D. Adrian Maldonado,
Mihai Anitescu
Abstract:
When there is a fault, the protection system automatically removes one or more transmission lines on a fast time scale of less than one minute. The outaged lines form a pattern in the transmission network. We extract these patterns from utility outage data, determine some key statistics of these patterns, and then show how to generate new patterns consistent with these statistics. The generated pa…
▽ More
When there is a fault, the protection system automatically removes one or more transmission lines on a fast time scale of less than one minute. The outaged lines form a pattern in the transmission network. We extract these patterns from utility outage data, determine some key statistics of these patterns, and then show how to generate new patterns consistent with these statistics. The generated patterns provide a new and easily feasible way to model the overall effect of the protection system at the scale of a large transmission system. This new generative modeling of protection is expected to contribute to simulations of disturbances in large grids so that they can better quantify the risk of blackouts. Analysis of the pattern sizes suggests an index that describes how much outages spread in the transmission network at the fast timescale.
△ Less
Submitted 6 May, 2025; v1 submitted 21 July, 2024;
originally announced July 2024.
-
Scalable Multi-Period AC Optimal Power Flow Utilizing GPUs with High Memory Capacities
Authors:
Sungho Shin,
Vishwas Rao,
Michel Schanen,
D. Adrian Maldonado,
Mihai Anitescu
Abstract:
This paper demonstrates the scalability of open-source GPU-accelerated nonlinear programming (NLP) frameworks -- ExaModels.jl and MadNLP.jl -- for solving multi-period alternating current (AC) optimal power flow (OPF) problems on GPUs with high memory capacities (e.g., NVIDIA GH200 with 480 GB of unified memory). There has been a growing interest in solving multi-period AC OPF problems, as the inc…
▽ More
This paper demonstrates the scalability of open-source GPU-accelerated nonlinear programming (NLP) frameworks -- ExaModels.jl and MadNLP.jl -- for solving multi-period alternating current (AC) optimal power flow (OPF) problems on GPUs with high memory capacities (e.g., NVIDIA GH200 with 480 GB of unified memory). There has been a growing interest in solving multi-period AC OPF problems, as the increasingly fluctuating electricity market requires operation planning over multiple periods. These problems, formerly deemed intractable, are now becoming technologically feasible to solve thanks to the advent of high-memory GPU hardware and accelerated NLP tools. This study evaluates the capability of these tools to tackle previously unsolvable multi-period AC OPF instances. Our numerical experiments, run on an NVIDIA GH200, demonstrate that we can solve a multi-period OPF instance with more than 10 million variables up to $10^{-4}$ precision in less than 10 minutes. These results demonstrate the efficacy of the GPU-accelerated NLP frameworks for the solution of extreme-scale multi-period OPF. We provide ExaModelsPower.jl, an open-source modeling tool for multi-period AC OPF models for GPUs.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
Data-Driven Estimation of Failure Probabilities in Correlated Structure-Preserving Stochastic Power System Models
Authors:
Hongli Zhao,
Tyler E. Maltba,
D. Adrian Maldonado,
Emil Constantinescu,
Mihai Anitescu
Abstract:
We propose a data-driven approach for propagating uncertainty in stochastic power grid simulations and apply it to the estimation of transmission line failure probabilities. A reduced-order equation governing the evolution of the observed line energy probability density function is derived from the Fokker--Planck equation of the full-order continuous Markov process. Our method consists of estimate…
▽ More
We propose a data-driven approach for propagating uncertainty in stochastic power grid simulations and apply it to the estimation of transmission line failure probabilities. A reduced-order equation governing the evolution of the observed line energy probability density function is derived from the Fokker--Planck equation of the full-order continuous Markov process. Our method consists of estimates produced by numerically integrating this reduced equation. Numerical experiments for scalar- and vector-valued energy functions are conducted using the classical multimachine model under spatiotemporally correlated noise perturbation. The method demonstrates a more sample-efficient approach for computing probabilities of tail events when compared with kernel density estimation. Moreover, it produces vastly more accurate estimates of joint event occurrence when compared with independent models.
△ Less
Submitted 4 January, 2024;
originally announced January 2024.
-
Data-driven Closures & Assimilation for Stiff Multiscale Random Dynamics
Authors:
Tyler E. Maltba,
Hongli Zhao,
D. Adrian Maldonado
Abstract:
We introduce a data-driven and physics-informed framework for propagating uncertainty in stiff, multiscale random ordinary differential equations (RODEs) driven by correlated (colored) noise. Unlike systems subjected to Gaussian white noise, a deterministic equation for the joint probability density function (PDF) of RODE state variables does not exist in closed form. Moreover, such an equation wo…
▽ More
We introduce a data-driven and physics-informed framework for propagating uncertainty in stiff, multiscale random ordinary differential equations (RODEs) driven by correlated (colored) noise. Unlike systems subjected to Gaussian white noise, a deterministic equation for the joint probability density function (PDF) of RODE state variables does not exist in closed form. Moreover, such an equation would require as many phase-space variables as there are states in the RODE system. To alleviate this curse of dimensionality, we instead derive exact, albeit unclosed, reduced-order PDF (RoPDF) equations for low-dimensional observables/quantities of interest. The unclosed terms take the form of state-dependent conditional expectations, which are directly estimated from data at sparse observation times. However, for systems exhibiting stiff, multiscale dynamics, data sparsity introduces regression discrepancies that compound during RoPDF evolution. This is overcome by introducing a kinetic-like defect term to the RoPDF equation, which is learned by assimilating in sparse, low-fidelity RoPDF estimates. Two assimilation methods are considered, namely nudging and deep neural networks, which are successfully tested against Monte Carlo simulations.
△ Less
Submitted 15 December, 2023;
originally announced December 2023.
-
Voltage-Dependent Electromechanical Wave Propagation Modeling for Dynamic Stability Analysis in Power Systems
Authors:
Somayeh Yarahmadi,
Daniel Adrian Maldonado,
Lamine Mili,
Junbo Zhao,
Mihai Anitescu
Abstract:
Accurate dynamic modeling of power systems is essential to assess the stability of electrical power systems when faced with disturbances, which can trigger cascading failures leading to blackouts. A continuum model proves to be effective in capturing Electromechanical Wave (EMW) propagation characteristics, including its velocity, arrival time, and deviations. Analyzing these characteristics enabl…
▽ More
Accurate dynamic modeling of power systems is essential to assess the stability of electrical power systems when faced with disturbances, which can trigger cascading failures leading to blackouts. A continuum model proves to be effective in capturing Electromechanical Wave (EMW) propagation characteristics, including its velocity, arrival time, and deviations. Analyzing these characteristics enables the assessment of the impacts of EMW on the performance of the protection system. Prior research has often modeled nonlinear EMW propagation through Partial Differential Equations (PDEs) within a homogeneous and uniform frame structure, assuming constant bus voltages across the entire power system. However, this assumption can produce inaccurate results. In this paper, we relax this assumption by introducing a second-order nonlinear hyperbolic EMW propagation equation model that accounts for voltage variations. Additionally, we present numerical solutions for the EMW propagation equation using the Lax-Wendroff integration method. To validate our approach, we conduct simulations on two test systems: a two-bus one-machine system and the New England 39-bus 10-machine system. The simulation results demonstrate the effectiveness of our proposed model and emphasize the importance of including the bus voltage equations in the analysis.
△ Less
Submitted 20 November, 2023;
originally announced November 2023.
-
Centralized calibration of power system dynamic models using variational data assimilation
Authors:
Ahmed Attia,
D. Adrian Maldonado,
Emil Constantinescu,
Mihai Anitescu
Abstract:
This paper presents a novel centralized, variational data assimilation approach for calibrating transient dynamic models in electrical power systems, focusing on load model parameters. With the increasing importance of inverter-based resources, assessing power systems' dynamic performance under disturbances has become challenging, necessitating robust model calibration methods. The proposed approa…
▽ More
This paper presents a novel centralized, variational data assimilation approach for calibrating transient dynamic models in electrical power systems, focusing on load model parameters. With the increasing importance of inverter-based resources, assessing power systems' dynamic performance under disturbances has become challenging, necessitating robust model calibration methods. The proposed approach expands on previous Bayesian frameworks by establishing a posterior distribution of parameters using an approximation around the maximum a posteriori value. We illustrate the efficacy of our method by generating events of varying intensity, highlighting its ability to capture the systems' evolution accurately and with associated uncertainty estimates. This research improves the precision of dynamic performance assessments in modern power systems, with potential applications in managing uncertainties and optimizing system operations.
△ Less
Submitted 13 November, 2023;
originally announced November 2023.
-
Efficient Computation of Power System Maximum Transient Linear Growth
Authors:
Daniel Adrian Maldonado,
Emil Constantinescu,
Junbo Zhao,
Mihai Anitescu
Abstract:
Existing methods to determine the stability of a power system to small perturbations are based on eigenvalue analysis and focus on the asymptotic (long-term) behavior of the power grid. During the preasymptotic (short-term) transient, however, the system can exhibit large growth that is not explained by eigenvalues alone. In this paper we propose a new framework to determine the maximum (optimal)…
▽ More
Existing methods to determine the stability of a power system to small perturbations are based on eigenvalue analysis and focus on the asymptotic (long-term) behavior of the power grid. During the preasymptotic (short-term) transient, however, the system can exhibit large growth that is not explained by eigenvalues alone. In this paper we propose a new framework to determine the maximum (optimal) preasymptotic growth using the singular value decomposition. The approach is tailored to the analysis of quantities of interest in power system dynamics, such as the set of rotor speed deviations. Matrix-free techniques are developed to avoid the explicit formation of dense matrices and enable the analysis of large-scale systems without reaching memory bounds. Extensive results carried out from small to very large-scale systems (e.g., 70k-bus system) verify the theoretical aspects of the technique.
△ Less
Submitted 20 February, 2023;
originally announced February 2023.
-
Parallel Interior-Point Solver for Block-Structured Nonlinear Programs on SIMD/GPU Architectures
Authors:
François Pacaud,
Michel Schanen,
Sungho Shin,
Daniel Adrian Maldonado,
Mihai Anitescu
Abstract:
We investigate how to port the standard interior-point method to new exascale architectures for block-structured nonlinear programs with state equations. Computationally, we decompose the interior-point algorithm into two successive operations: the evaluation of the derivatives and the solution of the associated Karush-Kuhn-Tucker (KKT) linear system. Our method accelerates both operations using t…
▽ More
We investigate how to port the standard interior-point method to new exascale architectures for block-structured nonlinear programs with state equations. Computationally, we decompose the interior-point algorithm into two successive operations: the evaluation of the derivatives and the solution of the associated Karush-Kuhn-Tucker (KKT) linear system. Our method accelerates both operations using two levels of parallelism. First, we distribute the computations on multiple processes using coarse parallelism. Second, each process uses a SIMD/GPU accelerator locally to accelerate the operations using fine-grained parallelism. The KKT system is reduced by eliminating the inequalities and the state variables from the corresponding equations, to a dense matrix encoding the sensitivities of the problem's degrees of freedom, drastically minimizing the memory exchange. We demonstrate the method's capability on the supercomputer Polaris, a testbed for the future exascale Aurora system. Each node is equipped with four GPUs, a setup amenable to our two-level approach. Our experiments on the stochastic optimal power flow problem show that the method can achieve a 50x speed-up compared to the state-of-the-art method.
△ Less
Submitted 12 January, 2023;
originally announced January 2023.
-
Learning the Evolution of Correlated Stochastic Power System Dynamics
Authors:
Tyler E. Maltba,
Vishwas Rao,
Daniel Adrian Maldonado
Abstract:
A machine learning technique is proposed for quantifying uncertainty in power system dynamics with spatiotemporally correlated stochastic forcing. We learn one-dimensional linear partial differential equations for the probability density functions of real-valued quantities of interest. The method is suitable for high-dimensional systems and helps to alleviate the curse of dimensionality.
A machine learning technique is proposed for quantifying uncertainty in power system dynamics with spatiotemporally correlated stochastic forcing. We learn one-dimensional linear partial differential equations for the probability density functions of real-valued quantities of interest. The method is suitable for high-dimensional systems and helps to alleviate the curse of dimensionality.
△ Less
Submitted 27 July, 2022;
originally announced July 2022.
-
Randomized Block Adaptive Linear System Solvers
Authors:
Vivak Patel,
Mohammad Jahangoshahi,
Daniel Adrian Maldonado
Abstract:
Randomized linear solvers randomly compress and solve a linear system with compelling theoretical convergence rates and computational complexities. However, such solvers suffer a substantial disconnect between their theoretical rates and actual efficiency in practice. Fortunately, these solvers are quite flexible and can be adapted to specific problems and computing environments to ensure high eff…
▽ More
Randomized linear solvers randomly compress and solve a linear system with compelling theoretical convergence rates and computational complexities. However, such solvers suffer a substantial disconnect between their theoretical rates and actual efficiency in practice. Fortunately, these solvers are quite flexible and can be adapted to specific problems and computing environments to ensure high efficiency in practice, even at the cost of lower effectiveness (i.e., having a slower theoretical rate of convergence). While highly efficient adapted solvers can be readily designed by application experts, will such solvers still converge and at what rate? To answer this, we distill three general criteria for randomized adaptive solvers, which, as we show, will guarantee a worst-case exponential rate of convergence of the solver applied to consistent and inconsistent linear systems irrespective of whether such systems are over-determined, under-determined or rank-deficient. As a result, we enable application experts to design randomized adaptive solvers that achieve efficiency and can be verified for effectiveness using our theory. We demonstrate our theory on twenty-six solvers, nine of which are novel or novel block extensions of existing methods to the best of our knowledge.
△ Less
Submitted 27 April, 2023; v1 submitted 4 April, 2022;
originally announced April 2022.
-
Condensed interior-point methods: porting reduced-space approaches on GPU hardware
Authors:
François Pacaud,
Sungho Shin,
Michel Schanen,
Daniel Adrian Maldonado,
Mihai Anitescu
Abstract:
The interior-point method (IPM) has become the workhorse method for nonlinear programming. The performance of IPM is directly related to the linear solver employed to factorize the Karush--Kuhn--Tucker (KKT) system at each iteration of the algorithm. When solving large-scale nonlinear problems, state-of-the art IPM solvers rely on efficient sparse linear solvers to solve the KKT system. Instead, w…
▽ More
The interior-point method (IPM) has become the workhorse method for nonlinear programming. The performance of IPM is directly related to the linear solver employed to factorize the Karush--Kuhn--Tucker (KKT) system at each iteration of the algorithm. When solving large-scale nonlinear problems, state-of-the art IPM solvers rely on efficient sparse linear solvers to solve the KKT system. Instead, we propose a novel reduced-space IPM algorithm that condenses the KKT system into a dense matrix whose size is proportional to the number of degrees of freedom in the problem. Depending on where the reduction occurs we derive two variants of the reduced-space method: linearize-then-reduce and reduce-then-linearize. We adapt their workflow so that the vast majority of computations are accelerated on GPUs. We provide extensive numerical results on the optimal power flow problem, comparing our GPU-accelerated reduced space IPM with Knitro and a hybrid full space IPM algorithm. By evaluating the derivatives on the GPU and solving the KKT system on the CPU, the hybrid solution is already significantly faster than the CPU-only solutions. The two reduced-space algorithms go one step further by solving the KKT system entirely on the GPU. As expected, the performance of the two reduction algorithms depends intrinsically on the number of available degrees of freedom: their performance is poor when the problem has many degrees of freedom, but the two algorithms are up to 3 times faster than Knitro as soon as the relative number of degrees of freedom becomes smaller.
△ Less
Submitted 22 March, 2022;
originally announced March 2022.
-
Batched Second-Order Adjoint Sensitivity for Reduced Space Methods
Authors:
François Pacaud,
Michel Schanen,
Daniel Adrian Maldonado,
Alexis Montoison,
Valentin Churavy,
Julian Samaroo,
Mihai Anitescu
Abstract:
This paper presents an efficient method for extracting the second-order sensitivities from a system of implicit nonlinear equations on upcoming graphical processing units (GPU) dominated computer systems. We design a custom automatic differentiation (AutoDiff) backend that targets highly parallel architectures by extracting the second-order information in batch. When the nonlinear equations are as…
▽ More
This paper presents an efficient method for extracting the second-order sensitivities from a system of implicit nonlinear equations on upcoming graphical processing units (GPU) dominated computer systems. We design a custom automatic differentiation (AutoDiff) backend that targets highly parallel architectures by extracting the second-order information in batch. When the nonlinear equations are associated to a reduced space optimization problem, we leverage the parallel reverse-mode accumulation in a batched adjoint-adjoint algorithm to compute efficiently the reduced Hessian of the problem. We apply the method to extract the reduced Hessian associated to the balance equations of a power network, and show on the largest instances that a parallel GPU implementation is 30 times faster than a sequential CPU reference based on UMFPACK.
△ Less
Submitted 1 January, 2022;
originally announced January 2022.
-
A Feasible Reduced Space Method for Real-Time Optimal Power Flow
Authors:
François Pacaud,
Daniel Adrian Maldonado,
Sungho Shin,
Michel Schanen,
Mihai Anitescu
Abstract:
We propose a novel feasible-path algorithm to solve the optimal power flow (OPF) problem for real-time use cases. The method augments the seminal work of Dommel and Tinney with second-order derivatives to work directly in the reduced space induced by the power flow equations. In the reduced space, the optimization problem includes only inequality constraints corresponding to the operational constr…
▽ More
We propose a novel feasible-path algorithm to solve the optimal power flow (OPF) problem for real-time use cases. The method augments the seminal work of Dommel and Tinney with second-order derivatives to work directly in the reduced space induced by the power flow equations. In the reduced space, the optimization problem includes only inequality constraints corresponding to the operational constraints. While the reduced formulation directly enforces the physical constraints, the operational constraints are softly enforced through Augmented Lagrangian penalty terms. In contrast to interior-point algorithms (state-of-the art for solving OPF), our algorithm maintains feasibility at each iteration, which makes it suitable for real-time application. By exploiting accelerator hardware (Graphic Processing Units) to compute the reduced Hessian, we show that the second-order method is numerically tractable and is effective to solve both static and real-time OPF problems.
△ Less
Submitted 6 October, 2021;
originally announced October 2021.
-
Trust-region approximation of extreme trajectories in power system dynamics
Authors:
Daniel Adrian Maldonado,
Emil Constantinescu,
Hong Zhang,
Vishwas Rao,
Mihai Anitescu
Abstract:
In this work we present a novel technique, based on a trust-region optimization algorithm and second-order trajectory sensitivities, to compute the extreme trajectories of power system dynamic simulations given a bounded set that represents parametric uncertainty. We show how this method, while remaining computationally efficient compared with sampling-based techniques, overcomes the limitations o…
▽ More
In this work we present a novel technique, based on a trust-region optimization algorithm and second-order trajectory sensitivities, to compute the extreme trajectories of power system dynamic simulations given a bounded set that represents parametric uncertainty. We show how this method, while remaining computationally efficient compared with sampling-based techniques, overcomes the limitations of previous sensitivity-based techniques to approximate the bounds of the trajectories, when the local approximation loses validity because of the nonlinearity. In addition, we show how this method can be adapted to account for those cases in which the initial conditions depend on the uncertain parameter. To conclude, we present several numerical experiments that showcase the accuracy and scalability of the technique, including a demonstration on the IEEE New England test system.
△ Less
Submitted 8 February, 2022; v1 submitted 30 June, 2021;
originally announced June 2021.
-
Convergence of Adaptive, Randomized, Iterative Linear Solvers
Authors:
Vivak Patel,
Mohammad Jahangoshahi,
Daniel Adrian Maldonado
Abstract:
Deterministic and randomized, row-action and column-action linear solvers have become increasingly popular owing to their simplicity, low computational and memory complexities, and ease of composition with other techniques. Moreover, in order to achieve high-performance, such solvers must often be adapted to the given problem structure and to the hardware platform on which the problem will be solv…
▽ More
Deterministic and randomized, row-action and column-action linear solvers have become increasingly popular owing to their simplicity, low computational and memory complexities, and ease of composition with other techniques. Moreover, in order to achieve high-performance, such solvers must often be adapted to the given problem structure and to the hardware platform on which the problem will be solved. Unfortunately, determining whether such adapted solvers will converge to a solution has required equally unique analyses. As a result, adapted, reliable solvers are slow to be developed and deployed. In this work, we provide a general set of assumptions under which such adapted solvers are guaranteed to converge with probability one, and provide worst case rates of convergence. As a result, we can provide practitioners with guidance on how to design highly adapted, randomized or deterministic, row-action or column-action linear solvers that are also guaranteed to converge.
△ Less
Submitted 26 April, 2021; v1 submitted 10 April, 2021;
originally announced April 2021.
-
Sequential Bayesian Parameter Estimation of Stochastic Dynamic Load Models
Authors:
Daniel Adrian Maldonado,
Vishwas Rao,
Mihai Anitescu,
Vivak Patel
Abstract:
In this paper we focus on the parameter estimation of dynamic load models with stochastic terms, in particular, load models where protection settings are uncertain, such as in aggregated air conditioning units. We show how the uncertainty in the aggregated protection characteristics can be formulated as a stochastic differential equation with process noise. We cast the parameter inversion within a…
▽ More
In this paper we focus on the parameter estimation of dynamic load models with stochastic terms, in particular, load models where protection settings are uncertain, such as in aggregated air conditioning units. We show how the uncertainty in the aggregated protection characteristics can be formulated as a stochastic differential equation with process noise. We cast the parameter inversion within a Bayesian parameter estimation framework, and we present methods to include process noise. We demonstrate the benefits of considering stochasticity in the parameter estimation and the risks of ignoring it.
△ Less
Submitted 29 April, 2020;
originally announced April 2020.
-
Probabilistic analysis of masked loads with aggregated photovoltaic production
Authors:
Shaohui Liu,
Daniel Adrian Maldonado,
Emil M. Constantinescu
Abstract:
In this paper we present a probabilistic analysis framework to estimate behind-the-meter photovoltaic generation in real time. We develop a forward model consisting of a spatiotemporal stochastic process that represents the photovoltaic generation and a stochastic differential equation with jumps that represents the demand. We employ this model to disaggregate the behind-the-meter photovoltaic gen…
▽ More
In this paper we present a probabilistic analysis framework to estimate behind-the-meter photovoltaic generation in real time. We develop a forward model consisting of a spatiotemporal stochastic process that represents the photovoltaic generation and a stochastic differential equation with jumps that represents the demand. We employ this model to disaggregate the behind-the-meter photovoltaic generation using net load and irradiance measurements.
△ Less
Submitted 21 April, 2020;
originally announced April 2020.
-
An Implicit Representation and Iterative Solution of Randomly Sketched Linear Systems
Authors:
Vivak Patel,
Mohammad Jahangoshahi,
Daniel Adrian Maldonado
Abstract:
Randomized linear system solvers have become popular as they have the potential to reduce floating point complexity while still achieving desirable convergence rates. One particularly promising class of methods, random sketching solvers, has achieved the best known computational complexity bounds in theory, but is blunted by two practical considerations: there is no clear way of choosing the size…
▽ More
Randomized linear system solvers have become popular as they have the potential to reduce floating point complexity while still achieving desirable convergence rates. One particularly promising class of methods, random sketching solvers, has achieved the best known computational complexity bounds in theory, but is blunted by two practical considerations: there is no clear way of choosing the size of the sketching matrix apriori; and there is a nontrivial storage cost of the sketched system. In this work, we make progress towards addressing these issues by implicitly generating the sketched system and solving it simultaneously through an iterative procedure. As a result, we replace the question of the size of the sketching matrix with determining appropriate stopping criteria; we also avoid the costs of explicitly representing the sketched linear system; and our implicit representation also solves the system at the same time, which controls the per-iteration computational costs.
Additionally, our approach allows us to generate a connection between random sketching methods and randomized iterative solvers (e.g., randomized Kaczmarz method, randomized Gauss-Seidel). As a consequence, we exploit this connection to (1) produce a stronger, more precise convergence theory for such randomized iterative solvers under arbitrary sampling schemes (i.i.d., adaptive, permutation, dependent, etc.), and (2) improve the rates of convergence of randomized iterative solvers at the expense of a user-determined increases in per-iteration computational and storage costs. We demonstrate these concepts on numerical examples on forty-nine distinct linear systems.
△ Less
Submitted 22 December, 2020; v1 submitted 26 April, 2019;
originally announced April 2019.
