Showing 1–34 of 34 results for author: Cohen, M B

A Family-Based Approach to Safety Cases for Controlled Airspaces in Small Uncrewed Aerial Systems

Authors: Michael C. Hunter, Usman Gohar, Myra B. Cohen, Robyn R. Lutz, Jane Cleland-Huang

Abstract: As small Uncrewed Aircraft Systems (sUAS) increasingly operate in the national airspace, safety concerns arise due to a corresponding rise in reported airspace violations and incidents, highlighting the need for a safe mechanism for sUAS entry control to manage the potential overload. This paper presents work toward our aim of establishing automated, customized safety-claim support for managing on… ▽ More As small Uncrewed Aircraft Systems (sUAS) increasingly operate in the national airspace, safety concerns arise due to a corresponding rise in reported airspace violations and incidents, highlighting the need for a safe mechanism for sUAS entry control to manage the potential overload. This paper presents work toward our aim of establishing automated, customized safety-claim support for managing on-entry requests from sUAS to enter controlled airspace. We describe our approach, Safety Case Software Product Line Engineering (SafeSPLE), which is a novel method to extend product-family techniques to on-entry safety cases. It begins with a hazard analysis and design of a safety case feature model defining key points in variation, followed by the creation of a parameterized safety case. We use these together to automate the generation of instances for specific sUAS. Finally we use a case study to demonstrate that the SafeSPLE method can be used to facilitate creation of safety cases for specific flights. △ Less

Submitted 4 February, 2025; originally announced February 2025.

Comments: Accepted at AIAA 2024

A Taxonomy of Real-World Defeaters in Safety Assurance Cases

Authors: Usman Gohar, Michael C. Hunter, Myra B. Cohen, Robyn R. Lutz

Abstract: The rise of cyber-physical systems in safety-critical domains calls for robust risk-evaluation frameworks. Assurance cases, often required by regulatory bodies, are a structured approach to demonstrate that a system meets its safety requirements. However, assurance cases are fraught with challenges, such as incomplete evidence and gaps in reasoning, called defeaters, that can call into question th… ▽ More The rise of cyber-physical systems in safety-critical domains calls for robust risk-evaluation frameworks. Assurance cases, often required by regulatory bodies, are a structured approach to demonstrate that a system meets its safety requirements. However, assurance cases are fraught with challenges, such as incomplete evidence and gaps in reasoning, called defeaters, that can call into question the credibility and robustness of assurance cases. Identifying these defeaters increases confidence in the assurance case and can prevent catastrophic failures. The search for defeaters in an assurance case, however, is not structured, and there is a need to standardize defeater analysis. The software engineering community thus could benefit from having a reusable classification of real-world defeaters in software assurance cases. In this paper, we conducted a systematic study of literature from the past 20 years. Using open coding, we derived a taxonomy with seven broad categories, laying the groundwork for standardizing the analysis and management of defeaters in safety-critical systems. We provide our artifacts as open source for the community to use and build upon, thus establishing a common framework for understanding defeaters. △ Less

Submitted 31 January, 2025; originally announced February 2025.

Comments: ICSE 2025, Workshop on Multi-disciplinary, Open, and integRatEd Requirements Engineering

Design and Implementation of ARA Wireless Living Lab for Rural Broadband and Applications

Authors: Taimoor Ul Islam, Joshua Ofori Boateng, Md Nadim, Guoying Zu, Mukaram Shahid, Xun Li, Tianyi Zhang, Salil Reddy, Wei Xu, Ataberk Atalar, Vincent Lee, Yung-Fu Chen, Evan Gosling, Elisabeth Permatasari, Christ Somiah, Zhibo Meng, Sarath Babu, Mohammed Soliman, Ali Hussain, Daji Qiao, Mai Zheng, Ozdal Boyraz, Yong Guan, Anish Arora, Mohamed Selim , et al. (6 additional authors not shown)

Abstract: To address the rural broadband challenge and to leverage the unique opportunities that rural regions provide for piloting advanced wireless applications, we design and implement the ARA wireless living lab for research and innovation in rural wireless systems and their applications in precision agriculture, community services, and so on. ARA focuses on the unique community, application, and econom… ▽ More To address the rural broadband challenge and to leverage the unique opportunities that rural regions provide for piloting advanced wireless applications, we design and implement the ARA wireless living lab for research and innovation in rural wireless systems and their applications in precision agriculture, community services, and so on. ARA focuses on the unique community, application, and economic context of rural regions, and it features the first-of-its-kind, real-world deployment of long-distance, high-capacity wireless x-haul and access platforms across a rural area of diameter over 30 km. With both software-defined radios and programmable COTS systems and through effective orchestration of these wireless resources with fiber as well as compute resources embedded end-to-end across user equipment, base stations, edge, and cloud, ARA offers programmability, performance, robustness, and heterogeneity at the same time, thus enabling rural-focused co-evolution of wireless and applications while helping advance the frontiers of wireless systems in domains such as O-RAN, NextG, and agriculture applications. Here we present the design principles and implementation strategies of ARA, characterize its performance and heterogeneity, and highlight example wireless and application experiments uniquely enabled by ARA. △ Less

Submitted 1 August, 2024; originally announced August 2024.

Comments: 17 pages, 18 figures

CoDefeater: Using LLMs To Find Defeaters in Assurance Cases

Authors: Usman Gohar, Michael C. Hunter, Robyn R. Lutz, Myra B. Cohen

Abstract: Constructing assurance cases is a widely used, and sometimes required, process toward demonstrating that safety-critical systems will operate safely in their planned environment. To mitigate the risk of errors and missing edge cases, the concept of defeaters - arguments or evidence that challenge claims in an assurance case - has been introduced. Defeaters can provide timely detection of weaknesse… ▽ More Constructing assurance cases is a widely used, and sometimes required, process toward demonstrating that safety-critical systems will operate safely in their planned environment. To mitigate the risk of errors and missing edge cases, the concept of defeaters - arguments or evidence that challenge claims in an assurance case - has been introduced. Defeaters can provide timely detection of weaknesses in the arguments, prompting further investigation and timely mitigations. However, capturing defeaters relies on expert judgment, experience, and creativity and must be done iteratively due to evolving requirements and regulations. This paper proposes CoDefeater, an automated process to leverage large language models (LLMs) for finding defeaters. Initial results on two systems show that LLMs can efficiently find known and unforeseen feasible defeaters to support safety analysts in enhancing the completeness and confidence of assurance cases. △ Less

Submitted 16 August, 2024; v1 submitted 18 July, 2024; originally announced July 2024.

Comments: ASE 2024 NIER

Towards Engineering Fair and Equitable Software Systems for Managing Low-Altitude Airspace Authorizations

Authors: Usman Gohar, Michael C. Hunter, Agnieszka Marczak-Czajka, Robyn R. Lutz, Myra B. Cohen, Jane Cleland-Huang

Abstract: Small Unmanned Aircraft Systems (sUAS) have gained widespread adoption across a diverse range of applications. This has introduced operational complexities within shared airspaces and an increase in reported incidents, raising safety concerns. In response, the U.S. Federal Aviation Administration (FAA) is developing a UAS Traffic Management (UTM) system to control access to airspace based on an sU… ▽ More Small Unmanned Aircraft Systems (sUAS) have gained widespread adoption across a diverse range of applications. This has introduced operational complexities within shared airspaces and an increase in reported incidents, raising safety concerns. In response, the U.S. Federal Aviation Administration (FAA) is developing a UAS Traffic Management (UTM) system to control access to airspace based on an sUAS's predicted ability to safely complete its mission. However, a fully automated system capable of swiftly approving or denying flight requests can be prone to bias and must consider safety, transparency, and fairness to diverse stakeholders. In this paper, we present an initial study that explores stakeholders' perspectives on factors that should be considered in an automated system. Results indicate flight characteristics and environmental conditions were perceived as most important but pilot and drone capabilities should also be considered. Further, several respondents indicated an aversion to any AI-supported automation, highlighting the need for full transparency in automated decision-making. Results provide a societal perspective on the challenges of automating UTM flight authorization decisions and help frame the ongoing design of a solution acceptable to the broader sUAS community. △ Less

Submitted 3 February, 2024; v1 submitted 14 January, 2024; originally announced January 2024.

Journal ref: ICSE-SEIS 2024

HIFuzz: Human Interaction Fuzzing for small Unmanned Aerial Vehicles

Authors: Theodore Chambers, Michael Vierhauser, Ankit Agrawal, Michael Murphy, Jason Matthew Brauer, Salil Purandare, Myra B. Cohen, Jane Cleland-Huang

Abstract: Small Unmanned Aerial Systems (sUAS) must meet rigorous safety standards when deployed in high-stress emergency response scenarios; however many reported accidents have involved humans in the loop. In this paper, we, therefore, present the HiFuzz testing framework, which uses fuzz testing to identify system vulnerabilities associated with human interactions. HiFuzz includes three distinct levels t… ▽ More Small Unmanned Aerial Systems (sUAS) must meet rigorous safety standards when deployed in high-stress emergency response scenarios; however many reported accidents have involved humans in the loop. In this paper, we, therefore, present the HiFuzz testing framework, which uses fuzz testing to identify system vulnerabilities associated with human interactions. HiFuzz includes three distinct levels that progress from a low-cost, limited-fidelity, large-scale, no-hazard environment, using fully simulated Proxy Human Agents, via an intermediate level, where proxy humans are replaced with real humans, to a high-stakes, high-cost, real-world environment. Through applying HiFuzz to an autonomous multi-sUAS system-under-test, we show that each test level serves a unique purpose in revealing vulnerabilities and making the system more robust with respect to human mistakes. While HiFuzz is designed for testing sUAS systems, we further discuss its potential for use in other Cyber-Physical Systems. △ Less

Submitted 7 April, 2024; v1 submitted 18 October, 2023; originally announced October 2023.

HyperGI: Automated Detection and Repair of Information Flow Leakage

Authors: Ibrahim Mesecan, Daniel Blackwell, David Clark, Myra B. Cohen, Justyna Petke

Abstract: Maintaining confidential information control in software is a persistent security problem where failure means secrets can be revealed via program behaviors. Information flow control techniques traditionally have been based on static or symbolic analyses -- limited in scalability and specialized to particular languages. When programs do leak secrets there are no approaches to automatically repair t… ▽ More Maintaining confidential information control in software is a persistent security problem where failure means secrets can be revealed via program behaviors. Information flow control techniques traditionally have been based on static or symbolic analyses -- limited in scalability and specialized to particular languages. When programs do leak secrets there are no approaches to automatically repair them unless the leak causes a functional test to fail. We present our vision for HyperGI, a genetic improvement framework tha detects, localizes and repairs information leakage. Key elements of HyperGI include (1) the use of two orthogonal test suites, (2) a dynamic leak detection approach which estimates and localizes potential leaks, and (3) a repair component that produces a candidate patch using genetic improvement. We demonstrate the successful use of HyperGI on several programs which have no failing functional tests. We manually examine the resulting patches and identify trade-offs and future directions for fully realizing our vision. △ Less

Submitted 26 August, 2021; originally announced August 2021.

Relative Lipschitzness in Extragradient Methods and a Direct Recipe for Acceleration

Authors: Michael B. Cohen, Aaron Sidford, Kevin Tian

Abstract: We show that standard extragradient methods (i.e. mirror prox and dual extrapolation) recover optimal accelerated rates for first-order minimization of smooth convex functions. To obtain this result we provide a fine-grained characterization of the convergence rates of extragradient methods for solving monotone variational inequalities in terms of a natural condition we call relative Lipschitzness… ▽ More We show that standard extragradient methods (i.e. mirror prox and dual extrapolation) recover optimal accelerated rates for first-order minimization of smooth convex functions. To obtain this result we provide a fine-grained characterization of the convergence rates of extragradient methods for solving monotone variational inequalities in terms of a natural condition we call relative Lipschitzness. We further generalize this framework to handle local and randomized notions of relative Lipschitzness and thereby recover rates for box-constrained $\ell_\infty$ regression based on area convexity and complexity bounds achieved by accelerated (randomized) coordinate descent for smooth convex function minimization. △ Less

Submitted 14 July, 2021; v1 submitted 12 November, 2020; originally announced November 2020.

Comments: 32 pages. This is the full version of a paper appearing in ITCS 2021. v2 addresses reviewer comments and adds citations

Genetic Improvement @ ICSE 2020

Authors: William B. Langdon, Westley Weimer, Justyna Petke, Erik Fredericks, Seongmin Lee, Emily Winter, Michail Basios, Myra B. Cohen, Aymeric Blot, Markus Wagner, Bobby R. Bruce, Shin Yoo, Simos Gerasimou, Oliver Krauss, Yu Huang, Michael Gerten

Abstract: Following Prof. Mark Harman of Facebook's keynote and formal presentations (which are recorded in the proceedings) there was a wide ranging discussion at the eighth international Genetic Improvement workshop, GI-2020 @ ICSE (held as part of the 42nd ACM/IEEE International Conference on Software Engineering on Friday 3rd July 2020). Topics included industry take up, human factors, explainabiloity (… ▽ More Following Prof. Mark Harman of Facebook's keynote and formal presentations (which are recorded in the proceedings) there was a wide ranging discussion at the eighth international Genetic Improvement workshop, GI-2020 @ ICSE (held as part of the 42nd ACM/IEEE International Conference on Software Engineering on Friday 3rd July 2020). Topics included industry take up, human factors, explainabiloity (explainability, justifyability, exploitability) and GI benchmarks. We also contrast various recent online approaches (e.g. SBST 2020) to holding virtual computer science conferences and workshops via the WWW on the Internet without face-2-face interaction. Finally we speculate on how the Coronavirus Covid-19 Pandemic will affect research next year and into the future. △ Less

Submitted 31 July, 2020; originally announced July 2020.

Comments: 7 pages, 2 figures. Write up of GI @ ICSE 2020 workshop. Submitted to ACM SIGSOFT Software Engineering Notes

A near-optimal algorithm for approximating the John Ellipsoid

Authors: Michael B. Cohen, Ben Cousins, Yin Tat Lee, Xin Yang

Abstract: We develop a simple and efficient algorithm for approximating the John Ellipsoid of a symmetric polytope. Our algorithm is near optimal in the sense that our time complexity matches the current best verification algorithm. We also provide the MATLAB code for further research. We develop a simple and efficient algorithm for approximating the John Ellipsoid of a symmetric polytope. Our algorithm is near optimal in the sense that our time complexity matches the current best verification algorithm. We also provide the MATLAB code for further research. △ Less

Submitted 18 February, 2020; v1 submitted 27 May, 2019; originally announced May 2019.

Comments: COLT 2019

Solving Directed Laplacian Systems in Nearly-Linear Time through Sparse LU Factorizations

Authors: Michael B. Cohen, Jonathan Kelner, Rasmus Kyng, John Peebles, Richard Peng, Anup B. Rao, Aaron Sidford

Abstract: We show how to solve directed Laplacian systems in nearly-linear time. Given a linear system in an $n \times n$ Eulerian directed Laplacian with $m$ nonzero entries, we show how to compute an $ε$-approximate solution in time $O(m \log^{O(1)} (n) \log (1/ε))$. Through reductions from [Cohen et al. FOCS'16] , this gives the first nearly-linear time algorithms for computing $ε$-approximate solutions… ▽ More We show how to solve directed Laplacian systems in nearly-linear time. Given a linear system in an $n \times n$ Eulerian directed Laplacian with $m$ nonzero entries, we show how to compute an $ε$-approximate solution in time $O(m \log^{O(1)} (n) \log (1/ε))$. Through reductions from [Cohen et al. FOCS'16] , this gives the first nearly-linear time algorithms for computing $ε$-approximate solutions to row or column diagonally dominant linear systems (including arbitrary directed Laplacians) and computing $ε$-approximations to various properties of random walks on directed graphs, including stationary distributions, personalized PageRank vectors, hitting times, and escape probabilities. These bounds improve upon the recent almost-linear algorithms of [Cohen et al. STOC'17], which gave an algorithm to solve Eulerian Laplacian systems in time $O((m+n2^{O(\sqrt{\log n \log \log n})})\log^{O(1)}(n ε^{-1}))$. To achieve our results, we provide a structural result that we believe is of independent interest. We show that Laplacians of all strongly connected directed graphs have sparse approximate LU-factorizations. That is, for every such directed Laplacian $ {\mathbf{L}}$, there is a lower triangular matrix $\boldsymbol{\mathit{\mathfrak{L}}}$ and an upper triangular matrix $\boldsymbol{\mathit{\mathfrak{U}}}$, each with at most $\tilde{O}(n)$ nonzero entries, such that their product $\boldsymbol{\mathit{\mathfrak{L}}} \boldsymbol{\mathit{\mathfrak{U}}}$ spectrally approximates $ {\mathbf{L}}$ in an appropriate norm. This claim can be viewed as an analogue of recent work on sparse Cholesky factorizations of Laplacians of undirected graphs. We show how to construct such factorizations in nearly-linear time and prove that, once constructed, they yield nearly-linear time algorithms for solving directed Laplacian systems. △ Less

Submitted 26 November, 2018; originally announced November 2018.

Comments: Appeared in FOCS 2018

Solving Linear Programs in the Current Matrix Multiplication Time

Authors: Michael B. Cohen, Yin Tat Lee, Zhao Song

Abstract: This paper shows how to solve linear programs of the form $\min_{Ax=b,x\geq0} c^\top x$ with $n$ variables in time $$O^*((n^ω+n^{2.5-α/2}+n^{2+1/6}) \log(n/δ))$$ where $ω$ is the exponent of matrix multiplication, $α$ is the dual exponent of matrix multiplication, and $δ$ is the relative accuracy. For the current value of $ω\sim2.37$ and $α\sim0.31$, our algorithm takes $O^*(n^ω \log(n/δ))$ time.… ▽ More This paper shows how to solve linear programs of the form $\min_{Ax=b,x\geq0} c^\top x$ with $n$ variables in time $$O^*((n^ω+n^{2.5-α/2}+n^{2+1/6}) \log(n/δ))$$ where $ω$ is the exponent of matrix multiplication, $α$ is the dual exponent of matrix multiplication, and $δ$ is the relative accuracy. For the current value of $ω\sim2.37$ and $α\sim0.31$, our algorithm takes $O^*(n^ω \log(n/δ))$ time. When $ω= 2$, our algorithm takes $O^*(n^{2+1/6} \log(n/δ))$ time. Our algorithm utilizes several new concepts that we believe may be of independent interest: $\bullet$ We define a stochastic central path method. $\bullet$ We show how to maintain a projection matrix $\sqrt{W}A^{\top}(AWA^{\top})^{-1}A\sqrt{W}$ in sub-quadratic time under $\ell_{2}$ multiplicative changes in the diagonal matrix $W$. △ Less

Submitted 19 October, 2020; v1 submitted 18 October, 2018; originally announced October 2018.

Comments: STOC 2019, JACM 2020

Constant Arboricity Spectral Sparsifiers

Authors: Timothy Chu, Michael B. Cohen, Jakub W. Pachocki, Richard Peng

Abstract: We show that every graph is spectrally similar to the union of a constant number of forests. Moreover, we show that Spielman-Srivastava sparsifiers are the union of O(logn) forests. This result can be used to estimate boundaries of small subsets of vertices in nearly optimal query time. We show that every graph is spectrally similar to the union of a constant number of forests. Moreover, we show that Spielman-Srivastava sparsifiers are the union of O(logn) forests. This result can be used to estimate boundaries of small subsets of vertices in nearly optimal query time. △ Less

Submitted 16 August, 2018; originally announced August 2018.

Metrical task systems on trees via mirror descent and unfair gluing

Authors: Sébastien Bubeck, Michael B. Cohen, James R. Lee, Yin Tat Lee

Abstract: We consider metrical task systems on tree metrics, and present an $O(\mathrm{depth} \times \log n)$-competitive randomized algorithm based on the mirror descent framework introduced in our prior work on the $k$-server problem. For the special case of hierarchically separated trees (HSTs), we use mirror descent to refine the standard approach based on gluing unfair metrical task systems. This yield… ▽ More We consider metrical task systems on tree metrics, and present an $O(\mathrm{depth} \times \log n)$-competitive randomized algorithm based on the mirror descent framework introduced in our prior work on the $k$-server problem. For the special case of hierarchically separated trees (HSTs), we use mirror descent to refine the standard approach based on gluing unfair metrical task systems. This yields an $O(\log n)$-competitive algorithm for HSTs, thus removing an extraneous $\log\log n$ in the bound of Fiat and Mendel (2003). Combined with well-known HST embedding theorems, this also gives an $O((\log n)^2)$-competitive randomized algorithm for every $n$-point metric space. △ Less

Submitted 25 November, 2020; v1 submitted 11 July, 2018; originally announced July 2018.

A Nearly-Linear Bound for Chasing Nested Convex Bodies

Authors: C. J. Argue, Sébastien Bubeck, Michael B. Cohen, Anupam Gupta, Yin Tat Lee

Abstract: Friedman and Linial introduced the convex body chasing problem to explore the interplay between geometry and competitive ratio in metrical task systems. In convex body chasing, at each time step $t \in \mathbb{N}$, the online algorithm receives a request in the form of a convex body $K_t \subseteq \mathbb{R}^d$ and must output a point $x_t \in K_t$. The goal is to minimize the total movement betwe… ▽ More Friedman and Linial introduced the convex body chasing problem to explore the interplay between geometry and competitive ratio in metrical task systems. In convex body chasing, at each time step $t \in \mathbb{N}$, the online algorithm receives a request in the form of a convex body $K_t \subseteq \mathbb{R}^d$ and must output a point $x_t \in K_t$. The goal is to minimize the total movement between consecutive output points, where the distance is measured in some given norm. This problem is still far from being understood, and recently Bansal et al. gave an algorithm for the nested version, where each convex body is contained within the previous one. We propose a different strategy which is $O(d \log d)$-competitive algorithm for this nested convex body chasing problem, improving substantially over previous work. Our algorithm works for any norm. This result is almost tight, given an $Ω(d)$ lower bound for the $\ell_{\infty}$. △ Less

Submitted 15 November, 2018; v1 submitted 22 June, 2018; originally announced June 2018.

On Acceleration with Noise-Corrupted Gradients

Authors: Michael B. Cohen, Jelena Diakonikolas, Lorenzo Orecchia

Abstract: Accelerated algorithms have broad applications in large-scale optimization, due to their generality and fast convergence. However, their stability in the practical setting of noise-corrupted gradient oracles is not well-understood. This paper provides two main technical contributions: (i) a new accelerated method AGDP that generalizes Nesterov's AGD and improves on the recent method AXGD (Diakonik… ▽ More Accelerated algorithms have broad applications in large-scale optimization, due to their generality and fast convergence. However, their stability in the practical setting of noise-corrupted gradient oracles is not well-understood. This paper provides two main technical contributions: (i) a new accelerated method AGDP that generalizes Nesterov's AGD and improves on the recent method AXGD (Diakonikolas & Orecchia, 2018), and (ii) a theoretical study of accelerated algorithms under noisy and inexact gradient oracles, which is supported by numerical experiments. This study leverages the simplicity of AGDP and its analysis to clarify the interaction between noise and acceleration and to suggest modifications to the algorithm that reduce the mean and variance of the error incurred due to the gradient noise. △ Less

Submitted 31 July, 2018; v1 submitted 31 May, 2018; originally announced May 2018.

Comments: Appeared in Proc. ICML'18; v2 corrects the statement of Corollary 3.9; v3 added references to concurrent work

An homotopy method for $\ell_p$ regression provably beyond self-concordance and in input-sparsity time

Authors: Sébastien Bubeck, Michael B. Cohen, Yin Tat Lee, Yuanzhi Li

Abstract: We consider the problem of linear regression where the $\ell_2^n$ norm loss (i.e., the usual least squares loss) is replaced by the $\ell_p^n$ norm. We show how to solve such problems up to machine precision in $O^*(n^{|1/2 - 1/p|})$ (dense) matrix-vector products and $O^*(1)$ matrix inversions, or alternatively in $O^*(n^{|1/2 - 1/p|})$ calls to a (sparse) linear system solver. This improves the… ▽ More We consider the problem of linear regression where the $\ell_2^n$ norm loss (i.e., the usual least squares loss) is replaced by the $\ell_p^n$ norm. We show how to solve such problems up to machine precision in $O^*(n^{|1/2 - 1/p|})$ (dense) matrix-vector products and $O^*(1)$ matrix inversions, or alternatively in $O^*(n^{|1/2 - 1/p|})$ calls to a (sparse) linear system solver. This improves the state of the art for any $p\not\in \{1,2,+\infty\}$. Furthermore we also propose a randomized algorithm solving such problems in {\em input sparsity time}, i.e., $O^*(Z + \mathrm{poly}(d))$ where $Z$ is the size of the input and $d$ is the number of variables. Such a result was only known for $p=2$. Finally we prove that these results lie outside the scope of the Nesterov-Nemirovski's theory of interior point methods by showing that any symmetric self-concordant barrier on the $\ell_p^n$ unit ball has self-concordance parameter $\tildeΩ(n)$. △ Less

Submitted 25 June, 2018; v1 submitted 3 November, 2017; originally announced November 2017.

Comments: 16 pages

k-server via multiscale entropic regularization

Authors: Sebastien Bubeck, Michael B. Cohen, James R. Lee, Yin Tat Lee, Aleksander Madry

Abstract: We present an $O((\log k)^2)$-competitive randomized algorithm for the $k$-server problem on hierarchically separated trees (HSTs). This is the first $o(k)$-competitive randomized algorithm for which the competitive ratio is independent of the size of the underlying HST. Our algorithm is designed in the framework of online mirror descent where the mirror map is a multiscale entropy. When combined… ▽ More We present an $O((\log k)^2)$-competitive randomized algorithm for the $k$-server problem on hierarchically separated trees (HSTs). This is the first $o(k)$-competitive randomized algorithm for which the competitive ratio is independent of the size of the underlying HST. Our algorithm is designed in the framework of online mirror descent where the mirror map is a multiscale entropy. When combined with Bartal's static HST embedding reduction, this leads to an $O((\log k)^2 \log n)$-competitive algorithm on any $n$-point metric space. We give a new dynamic HST embedding that yields an $O((\log k)^3 \log Δ)$-competitive algorithm on any metric space where the ratio of the largest to smallest non-zero distance is at most $Δ$. △ Less

Submitted 3 November, 2017; originally announced November 2017.

Sparsity, variance and curvature in multi-armed bandits

Authors: Sébastien Bubeck, Michael B. Cohen, Yuanzhi Li

Abstract: In (online) learning theory the concepts of sparsity, variance and curvature are well-understood and are routinely used to obtain refined regret and generalization bounds. In this paper we further our understanding of these concepts in the more challenging limited feedback scenario. We consider the adversarial multi-armed bandit and linear bandit settings and solve several open problems pertaining… ▽ More In (online) learning theory the concepts of sparsity, variance and curvature are well-understood and are routinely used to obtain refined regret and generalization bounds. In this paper we further our understanding of these concepts in the more challenging limited feedback scenario. We consider the adversarial multi-armed bandit and linear bandit settings and solve several open problems pertaining to the existence of algorithms with favorable regret bounds under the following assumptions: (i) sparsity of the individual losses, (ii) small variation of the loss sequence, and (iii) curvature of the action set. Specifically we show that (i) for $s$-sparse losses one can obtain $\tilde{O}(\sqrt{s T})$-regret (solving an open problem by Kwon and Perchet), (ii) for loss sequences with variation bounded by $Q$ one can obtain $\tilde{O}(\sqrt{Q})$-regret (solving an open problem by Kale and Hazan), and (iii) for linear bandit on an $\ell_p^n$ ball one can obtain $\tilde{O}(\sqrt{n T})$-regret for $p \in [1,2]$ and one has $\tildeΩ(n \sqrt{T})$-regret for $p>2$ (solving an open problem by Bubeck, Cesa-Bianchi and Kakade). A key new insight to obtain these results is to use regularizers satisfying more refined conditions than general self-concordance △ Less

Submitted 3 November, 2017; originally announced November 2017.

Comments: 18 pages

Matrix Scaling and Balancing via Box Constrained Newton's Method and Interior Point Methods

Authors: Michael B. Cohen, Aleksander Madry, Dimitris Tsipras, Adrian Vladu

Abstract: In this paper, we study matrix scaling and balancing, which are fundamental problems in scientific computing, with a long line of work on them that dates back to the 1960s. We provide algorithms for both these problems that, ignoring logarithmic factors involving the dimension of the input matrix and the size of its entries, both run in time $\widetilde{O}\left(m\log κ\log^2 (1/ε)\right)$ where… ▽ More In this paper, we study matrix scaling and balancing, which are fundamental problems in scientific computing, with a long line of work on them that dates back to the 1960s. We provide algorithms for both these problems that, ignoring logarithmic factors involving the dimension of the input matrix and the size of its entries, both run in time $\widetilde{O}\left(m\log κ\log^2 (1/ε)\right)$ where $ε$ is the amount of error we are willing to tolerate. Here, $κ$ represents the ratio between the largest and the smallest entries of the optimal scalings. This implies that our algorithms run in nearly-linear time whenever $κ$ is quasi-polynomial, which includes, in particular, the case of strictly positive matrices. We complement our results by providing a separate algorithm that uses an interior-point method and runs in time $\widetilde{O}(m^{3/2} \log (1/ε))$. In order to establish these results, we develop a new second-order optimization framework that enables us to treat both problems in a unified and principled manner. This framework identifies a certain generalization of linear system solving that we can use to efficiently minimize a broad class of functions, which we call second-order robust. We then show that in the context of the specific functions capturing matrix scaling and balancing, we can leverage and generalize the work on Laplacian system solving to make the algorithms obtained via this framework very efficient. △ Less

Submitted 21 August, 2017; v1 submitted 7 April, 2017; originally announced April 2017.

Comments: To appear in FOCS 2017

Almost-Linear-Time Algorithms for Markov Chains and New Spectral Primitives for Directed Graphs

Authors: Michael B. Cohen, Jonathan Kelner, John Peebles, Richard Peng, Anup Rao, Aaron Sidford, Adrian Vladu

Abstract: In this paper we introduce a notion of spectral approximation for directed graphs. While there are many potential ways one might define approximation for directed graphs, most of them are too strong to allow sparse approximations in general. In contrast, we prove that for our notion of approximation, such sparsifiers do exist, and we show how to compute them in almost linear time. Using this not… ▽ More In this paper we introduce a notion of spectral approximation for directed graphs. While there are many potential ways one might define approximation for directed graphs, most of them are too strong to allow sparse approximations in general. In contrast, we prove that for our notion of approximation, such sparsifiers do exist, and we show how to compute them in almost linear time. Using this notion of approximation, we provide a general framework for solving asymmetric linear systems that is broadly inspired by the work of [Peng-Spielman, STOC`14]. Applying this framework in conjunction with our sparsification algorithm, we obtain an almost linear time algorithm for solving directed Laplacian systems associated with Eulerian Graphs. Using this solver in the recent framework of [Cohen-Kelner-Peebles-Peng-Sidford-Vladu, FOCS`16], we obtain almost linear time algorithms for solving a directed Laplacian linear system, computing the stationary distribution of a Markov chain, computing expected commute times in a directed graph, and more. For each of these problems, our algorithms improves the previous best running times of $O((nm^{3/4} + n^{2/3} m) \log^{O(1)} (n κε^{-1}))$ to $O((m + n2^{O(\sqrt{\log{n}\log\log{n}})}) \log^{O(1)} (n κε^{-1}))$ where $n$ is the number of vertices in the graph, $m$ is the number of edges, $κ$ is a natural condition number associated with the problem, and $ε$ is the desired accuracy. We hope these results open the door for further studies into directed spectral graph theory, and will serve as a stepping stone for designing a new generation of fast algorithms for directed graphs. △ Less

Submitted 2 November, 2016; originally announced November 2016.

Faster Algorithms for Computing the Stationary Distribution, Simulating Random Walks, and More

Authors: Michael B. Cohen, Jon Kelner, John Peebles, Richard Peng, Aaron Sidford, Adrian Vladu

Abstract: In this paper, we provide faster algorithms for computing various fundamental quantities associated with random walks on a directed graph, including the stationary distribution, personalized PageRank vectors, hitting times, and escape probabilities. In particular, on a directed graph with $n$ vertices and $m$ edges, we show how to compute each quantity in time $\tilde{O}(m^{3/4}n+mn^{2/3})$, where… ▽ More In this paper, we provide faster algorithms for computing various fundamental quantities associated with random walks on a directed graph, including the stationary distribution, personalized PageRank vectors, hitting times, and escape probabilities. In particular, on a directed graph with $n$ vertices and $m$ edges, we show how to compute each quantity in time $\tilde{O}(m^{3/4}n+mn^{2/3})$, where the $\tilde{O}$ notation suppresses polylogarithmic factors in $n$, the desired accuracy, and the appropriate condition number (i.e. the mixing time or restart probability). Our result improves upon the previous fastest running times for these problems; previous results either invoke a general purpose linear system solver on a $n\times n$ matrix with $m$ non-zero entries, or depend polynomially on the desired error or natural condition number associated with the problem (i.e. the mixing time or restart probability). For sparse graphs, we obtain a running time of $\tilde{O}(n^{7/4})$, breaking the $O(n^{2})$ barrier of the best running time one could hope to achieve using fast matrix multiplication. We achieve our result by providing a similar running time improvement for solving directed Laplacian systems, a natural directed or asymmetric analog of the well studied symmetric or undirected Laplacian systems. We show how to solve such systems in time $\tilde{O}(m^{3/4}n+mn^{2/3})$, and efficiently reduce a broad range of problems to solving $\tilde{O}(1)$ directed Laplacian systems on Eulerian graphs. We hope these results and our analysis open the door for further study into directed spectral graph theory. △ Less

Submitted 2 November, 2016; v1 submitted 10 August, 2016; originally announced August 2016.

Geometric Median in Nearly Linear Time

Authors: Michael B. Cohen, Yin Tat Lee, Gary Miller, Jakub Pachocki, Aaron Sidford

Abstract: In this paper we provide faster algorithms for solving the geometric median problem: given $n$ points in $\mathbb{R}^{d}$ compute a point that minimizes the sum of Euclidean distances to the points. This is one of the oldest non-trivial problems in computational geometry yet despite an abundance of research the previous fastest algorithms for computing a $(1+ε)$-approximate geometric median were… ▽ More In this paper we provide faster algorithms for solving the geometric median problem: given $n$ points in $\mathbb{R}^{d}$ compute a point that minimizes the sum of Euclidean distances to the points. This is one of the oldest non-trivial problems in computational geometry yet despite an abundance of research the previous fastest algorithms for computing a $(1+ε)$-approximate geometric median were $O(d\cdot n^{4/3}ε^{-8/3})$ by Chin et. al, $\tilde{O}(d\exp{ε^{-4}\logε^{-1}})$ by Badoiu et. al, $O(nd+\mathrm{poly}(d,ε^{-1})$ by Feldman and Langberg, and $O((nd)^{O(1)}\log\frac{1}ε)$ by Parrilo and Sturmfels and Xue and Ye. In this paper we show how to compute a $(1+ε)$-approximate geometric median in time $O(nd\log^{3}\frac{1}ε)$ and $O(dε^{-2})$. While our $O(dε^{-2})$ is a fairly straightforward application of stochastic subgradient descent, our $O(nd\log^{3}\frac{1}ε)$ time algorithm is a novel long step interior point method. To achieve this running time we start with a simple $O((nd)^{O(1)}\log\frac{1}ε)$ time interior point method and show how to improve it, ultimately building an algorithm that is quite non-standard from the perspective of interior point literature. Our result is one of very few cases we are aware of outperforming traditional interior point theory and the only we are aware of using interior point methods to obtain a nearly linear time algorithm for a canonical optimization problem that traditionally requires superlinear time. We hope our work leads to further improvements in this line of research. △ Less

Submitted 16 June, 2016; originally announced June 2016.

Comments: Symposium on Theory of Computing (STOC) 2016

Negative-Weight Shortest Paths and Unit Capacity Minimum Cost Flow in $\tilde{O}(m^{10/7} \log W)$ Time

Authors: Michael B. Cohen, Aleksander Madry, Piotr Sankowski, Adrian Vladu

Abstract: In this paper, we study a set of combinatorial optimization problems on weighted graphs: the shortest path problem with negative weights, the weighted perfect bipartite matching problem, the unit-capacity minimum-cost maximum flow problem and the weighted perfect bipartite $b$-matching problem under the assumption that $\Vert b\Vert_1=O(m)$. We show that each one of these four problems can be solv… ▽ More In this paper, we study a set of combinatorial optimization problems on weighted graphs: the shortest path problem with negative weights, the weighted perfect bipartite matching problem, the unit-capacity minimum-cost maximum flow problem and the weighted perfect bipartite $b$-matching problem under the assumption that $\Vert b\Vert_1=O(m)$. We show that each one of these four problems can be solved in $\tilde{O}(m^{10/7}\log W)$ time, where $W$ is the absolute maximum weight of an edge in the graph, which gives the first in over 25 years polynomial improvement in their sparse-graph time complexity. At a high level, our algorithms build on the interior-point method-based framework developed by Madry (FOCS 2013) for solving unit-capacity maximum flow problem. We develop a refined way to analyze this framework, as well as provide new variants of the underlying preconditioning and perturbation techniques. Consequently, we are able to extend the whole interior-point method-based approach to make it applicable in the weighted graph regime. △ Less

Submitted 13 July, 2016; v1 submitted 5 May, 2016; originally announced May 2016.

Online Row Sampling

Authors: Michael B. Cohen, Cameron Musco, Jakub Pachocki

Abstract: Finding a small spectral approximation for a tall $n \times d$ matrix $A$ is a fundamental numerical primitive. For a number of reasons, one often seeks an approximation whose rows are sampled from those of $A$. Row sampling improves interpretability, saves space when $A$ is sparse, and preserves row structure, which is especially important, for example, when $A$ represents a graph. However, cor… ▽ More Finding a small spectral approximation for a tall $n \times d$ matrix $A$ is a fundamental numerical primitive. For a number of reasons, one often seeks an approximation whose rows are sampled from those of $A$. Row sampling improves interpretability, saves space when $A$ is sparse, and preserves row structure, which is especially important, for example, when $A$ represents a graph. However, correctly sampling rows from $A$ can be costly when the matrix is large and cannot be stored and processed in memory. Hence, a number of recent publications focus on row sampling in the streaming setting, using little more space than what is required to store the outputted approximation [KL13, KLM+14]. Inspired by a growing body of work on online algorithms for machine learning and data analysis, we extend this work to a more restrictive online setting: we read rows of $A$ one by one and immediately decide whether each row should be kept in the spectral approximation or discarded, without ever retracting these decisions. We present an extremely simple algorithm that approximates $A$ up to multiplicative error $ε$ and additive error $δ$ using $O(d \log d \log(ε||A||_2/δ)/ε^2)$ online samples, with memory overhead proportional to the cost of storing the spectral approximation. We also present an algorithm that uses $O(d^2$) memory but only requires $O(d\log(ε||A||_2/δ)/ε^2)$ samples, which we show is optimal. Our methods are clean and intuitive, allow for lower memory usage than prior work, and expose new theoretical properties of leverage score based matrix approximation. △ Less

Submitted 19 April, 2016; originally announced April 2016.

Ramanujan Graphs in Polynomial Time

Authors: Michael B. Cohen

Abstract: The recent work by Marcus, Spielman and Srivastava proves the existence of bipartite Ramanujan (multi)graphs of all degrees and all sizes. However, that paper did not provide a polynomial time algorithm to actually compute such graphs. Here, we provide a polynomial time algorithm to compute certain expected characteristic polynomials related to this construction. This leads to a deterministic poly… ▽ More The recent work by Marcus, Spielman and Srivastava proves the existence of bipartite Ramanujan (multi)graphs of all degrees and all sizes. However, that paper did not provide a polynomial time algorithm to actually compute such graphs. Here, we provide a polynomial time algorithm to compute certain expected characteristic polynomials related to this construction. This leads to a deterministic polynomial time algorithm to compute bipartite Ramanujan (multi)graphs of all degrees and all sizes. △ Less

Submitted 12 April, 2016; originally announced April 2016.

Input Sparsity Time Low-Rank Approximation via Ridge Leverage Score Sampling

Authors: Michael B. Cohen, Cameron Musco, Christopher Musco

Abstract: We present a new algorithm for finding a near optimal low-rank approximation of a matrix $A$ in $O(nnz(A))$ time. Our method is based on a recursive sampling scheme for computing a representative subset of $A$'s columns, which is then used to find a low-rank approximation. This approach differs substantially from prior $O(nnz(A))$ time algorithms, which are all based on fast Johnson-Lindenstraus… ▽ More We present a new algorithm for finding a near optimal low-rank approximation of a matrix $A$ in $O(nnz(A))$ time. Our method is based on a recursive sampling scheme for computing a representative subset of $A$'s columns, which is then used to find a low-rank approximation. This approach differs substantially from prior $O(nnz(A))$ time algorithms, which are all based on fast Johnson-Lindenstrauss random projections. It matches the guarantees of these methods while offering a number of advantages. Not only are sampling algorithms faster for sparse and structured data, but they can also be applied in settings where random projections cannot. For example, we give new single-pass streaming algorithms for the column subset selection and projection-cost preserving sample problems. Our method has also been used to give the fastest algorithms for provably approximating kernel matrices [MM16]. △ Less

Submitted 6 October, 2016; v1 submitted 23 November, 2015; originally announced November 2015.

Optimal approximate matrix product in terms of stable rank

Authors: Michael B. Cohen, Jelani Nelson, David P. Woodruff

Abstract: We prove, using the subspace embedding guarantee in a black box way, that one can achieve the spectral norm guarantee for approximate matrix multiplication with a dimensionality-reducing map having $m = O(\tilde{r}/\varepsilon^2)$ rows. Here $\tilde{r}$ is the maximum stable rank, i.e. squared ratio of Frobenius and operator norms, of the two matrices being multiplied. This is a quantitative impro… ▽ More We prove, using the subspace embedding guarantee in a black box way, that one can achieve the spectral norm guarantee for approximate matrix multiplication with a dimensionality-reducing map having $m = O(\tilde{r}/\varepsilon^2)$ rows. Here $\tilde{r}$ is the maximum stable rank, i.e. squared ratio of Frobenius and operator norms, of the two matrices being multiplied. This is a quantitative improvement over previous work of [MZ11, KVZ14], and is also optimal for any oblivious dimensionality-reducing map. Furthermore, due to the black box reliance on the subspace embedding property in our proofs, our theorem can be applied to a much more general class of sketching matrices than what was known before, in addition to achieving better bounds. For example, one can apply our theorem to efficient subspace embeddings such as the Subsampled Randomized Hadamard Transform or sparse subspace embeddings, or even with subspace embedding constructions that may be developed in the future. Our main theorem, via connections with spectral error matrix multiplication shown in prior work, implies quantitative improvements for approximate least squares regression and low rank approximation. Our main result has also already been applied to improve dimensionality reduction guarantees for $k$-means clustering [CEMMP14], and implies new results for nonparametric regression [YPW15]. We also separately point out that the proof of the "BSS" deterministic row-sampling result of [BSS12] can be modified to show that for any matrices $A, B$ of stable rank at most $\tilde{r}$, one can achieve the spectral norm guarantee for approximate matrix multiplication of $A^T B$ by deterministically sampling $O(\tilde{r}/\varepsilon^2)$ rows that can be found in polynomial time. The original result of [BSS12] was for rank instead of stable rank. Our observation leads to a stronger version of a main theorem of [KMST10]. △ Less

Submitted 2 March, 2016; v1 submitted 8 July, 2015; originally announced July 2015.

Comments: v3: minor edits; v2: fixed one step in proof of Theorem 9 which was wrong by a constant factor (see the new Lemma 5 and its use; final theorem unaffected)

Approximating Nearest Neighbor Distances

Authors: Michael B. Cohen, Brittany Terese Fasy, Gary L. Miller, Amir Nayyeri, Donald R. Sheehy, Ameya Velingker

Abstract: Several researchers proposed using non-Euclidean metrics on point sets in Euclidean space for clustering noisy data. Almost always, a distance function is desired that recognizes the closeness of the points in the same cluster, even if the Euclidean cluster diameter is large. Therefore, it is preferred to assign smaller costs to the paths that stay close to the input points. In this paper, we co… ▽ More Several researchers proposed using non-Euclidean metrics on point sets in Euclidean space for clustering noisy data. Almost always, a distance function is desired that recognizes the closeness of the points in the same cluster, even if the Euclidean cluster diameter is large. Therefore, it is preferred to assign smaller costs to the paths that stay close to the input points. In this paper, we consider the most natural metric with this property, which we call the nearest neighbor metric. Given a point set P and a path $γ$, our metric charges each point of $γ$ with its distance to P. The total charge along $γ$ determines its nearest neighbor length, which is formally defined as the integral of the distance to the input points along the curve. We describe a $(3+\varepsilon)$-approximation algorithm and a $(1+\varepsilon)$-approximation algorithm to compute the nearest neighbor metric. Both approximation algorithms work in near-linear time. The former uses shortest paths on a sparse graph using only the input points. The latter uses a sparse sample of the ambient space, to find good approximate geodesic paths. △ Less

Submitted 27 February, 2015; originally announced February 2015.

Comments: corrected author name

$\ell_p$ Row Sampling by Lewis Weights

Authors: Michael B. Cohen, Richard Peng

Abstract: We give a simple algorithm to efficiently sample the rows of a matrix while preserving the p-norms of its product with vectors. Given an $n$-by-$d$ matrix $\boldsymbol{\mathit{A}}$, we find with high probability and in input sparsity time an $\boldsymbol{\mathit{A}}'$ consisting of about $d \log{d}$ rescaled rows of $\boldsymbol{\mathit{A}}$ such that… ▽ More We give a simple algorithm to efficiently sample the rows of a matrix while preserving the p-norms of its product with vectors. Given an $n$-by-$d$ matrix $\boldsymbol{\mathit{A}}$, we find with high probability and in input sparsity time an $\boldsymbol{\mathit{A}}'$ consisting of about $d \log{d}$ rescaled rows of $\boldsymbol{\mathit{A}}$ such that $\| \boldsymbol{\mathit{A}} \boldsymbol{\mathit{x}} \|_1$ is close to $\| \boldsymbol{\mathit{A}}' \boldsymbol{\mathit{x}} \|_1$ for all vectors $\boldsymbol{\mathit{x}}$. We also show similar results for all $\ell_p$ that give nearly optimal sample bounds in input sparsity time. Our results are based on sampling by "Lewis weights", which can be viewed as statistical leverage scores of a reweighted matrix. We also give an elementary proof of the guarantees of this sampling process for $\ell_1$. △ Less

Submitted 1 December, 2014; originally announced December 2014.

Dimensionality Reduction for k-Means Clustering and Low Rank Approximation

Authors: Michael B. Cohen, Sam Elder, Cameron Musco, Christopher Musco, Madalina Persu

Abstract: We show how to approximate a data matrix $\mathbf{A}$ with a much smaller sketch $\mathbf{\tilde A}$ that can be used to solve a general class of constrained k-rank approximation problems to within $(1+ε)$ error. Importantly, this class of problems includes $k$-means clustering and unconstrained low rank approximation (i.e. principal component analysis). By reducing data points to just $O(k)$ dime… ▽ More We show how to approximate a data matrix $\mathbf{A}$ with a much smaller sketch $\mathbf{\tilde A}$ that can be used to solve a general class of constrained k-rank approximation problems to within $(1+ε)$ error. Importantly, this class of problems includes $k$-means clustering and unconstrained low rank approximation (i.e. principal component analysis). By reducing data points to just $O(k)$ dimensions, our methods generically accelerate any exact, approximate, or heuristic algorithm for these ubiquitous problems. For $k$-means dimensionality reduction, we provide $(1+ε)$ relative error results for many common sketching techniques, including random row projection, column selection, and approximate SVD. For approximate principal component analysis, we give a simple alternative to known algorithms that has applications in the streaming setting. Additionally, we extend recent work on column-based matrix reconstruction, giving column subsets that not only `cover' a good subspace for $\bv{A}$, but can be used directly to compute this subspace. Finally, for $k$-means clustering, we show how to achieve a $(9+ε)$ approximation by Johnson-Lindenstrauss projecting data points to just $O(\log k/ε^2)$ dimensions. This gives the first result that leverages the specific structure of $k$-means to achieve dimension independent of input size and sublinear in $k$. △ Less

Submitted 2 April, 2015; v1 submitted 24 October, 2014; originally announced October 2014.

Uniform Sampling for Matrix Approximation

Authors: Michael B. Cohen, Yin Tat Lee, Cameron Musco, Christopher Musco, Richard Peng, Aaron Sidford

Abstract: Random sampling has become a critical tool in solving massive matrix problems. For linear regression, a small, manageable set of data rows can be randomly selected to approximate a tall, skinny data matrix, improving processing time significantly. For theoretical performance guarantees, each row must be sampled with probability proportional to its statistical leverage score. Unfortunately, leverag… ▽ More Random sampling has become a critical tool in solving massive matrix problems. For linear regression, a small, manageable set of data rows can be randomly selected to approximate a tall, skinny data matrix, improving processing time significantly. For theoretical performance guarantees, each row must be sampled with probability proportional to its statistical leverage score. Unfortunately, leverage scores are difficult to compute. A simple alternative is to sample rows uniformly at random. While this often works, uniform sampling will eliminate critical row information for many natural instances. We take a fresh look at uniform sampling by examining what information it does preserve. Specifically, we show that uniform sampling yields a matrix that, in some sense, well approximates a large fraction of the original. While this weak form of approximation is not enough for solving linear regression directly, it is enough to compute a better approximation. This observation leads to simple iterative row sampling algorithms for matrix approximation that run in input-sparsity time and preserve row structure and sparsity at all intermediate steps. In addition to an improved understanding of uniform sampling, our main proof introduces a structural result of independent interest: we show that every matrix can be made to have low coherence by reweighting a small subset of its rows. △ Less

Submitted 21 August, 2014; originally announced August 2014.

Preconditioning in Expectation

Authors: Michael B. Cohen, Rasmus Kyng, Jakub W. Pachocki, Richard Peng, Anup Rao

Abstract: We show that preconditioners constructed by random sampling can perform well without meeting the standard requirements of iterative methods. When applied to graph Laplacians, this leads to ultra-sparsifiers that in expectation behave as the nearly-optimal ones given by [Kolla-Makarychev-Saberi-Teng STOC`10]. Combining this with the recursive preconditioning framework by [Spielman-Teng STOC`04] and… ▽ More We show that preconditioners constructed by random sampling can perform well without meeting the standard requirements of iterative methods. When applied to graph Laplacians, this leads to ultra-sparsifiers that in expectation behave as the nearly-optimal ones given by [Kolla-Makarychev-Saberi-Teng STOC`10]. Combining this with the recursive preconditioning framework by [Spielman-Teng STOC`04] and improved embedding algorithms, this leads to algorithms that solve symmetric diagonally dominant linear systems and electrical flow problems in expected time close to $m\log^{1/2}n$ . △ Less

Submitted 23 January, 2014; originally announced January 2014.

Stretching Stretch

Authors: Michael B. Cohen, Gary L. Miller, Jakub W. Pachocki, Richard Peng, Shen Chen Xu

Abstract: We give a generalized definition of stretch that simplifies the efficient construction of low-stretch embeddings suitable for graph algorithms. The generalization, based on discounting highly stretched edges by taking their $p$-th power for some $0 < p < 1$, is directly related to performances of existing algorithms. This discounting of high-stretch edges allows us to treat many classes of edges w… ▽ More We give a generalized definition of stretch that simplifies the efficient construction of low-stretch embeddings suitable for graph algorithms. The generalization, based on discounting highly stretched edges by taking their $p$-th power for some $0 < p < 1$, is directly related to performances of existing algorithms. This discounting of high-stretch edges allows us to treat many classes of edges with coarser granularity. It leads to a two-pass approach that combines bottom-up clustering and top-down decompositions to construct these embeddings in $\mathcal{O}(m\log\log{n})$ time. Our algorithm parallelizes readily and can also produce generalizations of low-stretch subgraphs. △ Less

Submitted 5 February, 2014; v1 submitted 10 January, 2014; originally announced January 2014.