Showing 1–2 of 2 results for author: Defazio, A J

Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems

Authors: Aaron J. Defazio, Tibério S. Caetano, Justin Domke

Abstract: Recent advances in optimization theory have shown that smooth strongly convex finite sums can be minimized faster than by treating them as a black box "batch" problem. In this work we introduce a new method in this class with a theoretical convergence rate four times faster than existing methods, for sums with sufficiently many terms. This method is also amendable to a sampling without replacement… ▽ More Recent advances in optimization theory have shown that smooth strongly convex finite sums can be minimized faster than by treating them as a black box "batch" problem. In this work we introduce a new method in this class with a theoretical convergence rate four times faster than existing methods, for sums with sufficiently many terms. This method is also amendable to a sampling without replacement scheme that in practice gives further speed-ups. We give empirical results showing state of the art performance. △ Less

Submitted 10 July, 2014; originally announced July 2014.

Journal ref: International Conference on Machine Learning 2014

A Convex Formulation for Learning Scale-Free Networks via Submodular Relaxation

Authors: Aaron J. Defazio, Tiberio S. Caetano

Abstract: A key problem in statistics and machine learning is the determination of network structure from data. We consider the case where the structure of the graph to be reconstructed is known to be scale-free. We show that in such cases it is natural to formulate structured sparsity inducing priors using submodular functions, and we use their Lovász extension to obtain a convex relaxation. For tractable… ▽ More A key problem in statistics and machine learning is the determination of network structure from data. We consider the case where the structure of the graph to be reconstructed is known to be scale-free. We show that in such cases it is natural to formulate structured sparsity inducing priors using submodular functions, and we use their Lovász extension to obtain a convex relaxation. For tractable classes such as Gaussian graphical models, this leads to a convex optimization problem that can be efficiently solved. We show that our method results in an improvement in the accuracy of reconstructed networks for synthetic data. We also show how our prior encourages scale-free reconstructions on a bioinfomatics dataset. △ Less

Submitted 10 July, 2014; originally announced July 2014.

Journal ref: Advances in Neural Information Processing Systems 25 (NIPS 2012)