Showing 1–7 of 7 results for author: Barré, M

A note on approximate accelerated forward-backward methods with absolute and relative errors, and possibly strongly convex objectives

Authors: Mathieu Barré, Adrien Taylor, Francis Bach

Abstract: In this short note, we provide a simple version of an accelerated forward-backward method (a.k.a. Nesterov's accelerated proximal gradient method) possibly relying on approximate proximal operators and allowing to exploit strong convexity of the objective function. The method supports both relative and absolute errors, and its behavior is illustrated on a set of standard numerical experiments. Usi… ▽ More In this short note, we provide a simple version of an accelerated forward-backward method (a.k.a. Nesterov's accelerated proximal gradient method) possibly relying on approximate proximal operators and allowing to exploit strong convexity of the objective function. The method supports both relative and absolute errors, and its behavior is illustrated on a set of standard numerical experiments. Using the same developments, we further provide a version of the accelerated proximal hybrid extragradient method of Monteiro and Svaiter (2013) possibly exploiting strong convexity of the objective function. △ Less

Submitted 21 January, 2022; v1 submitted 29 June, 2021; originally announced June 2021.

Comments: Published in Open Journal on Mathematical Optimization (https://ojmo.centre-mersenne.org/item/10.5802/ojmo.12.pdf). Code available at https://github.com/mathbarre/StronglyConvexForwardBackward

Journal ref: A note on approximate accelerated forward-backward methods with absolute and relative errors, and possibly strongly convex objectives. Open Journal of Mathematical Optimization, Volume 3 (2022), article no. 1, 15 p

Convergence of Constrained Anderson Acceleration

Authors: Mathieu Barré, Adrien Taylor, Alexandre d'Aspremont

Abstract: We prove non asymptotic linear convergence rates for the constrained Anderson acceleration extrapolation scheme. These guarantees come from new upper bounds on the constrained Chebyshev problem, which consists in minimizing the maximum absolute value of a polynomial on a bounded real interval with $l_1$ constraints on its coefficients vector. Constrained Anderson Acceleration has a numerical cost… ▽ More We prove non asymptotic linear convergence rates for the constrained Anderson acceleration extrapolation scheme. These guarantees come from new upper bounds on the constrained Chebyshev problem, which consists in minimizing the maximum absolute value of a polynomial on a bounded real interval with $l_1$ constraints on its coefficients vector. Constrained Anderson Acceleration has a numerical cost comparable to that of the original scheme. △ Less

Submitted 29 October, 2020; originally announced October 2020.

Averaging Atmospheric Gas Concentration Data using Wasserstein Barycenters

Authors: Mathieu Barré, Clément Giron, Matthieu Mazzolini, Alexandre d'Aspremont

Abstract: Hyperspectral satellite images report greenhouse gas concentrations worldwide on a daily basis. While taking simple averages of these images over time produces a rough estimate of relative emission rates, atmospheric transport means that simple averages fail to pinpoint the source of these emissions. We propose using Wasserstein barycenters coupled with weather data to average gas concentration da… ▽ More Hyperspectral satellite images report greenhouse gas concentrations worldwide on a daily basis. While taking simple averages of these images over time produces a rough estimate of relative emission rates, atmospheric transport means that simple averages fail to pinpoint the source of these emissions. We propose using Wasserstein barycenters coupled with weather data to average gas concentration data sets and better concentrate the mass around significant sources. △ Less

Submitted 6 October, 2020; originally announced October 2020.

Principled Analyses and Design of First-Order Methods with Inexact Proximal Operators

Authors: Mathieu Barré, Adrien Taylor, Francis Bach

Abstract: Proximal operations are among the most common primitives appearing in both practical and theoretical (or high-level) optimization methods. This basic operation typically consists in solving an intermediary (hopefully simpler) optimization problem. In this work, we survey notions of inaccuracies that can be used when solving those intermediary optimization problems. Then, we show that worst-case gu… ▽ More Proximal operations are among the most common primitives appearing in both practical and theoretical (or high-level) optimization methods. This basic operation typically consists in solving an intermediary (hopefully simpler) optimization problem. In this work, we survey notions of inaccuracies that can be used when solving those intermediary optimization problems. Then, we show that worst-case guarantees for algorithms relying on such inexact proximal operations can be systematically obtained through a generic procedure based on semidefinite programming. This methodology is primarily based on the approach introduced by Drori and Teboulle (2014) and on convex interpolation results, and allows producing non-improvable worst-case analyzes. In other words, for a given algorithm, the methodology generates both worst-case certificates (i.e., proofs) and problem instances on which those bounds are achieved. Relying on this methodology, we study numerical worst-case performances of a few basic methods relying on inexact proximal operations including accelerated variants, and design a variant with optimized worst-case behaviour. We further illustrate how to extend the approach to support strongly convex objectives by studying a simple relatively inexact proximal minimization method. △ Less

Submitted 29 June, 2021; v1 submitted 10 June, 2020; originally announced June 2020.

Comments: A-HPE and inexact forward-backward methods for strongly convex objectives were removed in this version, they can now be found in the work entitled "A note on approximate accelerated forward-backward methods with absolute and relative errors, and possibly strongly convex objectives". Code available at https://github.com/mathbarre/InexactProximalOperators/tree/version-2

Complexity Guarantees for Polyak Steps with Momentum

Authors: Mathieu Barré, Adrien Taylor, Alexandre d'Aspremont

Abstract: In smooth strongly convex optimization, knowledge of the strong convexity parameter is critical for obtaining simple methods with accelerated rates. In this work, we study a class of methods, based on Polyak steps, where this knowledge is substituted by that of the optimal value, $f_*$. We first show slightly improved convergence bounds than previously known for the classical case of simple gradie… ▽ More In smooth strongly convex optimization, knowledge of the strong convexity parameter is critical for obtaining simple methods with accelerated rates. In this work, we study a class of methods, based on Polyak steps, where this knowledge is substituted by that of the optimal value, $f_*$. We first show slightly improved convergence bounds than previously known for the classical case of simple gradient descent with Polyak steps, we then derive an accelerated gradient method with Polyak steps and momentum, along with convergence guarantees. △ Less

Submitted 3 July, 2020; v1 submitted 3 February, 2020; originally announced February 2020.

Comments: Accepted to COLT2020

Polyak Steps for Adaptive Fast Gradient Method

Authors: Mathieu Barré, Alexandre d'Aspremont

Abstract: Accelerated algorithms for minimizing smooth strongly convex functions usually require knowledge of the strong convexity parameter $μ$. In the case of an unknown $μ$, current adaptive techniques are based on restart schemes. When the optimal value $f^*$ is known, these strategies recover the accelerated linear convergence bound without additional grid search. In this paper we propose a new approac… ▽ More Accelerated algorithms for minimizing smooth strongly convex functions usually require knowledge of the strong convexity parameter $μ$. In the case of an unknown $μ$, current adaptive techniques are based on restart schemes. When the optimal value $f^*$ is known, these strategies recover the accelerated linear convergence bound without additional grid search. In this paper we propose a new approach that has the same bound without any restart, using an online estimation of strong convexity parameter. We show the robustness of the Fast Gradient Method when using a sequence of upper bounds on $μ$. We also present a good candidate for this estimate sequence and detail consistent empirical results. △ Less

Submitted 7 June, 2019; originally announced June 2019.

An M* Proxy for Sparse Recovery Performance

Authors: Mathieu Barré, Alexandre d'Aspremont

Abstract: This paper provides a new tractable lower bound for the sparse recovery threshold of sensing matrices. This lower bound is used as a proxy to quantify the quality of sensing matrices in two different applications. First, it serves as regularization for the classical dictionary learning problem in order to learn dictionaries with better generalisation properties on unseen data. Then, the proxy is u… ▽ More This paper provides a new tractable lower bound for the sparse recovery threshold of sensing matrices. This lower bound is used as a proxy to quantify the quality of sensing matrices in two different applications. First, it serves as regularization for the classical dictionary learning problem in order to learn dictionaries with better generalisation properties on unseen data. Then, the proxy is used to design sampling schemes for MRI acquisition that exhibit high reconstruction performances. △ Less

Submitted 14 December, 2020; v1 submitted 5 October, 2018; originally announced October 2018.