Skip to main content
Cornell University
We gratefully acknowledge support from the Simons Foundation, member institutions, and all contributors. Donate
arxiv logo > math > arXiv:1602.02915

Help | Advanced Search

arXiv logo
Cornell University Logo

quick links

  • Login
  • Help Pages
  • About

Mathematics > Optimization and Control

arXiv:1602.02915 (math)
[Submitted on 9 Feb 2016 (v1), last revised 30 Aug 2021 (this version, v6)]

Title:Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods

Authors:Guoyin Li, Ting Kei Pong
View a PDF of the paper titled Calculus of the exponent of Kurdyka-{\L}ojasiewicz inequality and its applications to linear convergence of first-order methods, by Guoyin Li and Ting Kei Pong
View PDF
Abstract:In this paper, we study the Kurdyka-Łojasiewicz (KL) exponent, an important quantity for analyzing the convergence rate of first-order methods. Specifically, we develop various calculus rules to deduce the KL exponent of new (possibly nonconvex and nonsmooth) functions formed from functions with known KL exponents. In addition, we show that the well-studied Luo-Tseng error bound together with a mild assumption on the separation of stationary values implies that the KL exponent is $\frac12$. The Luo-Tseng error bound is known to hold for a large class of concrete structured optimization problems, and thus we deduce the KL exponent of a large class of functions whose exponents were previously unknown. Building upon this and the calculus rules, we are then able to show that for many convex or nonconvex optimization models for applications such as sparse recovery, their objective function's KL exponent is $\frac12$. This includes the least squares problem with smoothly clipped absolute deviation (SCAD) regularization or minimax concave penalty (MCP) regularization and the logistic regression problem with $\ell_1$ regularization. Since many existing local convergence rate analysis for first-order methods in the nonconvex scenario relies on the KL exponent, our results enable us to obtain explicit convergence rate for various first-order methods when they are applied to a large variety of practical optimization models. Finally, we further illustrate how our results can be applied to establishing local linear convergence of the proximal gradient algorithm and the inertial proximal algorithm with constant step-sizes for some specific models that arise in sparse recovery.
Comments: The paper has been published in Foundations of Computational Mathematics: this https URL. In this update, we added the domain and range of g and h in Theorem 3.5 to remove ambiguity. See also the notes under the previous two updates for changes after the journal version is published
Subjects: Optimization and Control (math.OC); Machine Learning (stat.ML)
Cite as: arXiv:1602.02915 [math.OC]
  (or arXiv:1602.02915v6 [math.OC] for this version)
  https://doi.org/10.48550/arXiv.1602.02915
arXiv-issued DOI via DataCite
Related DOI: https://doi.org/10.1007/s10208-017-9366-8
DOI(s) linking to related resources

Submission history

From: Ting Kei Pong [view email]
[v1] Tue, 9 Feb 2016 09:50:00 UTC (30 KB)
[v2] Sat, 3 Sep 2016 06:21:17 UTC (36 KB)
[v3] Thu, 25 May 2017 05:28:35 UTC (40 KB)
[v4] Thu, 17 Aug 2017 13:57:45 UTC (40 KB)
[v5] Fri, 19 Jan 2018 03:40:29 UTC (40 KB)
[v6] Mon, 30 Aug 2021 03:25:06 UTC (40 KB)
Full-text links:

Access Paper:

    View a PDF of the paper titled Calculus of the exponent of Kurdyka-{\L}ojasiewicz inequality and its applications to linear convergence of first-order methods, by Guoyin Li and Ting Kei Pong
  • View PDF
  • TeX Source
  • Other Formats
view license
Current browse context:
math.OC
< prev   |   next >
new | recent | 2016-02
Change to browse by:
math
stat
stat.ML

References & Citations

  • NASA ADS
  • Google Scholar
  • Semantic Scholar
a export BibTeX citation Loading...

BibTeX formatted citation

×
Data provided by:

Bookmark

BibSonomy logo Reddit logo

Bibliographic and Citation Tools

Bibliographic Explorer (What is the Explorer?)
Connected Papers (What is Connected Papers?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)

Code, Data and Media Associated with this Article

alphaXiv (What is alphaXiv?)
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Hugging Face (What is Huggingface?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)

Demos

Replicate (What is Replicate?)
Hugging Face Spaces (What is Spaces?)
TXYZ.AI (What is TXYZ.AI?)

Recommenders and Search Tools

Influence Flower (What are Influence Flowers?)
CORE Recommender (What is CORE?)
  • Author
  • Venue
  • Institution
  • Topic

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)
  • About
  • Help
  • contact arXivClick here to contact arXiv Contact
  • subscribe to arXiv mailingsClick here to subscribe Subscribe
  • Copyright
  • Privacy Policy
  • Web Accessibility Assistance
  • arXiv Operational Status
    Get status notifications via email or slack