-
Heavy Ball and Nesterov Accelerations with Hessian-driven Damping for Nonconvex Optimization
Authors:
N. Hadjisavvas,
F. Lara,
R. T. Marcavillaca,
P. T. Vuong
Abstract:
In this work, we investigate a second-order dynamical system with Hessian-driven damping tailored for a class of nonconvex functions called strongly quasiconvex. Buil\-ding upon this continuous-time model, we derive two discrete-time gra\-dient-based algorithms through time discretizations. The first is a Heavy Ball method with Hessian correction, incorporating cur\-va\-tu\-re-dependent terms that…
▽ More
In this work, we investigate a second-order dynamical system with Hessian-driven damping tailored for a class of nonconvex functions called strongly quasiconvex. Buil\-ding upon this continuous-time model, we derive two discrete-time gra\-dient-based algorithms through time discretizations. The first is a Heavy Ball method with Hessian correction, incorporating cur\-va\-tu\-re-dependent terms that arise from discretizing the Hessian damping component. The second is a Nesterov-type accelerated method with adaptive momentum, fea\-tu\-ring correction terms that account for local curvature. Both algorithms aim to enhance stability and convergence performance, particularly by mi\-ti\-ga\-ting oscillations commonly observed in cla\-ssi\-cal momentum me\-thods. Furthermore, in both cases we establish li\-near convergence to the optimal solution for the iterates and functions values. Our approach highlights the rich interplay between continuous-time dynamics and discrete optimization algorithms in the se\-tting of strongly quasiconvex objectives. Numerical experiments are presented to support obtained results.
△ Less
Submitted 18 June, 2025;
originally announced June 2025.
-
Asymptotic Convergence Analysis of High-Order Proximal-Point Methods Beyond Sublinear Rates
Authors:
Masoud Ahookhosh,
Alfredo Iusem,
Alireza Kabgani,
Felipe Lara
Abstract:
This paper investigates the asymptotic convergence behavior of high-order proximal-point algorithms (HiPPA) toward global minimizers, extending the analysis beyond sublinear convergence rate results. Specifically, we consider the proximal operator of a lower semicontinuous function augmented with a $p$th-order regularization for $p>1$, and establish the convergence of HiPPA to a global minimizer w…
▽ More
This paper investigates the asymptotic convergence behavior of high-order proximal-point algorithms (HiPPA) toward global minimizers, extending the analysis beyond sublinear convergence rate results. Specifically, we consider the proximal operator of a lower semicontinuous function augmented with a $p$th-order regularization for $p>1$, and establish the convergence of HiPPA to a global minimizer with a particular focus on its convergence rate. To this end, we focus on minimizing the class of uniformly quasiconvex functions, including strongly convex, uniformly convex, and strongly quasiconvex functions as special cases. Our analysis reveals the following convergence behaviors of HiPPA when the uniform quasiconvexity modulus admits a power function of degree $q$ as a lower bound on an interval $\mathcal{I}$: (i) for $q\in (1,2]$ and $\mathcal{I}=[0,1)$, HiPPA exhibits local linear rate for $p\in (1,2)$; (ii) for $q=2$ and $\mathcal{I}=[0,\infty)$, HiPPA converges linearly for $p=2$; (iii) for $p=q>2$ and $\mathcal{I}=[0,\infty)$, HiPPA converges linearly; (iv) for $q\geq 2$ and $\mathcal{I}=[0,\infty)$, HiPPA achieves superlinear rate for $p>q$. Notably, to our knowledge, some of these results are novel, even in the context of strongly or uniformly convex functions, offering new insights into optimizing generalized convex problems.
△ Less
Submitted 26 May, 2025;
originally announced May 2025.
-
Delayed Feedback in Online Non-Convex Optimization: A Non-Stationary Approach with Applications
Authors:
Felipe Lara,
Cristian Vega
Abstract:
We study non-convex delayed-noise online optimization problems by evaluating dynamic regret in the non-stationary setting when the loss functions are quasar-convex. In particular, we consider scenarios involving quasar-convex functions either with a Lipschitz gradient or weakly smooth and, for each case, we ensure bounded dynamic regret in terms of cumulative path variation achieving sub-linear re…
▽ More
We study non-convex delayed-noise online optimization problems by evaluating dynamic regret in the non-stationary setting when the loss functions are quasar-convex. In particular, we consider scenarios involving quasar-convex functions either with a Lipschitz gradient or weakly smooth and, for each case, we ensure bounded dynamic regret in terms of cumulative path variation achieving sub-linear regret rates. Furthermore, we illustrate the flexibility of our framework by applying it to both theoretical settings such as zeroth-order (bandit) and also to practical applications with quadratic fractional functions. Moreover, we provide new examples of non-convex functions that are quasar-convex by proving that the class of differentiable strongly quasiconvex functions (Polyak 1966) are strongly quasar-convex on convex compact sets. Finally, several numerical experiments validate our theoretical findings, illustrating the effectiveness of our approach.
△ Less
Submitted 8 May, 2025; v1 submitted 18 December, 2024;
originally announced December 2024.
-
Strongly quasiconvex functions: what we know (so far)
Authors:
Sorin-Mihai Grad,
Felipe Lara,
Raúl T. Marcavillaca
Abstract:
Introduced by Polyak in 1966, the class of strongly quasiconvex functions includes some interesting nonconvex members, like the square root of the Euclidean norm or ratios with a nonnegative strongly convex numerator and a concave and positive denominator. This survey collects the vast majority of the results involving strongly quasiconvex functions available in the literature at the moment, prese…
▽ More
Introduced by Polyak in 1966, the class of strongly quasiconvex functions includes some interesting nonconvex members, like the square root of the Euclidean norm or ratios with a nonnegative strongly convex numerator and a concave and positive denominator. This survey collects the vast majority of the results involving strongly quasiconvex functions available in the literature at the moment, presenting, in particular, algorithms for minimizing such functions, and suggests some directions where additional investigations would be welcome.
△ Less
Submitted 30 October, 2024;
originally announced October 2024.
-
Weak sharp minima at infinity and solution stability in mathematical programming via asymptotic analysis
Authors:
Felipe Lara,
Nguyen Van Tuyen,
Tran Van Nghi
Abstract:
We develop sufficient conditions for the existence of the weak sharp minima at infinity property for nonsmooth optimization problems via asymptotic cones and generalized asymptotic functions. Next, we show that these conditions are also useful for studying the solution stability of nonconvex optimization problems under linear perturbations. Finally, we provide applications for a subclass of quasic…
▽ More
We develop sufficient conditions for the existence of the weak sharp minima at infinity property for nonsmooth optimization problems via asymptotic cones and generalized asymptotic functions. Next, we show that these conditions are also useful for studying the solution stability of nonconvex optimization problems under linear perturbations. Finally, we provide applications for a subclass of quasiconvex functions which is stable under linear additivity and includes the convex ones.
△ Less
Submitted 6 October, 2024;
originally announced October 2024.
-
Characterizations, Dynamical Systems and Gradient Methods for Strongly Quasiconvex Functions
Authors:
Felipe Lara,
Raúl T. Marcavillaca,
Phan T. Vuong
Abstract:
We study differentiable strongly quasiconvex functions for providing new properties for algorithmic and monotonicity purposes. Furthemore, we provide insights into the decreasing behaviour of strongly quasiconvex functions, applying this for establishing exponential convergence for first- and second-order gradient systems without relying on the usual Lipschitz continuity assumption on the gradient…
▽ More
We study differentiable strongly quasiconvex functions for providing new properties for algorithmic and monotonicity purposes. Furthemore, we provide insights into the decreasing behaviour of strongly quasiconvex functions, applying this for establishing exponential convergence for first- and second-order gradient systems without relying on the usual Lipschitz continuity assumption on the gradient of the function. The explicit discretization of the first-order dynamical system leads to the gradient descent method while discretization of the second-order dynamical system with viscous damping recovers the heavy ball method. We establish the linear convergence of both methods under suitable conditions on the parameters as well as comparisons with other classes of nonconvex functions used in the gradient descent literature.
△ Less
Submitted 4 October, 2024;
originally announced October 2024.
-
Regular Subgradients of Marginal Functions with Applications to Calculus and Bilevel Programming
Authors:
Le Phuoc Hai,
Felipe Lara,
Boris S. Mordukhovich
Abstract:
The paper addresses the study and applications of a broad class of extended-real-valued functions, known as optimal value or marginal functions, which are frequently appeared in variational analysis, parametric optimization, and a variety of applications. Functions of this type are intrinsically nonsmooth and require the usage of tools of generalized differentiation. The main results of this paper…
▽ More
The paper addresses the study and applications of a broad class of extended-real-valued functions, known as optimal value or marginal functions, which are frequently appeared in variational analysis, parametric optimization, and a variety of applications. Functions of this type are intrinsically nonsmooth and require the usage of tools of generalized differentiation. The main results of this paper provide novel evaluations and exact calculations of regular/Fréchet subgradients and their singular counterparts for general classes of marginal functions via their given data. The obtained results are applied to establishing new calculus rules for such subgradients and necessary optimality conditions in bilevel programming
△ Less
Submitted 4 February, 2025; v1 submitted 31 May, 2024;
originally announced May 2024.
-
Optimal Trade Characterizations in Multi-Asset Crypto-Financial Markets
Authors:
C. Escudero,
F. Lara,
M. Sama
Abstract:
This work focuses on the mathematical study of constant function market makers. We rigorously establish the conditions for optimal trading under the assumption of a quasilinear, but not necessarily convex (or concave), trade function. This generalizes previous results that used convexity, and also guarantees the robustness against arbitrage of so-designed automatic market makers. The theoretical r…
▽ More
This work focuses on the mathematical study of constant function market makers. We rigorously establish the conditions for optimal trading under the assumption of a quasilinear, but not necessarily convex (or concave), trade function. This generalizes previous results that used convexity, and also guarantees the robustness against arbitrage of so-designed automatic market makers. The theoretical results are illustrated by families of examples given by generalized means, and also by numerical simulations in certain concrete cases. These simulations along with the mathematical analysis suggest that the quasilinear-trade-function based automatic market makers might replicate the functioning of those based on convex functions, in particular regarding their resilience to arbitrage.
△ Less
Submitted 10 May, 2024;
originally announced May 2024.
-
HORSES3D: a high-order discontinuous Galerkin solver for flow simulations and multi-physics applications
Authors:
E. Ferrer,
G. Rubio,
G. Ntoukas,
W. Laskowski,
O. A. Mariño,
S. Colombo,
A. Mateo-Gabín,
F. Manrique de Lara,
D. Huergo,
J. Manzanero,
A. M. Rueda-Ramírez,
D. A. Kopriva,
E. Valero
Abstract:
We present the latest developments of our High-Order Spectral Element Solver (HORSES3D), an open source high-order discontinuous Galerkin framework, capable of solving a variety of flow applications, including compressible flows (with or without shocks), incompressible flows, various RANS and LES turbulence models, particle dynamics, multiphase flows, and aeroacoustics. We provide an overview of t…
▽ More
We present the latest developments of our High-Order Spectral Element Solver (HORSES3D), an open source high-order discontinuous Galerkin framework, capable of solving a variety of flow applications, including compressible flows (with or without shocks), incompressible flows, various RANS and LES turbulence models, particle dynamics, multiphase flows, and aeroacoustics. We provide an overview of the high-order spatial discretisation (including energy/entropy stable schemes) and anisotropic p-adaptation capabilities. The solver is parallelised using MPI and OpenMP showing good scalability for up to 1000 processors. Temporal discretisations include explicit, implicit, multigrid, and dual time-stepping schemes with efficient preconditioners. Additionally, we facilitate meshing and simulating complex geometries through a mesh-free immersed boundary technique. We detail the available documentation and the test cases included in the GitHub repository.
△ Less
Submitted 20 June, 2022;
originally announced June 2022.
-
On Gegenbauer Point Processes on the unit interval
Authors:
Carlos Beltrán,
Antonia M. Delgado,
Lidia Fernández,
Joaquín F. Sánchez Lara
Abstract:
In this note we compute the logarithmic energy of points in the unit interval $[-1,1]$ chosen from different Gegenbauer Determinantal Point Processes. We check that all the different families of Gegenbauer polynomials yield the same asymptotic result to third order, we compute exactly the value for Chebyshev polynomials and we give a closed expresion for the minimal possible logarithmic energy. Th…
▽ More
In this note we compute the logarithmic energy of points in the unit interval $[-1,1]$ chosen from different Gegenbauer Determinantal Point Processes. We check that all the different families of Gegenbauer polynomials yield the same asymptotic result to third order, we compute exactly the value for Chebyshev polynomials and we give a closed expresion for the minimal possible logarithmic energy. The comparison suggests that DPPs cannot match the value of the minimum beyond the third asymptotic term.
△ Less
Submitted 12 October, 2021;
originally announced October 2021.
-
An extension of the proximal point algorithm beyond convexity
Authors:
Sorin-Mihai Grad,
Felipe Lara
Abstract:
We introduce and investigate a new generalized convexity notion for functions called prox-convexity. The proximity operator of such a function is single-valued and firmly nonexpansive. We provide examples of (strongly) quasiconvex, weakly convex, and DC (difference of convex) functions that are prox-convex, however none of these classes fully contains the one of prox-convex functions or is include…
▽ More
We introduce and investigate a new generalized convexity notion for functions called prox-convexity. The proximity operator of such a function is single-valued and firmly nonexpansive. We provide examples of (strongly) quasiconvex, weakly convex, and DC (difference of convex) functions that are prox-convex, however none of these classes fully contains the one of prox-convex functions or is included into it. We show that the classical proximal point algorithm remains convergent when the convexity of the proper lower semicontinuous function to be minimized is relaxed to prox-convexity.
△ Less
Submitted 18 April, 2021;
originally announced April 2021.
-
Equilibrium problems in weakly admissible external fields created by pointwise charges
Authors:
Ramón Orive,
Joaquín F. Sánchez Lara,
Franck Wielonsky
Abstract:
The main subject of this paper is equilibrium problems on an unbounded conductor $Σ$ of the complex plane in the presence of a weakly admissible external field. An admissible external field $Q$ on $Σ$ satisfies, along with other mild conditions, the following growth property at infinity: $$\lim_{|x| \rightarrow \infty}(Q(x) - \log |x|) = +\infty.$$ This condition guarantees the existence and uniqu…
▽ More
The main subject of this paper is equilibrium problems on an unbounded conductor $Σ$ of the complex plane in the presence of a weakly admissible external field. An admissible external field $Q$ on $Σ$ satisfies, along with other mild conditions, the following growth property at infinity: $$\lim_{|x| \rightarrow \infty}(Q(x) - \log |x|) = +\infty.$$ This condition guarantees the existence and uniqueness of the equilibrium measure in the presence of $Q$, and the compactness of its support. In the last 10-15 years, several papers have dealt with weakly admissible external fields, in the sense that $Q$ satisfies a weaker condition at infinity, namely, $$\exists M\in(-\infty,\infty],\quad\liminf_{|x| \rightarrow \infty}(Q(x) - \log |x|) = M.$$ Under this last assumption, there still exists a unique equilibrium measure in the external field $Q$, but the support need not be a compact subset of $Σ$ anymore. In most examples considered in the literature the support is indeed unbounded. Our main goal in this paper is to illustrate this topic by means of a simple class of external fields on the real axis created by a pair of attractive and repellent charges in the complex plane, and to study the dynamics of the associated equilibrium measures as the strength of the charges evolves. As one of our findings, we exhibit configurations where the support of the equilibrium measure in a weakly admissible external field is a compact subset of the real axis. To achieve our goal, we extend some results from potential theory, known for admissible external fields, to the weakly admissible case. These new results may be of independent interest. Finally, the so--called signed equilibrium measure is an important tool in our analysis. Its relationship with the (positive) equilibrium measure is also explored.
△ Less
Submitted 7 March, 2019; v1 submitted 4 May, 2018;
originally announced May 2018.
-
Equilibrium measures in the presence of weak rational external fields
Authors:
Ramon Orive,
Joaquin F. Sanchez Lara
Abstract:
In this paper equilibrium measures in the presence of external fields created by fixed charges are analyzed. These external fields are a particular case of the so-called rational external fields (in the sense that their derivatives are rational functions). Along with some general results, a thorough analysis of the particular case of two fixed negative charges (``attractors') is presented. As for…
▽ More
In this paper equilibrium measures in the presence of external fields created by fixed charges are analyzed. These external fields are a particular case of the so-called rational external fields (in the sense that their derivatives are rational functions). Along with some general results, a thorough analysis of the particular case of two fixed negative charges (``attractors') is presented. As for the main tools used, this paper is a natural continuation of \cite{MOR2015}, where polynomial external fields were thoroughly studied, and \cite{OrSL2015}, where rational external fields with a polynomial part were considered. However, the absence of the polynomial part in the external fields analyzed in the current paper adds a considerable difficulty to solve the problem and justifies its separated treatment; moreover, it is noteworthy to point out the simplicity and beauty of the results obtained.
△ Less
Submitted 6 May, 2016;
originally announced May 2016.
