-
An inertial proximal splitting algorithm for hierarchical bilevel equilibria in Hilbert spaces
Authors:
Aicha Balhag,
Zakaria Mazgouri,
Hassan Riahi,
Michel Théra
Abstract:
In this article, we aim to approximate a solution to the bilevel equilibrium problem $\mathbf{(BEP})$ for short: find $\bar{x} \in \mathbf{S}_f$ such that $
g(\bar{x}, y) \geq 0, \,\, \forall y \in \mathbf{S}_f, $
where $
\mathbf{S}_f = \{ u \in \mathbf{K} : f(u, z) \geq 0, \forall z \in \mathbf{K} \}. $
Here, $\mathbf{K}$ is a closed convex subset of a real Hilbert space $\mathcal{H}$, and $f$ an…
▽ More
In this article, we aim to approximate a solution to the bilevel equilibrium problem $\mathbf{(BEP})$ for short: find $\bar{x} \in \mathbf{S}_f$ such that $
g(\bar{x}, y) \geq 0, \,\, \forall y \in \mathbf{S}_f, $
where $
\mathbf{S}_f = \{ u \in \mathbf{K} : f(u, z) \geq 0, \forall z \in \mathbf{K} \}. $
Here, $\mathbf{K}$ is a closed convex subset of a real Hilbert space $\mathcal{H}$, and $f$ and $g$ are two real-valued bifunctions defined on $\mathbf{K} \times \mathbf{K}$. We propose an inertial version of the proximal splitting algorithm introduced by Z. Chbani and H. Riahi: \textit{Weak and strong convergence of prox-penalization and splitting algorithms for bilevel equilibrium problems}. \textit{Numer. Algebra Control Optim.}, 3 (2013), pp. 353-366. Under suitable conditions, we establish the weak and strong convergence of the sequence generated by the proposed iterative method. We also report a numerical example illustrating our theoretical result.
△ Less
Submitted 28 February, 2025;
originally announced February 2025.
-
Weak and strong convergence of an inertial proximal method for solving bilevel monotone equilibrium problems
Authors:
AÏcha Balhag,
Zakaria Mazgouri,
Michel Théra
Abstract:
In this paper, we introduce an inertial proximal method for solving a bilevel problem involving two monotone equilibrium bifunctions in Hilbert spaces. Under suitable conditions and without any restrictive assumption on the trajectories, the weak and strong convergence of the sequence generated by the iterative method are established. Two particular cases illustrating the proposed method are there…
▽ More
In this paper, we introduce an inertial proximal method for solving a bilevel problem involving two monotone equilibrium bifunctions in Hilbert spaces. Under suitable conditions and without any restrictive assumption on the trajectories, the weak and strong convergence of the sequence generated by the iterative method are established. Two particular cases illustrating the proposed method are thereafter discussed with respect to hierarchical minimization problems and equilibrium problems under saddle point constraint. Furthermore, a numerical example is given to demonstrate the implementability of our algorithm. The algorithm and its convergence results improve and develop previous results in the field.
△ Less
Submitted 19 October, 2022;
originally announced October 2022.
-
Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the Hessian
Authors:
Hedy Attouch,
Aicha Balhag,
Zaki Chbani,
Hassan Riahi
Abstract:
In a Hilbert setting, for convex differentiable optimization, we consider accelerated gradient dynamics combining Tikhonov regularization with Hessian-driven damping. The Tikhonov regularization parameter is assumed to tend to zero as time tends to infinity, which preserves equilibria. The presence of the Tikhonov regularization term induces a strong convexity property which vanishes asymptoticall…
▽ More
In a Hilbert setting, for convex differentiable optimization, we consider accelerated gradient dynamics combining Tikhonov regularization with Hessian-driven damping. The Tikhonov regularization parameter is assumed to tend to zero as time tends to infinity, which preserves equilibria. The presence of the Tikhonov regularization term induces a strong convexity property which vanishes asymptotically. To take advantage of the exponential convergence rates attached to the heavy ball method in the strongly convex case, we consider the inertial dynamic where the viscous damping coefficient is taken proportional to the square root of the Tikhonov regularization parameter, and therefore also converges towards zero. Moreover, the dynamic involves a geometric damping which is driven by the Hessian of the function to be minimized, which induces a significant attenuation of the oscillations. Under an appropriate tuning of the parameters, based on Lyapunov's analysis, we show that the trajectories have at the same time several remarkable properties: they provide fast convergence of values, fast convergence of gradients towards zero, and strong convergence to the minimum norm minimizer. This study extends a previous paper by the authors where similar issues were examined but without the presence of Hessian driven damping.
△ Less
Submitted 31 March, 2022; v1 submitted 10 March, 2022;
originally announced March 2022.
-
Damped inertial dynamics with vanishing Tikhonov regularization: strong asymptotic convergence towards the minimum norm solution
Authors:
Hedy Attouch,
Aicha Balhag,
Zaki Chbani,
Hassan Riahi
Abstract:
In a Hilbert space, we provide a fast dynamic approach to the hierarchical minimization problem which consists in finding the minimum norm solution of a convex minimization problem. For this, we study the convergence properties of the trajectories generated by a damped inertial dynamic with Tikhonov regularization. When the time goes to infinity, the Tikhonov regularization parameter is supposed t…
▽ More
In a Hilbert space, we provide a fast dynamic approach to the hierarchical minimization problem which consists in finding the minimum norm solution of a convex minimization problem. For this, we study the convergence properties of the trajectories generated by a damped inertial dynamic with Tikhonov regularization. When the time goes to infinity, the Tikhonov regularization parameter is supposed to tend towards zero, not too fast, which is a key property to make the trajectories strongly converge towards the minimizer of $f$ of minimum norm. According to the structure of the heavy ball method for strongly convex functions, the viscous damping coefficient is proportional to the square root of the Tikhonov regularization parameter. Therefore, it also converges to zero, which will ensure rapid convergence of values. Precisely, under a proper tuning of these parameters, based on Lyapunov's analysis, we show that the trajectories strongly converge towards the minimizer of minimum norm, and we provide the convergence rate of the values. We show a trade off between the property of fast convergence of values, and the property of strong convergence towards the minimum norm solution. This study improves several previous works where this type of results was obtained under restrictive hypotheses.
△ Less
Submitted 31 July, 2021;
originally announced August 2021.
-
Fast convex optimization via inertial dynamics combining viscous and Hessian-driven damping with time rescaling
Authors:
Hedy Attouch,
Aicha Balhag,
Zaki Chbani,
Hassan Riahi
Abstract:
In a Hilbert setting, we develop fast methods for convex unconstrained optimization. We rely on the asymptotic behavior of an inertial system combining geometric damping with temporal scaling. The convex function to minimize enters the dynamic via its gradient. The dynamic includes three coefficients varying with time, one is a viscous damping coefficient, the second is attached to the Hessian-dri…
▽ More
In a Hilbert setting, we develop fast methods for convex unconstrained optimization. We rely on the asymptotic behavior of an inertial system combining geometric damping with temporal scaling. The convex function to minimize enters the dynamic via its gradient. The dynamic includes three coefficients varying with time, one is a viscous damping coefficient, the second is attached to the Hessian-driven damping, the third is a time scaling coefficient. We study the convergence rate of the values under general conditions involving the damping and the time scale coefficients. The obtained results are based on a new Lyapunov analysis and they encompass known results on the subject. We pay particular attention to the case of an asymptotically vanishing viscous damping, which is directly related to the accelerated gradient method of Nesterov. The Hessian-driven damping significantly reduces the oscillatory aspects. As a main result, we obtain an exponential rate of convergence of values without assuming the strong convexity of the objective function. The temporal discretization of these dynamics opens the gate to a large class of inertial optimization algorithms.
△ Less
Submitted 16 September, 2020;
originally announced September 2020.
