Showing 1–2 of 2 results for author: Lepage-Saucier, N

Anderson acceleration with adaptive relaxation for convergent fixed-point iterations

Authors: Nicolas Lepage-Saucier

Abstract: Two adaptive relaxation strategies are proposed for Anderson acceleration. They are specifically designed for applications in which mappings converge to a fixed point. Their superiority over alternative Anderson acceleration is demonstrated for linear contraction mappings. Both strategies perform well in three nonlinear fixed-point applications that include partial differential equations and the E… ▽ More Two adaptive relaxation strategies are proposed for Anderson acceleration. They are specifically designed for applications in which mappings converge to a fixed point. Their superiority over alternative Anderson acceleration is demonstrated for linear contraction mappings. Both strategies perform well in three nonlinear fixed-point applications that include partial differential equations and the EM algorithm. One strategy surpasses all other Anderson acceleration implementations tested in terms of computation time across various specifications, including composite Anderson acceleration. △ Less

Submitted 29 August, 2024; originally announced August 2024.

Comments: 23 pages, 9 figures

Alternating cyclic extrapolation methods for optimization algorithms

Authors: Nicolas Lepage-Saucier

Abstract: This article introduces new acceleration methods for fixed-point iterations. Extrapolations are computed using two or three mappings alternately and a new type of step length is proposed with good properties for nonlinear applications. The methods require no problem-specific adaptation and are especially efficient in high-dimensional contexts. Their computation uses few objective function evaluati… ▽ More This article introduces new acceleration methods for fixed-point iterations. Extrapolations are computed using two or three mappings alternately and a new type of step length is proposed with good properties for nonlinear applications. The methods require no problem-specific adaptation and are especially efficient in high-dimensional contexts. Their computation uses few objective function evaluations, no matrix inversion and little extra memory. A convergence analysis is followed by eight applications including gradient descent acceleration for constrained and unconstrained optimization. Performances are on par with or better than competitive alternatives. The algorithm is available as the Julia package SpeedMapping.jl. △ Less

Submitted 14 August, 2021; v1 submitted 11 April, 2021; originally announced April 2021.

Comments: 30 pages, 10 figures. Changes since previous version: Reorganized intro to improve readability. Made bound checking simpler. Changed the power method application. Added constrained minimization application. Added 9 CUTEst problems. Added interval-censored PH application. Updated software

MSC Class: 65K05 ACM Class: F.2.1