-
Relaxed and Inertial Nonlinear Forward-Backward with Momentum
Authors:
Fernando Roldán,
Cristian Vega
Abstract:
In this article, we study inertial algorithms for numerically solving monotone inclusions involving the sum of a maximally monotone and a cocoercive operator. In particular, we analyze the convergence of inertial and relaxed versions of the nonlinear forward-backward with momentum (NFBM). We propose an inertial version including a relaxation step, and a second version considering a double-inertial…
▽ More
In this article, we study inertial algorithms for numerically solving monotone inclusions involving the sum of a maximally monotone and a cocoercive operator. In particular, we analyze the convergence of inertial and relaxed versions of the nonlinear forward-backward with momentum (NFBM). We propose an inertial version including a relaxation step, and a second version considering a double-inertial step with additional momentum. By applying NFBM to specific monotone inclusions, we derive inertial and relaxed versions of algorithms such as forward-backward, forward-half-reflect-backward (FHRB), Chambolle-Pock, Condat-Vũ, among others, thereby recovering and extending previous results from the literature for solving monotone inclusions involving maximally monotone, cocoercive, monotone and Lipschitz, and linear bounded operators. We also present numerical experiments on image restoration, comparing the proposed inertial and relaxation algorithms. In particular, we compare the inertial FHRB with its non-inertial and momentum versions. Additionally, we compare the numerical convergence for larger step-sizes versus relaxation parameters and introduce a restart strategy that incorporates larger step-sizes and inertial steps to further enhance numerical convergence.
△ Less
Submitted 2 December, 2024;
originally announced December 2024.
-
Forward Primal-Dual Half-Forward Algorithm for Splitting Four Operators
Authors:
Fernando Roldán
Abstract:
In this article, we propose a splitting algorithm to find zeros of the sum of four maximally monotone operators in real Hilbert spaces. In particular, we consider a Lipschitzian operator, a cocoercive operator, and a linear composite term. In the case when the Lipschitzian operator is absent, our method reduces to the Condat-Vũ algorithm. On the other hand, when the linear composite term is absent…
▽ More
In this article, we propose a splitting algorithm to find zeros of the sum of four maximally monotone operators in real Hilbert spaces. In particular, we consider a Lipschitzian operator, a cocoercive operator, and a linear composite term. In the case when the Lipschitzian operator is absent, our method reduces to the Condat-Vũ algorithm. On the other hand, when the linear composite term is absent, the algorithm reduces to the Forward-Backward-Half-Forward algorithm (FBHF). Additionally, in each case, the set of step-sizes that guarantee the weak convergence of those methods are recovered. Therefore, our algorithm can be seen as a generalization of Condat-Vũ and FBHF. Moreover, we propose extensions and applications of our method in multivariate monotone inclusions and saddle point problems. Finally, we present a numerical experiment in image deblurring problems.
△ Less
Submitted 25 September, 2024; v1 submitted 26 October, 2023;
originally announced October 2023.
-
Solution of Mismatched Monotone+Lipschitz Inclusion Problems
Authors:
Emilie Chouzenoux,
Jean-Christophe Pesquet,
Fernando Roldán
Abstract:
In this article, we study the convergence of algorithms for solving monotone inclusions in the presence of adjoint mismatch. The adjoint mismatch arises when the adjoint of a linear operator is replaced by an approximation, due to computational or physical issues. This occurs in inverse problems, particularly in computed tomography. In real Hilbert spaces, monotone inclusion problems involving a m…
▽ More
In this article, we study the convergence of algorithms for solving monotone inclusions in the presence of adjoint mismatch. The adjoint mismatch arises when the adjoint of a linear operator is replaced by an approximation, due to computational or physical issues. This occurs in inverse problems, particularly in computed tomography. In real Hilbert spaces, monotone inclusion problems involving a maximally $ρ$-monotone operator, a cocoercive operator, and a Lipschitzian operator can be solved by the Forward-Backward-Half-Forward and the Forward-Douglas-Rachford-Forward methods. We investigate the case of a mismatched Lipschitzian operator. We propose variants of the two aforementioned methods to cope with the mismatch, and establish conditions under which the weak convergence to a solution is guaranteed for these variants. The proposed algorithms hence enable each iteration to be implemented with a possibly iteration-dependent approximation to the mismatch operator, thus allowing this operator to be modified at each iteration. Finally, we present numerical experiments on a computed tomography example in material science, showing the applicability of our theoretical findings.
△ Less
Submitted 9 November, 2023; v1 submitted 10 October, 2023;
originally announced October 2023.
-
Forward-Reflected-Backward and Shadow-Douglas--Rachford with partial inverse for Solving Monotone Inclusions
Authors:
Fernando Roldán
Abstract:
In this article, we study two methods for solving monotone inclusions in real Hilbert spaces involving the sum of a maximally monotone operator, a monotone-Lipschitzian operator, a cocoercive operator, and a normal cone to a vector subspace. Our algorithms split and exploits the intrinsic properties of each operator involved in the inclusion. We derive our methods by combining partial inverse tech…
▽ More
In this article, we study two methods for solving monotone inclusions in real Hilbert spaces involving the sum of a maximally monotone operator, a monotone-Lipschitzian operator, a cocoercive operator, and a normal cone to a vector subspace. Our algorithms split and exploits the intrinsic properties of each operator involved in the inclusion. We derive our methods by combining partial inverse techniques with the forward-reflected-backward algorithm and with the shadow-Douglas--Rachford algorithm, respectively. Our methods inherit the advantages of those methods, requiring only one activation of the Lipschitzian operator, one activation of the cocoercive operator, two projections onto the closed vector subspace, and one calculation of the resolvent of the maximally monotone operator. Additionally, to allow larger step-sizes in one of the proposed methods, we revisit FSDR by extending its convergence for larger step-sizes. Furthermore, we provide methods for solving monotone inclusions involving a sum of maximally monotone operators and for solving a system of primal-dual inclusions involving a mixture of sums, linear compositions, parallel sums, Lipschitzian operators, cocoercive operators, and normal cones. We apply our methods to constrained composite convex optimization problems as a specific example. Finally, in order to compare our methods with existing methods in the literature, we provide numerical experiments on constrained total variation least-squares optimization problems and computed tomography inverse problems. We obtain promising numerical results.
△ Less
Submitted 9 May, 2024; v1 submitted 27 May, 2023;
originally announced May 2023.
-
Four Operator Splitting via a Forward-Backward-Half-Forward Algorithm with Line Search
Authors:
Luis Briceño-Arias,
Fernando Roldán
Abstract:
In this article we provide a splitting method for solving monotone inclusions in a real Hilbert space involving four operators: a maximally monotone, a monotone-Lipschitzian, a cocoercive, and a monotone-continuous operator. The proposed method takes advantage of the intrinsic properties of each operator, generalizing the forward-back-half forward splitting and the Tseng's algorithm with line-sear…
▽ More
In this article we provide a splitting method for solving monotone inclusions in a real Hilbert space involving four operators: a maximally monotone, a monotone-Lipschitzian, a cocoercive, and a monotone-continuous operator. The proposed method takes advantage of the intrinsic properties of each operator, generalizing the forward-back-half forward splitting and the Tseng's algorithm with line-search. At each iteration, our algorithm defines the step-size by using a line search in which the monotone-Lipschitzian and the cocoercive operators need only one activation. We also derive a method for solving non-linearly constrained composite convex optimization problems in real Hilbert spaces. Finally, we implement our algorithm in a non-linearly constrained least-square problem, and we compare its performance with available methods in the literature.
△ Less
Submitted 18 July, 2022; v1 submitted 13 May, 2022;
originally announced May 2022.
-
Resolvent of the parallel composition and proximity operator of the infimal postcomposition
Authors:
Luis Briceño-Arias,
Fernando Roldán
Abstract:
In this paper we provide the resolvent computation of the parallel composition of a maximally monotone operator by a linear operator under mild assumptions. Connections with a modification of the warped resolvent are provided. In the context of convex optimization, we obtain the proximity operator of the infimal postcomposition of a convex function by a linear operator and we extend full range con…
▽ More
In this paper we provide the resolvent computation of the parallel composition of a maximally monotone operator by a linear operator under mild assumptions. Connections with a modification of the warped resolvent are provided. In the context of convex optimization, we obtain the proximity operator of the infimal postcomposition of a convex function by a linear operator and we extend full range conditions on the linear operator to mild qualification conditions. We also introduce a generalization of the proximity operator involving a general linear bounded operator leading to a generalization of Moreau's decomposition for composite convex optimization.
△ Less
Submitted 18 July, 2022; v1 submitted 14 September, 2021;
originally announced September 2021.
-
Forward-partial inverse-half-forward splitting algorithm for solving monotone inclusions
Authors:
Luis M. Briceño-Arias,
Jinjian Chen,
Fernando Roldán,
Yuchao Tang
Abstract:
In this paper we provide a splitting algorithm for solving coupled monotone inclusions in a real Hilbert space involving the sum of a normal cone to a vector subspace, a maximally monotone, a monotone-Lipschitzian, and a cocoercive operator. The proposed method takes advantage of the intrinsic properties of each operator and generalizes the method of partial inverses and the forward-backward-half…
▽ More
In this paper we provide a splitting algorithm for solving coupled monotone inclusions in a real Hilbert space involving the sum of a normal cone to a vector subspace, a maximally monotone, a monotone-Lipschitzian, and a cocoercive operator. The proposed method takes advantage of the intrinsic properties of each operator and generalizes the method of partial inverses and the forward-backward-half forward splitting, among other methods. At each iteration, our algorithm needs two computations of the Lipschitzian operator while the cocoercive operator is activated only once. By using product space techniques, we derive a method for solving a composite monotone primal-dual inclusions including linear operators and we apply it to solve constrained composite convex optimization problems. Finally, we apply our algorithm to a constrained total variation least-squares problem and we compare its performance with efficient methods in the literature.
△ Less
Submitted 7 February, 2022; v1 submitted 3 April, 2021;
originally announced April 2021.
-
Split-Douglas-Rachford algorithm for composite monotone inclusions and Split-ADMM
Authors:
Luis M. Briceño-Arias,
Fernando Roldán
Abstract:
In this paper we provide a generalization of the Douglas-Rachford splitting (DRS) and the primal-dual algorithm (Vu 2013, Condat 2013) for solving monotone inclusions in a real Hilbert space involving a general linear operator. The proposed method allows for primal and dual non-standard metrics and activates the linear operator separately from the monotone operators appearing in the inclusion. In…
▽ More
In this paper we provide a generalization of the Douglas-Rachford splitting (DRS) and the primal-dual algorithm (Vu 2013, Condat 2013) for solving monotone inclusions in a real Hilbert space involving a general linear operator. The proposed method allows for primal and dual non-standard metrics and activates the linear operator separately from the monotone operators appearing in the inclusion. In the simplest case when the linear operator has full range, it reduces to classical DRS. Moreover, the weak convergence of primal-dual sequences to a Kuhn-Tucker point is guaranteed, generalizing the main result in Svaiter (2011). Inspired by Gabay (1983), we also derive a new Split-ADMM (SADMM) by applying our method to the dual of a convex optimization problem involving a linear operator which can be expressed as the composition of two linear operators. The proposed SADMM activates one linear operator implicitly and the other one explicitly, and we recover ADMM when the latter is set as the identity. Connections and comparisons of our theoretical results with respect to the literature are provided for the main algorithm and SADMM. The flexibility and efficiency of both methods is illustrated via a numerical simulations in total variation image restoration and a sparse minimization problem.
△ Less
Submitted 21 September, 2021; v1 submitted 27 January, 2021;
originally announced January 2021.
-
Primal-dual splittings as fixed point iterations in the range of linear operators
Authors:
Luis Briceño-Arias,
Fernando Roldán
Abstract:
In this paper we study the relaxed primal-dual algorithm for solving composite monotone inclusions in real Hilbert spaces with critical preconditioners. Our approach is based in new results on the asymptotic behaviour of Krasnosel'skiĭ-Mann (KM) iterations defined in the range of monotone self-adjoint linear operators. These results generalize the convergence of classical KM iterations aiming at a…
▽ More
In this paper we study the relaxed primal-dual algorithm for solving composite monotone inclusions in real Hilbert spaces with critical preconditioners. Our approach is based in new results on the asymptotic behaviour of Krasnosel'skiĭ-Mann (KM) iterations defined in the range of monotone self-adjoint linear operators. These results generalize the convergence of classical KM iterations aiming at approximating fixed points. We prove that the relaxed primal-dual algorithm with critical preconditioners define KM iterations in the range of a particular monotone self-adjoint linear operator with non-trivial kernel. We then deduce from our fixed point approach that the shadows of primal-dual iterates on the range of the linear operator converges weakly to some point in this vector subspace from which we obtain a solution. This generalizes (Condat 2013 Theorem 3.3) to infinite dimensional relaxed primal-dual monotone inclusions involving critical preconditioners. The Douglas-Rachford splitting (DRS) is interpreted as a particular instance of the primal-dual algorithm when the step-sizes are critical and we recover classical results from this new perspective. We implement the relaxed primal-dual algorithm with critical preconditioners in total variation reconstruction and we illustrate its flexibility and efficiency.
△ Less
Submitted 10 August, 2021; v1 submitted 5 October, 2019;
originally announced October 2019.
