-
Circumcentering the Douglas--Rachford method
Authors:
Roger Behling,
Jose Yunier Bello Cruz,
Luiz-Rafael Santos
Abstract:
We introduce and study a geometric modification of the Douglas-Rach\-ford method called the Circumcentered-Douglas-Rachford method. This method iterates by taking the intersection of bisectors of reflection steps for solving certain classes of feasibility problems. The convergence analysis is established for best approximation problems involving two (affine) subspaces and both our theoretical and…
▽ More
We introduce and study a geometric modification of the Douglas-Rach\-ford method called the Circumcentered-Douglas-Rachford method. This method iterates by taking the intersection of bisectors of reflection steps for solving certain classes of feasibility problems. The convergence analysis is established for best approximation problems involving two (affine) subspaces and both our theoretical and numerical results compare favorably to the original Douglas-Rachford method. Under suitable conditions, it is shown that the linear rate of convergence of the Circumcentered-Douglas-Rachford method is at least the cosine of the Friedrichs angle between the (affine) subspaces, which is known to be the sharp rate for the Douglas-Rachford method. We also present a preliminary discussion on the Circumcentered-Douglas-Rachford method applied to the many set case and to examples featuring non-affine convex sets.
△ Less
Submitted 16 August, 2017; v1 submitted 21 April, 2017;
originally announced April 2017.
-
An inexact strategy for the projected gradient algorithm in vector optimization problems on variable ordered spaces
Authors:
Jose Yunier Bello Cruz,
Gemayqzel Bouza Allende
Abstract:
Variable order structures model situations in which the comparison between two points depends on a point-to-cone map. In this paper, an inexact projected gradient method for solving smooth constrained vector optimization problems on variable ordered spaces is presented. It is shown that every accumulation point of the generated sequence satisfies the first order necessary optimality condition. The…
▽ More
Variable order structures model situations in which the comparison between two points depends on a point-to-cone map. In this paper, an inexact projected gradient method for solving smooth constrained vector optimization problems on variable ordered spaces is presented. It is shown that every accumulation point of the generated sequence satisfies the first order necessary optimality condition. The convergence of all accumulation points to a weakly efficient point is established under suitable convexity assumptions for the objective function. The convergence results are also derived in the particular case in which the problem is unconstrained and if exact directions are taken as descent directions. Furthermore, we investigate the application of the proposed method to optimization models where the domain of the variable order map and the objective function are the same. In this case, similar concepts and convergence results are presented. Finally, some computational experiments designed to illustrate the behavior of the proposed inexact methods versus the exact ones (in terms of CPU time) are performed.
△ Less
Submitted 8 August, 2019; v1 submitted 6 January, 2017;
originally announced January 2017.
-
A semi-smooth Newton method for projection equations and linear complementarity problems with respect to the second order cone
Authors:
Jose Yunier Bello Cruz,
O. P. Ferreira,
S. Z. Nemeth,
L. F. Prudente
Abstract:
In this paper a special semi-smooth equation associated to the second order cone is studied. It is shown that, under mild assumptions, the semi-smooth Newton method applied to this equation is well-defined and the generated sequence is globally and Q-linearly convergent to a solution. As an application, the obtained results are used to study the linear second order cone complementarity problem, wi…
▽ More
In this paper a special semi-smooth equation associated to the second order cone is studied. It is shown that, under mild assumptions, the semi-smooth Newton method applied to this equation is well-defined and the generated sequence is globally and Q-linearly convergent to a solution. As an application, the obtained results are used to study the linear second order cone complementarity problem, with special emphasis on the particular case of positive definite matrices. Moreover, some computational experiments designed to investigate the practical viability of the method are presented.
△ Less
Submitted 14 October, 2016; v1 submitted 30 May, 2016;
originally announced May 2016.
-
A semi-smooth Newton method for a special piecewise linear system with application to positively constrained convex quadratic programming
Authors:
J. G. Barrios,
J. Y. Bello Cruz,
O. P. Ferreira,
S. Z. Németh
Abstract:
In this paper a special piecewise linear system is studied. It is shown that, under a mild assumption, the semi-smooth Newton method applied to this system is well defined and the method generates a sequence that converges linearly to a solution. Besides, we also show that the generated sequence is bounded, for any starting point, and a formula for any accumulation point of this sequence is presen…
▽ More
In this paper a special piecewise linear system is studied. It is shown that, under a mild assumption, the semi-smooth Newton method applied to this system is well defined and the method generates a sequence that converges linearly to a solution. Besides, we also show that the generated sequence is bounded, for any starting point, and a formula for any accumulation point of this sequence is presented. As an application, we study the convex quadratic programming problem under positive constraints. The numerical results suggest that the semi-smooth Newton method achieves accurate solutions to large scale problems in few iterations.
△ Less
Submitted 12 November, 2015; v1 submitted 6 August, 2015;
originally announced August 2015.
-
On the global convergence of the inexact semi-smooth Newton method for absolute value equation
Authors:
J. Y. Bello Cruz,
O. P. Ferreira,
L. F. Prudente
Abstract:
In this paper, we investigate global convergence properties of the inexact nonsmooth Newton method for solving the system of absolute value equations (AVE). Global $Q$-linear convergence is established under suitable assumptions. Moreover, we present some numerical experiments designed to investigate the practical viability of the proposed scheme.
In this paper, we investigate global convergence properties of the inexact nonsmooth Newton method for solving the system of absolute value equations (AVE). Global $Q$-linear convergence is established under suitable assumptions. Moreover, we present some numerical experiments designed to investigate the practical viability of the proposed scheme.
△ Less
Submitted 12 November, 2015; v1 submitted 6 August, 2015;
originally announced August 2015.
-
On the convergence of the forward-backward splitting method with linesearches
Authors:
J. Y. Bello Cruz,
T. T. A. Nghia
Abstract:
In this paper we focus on the convergence analysis of the forward-backward splitting method for solving nonsmooth optimization problems in Hilbert spaces when the objective function is the sum of two convex functions. Assuming that one of the functions is Fréchet differentiable and using two new linesearches, the weak convergence is established without any Lipschitz continuity assumption on the gr…
▽ More
In this paper we focus on the convergence analysis of the forward-backward splitting method for solving nonsmooth optimization problems in Hilbert spaces when the objective function is the sum of two convex functions. Assuming that one of the functions is Fréchet differentiable and using two new linesearches, the weak convergence is established without any Lipschitz continuity assumption on the gradient. Furthermore, we obtain many complexity results of cost values at the iterates when the stepsizes are bounded below by a positive constant.
△ Less
Submitted 14 October, 2016; v1 submitted 11 January, 2015;
originally announced January 2015.
-
Conditional extragradient algorithms for solving variational inequalities
Authors:
J. Y. Bello Cruz,
R. Díaz Millán,
Hung M. Phan
Abstract:
In this paper, we generalize the classical extragradient algorithm for solving variational inequality problems by utilizing nonzero normal vectors of the feasible set. In particular, conceptual algorithms are proposed with two different linesearchs. We then establish convergence results for these algorithms under mild assumptions. Our study suggests that nonzero normal vectors may significantly im…
▽ More
In this paper, we generalize the classical extragradient algorithm for solving variational inequality problems by utilizing nonzero normal vectors of the feasible set. In particular, conceptual algorithms are proposed with two different linesearchs. We then establish convergence results for these algorithms under mild assumptions. Our study suggests that nonzero normal vectors may significantly improve convergence if chosen appropriately.
△ Less
Submitted 22 February, 2019; v1 submitted 16 November, 2014;
originally announced November 2014.
-
On proximal subgradient splitting method for minimizing the sum of two nonsmooth convex functions
Authors:
Jose Yunier Bello Cruz
Abstract:
In this paper we present a variant of the proximal forward-backward splitting iteration for solving nonsmooth optimization problems in Hilbert spaces, when the objective function is the sum of two nondifferentiable convex functions. The proposed iteration, which will be called Proximal Subgradient Splitting Method, extends the classical subgradient iteration for important classes of problems, expl…
▽ More
In this paper we present a variant of the proximal forward-backward splitting iteration for solving nonsmooth optimization problems in Hilbert spaces, when the objective function is the sum of two nondifferentiable convex functions. The proposed iteration, which will be called Proximal Subgradient Splitting Method, extends the classical subgradient iteration for important classes of problems, exploiting the additive structure of the objective function. The weak convergence of the generated sequence was established using different stepsizes and under suitable assumptions. Moreover, we analyze the complexity of the iterates.
△ Less
Submitted 11 January, 2016; v1 submitted 20 October, 2014;
originally announced October 2014.
-
Optimal rates of convergence of matrices with applications
Authors:
Heinz H. Bauschke,
J. Y. Bello Cruz,
Tran T. A. Nghia,
Hung M. Phan,
Xianfu Wang
Abstract:
We present a systematic study on the linear convergence rates of the powers of (real or complex) matrices. We derive a characterization when the optimal convergence rate is attained. This characterization is given in terms of semi-simpleness of all eigenvalues having the second-largest modulus after 1. We also provide applications of our general results to analyze the optimal convergence rates for…
▽ More
We present a systematic study on the linear convergence rates of the powers of (real or complex) matrices. We derive a characterization when the optimal convergence rate is attained. This characterization is given in terms of semi-simpleness of all eigenvalues having the second-largest modulus after 1. We also provide applications of our general results to analyze the optimal convergence rates for several relaxed alternating projection methods and the generalized Douglas-Rachford splitting methods for finding the projection on the intersection of two subspaces. Numerical experiments confirm our convergence analysis.
△ Less
Submitted 2 July, 2014;
originally announced July 2014.
-
Subgradient algorithms for solving variable inequalities
Authors:
J. Y. Bello Cruz,
L. R. Lucambio Perez,
G. Bouza Allende
Abstract:
In this paper we consider the variable inequality problem, that is, to find a solution of the inclusion given by the sum of a function and a point-to-cone application. This problem can be seen as a generalization of the classical system inequality problem taking a variable order structure. Exploiting this special structure, we propose two variants of the subgradient algorithm for solving a system…
▽ More
In this paper we consider the variable inequality problem, that is, to find a solution of the inclusion given by the sum of a function and a point-to-cone application. This problem can be seen as a generalization of the classical system inequality problem taking a variable order structure. Exploiting this special structure, we propose two variants of the subgradient algorithm for solving a system of variable inequalities. The convergence analysis is given under convex-like conditions which, when the point-to-cone application is constant, contains the old subgradient schemes.
△ Less
Submitted 9 September, 2014; v1 submitted 23 March, 2014;
originally announced March 2014.
-
On Weak and Strong Convergence of the Projected Gradient Method for Convex Optimization in real Hilbert Spaces
Authors:
Jose Yunier Bello Cruz,
Welington de Oliveira
Abstract:
This work focuses on convergence analysis of the projected gradient method for solving constrained convex minimization problem in Hilbert spaces. We show that the sequence of points generated by the method employing the Armijo linesearch converges weakly to a solution of the considered convex optimization problem. Weak convergence is established by assuming convexity and Gateaux differentiability…
▽ More
This work focuses on convergence analysis of the projected gradient method for solving constrained convex minimization problem in Hilbert spaces. We show that the sequence of points generated by the method employing the Armijo linesearch converges weakly to a solution of the considered convex optimization problem. Weak convergence is established by assuming convexity and Gateaux differentiability of the objective function, whose Gateaux derivative is supposed to be uniformly continuous on bounded sets. Furthermore, we propose some modifications in the classical projected gradient method in order to obtain strong convergence. The new variant has the following desirable properties: the sequence of generated points is entirely contained in a ball with diameter equal to the distance between the initial point and the solution set, and the whole sequence converges strongly to the solution of the problem that lies closest to the initial iterate. Convergence analysis of both methods is presented without Lipschitz continuity assumption.
△ Less
Submitted 6 August, 2015; v1 submitted 24 February, 2014;
originally announced February 2014.
-
A Relaxed-Projection Splitting Algorithm for Variational Inequalities in Hilbert Spaces
Authors:
J. Y. Bello Cruz,
R. Diaz Millan
Abstract:
We introduce a relaxed-projection splitting algorithm for solving variational inequalities in Hilbert spaces for the sum of nonsmooth maximal monotone operators, where the feasible set is defined by a nonlinear and nonsmooth continuous convex function inequality. In our scheme, the orthogonal projections onto the feasible set are replaced by projections onto separating hyperplanes. Furthermore, ea…
▽ More
We introduce a relaxed-projection splitting algorithm for solving variational inequalities in Hilbert spaces for the sum of nonsmooth maximal monotone operators, where the feasible set is defined by a nonlinear and nonsmooth continuous convex function inequality. In our scheme, the orthogonal projections onto the feasible set are replaced by projections onto separating hyperplanes. Furthermore, each iteration of the proposed method consists of simple subgradient-like steps, which does not demand the solution of a nontrivial subproblem, using only individual operators, which exploits the structure of the problem. Assuming monotonicity of the individual operators and the existence of solutions, we prove that the generated sequence converges weakly to a solution.
△ Less
Submitted 29 December, 2015; v1 submitted 13 December, 2013;
originally announced December 2013.
-
The rate of linear convergence of the Douglas-Rachford algorithm for subspaces is the cosine of the Friedrichs angle
Authors:
Heinz H. Bauschke,
J. Y. Bello Cruz,
Tran T. A. Nghia,
Hung M. Phan,
Xianfu Wang
Abstract:
The Douglas-Rachford splitting algorithm is a classical optimization method that has found many applications. When specialized to two normal cone operators, it yields an algorithm for finding a point in the intersection of two convex sets. This method for solving feasibility problems has attracted a lot of attention due to its good performance even in nonconvex settings.
In this paper, we consid…
▽ More
The Douglas-Rachford splitting algorithm is a classical optimization method that has found many applications. When specialized to two normal cone operators, it yields an algorithm for finding a point in the intersection of two convex sets. This method for solving feasibility problems has attracted a lot of attention due to its good performance even in nonconvex settings.
In this paper, we consider the Douglas-Rachford algorithm for finding a point in the intersection of two subspaces. We prove that the method converges strongly to the projection of the starting point onto the intersection. Moreover, if the sum of the two subspaces is closed, then the convergence is linear with the rate being the cosine of the Friedrichs angle between the subspaces. Our results improve upon existing results in three ways: First, we identify the location of the limit and thus reveal the method as a best approximation algorithm; second, we quantify the rate of convergence, and third, we carry out our analysis in general (possibly infinite-dimensional) Hilbert space. We also provide various examples as well as a comparison with the classical method of alternating projections.
△ Less
Submitted 21 December, 2013; v1 submitted 18 September, 2013;
originally announced September 2013.
-
The inexact projected gradient method for quasiconvex vector optimization problems
Authors:
J. Y. Bello Cruz,
G. C. Bento,
G. Bouza Allende,
R. F. B. Costa
Abstract:
Vector optimization problems are a generalization of multiobjective optimization in which the preference order is related to an arbitrary closed and convex cone, rather than the nonnegative octant. Due to its real life applications, it is important to have practical solution approaches for computing. In this work, we consider the inexact projected gradient-like method for solving smooth constraine…
▽ More
Vector optimization problems are a generalization of multiobjective optimization in which the preference order is related to an arbitrary closed and convex cone, rather than the nonnegative octant. Due to its real life applications, it is important to have practical solution approaches for computing. In this work, we consider the inexact projected gradient-like method for solving smooth constrained vector optimization problems. Basically, we prove global convergence of any sequence produced by the method to a stationary point assuming that the objective function of the problem is $K$-quasiconvex, instead of the stronger $K$-convexity assumed in the literature.
△ Less
Submitted 1 December, 2013; v1 submitted 5 December, 2012;
originally announced December 2012.
-
On Polar Legendre Polynomials
Authors:
Héctor Pijeira Cabrera,
José Y. Bello Cruz,
Wilfredo Urbina
Abstract:
We introduce a new class of polynomials $\{P_{n}\}$, that we call polar Legendre polynomials, they appear as solutions of an inverse Gauss problem of equilibrium position of a field of forces with $n+1$ unit masses. We study algebraic, differential and asymptotic properties of this class of polynomials, that are simultaneously orthogonal with respect to a differential operator and a discrete-con…
▽ More
We introduce a new class of polynomials $\{P_{n}\}$, that we call polar Legendre polynomials, they appear as solutions of an inverse Gauss problem of equilibrium position of a field of forces with $n+1$ unit masses. We study algebraic, differential and asymptotic properties of this class of polynomials, that are simultaneously orthogonal with respect to a differential operator and a discrete-continuous Sobolev type inner product.
△ Less
Submitted 28 September, 2007;
originally announced September 2007.
