-
Proximal-point-like algorithms for abstract convex minimisation problems
Authors:
Reinier Díaz Millán,
Julien Ugon
Abstract:
In this paper we introduce two conceptual algorithms for minimising abstract convex functions. Both algorithms rely on solving a proximal-type subproblem with an abstract Bregman distance based proximal term. We prove their convergence when the set of abstract linear functions forms a linear space. This latter assumption can be relaxed to only require the set of abstract linear functions to be clo…
▽ More
In this paper we introduce two conceptual algorithms for minimising abstract convex functions. Both algorithms rely on solving a proximal-type subproblem with an abstract Bregman distance based proximal term. We prove their convergence when the set of abstract linear functions forms a linear space. This latter assumption can be relaxed to only require the set of abstract linear functions to be closed under the sum, which is a classical assumption in abstract convexity. We provide numerical examples on the minimisation of nonconvex functions with the presented algorithms.
△ Less
Submitted 5 February, 2024;
originally announced February 2024.
-
Extragradient method with feasible inexact projection to variational inequality problem
Authors:
R. Díaz Millán,
O. P. Ferreira,
J. Ugon
Abstract:
The variational inequality problem in finite-dimensional Euclidean space is addressed in this paper, and two inexact variants of the extragradient method are proposed to solve it. Instead of computing exact projections on the constraint set, as in previous versions extragradient method, the proposed methods compute feasible inexact projections on the constraint set using a relative error criterion…
▽ More
The variational inequality problem in finite-dimensional Euclidean space is addressed in this paper, and two inexact variants of the extragradient method are proposed to solve it. Instead of computing exact projections on the constraint set, as in previous versions extragradient method, the proposed methods compute feasible inexact projections on the constraint set using a relative error criterion. The first version of the proposed method provided is a counterpart to the classic form of the extragradient method with constant steps. In order to establish its convergence we need to assume that the operator is pseudo-monotone and Lipschitz continuous, as in the standard approach. For the second version, instead of a fixed step size, the method presented finds a suitable step size in each iteration by performing a line search. Like the classical extragradient method, the proposed method does just two projections into the feasible set in each iteration. A full convergence analysis is provided, with no Lipschitz continuity assumption of the operator defining the variational inequality problem.
△ Less
Submitted 21 June, 2024; v1 submitted 31 August, 2023;
originally announced September 2023.
-
Frank-Wolfe algorithm for DC optimization problem
Authors:
R. Díaz Millán,
O. P. Ferreira,
J. Ugon
Abstract:
In the present paper, we formulate two versions of Frank--Wolfe algorithm or conditional gradient method to solve the DC optimization problem with an adaptive step size. The DC objective function consists of two components; the first is thought to be differentiable with a continuous Lipschitz gradient, while the second is only thought to be convex. The second version is based on the first and empl…
▽ More
In the present paper, we formulate two versions of Frank--Wolfe algorithm or conditional gradient method to solve the DC optimization problem with an adaptive step size. The DC objective function consists of two components; the first is thought to be differentiable with a continuous Lipschitz gradient, while the second is only thought to be convex. The second version is based on the first and employs finite differences to approximate the gradient of the first component of the objective function. In contrast to past formulations that used the curvature/Lipschitz-type constant of the objective function, the step size computed does not require any constant associated with the components. For the first version, we established that the algorithm is well-defined of the algorithm and that every limit point of the generated sequence is a stationary point of the problem. We also introduce the class of weak-star-convex functions and show that, despite the fact that these functions are non-convex in general, the rate of convergence of the first version of the algorithm to minimize these functions is ${\cal O}(1/k)$. The finite difference used to approximate the gradient in the second version of the Frank-Wolfe algorithm is computed with the step-size adaptively updated using two previous iterations. Unlike previous applications of finite difference in the Frank-Wolfe algorithm, which provided approximate gradients with absolute error, the one used here provides us with a relative error, simplifying the algorithm analysis. In this case, we show that all limit points of the generated sequence for the second version of the Frank-Wolfe algorithm are stationary points for the problem under consideration, and we establish that the rate of convergence for the duality gap is ${\cal O}(1/\sqrt{k})$.
△ Less
Submitted 31 August, 2023;
originally announced August 2023.
-
A comparison of rational and neural network based approximations
Authors:
Vinesha Peiris,
Reinier Diaz Millan,
Nadezda Sukhorukova,
Julien Ugon
Abstract:
Rational and neural network based approximations are efficient tools in modern approximation. These approaches are able to produce accurate approximations to nonsmooth and non-Lipschitz functions, including multivariate domain functions. In this paper we compare the efficiency of function approximation using rational approximation, neural network and their combinations. It was found that rational…
▽ More
Rational and neural network based approximations are efficient tools in modern approximation. These approaches are able to produce accurate approximations to nonsmooth and non-Lipschitz functions, including multivariate domain functions. In this paper we compare the efficiency of function approximation using rational approximation, neural network and their combinations. It was found that rational approximation is superior to neural network based approaches with the same number of decision variables. Our numerical experiments demonstrate the efficiency of rational approximation, even when the number of approximation parameters (that is, the dimension of the corresponding optimisation problems) is small. Another important contribution of this paper lies in the improvement of rational approximation algorithms. Namely, the optimisation based algorithms for rational approximation can be adjusted to in such a way that the conditioning number of the constraint matrices are controlled. This simple adjustment enables us to work with high dimension optimisation problems and improve the design of the neural network. The main strength of neural networks is in their ability to handle models with a large number of variables: complex models are decomposed in several simple optimisation problems. Therefore the the large number of decision variables is in the nature of neural networks.
△ Less
Submitted 6 September, 2023; v1 submitted 8 March, 2023;
originally announced March 2023.
-
Variational properties of the abstract subdifferential operator
Authors:
Reinier Diàz Millàn,
Nadezda Sukhorukova,
Julien Ugon
Abstract:
Abstract convexity generalises classical convexity by considering the suprema of functions taken from an arbitrarily defined set of functions. These are called the abstract linear (abstract affine) functions. The purpose of this paper is to study the abstract subdifferential. We obtain a number of results on the calculus of this subdifferential: summation and composition rules, and prove that unde…
▽ More
Abstract convexity generalises classical convexity by considering the suprema of functions taken from an arbitrarily defined set of functions. These are called the abstract linear (abstract affine) functions. The purpose of this paper is to study the abstract subdifferential. We obtain a number of results on the calculus of this subdifferential: summation and composition rules, and prove that under some reasonable conditions the subdifferential is a maximal abstract monotone operator. Another contribution of this paper is a counterexample that demonstrates that the separation theorem between two abstract convex sets is generally not true. The lack of the extension of separation results to the case of abstract convexity is one of the obstacles in the development of abstract convexity based numerical methods.
△ Less
Submitted 5 April, 2024; v1 submitted 2 June, 2022;
originally announced June 2022.
-
Application and issues in abstract convexity
Authors:
Reinier Díaz Millán,
Nadezda Sukhorukova,
Julien Ugon
Abstract:
The theory of abstract convexity, also known as convexity without linearity, is an extension of the classical convex analysis. There are a number of remarkable results, mostly concerning duality, and some numerical methods, however, this area has not found many practical applications yet. In this paper we study the application of abstract convexity to function approximation. Another important rese…
▽ More
The theory of abstract convexity, also known as convexity without linearity, is an extension of the classical convex analysis. There are a number of remarkable results, mostly concerning duality, and some numerical methods, however, this area has not found many practical applications yet. In this paper we study the application of abstract convexity to function approximation. Another important research direction addressed in this paper is the connection with the so-called axiomatic convexity.
△ Less
Submitted 20 February, 2022;
originally announced February 2022.
-
The intrinsic core and minimal faces of convex sets in general vector spaces
Authors:
R. Díaz Millán,
Vera Roshchina
Abstract:
Intrinsic core generalises the finite-dimensional notion of the relative interior to arbitrary (real) vector spaces. Our main goal is to provide a self-contained overview of the key results pertaining to the intrinsic core and to elucidate the relations between intrinsic core and facial structure of convex sets in this general context. We gather several equivalent definitions of the intrinsic core…
▽ More
Intrinsic core generalises the finite-dimensional notion of the relative interior to arbitrary (real) vector spaces. Our main goal is to provide a self-contained overview of the key results pertaining to the intrinsic core and to elucidate the relations between intrinsic core and facial structure of convex sets in this general context. We gather several equivalent definitions of the intrinsic core, cover much of the folklore, review relevant recent results and present examples illustrating some of the phenomena specific to the facial structure of infinite-dimensional sets.
△ Less
Submitted 24 March, 2022; v1 submitted 16 July, 2021;
originally announced July 2021.
-
Approximate Douglas-Rachford algorithm for two-sets convex feasibility problems
Authors:
R. Díaz Millán,
O. P. Ferreira,
J. Ugon
Abstract:
In this paper, we propose a new algorithm combining the Douglas-Rachford (DR) algorithm and the Frank-Wolfe algorithm, also known as the conditional gradient (CondG) method, for solving the classic convex feasibility problem. Within the algorithm, which will be named {\it Approximate Douglas-Rachford (ApDR) algorithm}, the CondG method is used as a subroutine to compute feasible inexact projection…
▽ More
In this paper, we propose a new algorithm combining the Douglas-Rachford (DR) algorithm and the Frank-Wolfe algorithm, also known as the conditional gradient (CondG) method, for solving the classic convex feasibility problem. Within the algorithm, which will be named {\it Approximate Douglas-Rachford (ApDR) algorithm}, the CondG method is used as a subroutine to compute feasible inexact projections on the sets under consideration, and the ApDR iteration is defined based on the DR iteration. The ApDR algorithm generates two sequences, the main sequence, based on the DR iteration, and its corresponding shadow sequence. When the intersection of the feasible sets is nonempty, the main sequence converges to a fixed point of the usual DR operator, and the shadow sequence converges to the solution set. We provide some numerical experiments to illustrate the behaviour of the sequences produced by the proposed algorithm.
△ Less
Submitted 7 June, 2021; v1 submitted 27 May, 2021;
originally announced May 2021.
-
Multivariate approximation by polynomial and generalised rational functions
Authors:
R. Díaz Millán,
V. Peiris,
N. Sukhorukova,
J. Ugon
Abstract:
In this paper we develop an optimisation based approach to multivariate Chebyshev approximation on a finite grid. We consider two models: multivariate polynomial approximation and multivariate generalised rational approximation. In the second case the approximations are ratios of linear forms and the basis functions are not limited to monomials. It is already known that in the case of multivariate…
▽ More
In this paper we develop an optimisation based approach to multivariate Chebyshev approximation on a finite grid. We consider two models: multivariate polynomial approximation and multivariate generalised rational approximation. In the second case the approximations are ratios of linear forms and the basis functions are not limited to monomials. It is already known that in the case of multivariate polynomial approximation on a finite grid the corresponding optimisation problems can be reduced to solving a linear programming problem, while the area of multivariate rational approximation is not so well understood.In this paper we demonstrate that in the case of multivariate generalised rational approximation the corresponding optimisation problems are quasiconvex. This statement remains true even when the basis functions are not limited to monomials. Then we apply a bisection method, which is a general method for quasiconvex optimisation. This method converges to an optimal solution with given precision. We demonstrate that the convex feasibility problems appearing in the bisection method can be solved using linear programming. Finally, we compare the deviation error and computational time for multivariate polynomial and generalised rational approximation with the same number of decision variables.
△ Less
Submitted 27 January, 2021;
originally announced January 2021.
-
An algorithm for best generalised rational approximation of continuous functions
Authors:
R. Díaz Millán,
Nadezda Sukhorukova,
Julien Ugon
Abstract:
The motivation of this paper is the development of an optimisation method for solving optimisation problems appearing in Chebyshev rational and generalised rational approximation problems, where the approximations are constructed as ratios of linear forms (linear combinations of basis functions). The coefficients of the linear forms are subject to optimisation and the basis functions are continuou…
▽ More
The motivation of this paper is the development of an optimisation method for solving optimisation problems appearing in Chebyshev rational and generalised rational approximation problems, where the approximations are constructed as ratios of linear forms (linear combinations of basis functions). The coefficients of the linear forms are subject to optimisation and the basis functions are continuous function. It is known that the objective functions in generalised rational approximation problems are quasi-convex. In this paper we also prove a stronger result, the objective functions are pseudo-convex in the sense of Penot and Quang. Then we develop numerical methods, that are efficient for a wide range of pseudo-convex functions and test them on generalised rational approximation problems.
△ Less
Submitted 5 November, 2020;
originally announced November 2020.
-
Alternating conditional gradient method for convex feasibility problems
Authors:
R. Díaz Millán,
O. P. Ferreira,
L. F. Prudente
Abstract:
The classical convex feasibility problem in a finite dimensional Euclidean space is studied in the present paper. We are interested in two cases. First, we assume to know how to compute an exact project onto one of the sets involved and the other set is compact such that the conditional gradient (CondG) method can be used for computing efficiently an inexact projection on it. Second, we assume tha…
▽ More
The classical convex feasibility problem in a finite dimensional Euclidean space is studied in the present paper. We are interested in two cases. First, we assume to know how to compute an exact project onto one of the sets involved and the other set is compact such that the conditional gradient (CondG) method can be used for computing efficiently an inexact projection on it. Second, we assume that both sets involved are compact such that the CondG method can be used for computing efficiently inexact projections on them. We combine alternating projection method with CondG method to design a new method, which can be seen as an inexact feasible version of alternate projection method. The proposed method generates two different sequences belonging to each involved set, which converge to a point in the intersection of them whenever it is not empty. If the intersection is empty, then the sequences converge to points in the respective sets whose distance is equal to the distance between the sets in consideration.
△ Less
Submitted 9 December, 2019;
originally announced December 2019.
-
Comparing Averaged Relaxed Cutters and Projection Methods: Theory and Examples
Authors:
R. Díaz Millán,
Scott B. Lindstrom,
Vera Roshchina
Abstract:
We focus on the convergence analysis of averaged relaxations of cutters, specifically for variants that---depending upon how parameters are chosen---resemble \emph{alternating projections}, the \emph{Douglas--Rachford method}, \emph{relaxed reflect-reflect}, or the \emph{Peaceman--Rachford} method. Such methods are frequently used to solve convex feasibility problems. The standard convergence anal…
▽ More
We focus on the convergence analysis of averaged relaxations of cutters, specifically for variants that---depending upon how parameters are chosen---resemble \emph{alternating projections}, the \emph{Douglas--Rachford method}, \emph{relaxed reflect-reflect}, or the \emph{Peaceman--Rachford} method. Such methods are frequently used to solve convex feasibility problems. The standard convergence analyses of projection algorithms are based on the \emph{firm nonexpansivity} property of the relevant operators. However if the projections onto the constraint sets are replaced by cutters (projections onto separating hyperplanes), the firm nonexpansivity is lost. We provide a proof of convergence for a family of related averaged relaxed cutter methods under reasonable assumptions, relying on a simple geometric argument. This allows us to clarify fine details related to the allowable choice of the relaxation parameters, highlighting the distinction between the exact (firmly nonexpansive) and approximate (strongly quasi-nonexpansive) settings. We provide illustrative examples and discuss practical implementations of the method.
△ Less
Submitted 4 October, 2018;
originally announced October 2018.
-
Inexact proximal $ε$-subgradient methods for composite convex optimization problems
Authors:
Reinier Díaz Millán,
Majela Pentón Machado
Abstract:
We present two approximate versions of the proximal subgradient method for minimizing the sum of two convex functions (not necessarily differentiable). The algorithms involve, at each iteration, inexact evaluations of the proximal operator and approximate subgradients of the functions (namely: the $ε$-subgradients). The methods use different error criteria for approximating the proximal operators.…
▽ More
We present two approximate versions of the proximal subgradient method for minimizing the sum of two convex functions (not necessarily differentiable). The algorithms involve, at each iteration, inexact evaluations of the proximal operator and approximate subgradients of the functions (namely: the $ε$-subgradients). The methods use different error criteria for approximating the proximal operators. We provide an analysis of the convergence and rate of convergence properties of these methods, considering various stepsize rules, including both, diminishing and constant stepsizes. For the case where one of the functions is smooth, we propose an inexact accelerated version of the proximal gradient method, and prove that the optimal convergence rate for the function values can be achieved.
△ Less
Submitted 11 July, 2019; v1 submitted 25 May, 2018;
originally announced May 2018.
-
A projection algorithm for non-monotone variational inequalities
Authors:
Regina S. Burachik,
R. Díaz Millán
Abstract:
We introduce and study the convergence properties of a projection-type algorithm for solving the variational inequality problem for point-to-set operators. No monotoni\-city assumption is used in our analysis. The operator defining the problem is only assumed to be continuous in the point-to-set sense, i.e., inner- and outer-semicontinuous. Additionally, we assume non-emptiness of the so-called du…
▽ More
We introduce and study the convergence properties of a projection-type algorithm for solving the variational inequality problem for point-to-set operators. No monotoni\-city assumption is used in our analysis. The operator defining the problem is only assumed to be continuous in the point-to-set sense, i.e., inner- and outer-semicontinuous. Additionally, we assume non-emptiness of the so-called dual solution set. We prove that the whole sequence of iterates converges to a solution of the variational inequality. Moreover, we provide numerical experiments illustrating the behavior of our iterates. Through several examples, we provide a comparison with a recent similar algorithm.
△ Less
Submitted 28 November, 2017; v1 submitted 29 September, 2016;
originally announced September 2016.
-
Two algorithms for solving systems of inclusion problems
Authors:
R. Díaz Millán
Abstract:
The goal of this paper is to present two algorithms for solving systems of inclusion problems, with all component of the systems being a sum of two maximal monotone operators. The algorithms are variants of the forward-backward splitting method and one being a hybrid with the alternating projection method. They consist of approximating the solution sets involved in the problem by separating halfsp…
▽ More
The goal of this paper is to present two algorithms for solving systems of inclusion problems, with all component of the systems being a sum of two maximal monotone operators. The algorithms are variants of the forward-backward splitting method and one being a hybrid with the alternating projection method. They consist of approximating the solution sets involved in the problem by separating halfspaces which are a well-studied strategy. The schemes contain two part, the first one is an explicit Armijo-type search in the spirit of the extragradient-like methods for variational inequalities. The second part is the projection step, being this the main difference between the algorithms. While the first algorithm computes the projection onto the intersection of the separating halfspaces, the second choose one component of the system and project onto the separating halfspace of this case. In the iterative process, the forward-backward operator is computed once per inclusion problem, representing a relevant computational saving if compared with similar algorithms in the literature. The convergence analysis of the proposed methods is given assuming monotonicity in all operators, without Lipschitz continuity assumption. We also present some numerical experiments.
△ Less
Submitted 6 September, 2017; v1 submitted 28 September, 2016;
originally announced September 2016.
-
A hybrid method for solving systems of operator inclusion problems
Authors:
R. Díaz Millán
Abstract:
In this paper, we propose an algorithm combining the forward-backward splitting method and the alternative projection method for solving the system of splitting inclusion problem. We want to find a point in the interception of a finite number of sets that we don't know, the solution of each component of the system. The algorithm consists of approximate the sets involved in the problem by separates…
▽ More
In this paper, we propose an algorithm combining the forward-backward splitting method and the alternative projection method for solving the system of splitting inclusion problem. We want to find a point in the interception of a finite number of sets that we don't know, the solution of each component of the system. The algorithm consists of approximate the sets involved in the problem by separates halfspaces which are a known strategy. By finding these halfspaces in each iteration we use only one inclusion problem of the system. The iterations consist of two parts, the first contains an explicit Armijo-type search in the spirit of the extragradient-like methods for variational inequalities. In the iterative process, the operator forward-backward is computed only one time for each inclusion problem, this represents a great computational saving because the computational cost of this operator is nothing cheap. The second part consists of special projection step, projecting in the separating halfspace. The convergence analysis of the proposed scheme is given assuming monotonicity all operators, without any Lipschitz continuity assumption.
△ Less
Submitted 6 April, 2016;
originally announced April 2016.
-
A variant of forward-backward splitting method for the systems of inclusion problem
Authors:
R. Díaz Millán
Abstract:
In this paper, we propose variants of forward-backward splitting method for solving the system of splitting inclusion problem. We propose a conceptual algorithm containing three variants, each having a different projection steps. The algorithm consists in two parts, the first and main contains an explicit Armijo-type search in the spirit of the extragradient-like methods for variational inequaliti…
▽ More
In this paper, we propose variants of forward-backward splitting method for solving the system of splitting inclusion problem. We propose a conceptual algorithm containing three variants, each having a different projection steps. The algorithm consists in two parts, the first and main contains an explicit Armijo-type search in the spirit of the extragradient-like methods for variational inequalities. In the iterative process the operator forward-backward is computed only one time for each inclusion problem, this represent a great computational saving if we compare with Tseng's algorithm, because the computational cost of this operator is very high. The second part of the scheme consists in special projection steps. The convergence analysis of the proposed scheme is given assuming monotonicity on both operators, without assuming Lipschitz continuity on the forward operators.
△ Less
Submitted 1 January, 2016; v1 submitted 10 March, 2015;
originally announced March 2015.
-
Conditional extragradient algorithms for solving variational inequalities
Authors:
J. Y. Bello Cruz,
R. Díaz Millán,
Hung M. Phan
Abstract:
In this paper, we generalize the classical extragradient algorithm for solving variational inequality problems by utilizing nonzero normal vectors of the feasible set. In particular, conceptual algorithms are proposed with two different linesearchs. We then establish convergence results for these algorithms under mild assumptions. Our study suggests that nonzero normal vectors may significantly im…
▽ More
In this paper, we generalize the classical extragradient algorithm for solving variational inequality problems by utilizing nonzero normal vectors of the feasible set. In particular, conceptual algorithms are proposed with two different linesearchs. We then establish convergence results for these algorithms under mild assumptions. Our study suggests that nonzero normal vectors may significantly improve convergence if chosen appropriately.
△ Less
Submitted 22 February, 2019; v1 submitted 16 November, 2014;
originally announced November 2014.
-
A Relaxed-Projection Splitting Algorithm for Variational Inequalities in Hilbert Spaces
Authors:
J. Y. Bello Cruz,
R. Diaz Millan
Abstract:
We introduce a relaxed-projection splitting algorithm for solving variational inequalities in Hilbert spaces for the sum of nonsmooth maximal monotone operators, where the feasible set is defined by a nonlinear and nonsmooth continuous convex function inequality. In our scheme, the orthogonal projections onto the feasible set are replaced by projections onto separating hyperplanes. Furthermore, ea…
▽ More
We introduce a relaxed-projection splitting algorithm for solving variational inequalities in Hilbert spaces for the sum of nonsmooth maximal monotone operators, where the feasible set is defined by a nonlinear and nonsmooth continuous convex function inequality. In our scheme, the orthogonal projections onto the feasible set are replaced by projections onto separating hyperplanes. Furthermore, each iteration of the proposed method consists of simple subgradient-like steps, which does not demand the solution of a nontrivial subproblem, using only individual operators, which exploits the structure of the problem. Assuming monotonicity of the individual operators and the existence of solutions, we prove that the generated sequence converges weakly to a solution.
△ Less
Submitted 29 December, 2015; v1 submitted 13 December, 2013;
originally announced December 2013.
