-
Safe global optimization of expensive noisy black-box functions in the $δ$-Lipschitz framework
Authors:
Yaroslav D. Sergeyev,
Antonio Candelieri,
Dmitri E. Kvasov,
Riccardo Perego
Abstract:
In this paper, the problem of safe global maximization (it should not be confused with robust optimization) of expensive noisy black-box functions satisfying the Lipschitz condition is considered. The notion "safe" means that the objective function $f(x)$ during optimization should not violate a "safety" threshold, for instance, a certain a priori given value $h$ in a maximization problem. Thus, a…
▽ More
In this paper, the problem of safe global maximization (it should not be confused with robust optimization) of expensive noisy black-box functions satisfying the Lipschitz condition is considered. The notion "safe" means that the objective function $f(x)$ during optimization should not violate a "safety" threshold, for instance, a certain a priori given value $h$ in a maximization problem. Thus, any new function evaluation (possibly corrupted by noise) must be performed at "safe points" only, namely, at points $y$ for which it is known that the objective function $f(y) > h$. The main difficulty here consists in the fact that the used optimization algorithm should ensure that the safety constraint will be satisfied at a point $y$ before evaluation of $f(y)$ will be executed. Thus, it is required both to determine the safe region $Ω$ within the search domain~$D$ and to find the global maximum within $Ω$. An additional difficulty consists in the fact that these problems should be solved in the presence of the noise. This paper starts with a theoretical study of the problem and it is shown that even though the objective function $f(x)$ satisfies the Lipschitz condition, traditional Lipschitz minorants and majorants cannot be used due to the presence of the noise. Then, a $δ$-Lipschitz framework and two algorithms using it are proposed to solve the safe global maximization problem. The first method determines the safe area within the search domain and the second one executes the global maximization over the found safe region. For both methods a number of theoretical results related to their functioning and convergence is established. Finally, numerical experiments confirming the reliability of the proposed procedures are performed.
△ Less
Submitted 15 August, 2020; v1 submitted 15 August, 2019;
originally announced August 2019.
-
On strong homogeneity of a class of global optimization algorithms working with infinite and infinitesimal scales
Authors:
Yaroslav D. Sergeyev,
Dmitri E. Kvasov,
Marat S. Mukhametzhanov
Abstract:
The necessity to find the global optimum of multiextremal functions arises in many applied problems where finding local solutions is insufficient. One of the desirable properties of global optimization methods is \emph{strong homogeneity} meaning that a method produces the same sequences of points where the objective function is evaluated independently both of multiplication of the function by a s…
▽ More
The necessity to find the global optimum of multiextremal functions arises in many applied problems where finding local solutions is insufficient. One of the desirable properties of global optimization methods is \emph{strong homogeneity} meaning that a method produces the same sequences of points where the objective function is evaluated independently both of multiplication of the function by a scaling constant and of adding a shifting constant. In this paper, several aspects of global optimization using strongly homogeneous methods are considered. First, it is shown that even if a method possesses this property theoretically, numerically very small and large scaling constants can lead to ill-conditioning of the scaled problem. Second, a new class of global optimization problems where the objective function can have not only finite but also infinite or infinitesimal Lipschitz constants is introduced. Third, the strong homogeneity of several Lipschitz global optimization algorithms is studied in the framework of the Infinity Computing paradigm allowing one to work \emph{numerically} with a variety of infinities and infinitesimals. Fourth, it is proved that a class of efficient univariate methods enjoys this property for finite, infinite and infinitesimal scaling and shifting constants. Finally, it is shown that in certain cases the usage of numerical infinities and infinitesimals can avoid ill-conditioning produced by scaling. Numerical experiments illustrating theoretical results are described.
△ Less
Submitted 13 January, 2018;
originally announced January 2018.
-
Deterministic approaches for solving practical black-box global optimization problems
Authors:
Dmitri E. Kvasov,
Yaroslav D. Sergeyev
Abstract:
In many important design problems, some decisions should be made by finding the global optimum of a multiextremal objective function subject to a set of constrains. Frequently, especially in engineering applications, the functions involved in optimization process are black-box with unknown analytical representations and hard to evaluate. Such computationally challenging decision-making problems of…
▽ More
In many important design problems, some decisions should be made by finding the global optimum of a multiextremal objective function subject to a set of constrains. Frequently, especially in engineering applications, the functions involved in optimization process are black-box with unknown analytical representations and hard to evaluate. Such computationally challenging decision-making problems often cannot be solved by traditional optimization techniques based on strong suppositions about the problem (convexity, differentiability, etc.). Nature and evolutionary inspired metaheuristics are also not always successful in finding global solutions to these problems due to their multiextremal character. In this paper, some innovative and powerful deterministic approaches developed by the authors to construct numerical methods for solving the mentioned problems are surveyed. Their efficiency is shown on solving both the classes of random test problems and some practical engineering tasks.
△ Less
Submitted 15 September, 2015;
originally announced September 2015.
-
A deterministic global optimization using smooth diagonal auxiliary functions
Authors:
Yaroslav D. Sergeyev,
Dmitri E. Kvasov
Abstract:
In many practical decision-making problems it happens that functions involved in optimization process are black-box with unknown analytical representations and hard to evaluate. In this paper, a global optimization problem is considered where both the goal function~$f(x)$ and its gradient $f'(x)$ are black-box functions. It is supposed that $f'(x)$ satisfies the Lipschitz condition over the search…
▽ More
In many practical decision-making problems it happens that functions involved in optimization process are black-box with unknown analytical representations and hard to evaluate. In this paper, a global optimization problem is considered where both the goal function~$f(x)$ and its gradient $f'(x)$ are black-box functions. It is supposed that $f'(x)$ satisfies the Lipschitz condition over the search hyperinterval with an unknown Lipschitz constant~$K$. A new deterministic `Divide-the-Best' algorithm based on efficient diagonal partitions and smooth auxiliary functions is proposed in its basic version, its convergence conditions are studied and numerical experiments executed on eight hundred test functions are presented.
△ Less
Submitted 15 September, 2015;
originally announced September 2015.
-
Lipschitz gradients for global optimization in a one-point-based partitioning scheme
Authors:
Dmitri E. Kvasov,
Yaroslav D. Sergeyev
Abstract:
A global optimization problem is studied where the objective function $f(x)$ is a multidimensional black-box function and its gradient $f'(x)$ satisfies the Lipschitz condition over a hyperinterval with an unknown Lipschitz constant $K$. Different methods for solving this problem by using an a priori given estimate of $K$, its adaptive estimates, and adaptive estimates of local Lipschitz constants…
▽ More
A global optimization problem is studied where the objective function $f(x)$ is a multidimensional black-box function and its gradient $f'(x)$ satisfies the Lipschitz condition over a hyperinterval with an unknown Lipschitz constant $K$. Different methods for solving this problem by using an a priori given estimate of $K$, its adaptive estimates, and adaptive estimates of local Lipschitz constants are known in the literature. Recently, the authors have proposed a one-dimensional algorithm working with multiple estimates of the Lipschitz constant for $f'(x)$ (the existence of such an algorithm was a challenge for 15 years). In this paper, a new multidimensional geometric method evolving the ideas of this one-dimensional scheme and using an efficient one-point-based partitioning strategy is proposed. Numerical experiments executed on 800 multidimensional test functions demonstrate quite a promising performance in comparison with popular DIRECT-based methods.
△ Less
Submitted 15 July, 2013;
originally announced July 2013.
-
A One-Dimensional Local Tuning Algorithm for Solving GO Problems with Partially Defined Constraints
Authors:
Yaroslav D. Sergeyev,
Dmitri E. Kvasov,
Falah M. H. Khalaf
Abstract:
Lipschitz one-dimensional constrained global optimization (GO) problems where both the objective function and constraints can be multiextremal and non-differentiable are considered in this paper. Problems, where the constraints are verified in an a priori given order fixed by the nature of the problem are studied. Moreover, if a constraint is not satisfied at a point, then the remaining constraint…
▽ More
Lipschitz one-dimensional constrained global optimization (GO) problems where both the objective function and constraints can be multiextremal and non-differentiable are considered in this paper. Problems, where the constraints are verified in an a priori given order fixed by the nature of the problem are studied. Moreover, if a constraint is not satisfied at a point, then the remaining constraints and the objective function can be undefined at this point. The constrained problem is reduced to a discontinuous unconstrained problem by the index scheme without introducing additional parameters or variables. A new geometric method using adaptive estimates of local Lipschitz constants is introduced. The estimates are calculated by using the local tuning technique proposed recently. Numerical experiments show quite a satisfactory performance of the new method in comparison with the penalty approach and a method using a priori given Lipschitz constants.
△ Less
Submitted 26 July, 2011;
originally announced July 2011.
-
Local tuning and partition strategies for diagonal GO methods
Authors:
Dmitri E. Kvasov,
Clara Pizzuti,
Yaroslav D. Sergeyev
Abstract:
In this paper, global optimization (GO) Lipschitz problems are considered where the multi-dimensional multiextremal objective function is determined over a hyperinterval. An efficient one-dimensional GO method using local tuning on the behavior of the objective function is generalized to the multi-dimensional case by the diagonal approach using two partition strategies. Global convergence conditio…
▽ More
In this paper, global optimization (GO) Lipschitz problems are considered where the multi-dimensional multiextremal objective function is determined over a hyperinterval. An efficient one-dimensional GO method using local tuning on the behavior of the objective function is generalized to the multi-dimensional case by the diagonal approach using two partition strategies. Global convergence conditions are established for the obtained diagonal geometric methods. Results of a wide numerical comparison show a strong acceleration reached by the new methods working with estimates of the local Lipschitz constants over different subregions of the search domain in comparison with the traditional approach.
△ Less
Submitted 18 March, 2011;
originally announced March 2011.
-
Software for Generation of Classes of Test Functions with Known Local and Global Minima for Global Optimization
Authors:
Marco Gaviano,
Dmitri E. Kvasov,
Daniela Lera,
Yaroslav D. Sergeyev
Abstract:
A procedure for generating non-differentiable, continuously differentiable, and twice continuously differentiable classes of test functions for multiextremal multidimensional box-constrained global optimization and a corresponding package of C subroutines are presented. Each test class consists of 100 functions. Test functions are generated by defining a convex quadratic function systematically di…
▽ More
A procedure for generating non-differentiable, continuously differentiable, and twice continuously differentiable classes of test functions for multiextremal multidimensional box-constrained global optimization and a corresponding package of C subroutines are presented. Each test class consists of 100 functions. Test functions are generated by defining a convex quadratic function systematically distorted by polynomials in order to introduce local minima. To determine a class, the user defines the following parameters: (i) problem dimension, (ii) number of local minima, (iii) value of the global minimum, (iv) radius of the attraction region of the global minimizer, (v) distance from the global minimizer to the vertex of the quadratic function. Then, all other necessary parameters are generated randomly for all 100 functions of the class. Full information about each test function including locations and values of all local minima is supplied to the user. Partial derivatives are also generated where possible.
△ Less
Submitted 14 March, 2011;
originally announced March 2011.
-
Global Search Based on Efficient Diagonal Partitions and a set of Lipschitz Constants
Authors:
Yaroslav D. Sergeyev,
Dmitri E. Kvasov
Abstract:
In the paper, the global optimization problem of a multidimensional "black-box" function satisfying the Lipschitz condition over a hyperinterval with an unknown Lipschitz constant is considered. A new efficient algorithm for solving this problem is presented. At each iteration of the method a number of possible Lipschitz constants is chosen from a set of values varying from zero to infinity. This…
▽ More
In the paper, the global optimization problem of a multidimensional "black-box" function satisfying the Lipschitz condition over a hyperinterval with an unknown Lipschitz constant is considered. A new efficient algorithm for solving this problem is presented. At each iteration of the method a number of possible Lipschitz constants is chosen from a set of values varying from zero to infinity. This idea is unified with an efficient diagonal partition strategy. A novel technique balancing usage of local and global information during partitioning is proposed. A new procedure for finding lower bounds of the objective function over hyperintervals is also considered. It is demonstrated by extensive numerical experiments performed on more than 1600 multidimensional test functions that the new algorithm shows a very promising performance.
△ Less
Submitted 10 March, 2011;
originally announced March 2011.
