-
Towards the theory of strong minimum. A view from variational analysis
Authors:
A. D. Ioffe
Abstract:
The key element of the approach to the theory of necessary conditions in optimal control discussed in the paper is reduction of the original constrained problem to unconstrained minimization with subsequent application of a suitable mechanism of local analysis to characterize minima of (necessarily nonsmooth) functionals that appear after reduction. Using unconstrained minimization at the crucial…
▽ More
The key element of the approach to the theory of necessary conditions in optimal control discussed in the paper is reduction of the original constrained problem to unconstrained minimization with subsequent application of a suitable mechanism of local analysis to characterize minima of (necessarily nonsmooth) functionals that appear after reduction. Using unconstrained minimization at the crucial step of obtaining necessary conditions definitely facilitates studies of new phenomena and allows to get more transparent and technically simple proofs of known results. In the paper we offer a new proof of the maximum principle for a nonsmooth optimal control problem (in the standard Pontryagin form) with state constraints and then prove a new second order condition for a strong minimum in the same problem but with data differentiable with respect to the state and control variables. The role of variational analysis is twofold. Conceptually, the main considerations behind the reduction are connected with metric regularity and Ekeland's principle. On the other hand, technically, calculation of subdifferentials of components of the functionals that appear after the reduction is an essential part of the proofs.
△ Less
Submitted 25 June, 2019; v1 submitted 24 April, 2019;
originally announced April 2019.
-
Nonsmooth optimization using Taylor-like models: error bounds, convergence, and termination criteria
Authors:
Dmitriy Drusvyatskiy,
Alexander D. Ioffe,
Adrian S. Lewis
Abstract:
We consider optimization algorithms that successively minimize simple Taylor-like models of the objective function. Methods of Gauss-Newton type for minimizing the composition of a convex function and a smooth map are common examples. Our main result is an explicit relationship between the step-size of any such algorithm and the slope of the function at a nearby point. Consequently, we (1) show th…
▽ More
We consider optimization algorithms that successively minimize simple Taylor-like models of the objective function. Methods of Gauss-Newton type for minimizing the composition of a convex function and a smooth map are common examples. Our main result is an explicit relationship between the step-size of any such algorithm and the slope of the function at a nearby point. Consequently, we (1) show that the step-sizes can be reliably used to terminate the algorithm, (2) prove that as long as the step-sizes tend to zero, every limit point of the iterates is stationary, and (3) show that conditions, akin to classical quadratic growth, imply that the step-sizes linearly bound the distance of the iterates to the solution set. The latter so-called error bound property is typically used to establish linear (or faster) convergence guarantees. Analogous results hold when the step-size is replaced by the square root of the decrease in the model's value. We complete the paper with extensions to when the models are minimized only inexactly.
△ Less
Submitted 11 October, 2016;
originally announced October 2016.
-
Polyhedrality,Complementarity and Regularity with Applications to Variational Inequalities over Polyhedral Sets
Authors:
Alexander D. Ioffe
Abstract:
The regularity theory for variational inequalities over polyhedral sets developed in a series of papers by Robinson, Ralph and Dontchev-Rockafellar in the 90s has long become classics of variational analysis. But in the available proofs of almost all main results, fairly nontrivial as they are, techniques of variational analysis do not play a significant part. In the paper we develop a new approac…
▽ More
The regularity theory for variational inequalities over polyhedral sets developed in a series of papers by Robinson, Ralph and Dontchev-Rockafellar in the 90s has long become classics of variational analysis. But in the available proofs of almost all main results, fairly nontrivial as they are, techniques of variational analysis do not play a significant part. In the paper we develop a new approach that allows to obtain some generalizations of the the mentioned results without invoking anything beyond elementary geometry of convex polyhedra and some basic facts of the theory of metric regularity.
△ Less
Submitted 30 August, 2015; v1 submitted 26 August, 2015;
originally announced August 2015.
-
Metric Regularity. Theory and Applications - a survey
Authors:
A. D. Ioffe
Abstract:
Metric regularity has emerged during last 2-3 decades as one of the central concepts of variational analysis. The roots of this concept go back to a circle of fundamental regularity ideas of classical analysis embodied in such results as the implicit function theorem, Banach open mapping theorem, theorems of Lyusternik and Graves, on the one hand, and the Sard theorem and the Thom-Smale transversa…
▽ More
Metric regularity has emerged during last 2-3 decades as one of the central concepts of variational analysis. The roots of this concept go back to a circle of fundamental regularity ideas of classical analysis embodied in such results as the implicit function theorem, Banach open mapping theorem, theorems of Lyusternik and Graves, on the one hand, and the Sard theorem and the Thom-Smale transversality theory, on the other. The three principal themes that are in the focus of attention are: (a) regularity criteria (containing quantitative estimates for rates of regularity) including formal comparisons of their relative power and precision; (b) stability problems relating to the effect of perturbations of the mapping on its regularity properties, on the one hand, and to solutions of equations, inclusions etc. on the other; (c) role of metric regularity in analysis and optimization. All of them are studied at three levels of generality: the general theory for (set-valued) mappings between metric spaces is followed by a detailed study of Banach and finite dimensional theories. There is a number of new results, both theoretical and relating to applications, and some known results are supplied with new, usually simpler, proofs.
△ Less
Submitted 24 October, 2015; v1 submitted 29 May, 2015;
originally announced May 2015.
-
Generic minimizing behavior in semi-algebraic optimization
Authors:
D. Drusvyatskiy,
A. D. Ioffe,
A. S. Lewis
Abstract:
We present a theorem of Sard type for semi-algebraic set-valued mappings whose graphs have dimension no larger than that of their range space: the inverse of such a mapping admits a single-valued analytic localization around any pair in the graph, for a generic value parameter. This simple result yields a transparent and unified treatment of generic properties of semi-algebraic optimization proble…
▽ More
We present a theorem of Sard type for semi-algebraic set-valued mappings whose graphs have dimension no larger than that of their range space: the inverse of such a mapping admits a single-valued analytic localization around any pair in the graph, for a generic value parameter. This simple result yields a transparent and unified treatment of generic properties of semi-algebraic optimization problems: "typical" semi-algebraic problems have finitely many critical points, around each of which they admit a unique "active manifold" (analogue of an active set in nonlinear optimization); moreover, such critical points satisfy strict complementarity and second-order sufficient conditions for optimality are indeed necessary.
△ Less
Submitted 28 April, 2015;
originally announced April 2015.
-
Transversality and alternating projections for nonconvex sets
Authors:
D. Drusvyatskiy,
A. D. Ioffe,
A. S. Lewis
Abstract:
We consider the method of alternating projections for finding a point in the intersection of two closed sets, possibly nonconvex. Assuming only the standard transversality condition (or a weaker version thereof), we prove local linear convergence. When the two sets are semi-algebraic and bounded, but not necessarily transversal, we nonetheless prove subsequence convergence.
We consider the method of alternating projections for finding a point in the intersection of two closed sets, possibly nonconvex. Assuming only the standard transversality condition (or a weaker version thereof), we prove local linear convergence. When the two sets are semi-algebraic and bounded, but not necessarily transversal, we nonetheless prove subsequence convergence.
△ Less
Submitted 11 August, 2016; v1 submitted 29 January, 2014;
originally announced January 2014.
-
Quadratic growth and critical point stability of semi-algebraic functions
Authors:
D. Drusvyatskiy,
A. D. Ioffe
Abstract:
We show that quadratic growth of a semi-algebraic function is equivalent to strong metric subregularity of the subdifferential --- a kind of stability of generalized critical points. In contrast, this equivalence can easily fail outside of the semi-algebraic setting. As a consequence, we derive necessary conditions and sufficient conditions for optimality in subdifferential terms.
We show that quadratic growth of a semi-algebraic function is equivalent to strong metric subregularity of the subdifferential --- a kind of stability of generalized critical points. In contrast, this equivalence can easily fail outside of the semi-algebraic setting. As a consequence, we derive necessary conditions and sufficient conditions for optimality in subdifferential terms.
△ Less
Submitted 20 December, 2014; v1 submitted 5 September, 2013;
originally announced September 2013.
-
Curves of Descent
Authors:
D. Drusvyatskiy,
A. D. Ioffe,
A. S. Lewis
Abstract:
Steepest descent is central in variational mathematics. We present a new transparent existence proof for curves of near-maximal slope --- an influential notion of steepest descent in a nonsmooth setting. We moreover show that for semi-algebraic functions --- prototypical nonpathological functions in nonsmooth optimization --- such curves are precisely the solutions of subgradient dynamical systems…
▽ More
Steepest descent is central in variational mathematics. We present a new transparent existence proof for curves of near-maximal slope --- an influential notion of steepest descent in a nonsmooth setting. We moreover show that for semi-algebraic functions --- prototypical nonpathological functions in nonsmooth optimization --- such curves are precisely the solutions of subgradient dynamical systems.
△ Less
Submitted 7 May, 2013; v1 submitted 5 December, 2012;
originally announced December 2012.
-
Clarke subgradients for directionally Lipschitzian stratifiable functions
Authors:
Dmitriy Drusvyatskiy,
Alexander D. Ioffe,
Adrian S. Lewis
Abstract:
Using a geometric argument, we show that under a reasonable continuity condition, the Clarke subdifferential of a semi-algebraic (or more generally stratifiable) directionally Lipschitzian function admits a simple form: the normal cone to the domain and limits of gradients generate the entire Clarke subdifferential. The characterization formula we obtain unifies various apparently disparate result…
▽ More
Using a geometric argument, we show that under a reasonable continuity condition, the Clarke subdifferential of a semi-algebraic (or more generally stratifiable) directionally Lipschitzian function admits a simple form: the normal cone to the domain and limits of gradients generate the entire Clarke subdifferential. The characterization formula we obtain unifies various apparently disparate results that have appeared in the literature. Our techniques also yield a simplified proof that closed semialgebraic functions on $\R^n$ have a limiting subdifferential graph of uniform local dimension $n$.
△ Less
Submitted 15 November, 2012;
originally announced November 2012.
-
The dimension of semialgebraic subdifferential graphs
Authors:
Dmitriy Drusvyatskiy,
Alexander D. Ioffe,
Adrian S. Lewis
Abstract:
Examples exist of extended-real-valued closed functions on ${\bf R}^n$ whose subdifferentials (in the standard, limiting sense) have large graphs. By contrast, if such a function is semi-algebraic, then its subdifferential graph must have everywhere constant local dimension $n$. This result is related to a celebrated theorem of Minty, and surprisingly may fail for the Clarke subdifferential.
Examples exist of extended-real-valued closed functions on ${\bf R}^n$ whose subdifferentials (in the standard, limiting sense) have large graphs. By contrast, if such a function is semi-algebraic, then its subdifferential graph must have everywhere constant local dimension $n$. This result is related to a celebrated theorem of Minty, and surprisingly may fail for the Clarke subdifferential.
△ Less
Submitted 20 February, 2011;
originally announced February 2011.
-
A Sard theorem for Tame Set-Valued mappings
Authors:
A. D. Ioffe
Abstract:
If $F$ is a set-valued mapping from $\R^n$ into $\R^m$ with closed graph, then $y\in \R^m$ is a critical value of $F$ if for some $x$ with $y\in F(x)$, $F$ is not metrically regular at $(x,y)$. We prove that the set of critical values of a set-valued mapping whose graph is a definable (tame) set in an $o$-minimal structure containing additions and multiplications is a set of dimension not greate…
▽ More
If $F$ is a set-valued mapping from $\R^n$ into $\R^m$ with closed graph, then $y\in \R^m$ is a critical value of $F$ if for some $x$ with $y\in F(x)$, $F$ is not metrically regular at $(x,y)$. We prove that the set of critical values of a set-valued mapping whose graph is a definable (tame) set in an $o$-minimal structure containing additions and multiplications is a set of dimension not greater than $m-1$ (resp. a porous set). As a corollary of this result we get that the collection of asymptotically critical values of a semialgebraic set-valued mapping has dimension not greater than $m-1$, thus extending to such mappings a corresponding result by Kurdyka-Orro-Simon for $C^1$ semialgebraic mappings. We also give an independent proof of the fact that a definable continuous real-valued function is constant on components of the set of its subdifferentiably critical points, thus extending to all definable functions a recent result of Bolte-Daniilidis-Lewis for globally subanalytic functions.
△ Less
Submitted 27 July, 2006;
originally announced July 2006.
