-
Extended SQP Methods in Nonsmooth Difference Programming Applied to Problems with Variational Inequality Constraints
Authors:
Boris S. Mordukhovich,
Yixia Song,
Shangzhi Zeng,
Jin Zhang
Abstract:
This paper explores a new class of constrained difference programming problems, where the objective and constraints are formulated as differences of functions, without requiring their convexity. To investigate such problems, novel variants of the extended sequential quadratic method are introduced. These algorithms iteratively solve strongly convex quadratic subproblems constructed via linear appr…
▽ More
This paper explores a new class of constrained difference programming problems, where the objective and constraints are formulated as differences of functions, without requiring their convexity. To investigate such problems, novel variants of the extended sequential quadratic method are introduced. These algorithms iteratively solve strongly convex quadratic subproblems constructed via linear approximations of the given data by using their gradients and subgradients. The convergence of the proposed methods is rigorously analyzed by employing, in particular, the Polyak-Łojasiewicz-Kurdyka property that ensures global convergence for various classes of functions in the problem formulation, e.g., semialgebraic ones. The original framework is further extended to address difference programming problems with variational inequality (VI) constraints. By reformulating VI constraints via regularized gap functions, such problems are naturally embedded into constrained difference programming that leads us to direct applications of the proposed algorithms. Numerical experiments for the class of continuous network design problems demonstrate the efficiency of the new methods.
△ Less
Submitted 7 April, 2025;
originally announced April 2025.
-
The Least Singular Value Function in Variational Analysis
Authors:
Mario Jelitte,
Boris S. Mordukhovich
Abstract:
Metric regularity is among the central concepts of nonlinear and variational analysis, constrained optimization, and their numerous applications. However, metric regularity can be elusive for some important ill-posed classes of problems including polynomial equations, parametric variational systems, smooth reformulations of complementarity systems with degenerate solutions, etc. The study of stabi…
▽ More
Metric regularity is among the central concepts of nonlinear and variational analysis, constrained optimization, and their numerous applications. However, metric regularity can be elusive for some important ill-posed classes of problems including polynomial equations, parametric variational systems, smooth reformulations of complementarity systems with degenerate solutions, etc. The study of stability issues for such problems can often not rely on the machinery of first-order variational analysis, and so higher-order regularity concepts have been proposed in recent years. In this paper, we investigate some notions of mixed-order regularity by using advanced tools of first-order and second-order variational analysis and generalized differentiation of both primal and dual types. Efficient characterizations of such mixed-order regularity concepts are established by employing a fresh notion of the least singular value function. The obtained conditions are applied to deriving constructive criteria for mixed-order regularity in coupled constraint and variational systems.
△ Less
Submitted 25 March, 2025;
originally announced March 2025.
-
Optimal Control of Nonconvex Sweeping Processes with Variable Time via Finite-Difference Approximations
Authors:
Tan H. Cao,
Boris S. Mordukhovich,
Dao Nguyen,
Trang Nguyen,
Nguyen N. Thieu
Abstract:
The paper is devoted to the study of a new class of optimal control problems for nonsmooth dynamical systems governed by nonconvex discontinuous differential inclusions of the sweeping type with involving variable time into optimization. We develop a novel version of the method of discrete approximations of its own qualitative and numerical importance with establishing its well-posedness and stron…
▽ More
The paper is devoted to the study of a new class of optimal control problems for nonsmooth dynamical systems governed by nonconvex discontinuous differential inclusions of the sweeping type with involving variable time into optimization. We develop a novel version of the method of discrete approximations of its own qualitative and numerical importance with establishing its well-posedness and strong convergence to optimal solutions of the controlled sweeping process. Using advanced tools of variational analysis and generalized differentiation leads us to deriving new necessary conditions for optimal solutions to discrete approximation problems, which serve as suboptimality conditions for the original continuous-time controlled sweeping process. The obtained results are applied to a class of motion models of practical interest, where the established necessary conditions are used to investigate the agents' interactions and to develop an algorithm for calculating optimal solutions.
△ Less
Submitted 4 March, 2025; v1 submitted 1 March, 2025;
originally announced March 2025.
-
Characterizations of Variational Convexity and Tilt Stability via Quadratic Bundles
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Vo Thanh Phat,
Le Duc Viet
Abstract:
In this paper, we establish characterizations of variational $s$-convexity and tilt stability for prox-regular functions in the absence of subdifferential continuity via quadratic bundles, a kind of primal-dual generalized second-order derivatives recently introduced by Rockafellar. Deriving such characterizations in the effective pointbased form requires a certain revision of quadratic bundles in…
▽ More
In this paper, we establish characterizations of variational $s$-convexity and tilt stability for prox-regular functions in the absence of subdifferential continuity via quadratic bundles, a kind of primal-dual generalized second-order derivatives recently introduced by Rockafellar. Deriving such characterizations in the effective pointbased form requires a certain revision of quadratic bundles investigated below. Our device is based on the notion of generalized twice differentiability and its novel characterization via classical twice differentiability of the associated Moreau envelopes combined with various limiting procedures for functions and sets.
△ Less
Submitted 8 January, 2025;
originally announced January 2025.
-
Generalized Twice Differentiability and Quadratic Bundles in Second-Order Variational Analysis
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Vo Thanh Phat,
Le Duc Viet
Abstract:
In this paper, we investigate the concepts of generalized twice differentiability and quadratic bundles of nonsmooth functions that have been very recently proposed by Rockafellar in the framework of second-order variational analysis. These constructions, in contrast to second-order subdifferentials, are defined in primal spaces. We develop new techniques to study generalized twice differentiabili…
▽ More
In this paper, we investigate the concepts of generalized twice differentiability and quadratic bundles of nonsmooth functions that have been very recently proposed by Rockafellar in the framework of second-order variational analysis. These constructions, in contrast to second-order subdifferentials, are defined in primal spaces. We develop new techniques to study generalized twice differentiability for a broad class of prox-regular functions, establish their novel characterizations. Subsequently, quadratic bundles of prox-regular functions are shown to be nonempty, which provides the ground of potential applications in variational analysis and optimization.
△ Less
Submitted 3 January, 2025;
originally announced January 2025.
-
An Inexact Boosted Difference of Convex Algorithm for Nondifferentiable Functions
Authors:
Orizon P. Ferreira,
Boris S. Mordukhovich,
Wilkreffy M. S. Santos,
João Carlos O. Souza
Abstract:
In this paper, we introduce an inexact approach to the Boosted Difference of Convex Functions Algorithm (BDCA) for solving nonconvex and nondifferentiable problems involving the difference of two convex functions (DC functions). Specifically, when the first DC component is differentiable and the second may be nondifferentiable, BDCA utilizes the solution from the subproblem of the DC Algorithm (DC…
▽ More
In this paper, we introduce an inexact approach to the Boosted Difference of Convex Functions Algorithm (BDCA) for solving nonconvex and nondifferentiable problems involving the difference of two convex functions (DC functions). Specifically, when the first DC component is differentiable and the second may be nondifferentiable, BDCA utilizes the solution from the subproblem of the DC Algorithm (DCA) to define a descent direction for the objective function. A monotone linesearch is then performed to find a new point that improves the objective function relative to the subproblem solution. This approach enhances the performance of DCA. However, if the first DC component is nondifferentiable, the BDCA direction may become an ascent direction, rendering the monotone linesearch ineffective. To address this, we propose an Inexact nonmonotone Boosted Difference of Convex Algorithm (InmBDCA). This algorithm incorporates two main features of inexactness: First, the subproblem therein is solved approximately allowing us for a controlled relative error tolerance in defining the linesearch direction. Second, an inexact nonmonotone linesearch scheme is used to determine the step size for the next iteration. Under suitable assumptions, we demonstrate that InmBDCA is well-defined, with any accumulation point of the sequence generated by InmBDCA being a critical point of the problem. We also provide iteration-complexity bounds for the algorithm. Numerical experiments show that InmBDCA outperforms both the nonsmooth BDCA (nmBDCA) and the monotone version of DCA in practical scenarios.
△ Less
Submitted 7 December, 2024;
originally announced December 2024.
-
Subdifferential Calculus for Ordered Set-Valued Mappings between Infinite-Dimensional Spaces
Authors:
Boris S. Mordukhovich,
Oanh Nguyen
Abstract:
The paper is devoted to developing subdifferential theory for set-valued mappings taking values in ordered infinite-dimensional spaces. This study is motivated by applications to problems of vector and set optimization with various constraints in infinite dimensions. The main results establish new sum and chain rules for major subdifferential constructions associated with ordered set-valued mappin…
▽ More
The paper is devoted to developing subdifferential theory for set-valued mappings taking values in ordered infinite-dimensional spaces. This study is motivated by applications to problems of vector and set optimization with various constraints in infinite dimensions. The main results establish new sum and chain rules for major subdifferential constructions associated with ordered set-valued mappings under appropriate qualification and sequentially normal compactness conditions.
△ Less
Submitted 15 October, 2024;
originally announced October 2024.
-
Discrete approximations and optimality conditions for integro-differential inclusions
Authors:
Abderrahim Bouach,
Tahar Haddad,
Boris S. Mordukhovich
Abstract:
This paper addresses a new class of generalized Bolza problems governed by nonconvex integro-differential inclusions with endpoint constraints on trajectories, where the integral terms are given in the general (with time-dependent integrands in the dynamics) Volterra form. We pursue here a threefold goal. First we construct well-posed approximations of continuous-time integro-differential systems…
▽ More
This paper addresses a new class of generalized Bolza problems governed by nonconvex integro-differential inclusions with endpoint constraints on trajectories, where the integral terms are given in the general (with time-dependent integrands in the dynamics) Volterra form. We pursue here a threefold goal. First we construct well-posed approximations of continuous-time integro-differential systems by their discrete-time counterparts with showing that any feasible solution to the original system can be strongly approximated in the $W^{1,2}$-norm topology by piecewise-linear extensions of feasible discrete trajectories. This allows us to verify in turn the strong convergence of discrete optimal solutions to a prescribed local minimizer for the original problem. Facing intrinsic nonsmoothness of original integro-differential problem and its discrete approximations, we employ appropriate tools of generalized differentiation in variational analysis to derive necessary optimality conditions for discrete-time problems (which is our second goal) and finally accomplish our third goal to obtain necessary conditions for the original continuous-time problems by passing to the limit from discrete approximations. In this way we establish, in particular, a novel necessary optimality condition of the Volterra type, which is the crucial result for dynamic optimization of integro-differential inclusions.
△ Less
Submitted 4 October, 2024; v1 submitted 5 August, 2024;
originally announced August 2024.
-
Coderivative-Based Newton Methods with Wolfe Linesearch for Nonsmooth Optimization
Authors:
Miantao Chao,
Boris S. Mordukhovich,
Zijian Shi,
Jin Zhang
Abstract:
This paper introduces and develops novel coderivative-based Newton methods with Wolfe linesearch conditions to solve various classes of problems in nonsmooth optimization. We first propose a generalized regularized Newton method with Wolfe linesearch (GRNM-W) for unconstrained $C^{1,1}$ minimization problems (which are second-order nonsmooth) and establish global as well as local superlinear conve…
▽ More
This paper introduces and develops novel coderivative-based Newton methods with Wolfe linesearch conditions to solve various classes of problems in nonsmooth optimization. We first propose a generalized regularized Newton method with Wolfe linesearch (GRNM-W) for unconstrained $C^{1,1}$ minimization problems (which are second-order nonsmooth) and establish global as well as local superlinear convergence of their iterates. To deal with convex composite minimization problems (which are first-order nonsmooth and can be constrained), we combine the proposed GRNM-W with two algorithmic frameworks: the forward-backward envelope and the augmented Lagrangian method resulting in the two new algorithms called CNFB and CNAL, respectively. Finally, we present numerical results to solve Lasso and support vector machine problems appearing in, e.g., machine learning and statistics, which demonstrate the efficiency of the proposed algorithms.
△ Less
Submitted 3 July, 2024; v1 submitted 2 July, 2024;
originally announced July 2024.
-
Convergence of Descent Optimization Algorithms under Polyak-Łojasiewicz-Kurdyka Conditions
Authors:
G. C. Bento,
B. S. Mordukhovich,
T. S. Mota,
Yu. Nesterov
Abstract:
This paper develops a comprehensive convergence analysis for generic classes of descent algorithms in nonsmooth and nonconvex optimization under several conditions of the Polyak-Łojasiewicz-Kurdyka (PLK) type. Along other results, we prove the finite termination of generic algorithms under the PLK conditions with lower exponents. Specifications are given to establish new convergence rates for inex…
▽ More
This paper develops a comprehensive convergence analysis for generic classes of descent algorithms in nonsmooth and nonconvex optimization under several conditions of the Polyak-Łojasiewicz-Kurdyka (PLK) type. Along other results, we prove the finite termination of generic algorithms under the PLK conditions with lower exponents. Specifications are given to establish new convergence rates for inexact reduced gradient methods and some versions of the boosted algorithm in DC programming. It is revealed, e.g., that the lower exponent PLK conditions for a broad class of difference programs are incompatible with the gradient Lipschitz continuity for the plus function around a local minimizer. On the other hand, we show that the above inconsistency observation may fail if the Lipschitz continuity is replaced by merely the gradient continuity.
△ Less
Submitted 12 February, 2025; v1 submitted 30 June, 2024;
originally announced July 2024.
-
Regular Subgradients of Marginal Functions with Applications to Calculus and Bilevel Programming
Authors:
Le Phuoc Hai,
Felipe Lara,
Boris S. Mordukhovich
Abstract:
The paper addresses the study and applications of a broad class of extended-real-valued functions, known as optimal value or marginal functions, which are frequently appeared in variational analysis, parametric optimization, and a variety of applications. Functions of this type are intrinsically nonsmooth and require the usage of tools of generalized differentiation. The main results of this paper…
▽ More
The paper addresses the study and applications of a broad class of extended-real-valued functions, known as optimal value or marginal functions, which are frequently appeared in variational analysis, parametric optimization, and a variety of applications. Functions of this type are intrinsically nonsmooth and require the usage of tools of generalized differentiation. The main results of this paper provide novel evaluations and exact calculations of regular/Fréchet subgradients and their singular counterparts for general classes of marginal functions via their given data. The obtained results are applied to establishing new calculus rules for such subgradients and necessary optimality conditions in bilevel programming
△ Less
Submitted 4 February, 2025; v1 submitted 31 May, 2024;
originally announced May 2024.
-
Optimal control of perturbed sweeping processes with applications to general robotics models
Authors:
Giovanni Colombo,
Boris S. Mordukhovich,
Dao Nguyen,
Trang Nguyen,
Norma Ortiz-Robinson
Abstract:
This paper primarily focuses on the practical applications of optimal control theory for perturbed sweeping processes within the realm of robotics dynamics. By describing these models as controlled sweeping processes with pointwise control and state constraints and by employing necessary optimality conditions for such systems, we formulate optimal control problems suitable to these models and deve…
▽ More
This paper primarily focuses on the practical applications of optimal control theory for perturbed sweeping processes within the realm of robotics dynamics. By describing these models as controlled sweeping processes with pointwise control and state constraints and by employing necessary optimality conditions for such systems, we formulate optimal control problems suitable to these models and develop numerical algorithms for their solving. Subsequently, we use the Python Dynamic Optimization library GEKKO to simulate solutions to the posed robotics problems in the case of any fixed number of robots under different initial conditions.
△ Less
Submitted 14 May, 2024;
originally announced May 2024.
-
Relationships between Global and Local Monotonicity of Operators
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Juan Enrique Martínez-Legaz,
Boris S. Mordukhovich
Abstract:
The paper is devoted to establishing relationships between global and local monotonicity, as well as their maximality versions, for single-valued and set-valued mappings between finite-dimensional and infinite-dimensional spaces. We first show that for single-valued operators with convex domains in locally convex topological spaces, their continuity ensures that their global monotonicity agrees wi…
▽ More
The paper is devoted to establishing relationships between global and local monotonicity, as well as their maximality versions, for single-valued and set-valued mappings between finite-dimensional and infinite-dimensional spaces. We first show that for single-valued operators with convex domains in locally convex topological spaces, their continuity ensures that their global monotonicity agrees with the local one around any point of the graph. This also holds for set-valued mappings defined on the real line under a certain connectedness condition. The situation is different for set-valued operators in multidimensional spaces as demonstrated by an example of locally monotone operator on the plane that is not globally monotone. Finally, we invoke coderivative criteria from variational analysis to characterize both global and local maximal monotonicity of set-valued operators in Hilbert spaces to verify the equivalence between these monotonicity properties under the closed-graph and global hypomonotonicity assumptions.
△ Less
Submitted 29 March, 2024;
originally announced March 2024.
-
A Refined Proximal Algorithm for Nonconvex Multiobjective Optimization in Hilbert Spaces
Authors:
G. C. Bento,
J. X. Cruz Neto,
J. O. Lopes,
B. S. Mordukhovich,
P. R. Silva Filho
Abstract:
This paper is devoted to general nonconvex problems of multiobjective optimization in Hilbert spaces. Based on Mordukhovich's limiting subgradients, we define a new notion of Pareto critical points for such problems, establish necessary optimality conditions for them, and then employ these conditions to develop a refined version of the vectorial proximal point algorithm with providing its detailed…
▽ More
This paper is devoted to general nonconvex problems of multiobjective optimization in Hilbert spaces. Based on Mordukhovich's limiting subgradients, we define a new notion of Pareto critical points for such problems, establish necessary optimality conditions for them, and then employ these conditions to develop a refined version of the vectorial proximal point algorithm with providing its detailed convergence analysis. The obtained results largely extend those initiated by Bonnel, Iusem and Svaiter \cite{Bonnel2005} for convex vector optimization problems and by Bento et al. \cite{Bento2018} for nonconvex finite-dimensional problems in terms of Clarke's generalized gradients.
△ Less
Submitted 14 March, 2024;
originally announced March 2024.
-
Coderivative-Based Newton Methods in Structured Nonconvex and Nonsmooth Optimization
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
This paper proposes and develops new Newton-type methods to solve structured nonconvex and nonsmooth optimization problems with justifying their fast local and global convergence by means of advanced tools of variational analysis and generalized differentiation. The objective functions belong to a broad class of prox-regular functions with specification to constrained optimization of nonconvex str…
▽ More
This paper proposes and develops new Newton-type methods to solve structured nonconvex and nonsmooth optimization problems with justifying their fast local and global convergence by means of advanced tools of variational analysis and generalized differentiation. The objective functions belong to a broad class of prox-regular functions with specification to constrained optimization of nonconvex structured sums. We also develop a novel line search method, which is an extension of the proximal gradient algorithm while allowing us to globalize the proposed coderivative-based Newton methods by incorporating the machinery of forward-backward envelopes. Applications and numerical experiments, which are provided for nonconvex least squares regression models, Student's t-regression, and image restoration problems, demonstrate the efficiency of the proposed algorithms
△ Less
Submitted 20 May, 2025; v1 submitted 7 March, 2024;
originally announced March 2024.
-
Fundamental Convergence Analysis of Sharpness-Aware Minimization
Authors:
Pham Duy Khanh,
Hoang-Chau Luong,
Boris S. Mordukhovich,
Dat Ba Tran
Abstract:
The paper investigates the fundamental convergence properties of Sharpness-Aware Minimization (SAM), a recently proposed gradient-based optimization method [Foret et al., 2021] that significantly improves the generalization of deep neural networks. The convergence properties, including the stationarity of accumulation points, the convergence of the sequence of gradients to the origin, the sequence…
▽ More
The paper investigates the fundamental convergence properties of Sharpness-Aware Minimization (SAM), a recently proposed gradient-based optimization method [Foret et al., 2021] that significantly improves the generalization of deep neural networks. The convergence properties, including the stationarity of accumulation points, the convergence of the sequence of gradients to the origin, the sequence of function values to the optimal value, and the sequence of iterates to the optimal solution, are established for the method. The universality of the provided convergence analysis, based on inexact gradient descent frameworks Khanh et al. [2023b], allows its extensions to efficient normalized versions of SAM such as F-SAM [Li et al., 2024], VaSSO [Li and Giannakis, 2023], RSAM [Liu et al., 2022], and to the unnormalized versions of SAM such as USAM [Andriushchenko and Flammarion, 2022]. Numerical experiments are conducted on classification tasks using deep learning models to confirm the practical aspects of our analysis.
△ Less
Submitted 19 October, 2024; v1 submitted 15 January, 2024;
originally announced January 2024.
-
Stability Criteria and Calculus Rules via Conic Contingent Coderivatives in Banach Spaces
Authors:
Boris S. Mordukhovich,
Pengcheng Wu,
Xiaoqi Yang
Abstract:
This paper addresses the study of novel constructions of variational analysis and generalized differentiation that are appropriate for characterizing robust stability properties of constrained set-valued mappings/multifunctions between Banach spaces important in optimization theory and its applications. Our tools of generalized differentiation revolves around the newly introduced concept of…
▽ More
This paper addresses the study of novel constructions of variational analysis and generalized differentiation that are appropriate for characterizing robust stability properties of constrained set-valued mappings/multifunctions between Banach spaces important in optimization theory and its applications. Our tools of generalized differentiation revolves around the newly introduced concept of $\varepsilon$-regular normal cone to sets and associated coderivative notions for set-valued mappings. Based on these constructions, we establish several characterizations of the central stability notion known as the relative Lipschitz-like property of set-valued mappings in infinite dimensions. Applying a new version of the constrained extremal principle of variational analysis, we develop comprehensive sum and chain rules for our major constructions of conic contingent coderivatives for multifunctions between appropriate classes of Banach spaces.
△ Less
Submitted 9 January, 2024;
originally announced January 2024.
-
Second-Order Subdifferential Optimality Conditions in Nonsmooth Optimization
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
The paper is devoted to deriving novel second-order necessary and sufficient optimality conditions for local minimizers in rather general classes of nonsmooth unconstrained and constrained optimization problems in finite-dimensional spaces. The established conditions are expressed in terms of second-order subdifferentials of lower semicontinuous functions and mainly concern prox-regular objectives…
▽ More
The paper is devoted to deriving novel second-order necessary and sufficient optimality conditions for local minimizers in rather general classes of nonsmooth unconstrained and constrained optimization problems in finite-dimensional spaces. The established conditions are expressed in terms of second-order subdifferentials of lower semicontinuous functions and mainly concern prox-regular objectives that cover a large territory in nonsmooth optimization and its applications. Our tools are based on the machinery of variational analysis and second-order generalized differentiation. The obtained general results are applied to problems of nonlinear programming, where the derived second-order optimality conditions are new even for problems with twice continuously differential data, being expressed there in terms of the classical Hessian matrices.
△ Less
Submitted 4 January, 2025; v1 submitted 26 December, 2023;
originally announced December 2023.
-
Local Minimizers of Nonconvex Functions in Banach Spaces via Moreau Envelopes
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
This paper investigates the preservation of local minimizers and strong minimizers of extended-real-valued lower semicontinuous functions under taking their Moreau envelopes. We address a general setting of Banach spaces, while all the obtained results are new even for functions in finite dimensions. Our main motivation came from applications to numerical methods in nonsmooth optimization dealing…
▽ More
This paper investigates the preservation of local minimizers and strong minimizers of extended-real-valued lower semicontinuous functions under taking their Moreau envelopes. We address a general setting of Banach spaces, while all the obtained results are new even for functions in finite dimensions. Our main motivation came from applications to numerical methods in nonsmooth optimization dealing with broad classes of nondifferentiable and nonconvex functions. The paper also formulates and discusses some open questions stemming from this study.
△ Less
Submitted 30 November, 2023;
originally announced November 2023.
-
General Derivative-Free Optimization Methods under Global and Local Lipschitz Continuity of Gradients
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Dat Ba Tran
Abstract:
This paper addresses the study of derivative-free smooth optimization problems, where the gradient information on the objective function is unavailable. Two novel general derivative-free methods are proposed and developed for minimizing such functions with either global or local Lipschitz continuous gradients. The newly developed methods use gradient approximations based on finite differences, whe…
▽ More
This paper addresses the study of derivative-free smooth optimization problems, where the gradient information on the objective function is unavailable. Two novel general derivative-free methods are proposed and developed for minimizing such functions with either global or local Lipschitz continuous gradients. The newly developed methods use gradient approximations based on finite differences, where finite difference intervals are automatically adapted to the magnitude of the exact gradients without knowing them exactly. The suggested algorithms achieve fundamental convergence results, including stationarity of accumulation points in general settings as well as global convergence with constructive convergence rates when the Kurdyka-Łojasiewicz property is imposed. The local convergence of the proposed algorithms to nonisolated local minimizers, along with their local convergence rates, is also analyzed under this property. Numerical experiences involving various convex, nonconvex, noiseless, and noisy functions demonstrate that the new methods exhibit essential advantages over other state-of-the-art methods in derivative-free optimization.
△ Less
Submitted 28 November, 2023;
originally announced November 2023.
-
Optimal control of sweeping processes in unmanned surface vehicle and nanoparticle modeling
Authors:
Boris S. Mordukhovich,
Dao Nguyen,
Trang Nguyen
Abstract:
This paper addresses novel applications to practical modeling of the newly developed theory of necessary optimality conditions in controlled sweeping/Moreau processes with free time and pointwise control and state constraints. Problems of this type appear, in particular, in dynamical models dealing with unmanned surface vehicles (USVs) and nanoparticles. We formulate optimal control problems for a…
▽ More
This paper addresses novel applications to practical modeling of the newly developed theory of necessary optimality conditions in controlled sweeping/Moreau processes with free time and pointwise control and state constraints. Problems of this type appear, in particular, in dynamical models dealing with unmanned surface vehicles (USVs) and nanoparticles. We formulate optimal control problems for a general class of such dynamical systems and show that the developed necessary optimality conditions for constrained free-time controlled sweeping processes lead us to designing efficient procedures to solve practical models of this class. Moreover, the paper contains numerical calculations of optimal solutions to marine USVs and nanoparticle models in specific situations. Overall, this study contributes to the advancement of optimal control theory for constrained sweeping processes and its practical applications in the fields of marine USVs and nanoparticle modeling.
△ Less
Submitted 21 November, 2023;
originally announced November 2023.
-
Discrete Approximations and Optimality Conditions for Controlled Free-Time Sweeping Processes
Authors:
Giovanni Colombo,
Boris S. Mordukhovich,
Dao Nguyen,
Trang Nguyen
Abstract:
The paper is devoted to the study of a new class of optimal control problems governed by discontinuous constrained differential inclusions of the sweeping type with involving the duration of the dynamic process into optimization. We develop a novel version of the method of discrete approximations of its own qualitative and numerical values with establishing its well-posedness and strong convergenc…
▽ More
The paper is devoted to the study of a new class of optimal control problems governed by discontinuous constrained differential inclusions of the sweeping type with involving the duration of the dynamic process into optimization. We develop a novel version of the method of discrete approximations of its own qualitative and numerical values with establishing its well-posedness and strong convergence to optimal solutions of the controlled sweeping process. Using advanced tools of first-order and second-order variational analysis and generalized differentiation allows us to derive new necessary conditions for optimal solutions of the discrete-time problems and then, by passing to the limit in the discretization procedure, for designated local minimizers in the original problem of sweeping optimal control. The obtained results are illustrated by a numerical example.
△ Less
Submitted 16 October, 2023;
originally announced October 2023.
-
Variational and Strong Variational Convexity in Infinite-Dimensional Variational Analysis
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
This paper is devoted to a systematic study and characterizations of the fundamental notions of variational and strong variational convexity for lower semicontinuous functions. While these notions have been quite recently introduced by Rockafellar, the importance of them has been already recognized and documented in finite-dimensional variational analysis and optimization. Here we address general…
▽ More
This paper is devoted to a systematic study and characterizations of the fundamental notions of variational and strong variational convexity for lower semicontinuous functions. While these notions have been quite recently introduced by Rockafellar, the importance of them has been already recognized and documented in finite-dimensional variational analysis and optimization. Here we address general infinite-dimensional settings and derive comprehensive characterizations of both variational and strong variational convexity notions by developing novel techniques, which are essentially different from finite-dimensional counterparts. As a consequence of the obtained characterizations, we establish new quantitative and qualitative relationships between strong variational convexity and tilt stability of local minimizers in appropriate frameworks of Banach spaces.
△ Less
Submitted 24 September, 2023;
originally announced September 2023.
-
Local Maximal Monotonicity in Variational Analysis and Optimization
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
The paper is devoted to a systematic study and characterizations of notions of local maximal monotonicity and their strong counterparts for set-valued operators that appear in variational analysis, optimization, and their applications. We obtain novel resolvent characterizations of these notions together with efficient conditions for their preservation under summation in broad infinite-dimensional…
▽ More
The paper is devoted to a systematic study and characterizations of notions of local maximal monotonicity and their strong counterparts for set-valued operators that appear in variational analysis, optimization, and their applications. We obtain novel resolvent characterizations of these notions together with efficient conditions for their preservation under summation in broad infinite-dimensional settings. Further characterizations of these notions are derived by using generalized differentiation of variational analysis in the framework of Hilbert spaces.
△ Less
Submitted 27 August, 2023;
originally announced August 2023.
-
A New Inexact Gradient Descent Method with Applications to Nonsmooth Convex Optimization
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Dat Ba Tran
Abstract:
The paper proposes and develops a novel inexact gradient method (IGD) for minimizing C1-smooth functions with Lipschitzian gradients, i.e., for problems of C1,1 optimization. We show that the sequence of gradients generated by IGD converges to zero. The convergence of iterates to stationary points is guaranteed under the Kurdyka- Lojasiewicz (KL) property of the objective function with convergence…
▽ More
The paper proposes and develops a novel inexact gradient method (IGD) for minimizing C1-smooth functions with Lipschitzian gradients, i.e., for problems of C1,1 optimization. We show that the sequence of gradients generated by IGD converges to zero. The convergence of iterates to stationary points is guaranteed under the Kurdyka- Lojasiewicz (KL) property of the objective function with convergence rates depending on the KL exponent. The newly developed IGD is applied to designing two novel gradient-based methods of nonsmooth convex optimization such as the inexact proximal point methods (GIPPM) and the inexact augmented Lagrangian method (GIALM) for convex programs with linear equality constraints. These two methods inherit global convergence properties from IGD and are confirmed by numerical experiments to have practical advantages over some well-known algorithms of nonsmooth convex optimization.
△ Less
Submitted 11 January, 2024; v1 submitted 15 March, 2023;
originally announced March 2023.
-
Coderivative-Based Semi-Newton Method in Nonsmooth Difference Programming
Authors:
Francisco J. Aragón-Artacho,
Boris S. Mordukhovich,
Pedro Pérez-Aros
Abstract:
This paper addresses the study of a new class of nonsmooth optimization problems, where the objective is represented as a difference of two generally nonconvex functions. We propose and develop a novel Newton-type algorithm to solving such problems, which is based on the coderivative generated second-order subdifferential (generalized Hessian) and employs advanced tools of variational analysis. We…
▽ More
This paper addresses the study of a new class of nonsmooth optimization problems, where the objective is represented as a difference of two generally nonconvex functions. We propose and develop a novel Newton-type algorithm to solving such problems, which is based on the coderivative generated second-order subdifferential (generalized Hessian) and employs advanced tools of variational analysis. Well-posedness properties of the proposed algorithm are derived under fairly general requirements, while constructive convergence rates are established by using additional assumptions including the Kurdyka--Łojasiewicz condition. We provide applications of the main algorithm to solving a general class of nonsmooth nonconvex problems of structured optimization that encompasses, in particular, optimization problems with explicit constraints. Finally, applications and numerical experiments are given for solving practical problems that arise in biochemical models, constrained quadratic programming, etc., where advantages of our algorithms are demonstrated in comparison with some known techniques and results.
△ Less
Submitted 9 January, 2023;
originally announced January 2023.
-
Variational Convexity of Functions in Banach Spaces
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
This paper addresses the study and characterizations of variational convexity of extended-real-valued functions on Banach spaces. This notion has been recently introduced by Rockafellar, and its importance has been already realized and applied to continuous optimization problems in finite-dimensional spaces. Variational convexity in infinite-dimensional spaces, which is studied here for the first…
▽ More
This paper addresses the study and characterizations of variational convexity of extended-real-valued functions on Banach spaces. This notion has been recently introduced by Rockafellar, and its importance has been already realized and applied to continuous optimization problems in finite-dimensional spaces. Variational convexity in infinite-dimensional spaces, which is studied here for the first time, is significantly more involved and requires the usage of powerful tools of geometric functional analysis together with variational analysis and generalized differentiation in Banach spaces.
△ Less
Submitted 27 August, 2023; v1 submitted 13 December, 2022;
originally announced December 2022.
-
Relative Well-Posedness of Truncated Constrained Systems Accompanied by Variational Calculus
Authors:
Boris S. Mordukhovich,
Pengcheng Wu,
Xiaoqi Yang
Abstract:
The paper concerns foundations of sensitivity and stability analysis in optimization and related areas, being primarily addressed truncated constrained systems. We consider general models, which are described by multifunctions between Banach spaces and concentrate on characterizing their well-posedness properties that revolve around Lipschitz stability and metric regularity relative to sets. Invok…
▽ More
The paper concerns foundations of sensitivity and stability analysis in optimization and related areas, being primarily addressed truncated constrained systems. We consider general models, which are described by multifunctions between Banach spaces and concentrate on characterizing their well-posedness properties that revolve around Lipschitz stability and metric regularity relative to sets. Invoking tools of variational analysis and generalized differentiation, we introduce new robust notions of relative contingent coderivatives. The novel machinery of variational analysis leads us to establishing complete characterizations of such properties and developing basic rules of variational calculus interrelated with the obtained characterizations of well-posedness. Most of the our results valid in general infinite-dimensional settings are also new in finite dimensions.
△ Less
Submitted 29 April, 2025; v1 submitted 5 December, 2022;
originally announced December 2022.
-
Optimal Control of a Quasi-Variational Sweeping Process
Authors:
Harbir Antil,
Rafael Arndt,
Boris S. Mordukhovich,
Dao Nguyen,
Carlos N. Rautenberg
Abstract:
The paper addresses the study of a class of evolutionary quasi-variational inequalities of the parabolic type arising in the formation and growth models of granular and cohensionless materials. Such models and their mathematical descriptions are highly challenging and require powerful tools of their analysis and implementation. We formulate a space-time continuous optimal control problem for a bas…
▽ More
The paper addresses the study of a class of evolutionary quasi-variational inequalities of the parabolic type arising in the formation and growth models of granular and cohensionless materials. Such models and their mathematical descriptions are highly challenging and require powerful tools of their analysis and implementation. We formulate a space-time continuous optimal control problem for a basic model of this type, develop several regularization and approximation procedures, and establish the existence of optimal solutions for the time-continuous and space-discrete problem. Viewing a version of this problem as a controlled quasi-variational sweeping process leads us to deriving necessary optimality conditions for the fully discrete problem by using the advanced machinery of variational analysis and generalized differentiation.
△ Less
Submitted 31 August, 2022;
originally announced September 2022.
-
Variational Convexity of Functions and Variational Sufficiency in Optimization
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
The paper is devoted to the study, characterizations, and applications of variational convexity of functions, the property that has been recently introduced by Rockafellar together with its strong counterpart. First we show that these variational properties of an extended-real-valued function are equivalent to, respectively, the conventional (local) convexity and strong convexity of its Moreau env…
▽ More
The paper is devoted to the study, characterizations, and applications of variational convexity of functions, the property that has been recently introduced by Rockafellar together with its strong counterpart. First we show that these variational properties of an extended-real-valued function are equivalent to, respectively, the conventional (local) convexity and strong convexity of its Moreau envelope. Then we derive new characterizations of both variational convexity and variational strong convexity of general functions via their second-order subdifferentials (generalized Hessians), which are coderivatives of subgradient mappings. We also study relationships of these notions with local minimizers and tilt-stable local minimizers. The obtained results are used for characterizing related notions of variational and strong variational sufficiency in composite optimization with applications to nonlinear programming.
△ Less
Submitted 26 January, 2023; v1 submitted 30 August, 2022;
originally announced August 2022.
-
Inexact reduced gradient methods in nonconvex optimization
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Dat Ba Tran
Abstract:
This paper proposes and develops new linesearch methods with inexact gradient information for finding stationary points of nonconvex continuously differentiable functions on finite-dimensional spaces. Some abstract convergence results for a broad class of linesearch methods are stablished. A general scheme for inexact reduced gradient (IRG) methods is proposed, where the errors in the gradient app…
▽ More
This paper proposes and develops new linesearch methods with inexact gradient information for finding stationary points of nonconvex continuously differentiable functions on finite-dimensional spaces. Some abstract convergence results for a broad class of linesearch methods are stablished. A general scheme for inexact reduced gradient (IRG) methods is proposed, where the errors in the gradient approximation automatically adapt with the magnitudes of the exact gradients. The sequences of iterations are shown to obtain stationary accumulation points when different stepsize selections are employed. Convergence results with constructive convergence rates for the developed IRG methods are established under the Kurdyka- Lojasiewicz property. The obtained results for the IRG methods are confirmed by encouraging numerical experiments, which demonstrate advantages of automatically controlled errors in IRG methods over other frequently used error selections.
△ Less
Submitted 30 December, 2022; v1 submitted 4 April, 2022;
originally announced April 2022.
-
Coincidence Points of Parameterized Generalized Equations with Applications to Optimal Value Functions
Authors:
Aram V. Arutyunov,
Boris S. Mordukhovich,
Sergey E. Zhukovskiy
Abstract:
The paper studies coincidence points of parameterized set-valued mappings (multifunctions), which provide an extended framework to cover several important topics in variational analysis and optimization that include the existence of solutions of parameterized generalized equations, implicit function and fixed-point theorems, optimal value functions in parametric optimization, etc. Using the advanc…
▽ More
The paper studies coincidence points of parameterized set-valued mappings (multifunctions), which provide an extended framework to cover several important topics in variational analysis and optimization that include the existence of solutions of parameterized generalized equations, implicit function and fixed-point theorems, optimal value functions in parametric optimization, etc. Using the advanced machinery of variational analysis and generalized differentiation that furnishes complete characterizations of well-posedness properties of multifunctions, we establish a general theorem ensuring the existence of parameter-dependent coincidence point mappings with explicit error bounds for parameterized multifunctions between infinite-dimensional spaces. The obtained major result yields a new implicit function theorem and allows us to derive efficient conditions for semicontinuity and continuity of optimal value functions associated with parametric minimization problems subject to constraints governed by parameterized generalized equations.
△ Less
Submitted 21 March, 2022;
originally announced March 2022.
-
Sensitivity Analysis of Stochastic Constraint and Variational Systems via Generalized Differentiation
Authors:
Boris S. Mordukhovich,
Pedro Pérez-Aros
Abstract:
This paper conducts sensitivity analysis of random constraint and variational systems related to stochastic optimization and variational inequalities. We establish efficient conditions for well-posedness, in the sense of robust Lipschitzian stability and/or metric regularity, of such systems by employing and developing coderivative characterizations of well-posedness properties for random multifun…
▽ More
This paper conducts sensitivity analysis of random constraint and variational systems related to stochastic optimization and variational inequalities. We establish efficient conditions for well-posedness, in the sense of robust Lipschitzian stability and/or metric regularity, of such systems by employing and developing coderivative characterizations of well-posedness properties for random multifunctions and efficiently evaluating coderivatives of special classes of random integral set-valued mappings that naturally emerge in stochastic programming and stochastic variational inequalities.
△ Less
Submitted 10 December, 2021;
originally announced December 2021.
-
Controlled polyhedral sweeping processes: existence, stability, and optimality conditions
Authors:
René Henrion,
Abderrahim Jourani,
Boris S. Mordukhovich
Abstract:
This paper is mainly devoted to the study of controlled sweeping processes with polyhedral moving sets in Hilbert spaces. Based on a detailed analysis of truncated Hausdorff distances between moving polyhedra, we derive new existence and uniqueness theorems for sweeping trajectories corresponding to various classes of control functions acting in moving sets. Then we establish quantitative stabilit…
▽ More
This paper is mainly devoted to the study of controlled sweeping processes with polyhedral moving sets in Hilbert spaces. Based on a detailed analysis of truncated Hausdorff distances between moving polyhedra, we derive new existence and uniqueness theorems for sweeping trajectories corresponding to various classes of control functions acting in moving sets. Then we establish quantitative stability results, which provide efficient estimates on the sweeping trajectory dependence on controls and initial values. Our final topic, accomplished in finite-dimensional state spaces, is deriving new necessary optimality and suboptimality conditions for sweeping control systems with endpoint constrains by using constructive discrete approximations.
△ Less
Submitted 3 December, 2021;
originally announced December 2021.
-
Optimization of Controlled Free-Time Sweeping Processes with Applications to Marine Surface Vehicle Modeling
Authors:
Tan H. Cao,
Nathalie T. Khalil,
Boris S. Mordukhovich,
Dao Nguyen,
Trang Nguyen,
Fernando Lobo Pereira
Abstract:
The paper is devoted to a free-time optimal control problem for sweeping processes. We develop a constructive finite-difference approximation procedure that allows us to establish necessary optimality conditions for discrete optimal solutions and then show how these optimality conditions are applied to solving a controlled marine surface vehicle model.
The paper is devoted to a free-time optimal control problem for sweeping processes. We develop a constructive finite-difference approximation procedure that allows us to establish necessary optimality conditions for discrete optimal solutions and then show how these optimality conditions are applied to solving a controlled marine surface vehicle model.
△ Less
Submitted 7 May, 2021;
originally announced May 2021.
-
Crowd motion paradigm modeled by a bilevel sweeping control problem
Authors:
Tan H. Cao,
Nathalie T. Khalil,
Boris S. Mordukhovich,
Dao Nguyen,
Fernando Lobo Pereira
Abstract:
This article concerns an optimal crowd motion control problem in which the crowd features a structure given by its organization into N groups (participants) each one spatially confined in a set. The overall optimal control problem consists in driving the ensemble of sets as close as possible to a given point (the 'exit') while the population in each set minimizes its control effort subject to its…
▽ More
This article concerns an optimal crowd motion control problem in which the crowd features a structure given by its organization into N groups (participants) each one spatially confined in a set. The overall optimal control problem consists in driving the ensemble of sets as close as possible to a given point (the 'exit') while the population in each set minimizes its control effort subject to its sweeping dynamics with a controlled state dependent velocity drift. In order to capture the conflict between the goal of the overall population and those of the various groups, the problem is cast as a bilevel optimization framework. A key challenge of this problem consists in bringing together two quite different paradigms: bilevel programming and sweeping dynamics with a controlled drift. Necessary conditions of optimality in the form of a Maximum Principle of Pontryagin in the Gamkrelidze framework are derived. These conditions are then used to solve a simple illustrative example with two participants, emphasizing the interaction between them.
△ Less
Submitted 19 April, 2021;
originally announced April 2021.
-
Optimal control of nonconvex integro-differential sweeping processes
Authors:
Abderrahim Bouach,
Tahar Haddad,
Boris S. Mordukhovich
Abstract:
This paper is devoted to the study, for the first time in the literature, of optimal control problems for sweeping processes governed by integro-differential inclusions of the Volterra type with different classes of control functions acting in nonconvex moving sets, external dynamic perturbations, and integral parts of the sweeping dynamics. We establish the existence of optimal solutions and then…
▽ More
This paper is devoted to the study, for the first time in the literature, of optimal control problems for sweeping processes governed by integro-differential inclusions of the Volterra type with different classes of control functions acting in nonconvex moving sets, external dynamic perturbations, and integral parts of the sweeping dynamics. We establish the existence of optimal solutions and then obtain necessary optimality conditions for a broad class of local minimizers in such problems. Our approach to deriving necessary optimality conditions is based on the method of discrete approximations married to basic constructions and calculus rules of first-order and second-order variational analysis and generalized differentiation. The obtained necessary optimality conditions are expressed entirely in terms of the problem data and are illustrated by nontrivial examples that include applications to optimal control models of non-regular electrical circuits.
△ Less
Submitted 28 March, 2021;
originally announced March 2021.
-
Vector Optimization with Domination Structures: Variational Principles and Applications
Authors:
Truong Q. Bao,
Boris S. Mordukhovich,
Antoine Soubeyran,
Christiane Tammer
Abstract:
This paper addresses a large class of vector optimization problems in infinite-dimensional spaces with respect to two important binary relations derived from domination structures. Motivated by theoretical challenges as well as by applications to some models in behavioral sciences, we establish new variational principles that can be viewed as far-going extensions of the Ekeland variational princip…
▽ More
This paper addresses a large class of vector optimization problems in infinite-dimensional spaces with respect to two important binary relations derived from domination structures. Motivated by theoretical challenges as well as by applications to some models in behavioral sciences, we establish new variational principles that can be viewed as far-going extensions of the Ekeland variational principle to cover domination vector settings. Our approach combines advantages of both primal and dual techniques in variational analysis with providing useful suficient conditions for the existence of variational traps in behavioral science models with variable domination structures.
△ Less
Submitted 16 February, 2021;
originally announced February 2021.
-
Generalized Leibniz rules and Lipschitzian stability for expected-integral mappings
Authors:
Boris S. Mordukhovich,
Pedro Pérez-Aros
Abstract:
This paper is devoted to the study of the expected-integral multifunctions given in the form \begin{equation*} \operatorname{E}_Φ(x):=\int_TΦ_t(x)dμ, \end{equation*} where $Φ\colon T\times\mathbb{R}^n \rightrightarrows \mathbb{R}^m$ is a set-valued mapping on a measure space $(T,\mathcal{A},μ)$. Such multifunctions appear in applications to stochastic programming, which require developing efficien…
▽ More
This paper is devoted to the study of the expected-integral multifunctions given in the form \begin{equation*} \operatorname{E}_Φ(x):=\int_TΦ_t(x)dμ, \end{equation*} where $Φ\colon T\times\mathbb{R}^n \rightrightarrows \mathbb{R}^m$ is a set-valued mapping on a measure space $(T,\mathcal{A},μ)$. Such multifunctions appear in applications to stochastic programming, which require developing efficient calculus rules of generalized differentiation. Major calculus rules are developed in this paper for coderivatives of multifunctions $\operatorname{E}_Φ$ and second-order subdifferentials of the corresponding expected-integral functionals with applications to constraint systems arising in stochastic programming. The paper is self-contained with presenting in the preliminaries some needed results on sequential first-order subdifferential calculus of expected-integral functionals taken from the first paper of this series.
△ Less
Submitted 18 June, 2021; v1 submitted 17 January, 2021;
originally announced January 2021.
-
Optimization of Fully Controlled Sweeping Processes
Authors:
Tan H. Cao,
Giovanni Colombo,
Boris S. Mordukhovich,
Dao Nguyen
Abstract:
The paper is devoted to deriving necessary optimality conditions in a general optimal control problem for dynamical systems governed by controlled sweeping processes with hard-constrained control actions entering both polyhedral moving sets and additive perturbations. By using the first-order and mainly second-order tools of variational analysis and generalized differentiation, we develop a well-p…
▽ More
The paper is devoted to deriving necessary optimality conditions in a general optimal control problem for dynamical systems governed by controlled sweeping processes with hard-constrained control actions entering both polyhedral moving sets and additive perturbations. By using the first-order and mainly second-order tools of variational analysis and generalized differentiation, we develop a well-posed method of discrete approximations, obtain optimality conditions for solutions to discrete-time control systems, and then establish by passing to the limit verifiable necessary optimality conditions for local minimizers of the original controlled sweeping process that are expressed entirely in terms of its given data. The efficiency of the obtained necessary optimality conditions for the sweeping dynamics is illustrated by solving three nontrivial examples of their own interest.
△ Less
Submitted 16 March, 2021; v1 submitted 21 December, 2020;
originally announced December 2020.
-
A Globally Convergent Proximal Newton-Type Method in Nonsmooth Convex Optimization
Authors:
Boris S. Mordukhovich,
Xiaoming Yuan,
Shangzhi Zeng,
Jin Zhang
Abstract:
The paper proposes and justifies a new algorithm of the proximal Newton type to solve a broad class of nonsmooth composite convex optimization problems without strong convexity assumptions. Based on advanced notions and techniques of variational analysis, we establish implementable results on the global convergence of the proposed algorithm as well as its local convergence with superlinear and qua…
▽ More
The paper proposes and justifies a new algorithm of the proximal Newton type to solve a broad class of nonsmooth composite convex optimization problems without strong convexity assumptions. Based on advanced notions and techniques of variational analysis, we establish implementable results on the global convergence of the proposed algorithm as well as its local convergence with superlinear and quadratic rates. For certain structured problems, the obtained local convergence conditions do not require the local Lipschitz continuity of the corresponding Hessian mappings that is a crucial assumption used in the literature to ensure a superlinear convergence of other algorithms of the proximal Newton type. The conducted numerical experiments of solving the $l_1$ regularized logistic regression model illustrate the possibility of applying the proposed algorithm to deal with practically important problems.
△ Less
Submitted 28 February, 2022; v1 submitted 16 November, 2020;
originally announced November 2020.
-
Generalized Sequential Differential Calculus for Expected-Integral Functionals
Authors:
Boris S. Mordukhovich,
Pedro Pérez-Aros
Abstract:
Motivated by applications to stochastic programming, we introduce and study the expected-integral functionals, which are mappings given in an integral form depending on two variables, the first a finite dimensional decision vector and the second one an integrable function. The main goal of this paper is to establish sequential versions of Leibniz's rule for regular subgradients by employing and de…
▽ More
Motivated by applications to stochastic programming, we introduce and study the expected-integral functionals, which are mappings given in an integral form depending on two variables, the first a finite dimensional decision vector and the second one an integrable function. The main goal of this paper is to establish sequential versions of Leibniz's rule for regular subgradients by employing and developing appropriate tools of variational analysis.
△ Less
Submitted 13 June, 2021; v1 submitted 25 September, 2020;
originally announced September 2020.
-
Augmented Lagrangian Method for Second-Order Cone Programs under Second-Order Sufficiency
Authors:
Nguyen T. V. Hang,
Boris S. Mordukhovich,
M. Ebrahim Sarabi
Abstract:
This paper addresses problems of second-order cone programming important in optimization theory and applications. The main attention is paid to the augmented Lagrangian method (ALM) for such problems considered in both exact and inexact forms. Using generalized differential tools of second-order variational analysis, we formulate the corresponding version of second-order sufficiency and use it to…
▽ More
This paper addresses problems of second-order cone programming important in optimization theory and applications. The main attention is paid to the augmented Lagrangian method (ALM) for such problems considered in both exact and inexact forms. Using generalized differential tools of second-order variational analysis, we formulate the corresponding version of second-order sufficiency and use it to establish, among other results, the uniform second-order growth condition for the augmented Lagrangian. The latter allows us to justify the solvability of subproblems in the ALM and to prove the linear primal-dual convergence of this method.
△ Less
Submitted 5 July, 2021; v1 submitted 8 May, 2020;
originally announced May 2020.
-
Discrete Approximations and Optimal Control of Nonsmooth Perturbed Sweeping Processes
Authors:
Boris S. Mordukhovich,
Dao Nguyen
Abstract:
The main goal of this paper is developing the method of discrete approximations to derive necessary optimality conditions for a class of constrained sweeping processes with nonsmooth perturbations. Optimal control problems for sweeping processes have been recently recognized among the most interesting and challenging problems in modern control theory for discontinuous differential inclusions with…
▽ More
The main goal of this paper is developing the method of discrete approximations to derive necessary optimality conditions for a class of constrained sweeping processes with nonsmooth perturbations. Optimal control problems for sweeping processes have been recently recognized among the most interesting and challenging problems in modern control theory for discontinuous differential inclusions with irregular dynamics and implicit state constrained, while deriving necessary optimality conditions for their local minimizers have been significantly based on the smoothness of controlled dynamic perturbations. To overcome these difficulties, we use the method of discrete approximations and employ advanced tools of second-order variational analysis. This approach allows us to obtain new necessary optimality conditions for nonsmooth and nonconvex discrete-time problems of the sweeping type. Then we employ the obtained conditions and the strong convergence of discrete approximations to establish novel results for original nonsmooth sweeping control problems that include extended Euler-Lagrange and maximization conditions for local minimizers. Finally, we present applications of the obtained results to solving a controlled mobile robot model with a nonsmooth sweeping dynamics that is of some practical interest.
△ Less
Submitted 12 May, 2020; v1 submitted 28 April, 2020;
originally announced April 2020.
-
Optimization and Discrete Approximation of Sweeping Processes with Controlled Moving Sets and Perturbations
Authors:
Tan H. Cao,
Giovanni Colombo,
Boris S. Mordukhovich,
Dao Nguyen
Abstract:
This paper addresses a new class of optimal control problems for perturbed sweeping processes with measurable controls in additive perturbations of the dynamics and smooth controls in polyhedral moving sets. We develop a constructive discrete approximation procedure that allows us to strongly approximate any feasible trajectory of the controlled sweeping process by feasible discrete trajectories a…
▽ More
This paper addresses a new class of optimal control problems for perturbed sweeping processes with measurable controls in additive perturbations of the dynamics and smooth controls in polyhedral moving sets. We develop a constructive discrete approximation procedure that allows us to strongly approximate any feasible trajectory of the controlled sweeping process by feasible discrete trajectories and also establish a $W^{1,2}$-strong convergence of optimal trajectories for discretized control problems to a given local minimizer of the original continuous-time sweeping control problem of the Bolza type. Employing advanced tools of first-order and second-order variational analysis and generalized differentiation, we derive necessary optimality conditions for discrete optimal solutions under fairly general assumptions formulated entirely in terms of the given data. The obtained results give us efficient suboptimality ("almost optimality") conditions for the original sweeping control problem that are illustrated by a nontrivial numerical example.
△ Less
Submitted 13 February, 2020;
originally announced February 2020.
-
Feedback Stabilization of Nonlinear Control Systems by Composition Operators
Authors:
Bryce A. Christopherson,
Boris S. Mordukhovich,
Farhad Jafari
Abstract:
Feedback asymptotic stabilization of control systems is an important topic of control theory and applications. Broadly speaking, if the system $\dot{x} = f(x,u)$ is locally asymptotically stabilizable, then there exists a feedback control $u(x)$ ensuring the convergence to an equilibrium for any trajectory starting from a point sufficiently close to the equilibrium state. In this paper, we develop…
▽ More
Feedback asymptotic stabilization of control systems is an important topic of control theory and applications. Broadly speaking, if the system $\dot{x} = f(x,u)$ is locally asymptotically stabilizable, then there exists a feedback control $u(x)$ ensuring the convergence to an equilibrium for any trajectory starting from a point sufficiently close to the equilibrium state. In this paper, we develop a reasonably natural and general composition operator approach to stabilizability. To begin with, we provide an extension of the classical Hautus lemma to the generalized context of composition operators and show that Brockett's theorem is still necessary for local asymptotic stabilizability in this generalized framework. Further, we employ a powerful version of the implicit function theorem--as given by Jittorntrum and Kumagai--to cover stabilization without differentiability requirements in this expanded context. Employing the obtained characterizations, we establish relationships between stabilizability in the conventional sense and in the generalized composition operator sense. This connection allows us to show that the stabilizability of a control system is equivalent to the stability of an associated system. That is, we reduce the question of stabilizability to that of stability.
△ Less
Submitted 18 September, 2021; v1 submitted 23 January, 2020;
originally announced January 2020.
-
Algebraic Core and Convex Calculus without Topology
Authors:
Dang Van Cuong,
Boris S. Mordukhovich,
Nguyen Mau Nam,
Addison Cartmell
Abstract:
In this paper we study the concept of algebraic core for convex sets in general vector spaces without any topological structure and then present its applications to problems of convex analysis and optimization. Deriving the equivalence between the Hahn-Banach theorem and and a simple version of the separation theorem of convex sets in vector spaces allows us to develop a geometric approach to gene…
▽ More
In this paper we study the concept of algebraic core for convex sets in general vector spaces without any topological structure and then present its applications to problems of convex analysis and optimization. Deriving the equivalence between the Hahn-Banach theorem and and a simple version of the separation theorem of convex sets in vector spaces allows us to develop a geometric approach to generalized differential calculus for convex sets, set-valued mappings, and extended-real-valued functions with qualification conditions formulated in terms of algebraic cores for such objects. We also obtain a precise formula for computing the subdifferential of optimal value functions associated with convex problems of parametric optimization in vector spaces. Functions of this type play a crucial role in many aspects of convex optimization and its applications.
△ Less
Submitted 5 February, 2020; v1 submitted 1 January, 2020;
originally announced January 2020.
-
New extremal principles with applications to stochastic and semi-infinite programming
Authors:
Boris S. Mordukhovich,
Pedro Pérez-Aros
Abstract:
This paper develops new extremal principles of variational analysis that are motivated by applications to constrained problems of stochastic programming and semi-infinite programming without smoothness and/or convexity assumptions. These extremal principles concern measurable set-valued mappings/multifunctions with values in finite-dimensional spaces and are established in both approximate and exa…
▽ More
This paper develops new extremal principles of variational analysis that are motivated by applications to constrained problems of stochastic programming and semi-infinite programming without smoothness and/or convexity assumptions. These extremal principles concern measurable set-valued mappings/multifunctions with values in finite-dimensional spaces and are established in both approximate and exact forms. The obtained principles are instrumental to derive via variational approaches integral representations and upper estimates of regular and limiting normals cones to essential intersections of sets defined by measurable multifunctions, which are in turn crucial for novel applications to stochastic and semi-infinite programming.
△ Less
Submitted 22 July, 2020; v1 submitted 25 November, 2019;
originally announced November 2019.
-
Subdifferentials of Value Functions in Nonconvex Dynamic Programming for Nonstationary Stochastic Processes
Authors:
Boris S. Mordukhovich,
Nobusumi Sagara
Abstract:
The main goal of this paper is to apply the machinery of variational analysis and generalized differentiation to study infinite horizon stochastic dynamic programming (DP) with discrete time in the Banach space setting without convexity assumptions. Unlike to standard stochastic DP with stationary Markov processes, we investigate here stochastic DP in $L^p$ spaces to deal with nonstationary stocha…
▽ More
The main goal of this paper is to apply the machinery of variational analysis and generalized differentiation to study infinite horizon stochastic dynamic programming (DP) with discrete time in the Banach space setting without convexity assumptions. Unlike to standard stochastic DP with stationary Markov processes, we investigate here stochastic DP in $L^p$ spaces to deal with nonstationary stochastic processes, which describe a more flexible learning procedure for the decision-maker. Our main concern is to calculate generalized subgradients of the corresponding value function and to derive necessary conditions for optimality in terms of the stochastic Euler inclusion under appropriate Lipschitzian assumptions. The usage of the subdifferential formula for integral functionals on $L^p$ spaces allows us, in particular, to find verifiable conditions to ensure smoothness of the value function without any convexity and/or interiority assumptions.
△ Less
Submitted 1 September, 2019;
originally announced September 2019.
-
Parabolic Regularity in Geometric Variational Analysis
Authors:
Ashkan Mohammadi,
Boris S. Mordukhovich,
M. Ebrahim Sarabi
Abstract:
The paper is mainly devoted to systematic developments and applications of geometric aspects of second-order variational analysis that are revolved around the concept of parabolic regularity of sets. This concept has been known in variational analysis for more than two decades while being largely underinvestigated. We discover here that parabolic regularity is the key to derive new calculus rules…
▽ More
The paper is mainly devoted to systematic developments and applications of geometric aspects of second-order variational analysis that are revolved around the concept of parabolic regularity of sets. This concept has been known in variational analysis for more than two decades while being largely underinvestigated. We discover here that parabolic regularity is the key to derive new calculus rules and computation formulas for major second-order generalized differential constructions of variational analysis in connection with some properties of sets that go back to classical differential geometry and geometric measure theory. The established results of second-order variational analysis and generalized differentiation, being married to the developed calculus of parabolic regularity, allow us to obtain novel applications to both qualitative and quantitative/numerical aspects of constrained optimization including second-order optimality conditions, augmented Lagrangians, etc. under weak constraint qualifications.
△ Less
Submitted 16 June, 2020; v1 submitted 31 August, 2019;
originally announced September 2019.
