-
Convergence of First-Order Algorithms with Momentum from the Perspective of an Inexact Gradient Descent Method
Authors:
Pham Duy Khanh,
Boris Mordukhovich,
Dat Ba Tran
Abstract:
This paper introduces a novel inexact gradient descent method with momentum (IGDm) considered as a general framework for various first-order methods with momentum. This includes, in particular, the inexact proximal point method (IPPm), extragradient method (EGm), and sharpness-aware minimization (SAMm). Asymptotic convergence properies of IGDm are established under both global and local assumption…
▽ More
This paper introduces a novel inexact gradient descent method with momentum (IGDm) considered as a general framework for various first-order methods with momentum. This includes, in particular, the inexact proximal point method (IPPm), extragradient method (EGm), and sharpness-aware minimization (SAMm). Asymptotic convergence properies of IGDm are established under both global and local assumptions on objective functions with providing constructive convergence rates depending on the Polyak-Łojasiewicz-Kurdyka (PLK) conditions for the objective function. Global convergence of EGm and SAMm for general smooth functions and of IPPM for weakly convex functions is derived in this way. Moreover, local convergence properties of EGm and SAMm for locally smooth functions as well as of IPPm for prox-regular functions are established. Numerical experiments for derivative-free optimization problems are conducted to confirm the efficiency of the momentum effects of the developed methods under inexactness of gradient computations
△ Less
Submitted 5 May, 2025;
originally announced May 2025.
-
Characterizations of Variational Convexity and Tilt Stability via Quadratic Bundles
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Vo Thanh Phat,
Le Duc Viet
Abstract:
In this paper, we establish characterizations of variational $s$-convexity and tilt stability for prox-regular functions in the absence of subdifferential continuity via quadratic bundles, a kind of primal-dual generalized second-order derivatives recently introduced by Rockafellar. Deriving such characterizations in the effective pointbased form requires a certain revision of quadratic bundles in…
▽ More
In this paper, we establish characterizations of variational $s$-convexity and tilt stability for prox-regular functions in the absence of subdifferential continuity via quadratic bundles, a kind of primal-dual generalized second-order derivatives recently introduced by Rockafellar. Deriving such characterizations in the effective pointbased form requires a certain revision of quadratic bundles investigated below. Our device is based on the notion of generalized twice differentiability and its novel characterization via classical twice differentiability of the associated Moreau envelopes combined with various limiting procedures for functions and sets.
△ Less
Submitted 8 January, 2025;
originally announced January 2025.
-
Generalized Twice Differentiability and Quadratic Bundles in Second-Order Variational Analysis
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Vo Thanh Phat,
Le Duc Viet
Abstract:
In this paper, we investigate the concepts of generalized twice differentiability and quadratic bundles of nonsmooth functions that have been very recently proposed by Rockafellar in the framework of second-order variational analysis. These constructions, in contrast to second-order subdifferentials, are defined in primal spaces. We develop new techniques to study generalized twice differentiabili…
▽ More
In this paper, we investigate the concepts of generalized twice differentiability and quadratic bundles of nonsmooth functions that have been very recently proposed by Rockafellar in the framework of second-order variational analysis. These constructions, in contrast to second-order subdifferentials, are defined in primal spaces. We develop new techniques to study generalized twice differentiability for a broad class of prox-regular functions, establish their novel characterizations. Subsequently, quadratic bundles of prox-regular functions are shown to be nonempty, which provides the ground of potential applications in variational analysis and optimization.
△ Less
Submitted 3 January, 2025;
originally announced January 2025.
-
Lipschitz Modulus of Convex Functions via Function Values
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Vo Thanh Phat,
Le Duc Viet
Abstract:
In this note, we establish the Lipschitz continuity of finite-dimensional globally convex functions on all given balls and global Lipschitz continuity for eligible functions of that type. The Lipschitz constants in both situations draw information solely from function values, and the global Lipschitz modulus is found when it exists. Some examples of classes of globally Lipschitz continuous convex…
▽ More
In this note, we establish the Lipschitz continuity of finite-dimensional globally convex functions on all given balls and global Lipschitz continuity for eligible functions of that type. The Lipschitz constants in both situations draw information solely from function values, and the global Lipschitz modulus is found when it exists. Some examples of classes of globally Lipschitz continuous convex functions beside the norms are also provided along with their global Lipschitz modulus.
△ Less
Submitted 1 August, 2024;
originally announced August 2024.
-
Relationships between Global and Local Monotonicity of Operators
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Juan Enrique Martínez-Legaz,
Boris S. Mordukhovich
Abstract:
The paper is devoted to establishing relationships between global and local monotonicity, as well as their maximality versions, for single-valued and set-valued mappings between finite-dimensional and infinite-dimensional spaces. We first show that for single-valued operators with convex domains in locally convex topological spaces, their continuity ensures that their global monotonicity agrees wi…
▽ More
The paper is devoted to establishing relationships between global and local monotonicity, as well as their maximality versions, for single-valued and set-valued mappings between finite-dimensional and infinite-dimensional spaces. We first show that for single-valued operators with convex domains in locally convex topological spaces, their continuity ensures that their global monotonicity agrees with the local one around any point of the graph. This also holds for set-valued mappings defined on the real line under a certain connectedness condition. The situation is different for set-valued operators in multidimensional spaces as demonstrated by an example of locally monotone operator on the plane that is not globally monotone. Finally, we invoke coderivative criteria from variational analysis to characterize both global and local maximal monotonicity of set-valued operators in Hilbert spaces to verify the equivalence between these monotonicity properties under the closed-graph and global hypomonotonicity assumptions.
△ Less
Submitted 29 March, 2024;
originally announced March 2024.
-
Coderivative-Based Newton Methods in Structured Nonconvex and Nonsmooth Optimization
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
This paper proposes and develops new Newton-type methods to solve structured nonconvex and nonsmooth optimization problems with justifying their fast local and global convergence by means of advanced tools of variational analysis and generalized differentiation. The objective functions belong to a broad class of prox-regular functions with specification to constrained optimization of nonconvex str…
▽ More
This paper proposes and develops new Newton-type methods to solve structured nonconvex and nonsmooth optimization problems with justifying their fast local and global convergence by means of advanced tools of variational analysis and generalized differentiation. The objective functions belong to a broad class of prox-regular functions with specification to constrained optimization of nonconvex structured sums. We also develop a novel line search method, which is an extension of the proximal gradient algorithm while allowing us to globalize the proposed coderivative-based Newton methods by incorporating the machinery of forward-backward envelopes. Applications and numerical experiments, which are provided for nonconvex least squares regression models, Student's t-regression, and image restoration problems, demonstrate the efficiency of the proposed algorithms
△ Less
Submitted 20 May, 2025; v1 submitted 7 March, 2024;
originally announced March 2024.
-
Fundamental Convergence Analysis of Sharpness-Aware Minimization
Authors:
Pham Duy Khanh,
Hoang-Chau Luong,
Boris S. Mordukhovich,
Dat Ba Tran
Abstract:
The paper investigates the fundamental convergence properties of Sharpness-Aware Minimization (SAM), a recently proposed gradient-based optimization method [Foret et al., 2021] that significantly improves the generalization of deep neural networks. The convergence properties, including the stationarity of accumulation points, the convergence of the sequence of gradients to the origin, the sequence…
▽ More
The paper investigates the fundamental convergence properties of Sharpness-Aware Minimization (SAM), a recently proposed gradient-based optimization method [Foret et al., 2021] that significantly improves the generalization of deep neural networks. The convergence properties, including the stationarity of accumulation points, the convergence of the sequence of gradients to the origin, the sequence of function values to the optimal value, and the sequence of iterates to the optimal solution, are established for the method. The universality of the provided convergence analysis, based on inexact gradient descent frameworks Khanh et al. [2023b], allows its extensions to efficient normalized versions of SAM such as F-SAM [Li et al., 2024], VaSSO [Li and Giannakis, 2023], RSAM [Liu et al., 2022], and to the unnormalized versions of SAM such as USAM [Andriushchenko and Flammarion, 2022]. Numerical experiments are conducted on classification tasks using deep learning models to confirm the practical aspects of our analysis.
△ Less
Submitted 19 October, 2024; v1 submitted 15 January, 2024;
originally announced January 2024.
-
Second-Order Subdifferential Optimality Conditions in Nonsmooth Optimization
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
The paper is devoted to deriving novel second-order necessary and sufficient optimality conditions for local minimizers in rather general classes of nonsmooth unconstrained and constrained optimization problems in finite-dimensional spaces. The established conditions are expressed in terms of second-order subdifferentials of lower semicontinuous functions and mainly concern prox-regular objectives…
▽ More
The paper is devoted to deriving novel second-order necessary and sufficient optimality conditions for local minimizers in rather general classes of nonsmooth unconstrained and constrained optimization problems in finite-dimensional spaces. The established conditions are expressed in terms of second-order subdifferentials of lower semicontinuous functions and mainly concern prox-regular objectives that cover a large territory in nonsmooth optimization and its applications. Our tools are based on the machinery of variational analysis and second-order generalized differentiation. The obtained general results are applied to problems of nonlinear programming, where the derived second-order optimality conditions are new even for problems with twice continuously differential data, being expressed there in terms of the classical Hessian matrices.
△ Less
Submitted 4 January, 2025; v1 submitted 26 December, 2023;
originally announced December 2023.
-
Local Minimizers of Nonconvex Functions in Banach Spaces via Moreau Envelopes
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
This paper investigates the preservation of local minimizers and strong minimizers of extended-real-valued lower semicontinuous functions under taking their Moreau envelopes. We address a general setting of Banach spaces, while all the obtained results are new even for functions in finite dimensions. Our main motivation came from applications to numerical methods in nonsmooth optimization dealing…
▽ More
This paper investigates the preservation of local minimizers and strong minimizers of extended-real-valued lower semicontinuous functions under taking their Moreau envelopes. We address a general setting of Banach spaces, while all the obtained results are new even for functions in finite dimensions. Our main motivation came from applications to numerical methods in nonsmooth optimization dealing with broad classes of nondifferentiable and nonconvex functions. The paper also formulates and discusses some open questions stemming from this study.
△ Less
Submitted 30 November, 2023;
originally announced November 2023.
-
General Derivative-Free Optimization Methods under Global and Local Lipschitz Continuity of Gradients
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Dat Ba Tran
Abstract:
This paper addresses the study of derivative-free smooth optimization problems, where the gradient information on the objective function is unavailable. Two novel general derivative-free methods are proposed and developed for minimizing such functions with either global or local Lipschitz continuous gradients. The newly developed methods use gradient approximations based on finite differences, whe…
▽ More
This paper addresses the study of derivative-free smooth optimization problems, where the gradient information on the objective function is unavailable. Two novel general derivative-free methods are proposed and developed for minimizing such functions with either global or local Lipschitz continuous gradients. The newly developed methods use gradient approximations based on finite differences, where finite difference intervals are automatically adapted to the magnitude of the exact gradients without knowing them exactly. The suggested algorithms achieve fundamental convergence results, including stationarity of accumulation points in general settings as well as global convergence with constructive convergence rates when the Kurdyka-Łojasiewicz property is imposed. The local convergence of the proposed algorithms to nonisolated local minimizers, along with their local convergence rates, is also analyzed under this property. Numerical experiences involving various convex, nonconvex, noiseless, and noisy functions demonstrate that the new methods exhibit essential advantages over other state-of-the-art methods in derivative-free optimization.
△ Less
Submitted 28 November, 2023;
originally announced November 2023.
-
Variational and Strong Variational Convexity in Infinite-Dimensional Variational Analysis
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
This paper is devoted to a systematic study and characterizations of the fundamental notions of variational and strong variational convexity for lower semicontinuous functions. While these notions have been quite recently introduced by Rockafellar, the importance of them has been already recognized and documented in finite-dimensional variational analysis and optimization. Here we address general…
▽ More
This paper is devoted to a systematic study and characterizations of the fundamental notions of variational and strong variational convexity for lower semicontinuous functions. While these notions have been quite recently introduced by Rockafellar, the importance of them has been already recognized and documented in finite-dimensional variational analysis and optimization. Here we address general infinite-dimensional settings and derive comprehensive characterizations of both variational and strong variational convexity notions by developing novel techniques, which are essentially different from finite-dimensional counterparts. As a consequence of the obtained characterizations, we establish new quantitative and qualitative relationships between strong variational convexity and tilt stability of local minimizers in appropriate frameworks of Banach spaces.
△ Less
Submitted 24 September, 2023;
originally announced September 2023.
-
Local Maximal Monotonicity in Variational Analysis and Optimization
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
The paper is devoted to a systematic study and characterizations of notions of local maximal monotonicity and their strong counterparts for set-valued operators that appear in variational analysis, optimization, and their applications. We obtain novel resolvent characterizations of these notions together with efficient conditions for their preservation under summation in broad infinite-dimensional…
▽ More
The paper is devoted to a systematic study and characterizations of notions of local maximal monotonicity and their strong counterparts for set-valued operators that appear in variational analysis, optimization, and their applications. We obtain novel resolvent characterizations of these notions together with efficient conditions for their preservation under summation in broad infinite-dimensional settings. Further characterizations of these notions are derived by using generalized differentiation of variational analysis in the framework of Hilbert spaces.
△ Less
Submitted 27 August, 2023;
originally announced August 2023.
-
Inexact proximal methods for weakly convex functions
Authors:
Pham Duy Khanh,
Boris Mordukhovich,
Vo Thanh Phat,
Dat Ba Tran
Abstract:
This paper proposes and develops inexact proximal methods for finding stationary points of the sum of a smooth function and a nonsmooth weakly convex one, where an error is present in the calculation of the proximal mapping of the nonsmooth term. A general framework for finding zeros of a continuous mapping is derived from our previous paper on this subject to establish convergence properties of t…
▽ More
This paper proposes and develops inexact proximal methods for finding stationary points of the sum of a smooth function and a nonsmooth weakly convex one, where an error is present in the calculation of the proximal mapping of the nonsmooth term. A general framework for finding zeros of a continuous mapping is derived from our previous paper on this subject to establish convergence properties of the inexact proximal point method when the smooth term is vanished and of the inexact proximal gradient method when the smooth term satisfies a descent condition. The inexact proximal point method achieves global convergence with constructive convergence rates when the Moreau envelope of the objective function satisfies the Kurdyka-Lojasiewicz (KL) property. Meanwhile, when the smooth term is twice continuously differentiable with a Lipschitz continuous gradient and a differentiable approximation of the objective function satisfies the KL property, the inexact proximal gradient method achieves the global convergence of iterates with constructive convergence rates.
△ Less
Submitted 4 August, 2023; v1 submitted 28 July, 2023;
originally announced July 2023.
-
A New Inexact Gradient Descent Method with Applications to Nonsmooth Convex Optimization
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Dat Ba Tran
Abstract:
The paper proposes and develops a novel inexact gradient method (IGD) for minimizing C1-smooth functions with Lipschitzian gradients, i.e., for problems of C1,1 optimization. We show that the sequence of gradients generated by IGD converges to zero. The convergence of iterates to stationary points is guaranteed under the Kurdyka- Lojasiewicz (KL) property of the objective function with convergence…
▽ More
The paper proposes and develops a novel inexact gradient method (IGD) for minimizing C1-smooth functions with Lipschitzian gradients, i.e., for problems of C1,1 optimization. We show that the sequence of gradients generated by IGD converges to zero. The convergence of iterates to stationary points is guaranteed under the Kurdyka- Lojasiewicz (KL) property of the objective function with convergence rates depending on the KL exponent. The newly developed IGD is applied to designing two novel gradient-based methods of nonsmooth convex optimization such as the inexact proximal point methods (GIPPM) and the inexact augmented Lagrangian method (GIALM) for convex programs with linear equality constraints. These two methods inherit global convergence properties from IGD and are confirmed by numerical experiments to have practical advantages over some well-known algorithms of nonsmooth convex optimization.
△ Less
Submitted 11 January, 2024; v1 submitted 15 March, 2023;
originally announced March 2023.
-
Variational Convexity of Functions in Banach Spaces
Authors:
Pham Duy Khanh,
Vu Vinh Huy Khoa,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
This paper addresses the study and characterizations of variational convexity of extended-real-valued functions on Banach spaces. This notion has been recently introduced by Rockafellar, and its importance has been already realized and applied to continuous optimization problems in finite-dimensional spaces. Variational convexity in infinite-dimensional spaces, which is studied here for the first…
▽ More
This paper addresses the study and characterizations of variational convexity of extended-real-valued functions on Banach spaces. This notion has been recently introduced by Rockafellar, and its importance has been already realized and applied to continuous optimization problems in finite-dimensional spaces. Variational convexity in infinite-dimensional spaces, which is studied here for the first time, is significantly more involved and requires the usage of powerful tools of geometric functional analysis together with variational analysis and generalized differentiation in Banach spaces.
△ Less
Submitted 27 August, 2023; v1 submitted 13 December, 2022;
originally announced December 2022.
-
Variational Convexity of Functions and Variational Sufficiency in Optimization
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Vo Thanh Phat
Abstract:
The paper is devoted to the study, characterizations, and applications of variational convexity of functions, the property that has been recently introduced by Rockafellar together with its strong counterpart. First we show that these variational properties of an extended-real-valued function are equivalent to, respectively, the conventional (local) convexity and strong convexity of its Moreau env…
▽ More
The paper is devoted to the study, characterizations, and applications of variational convexity of functions, the property that has been recently introduced by Rockafellar together with its strong counterpart. First we show that these variational properties of an extended-real-valued function are equivalent to, respectively, the conventional (local) convexity and strong convexity of its Moreau envelope. Then we derive new characterizations of both variational convexity and variational strong convexity of general functions via their second-order subdifferentials (generalized Hessians), which are coderivatives of subgradient mappings. We also study relationships of these notions with local minimizers and tilt-stable local minimizers. The obtained results are used for characterizing related notions of variational and strong variational sufficiency in composite optimization with applications to nonlinear programming.
△ Less
Submitted 26 January, 2023; v1 submitted 30 August, 2022;
originally announced August 2022.
-
Inexact reduced gradient methods in nonconvex optimization
Authors:
Pham Duy Khanh,
Boris S. Mordukhovich,
Dat Ba Tran
Abstract:
This paper proposes and develops new linesearch methods with inexact gradient information for finding stationary points of nonconvex continuously differentiable functions on finite-dimensional spaces. Some abstract convergence results for a broad class of linesearch methods are stablished. A general scheme for inexact reduced gradient (IRG) methods is proposed, where the errors in the gradient app…
▽ More
This paper proposes and develops new linesearch methods with inexact gradient information for finding stationary points of nonconvex continuously differentiable functions on finite-dimensional spaces. Some abstract convergence results for a broad class of linesearch methods are stablished. A general scheme for inexact reduced gradient (IRG) methods is proposed, where the errors in the gradient approximation automatically adapt with the magnitudes of the exact gradients. The sequences of iterations are shown to obtain stationary accumulation points when different stepsize selections are employed. Convergence results with constructive convergence rates for the developed IRG methods are established under the Kurdyka- Lojasiewicz property. The obtained results for the IRG methods are confirmed by encouraging numerical experiments, which demonstrate advantages of automatically controlled errors in IRG methods over other frequently used error selections.
△ Less
Submitted 30 December, 2022; v1 submitted 4 April, 2022;
originally announced April 2022.
-
Globally Convergent Coderivative-Based Generalized Newton Methods in Nonsmooth Optimization
Authors:
Pham Duy Khanh,
Boris Mordukhovich,
Vo Thanh Phat,
Dat Ba Tran
Abstract:
This paper proposes and justifies two globally convergent Newton-type methods to solve unconstrained and constrained problems of nonsmooth optimization by using tools of variational analysis and generalized differentiation. Both methods are coderivative-based and employ generalized Hessians (coderivatives of subgradient mappings) associated with objective functions, which are either of class…
▽ More
This paper proposes and justifies two globally convergent Newton-type methods to solve unconstrained and constrained problems of nonsmooth optimization by using tools of variational analysis and generalized differentiation. Both methods are coderivative-based and employ generalized Hessians (coderivatives of subgradient mappings) associated with objective functions, which are either of class $\mathcal{C}^{1,1}$, or are represented in the form of convex composite optimization, where one of the terms may be extended-real-valued. The proposed globally convergent algorithms are of two types. The first one extends the damped Newton method and requires positive-definiteness of the generalized Hessians for its well-posedness and efficient performance, while the other algorithm is of {the regularized Newton type} being well-defined when the generalized Hessians are merely positive-semidefinite. The obtained convergence rates for both methods are at least linear, but become superlinear under the semismooth$^*$ property of subgradient mappings. Problems of convex composite optimization are investigated with and without the strong convexity assumption {on smooth parts} of objective functions by implementing the machinery of forward-backward envelopes. Numerical experiments are conducted for Lasso problems and for box constrained quadratic programs with providing performance comparisons of the new algorithms and some other first-order and second-order methods that are highly recognized in nonsmooth optimization.
△ Less
Submitted 25 April, 2023; v1 submitted 5 September, 2021;
originally announced September 2021.
-
Generalized Damped Newton Algorithms in Nonsmooth Optimization via Second-Order Subdifferentials
Authors:
Pham Duy Khanh,
Boris Mordukhovich,
Vo Thanh Phat,
Dat Ba Tran
Abstract:
The paper proposes and develops new globally convergent algorithms of the generalized damped Newton type for solving important classes of nonsmooth optimization problems. These algorithms are based on the theory and calculations of second-order subdifferentials of nonsmooth functions with employing the machinery of second-order variational analysis and generalized differentiation. First we develop…
▽ More
The paper proposes and develops new globally convergent algorithms of the generalized damped Newton type for solving important classes of nonsmooth optimization problems. These algorithms are based on the theory and calculations of second-order subdifferentials of nonsmooth functions with employing the machinery of second-order variational analysis and generalized differentiation. First we develop a globally superlinearly convergent damped Newton-type algorithm for the class of continuously differentiable functions with Lipschitzian gradients, which are nonsmooth of second order. Then we design such a globally convergent algorithm to solve a structured class of nonsmooth quadratic composite problems with extended-real-valued cost functions, which typically arise in machine learning and statistics. Finally, we present the results of numerical experiments and compare the performance of our main algorithm applied to an important class of Lasso problems with those achieved by other first-order and second-order optimization algorithms.
△ Less
Submitted 18 January, 2022; v1 submitted 25 January, 2021;
originally announced January 2021.
-
A Generalized Newton Method for Subgradient Systems
Authors:
Pham Duy Khanh,
Boris Mordukhovich,
Vo Thanh Phat
Abstract:
This paper proposes and develops a new Newton-type algorithm to solve subdifferential inclusions defined by subgradients of extended-real-valued prox-regular functions. The proposed algorithm is formulated in terms of the second-order subdifferential of such functions that enjoys extensive calculus rules and can be efficiently computed for broad classes of extended-real-valued functions. Based on…
▽ More
This paper proposes and develops a new Newton-type algorithm to solve subdifferential inclusions defined by subgradients of extended-real-valued prox-regular functions. The proposed algorithm is formulated in terms of the second-order subdifferential of such functions that enjoys extensive calculus rules and can be efficiently computed for broad classes of extended-real-valued functions. Based on this and on metric regularity and subregularity properties of subgradient mappings, we establish verifiable conditions ensuring well-posedness of the proposed algorithm and its local superlinear convergence. The obtained results are also new for the class of equations defined by continuously differentiable functions with Lipschitzian gradients (${\cal C}^{1,1}$ functions), which is the underlying case of our consideration. The developed algorithms for prox-regular functions and its extension to a structured class of composite functions are formulated in terms of proximal mappings and forward-backward envelopes. Besides numerous illustrative examples and comparison with known algorithms for ${\cal C}^{1,1}$ functions and generalized equations, the paper presents applications of the proposed algorithms to regularized least square problems arising in statistics, machine learning, and related disciplines.
△ Less
Submitted 14 September, 2022; v1 submitted 22 September, 2020;
originally announced September 2020.
-
Faces and Support Functions for the Values of Maximal Monotone Operators
Authors:
Bao Tran Nguyen,
Pham Duy Khanh
Abstract:
Representation formulas for faces and support functions of the values of maximal monotone operators are established in two cases: either the operators are defined on uniformly Banach spaces with uniformly convex duals, or their domains have nonempty interiors on reflexive real Banach spaces.
Faces and support functions are characterized by the limit values of the minimal-norm selections of maxim…
▽ More
Representation formulas for faces and support functions of the values of maximal monotone operators are established in two cases: either the operators are defined on uniformly Banach spaces with uniformly convex duals, or their domains have nonempty interiors on reflexive real Banach spaces.
Faces and support functions are characterized by the limit values of the minimal-norm selections of maximal monotone operators in the first case while in the second case they are represented by the limit values of any selection of maximal monotone operators. These obtained formulas are applied to study the structure of maximal monotone operators: the local unique determination from their minimal-norm selections, the local and global decompositions, and the unique determination on dense subsets of their domains.
△ Less
Submitted 21 February, 2020; v1 submitted 12 November, 2019;
originally announced November 2019.
-
Lipschitz Continuity of Convex Functions
Authors:
Bao Tran Nguyen,
Pham Duy Khanh
Abstract:
We provide some necessary and sufficient conditions for a proper lower semicontinuous convex function, defined on a real Banach space, to be locally or globally Lipschitz continuous. Our criteria rely on the existence of a bounded selection of the subdifferential mapping and the intersections of the subdifferential mapping and the normal cone operator to the domain of the given function. Moreover,…
▽ More
We provide some necessary and sufficient conditions for a proper lower semicontinuous convex function, defined on a real Banach space, to be locally or globally Lipschitz continuous. Our criteria rely on the existence of a bounded selection of the subdifferential mapping and the intersections of the subdifferential mapping and the normal cone operator to the domain of the given function. Moreover, we also point out that the Lipschitz continuity of the given function on an open and bounded (not necessarily convex) set can be characterized via the existence of a bounded selection of the subdifferential mapping on the boundary of the given set and as a consequence it is equivalent to the local Lipschitz continuity at every point on the boundary of that set. Our results are applied to extend a Lipschitz and convex function to the whole space and to study the Lipschitz continuity of its Moreau envelope functions.
△ Less
Submitted 12 November, 2019;
originally announced November 2019.
-
On Second-order Conditions for Quasiconvexity and Pseudoconvexity of $\mathcal{C}^{1,1}$-smooth Functions
Authors:
Pham Duy Khanh,
Vo Thanh Phat
Abstract:
For a $\mathcal{C}^2$-smooth function on a finite-dimensional space, a necessary condition for its quasiconvexity is the positive semidefiniteness of its Hessian matrix on the subspace orthogonal to its gradient, whereas a sufficient condition for its strict pseudoconvexity is the positive definiteness of its Hessian matrix on the subspace orthogonal to its gradient. Our aim in this paper is to ex…
▽ More
For a $\mathcal{C}^2$-smooth function on a finite-dimensional space, a necessary condition for its quasiconvexity is the positive semidefiniteness of its Hessian matrix on the subspace orthogonal to its gradient, whereas a sufficient condition for its strict pseudoconvexity is the positive definiteness of its Hessian matrix on the subspace orthogonal to its gradient. Our aim in this paper is to extend those conditions for $\mathcal{C}^{1,1}$-smooth functions by using the Fréchet and Mordukhovich second-order subdifferentials.
△ Less
Submitted 30 October, 2018;
originally announced October 2018.
-
Variational Inequalities Governed By Strongly Pseudomonotone Operators
Authors:
Pham Tien Kha,
Pham Duy Khanh
Abstract:
Qualitative and quantitative aspects for variational inequalities governed by strongly pseudomonotone operators on Hilbert space are investigated in this paper. First, we establish a global error bound for the solution set of the given problem with the residual function being the normal map. Second, we will prove that the iterative sequences generated by gradient projection method (GPM) with steps…
▽ More
Qualitative and quantitative aspects for variational inequalities governed by strongly pseudomonotone operators on Hilbert space are investigated in this paper. First, we establish a global error bound for the solution set of the given problem with the residual function being the normal map. Second, we will prove that the iterative sequences generated by gradient projection method (GPM) with stepsizes forming a non-summable diminishing sequence of positive real numbers converge to the unique solution of the problem when the operator is bounded over the constraint set. Two counter-examples are given to show the necessity of the boundedness assumption and the variation of stepsizes. We also analyze the convergence rate of the iterative sequences generated by this method. Finally, we give an in-depth comparison between our algorithm and a recent related algorithm through several numerical experiments.
△ Less
Submitted 6 October, 2020; v1 submitted 30 July, 2018;
originally announced July 2018.
-
Characterizations of Nonsmooth Robustly Quasiconvex Functions
Authors:
Hoa T. Bui,
Pham Duy Khanh,
Tran Thi Tu Trinh
Abstract:
Two criteria for the robust quasiconvexity of lower semicontinuous functions are established in terms of Fréchet subdifferentials in Asplund spaces.
Two criteria for the robust quasiconvexity of lower semicontinuous functions are established in terms of Fréchet subdifferentials in Asplund spaces.
△ Less
Submitted 18 April, 2018;
originally announced April 2018.
