-
A twice continuously differentiable penalty function for nonlinear semidefinite programming problems and its application
Authors:
Yuya Yamakawa
Abstract:
This paper presents a twice continuously differentiable penalty function for nonlinear semidefinite programming problems. In some optimization methods, such as penalty methods and augmented Lagrangian methods, their convergence property can be ensured by incorporating a penalty function into them, and hence several types of penalty functions have been proposed. In particular, these functions are d…
▽ More
This paper presents a twice continuously differentiable penalty function for nonlinear semidefinite programming problems. In some optimization methods, such as penalty methods and augmented Lagrangian methods, their convergence property can be ensured by incorporating a penalty function into them, and hence several types of penalty functions have been proposed. In particular, these functions are designed to apply optimization methods to find first-order stationary points. Meanwhile, in recent years, second-order sequential optimality, such as Approximate Karush-Kuhn-Tucker2 (AKKT2) and Complementarity AKKT2 (CAKKT2) conditions, has been introduced, and the development of methods for such second-order stationary points would be required in future research. However, existing well-known penalty functions have low compatibility with such methods because they are not twice continuously differentiable. In contrast, the proposed function is expected to have a high affinity for methods to find second-order stationary points. To verify the high affinity, we also present a practical penalty method to find points that satisfy the AKKT and CAKKT conditions by exploiting the proposed function and show their convergence properties.
△ Less
Submitted 24 September, 2025;
originally announced September 2025.
-
Second-order sequential optimality conditions for nonlinear semidefinite optimization problems
Authors:
Huimin Li,
Yuya Yamakawa,
Ellen H. Fukuda
Abstract:
Sequential optimality conditions play an important role in constrained optimization since they provide necessary conditions without requiring constraint qualifications (CQs). This paper introduces a second-order extension of the Approximate Karush-Kuhn-Tucker (AKKT) conditions, referred to as AKKT2, for nonlinear semidefinite optimization problems (NSDP). In particular, we provide a formal definit…
▽ More
Sequential optimality conditions play an important role in constrained optimization since they provide necessary conditions without requiring constraint qualifications (CQs). This paper introduces a second-order extension of the Approximate Karush-Kuhn-Tucker (AKKT) conditions, referred to as AKKT2, for nonlinear semidefinite optimization problems (NSDP). In particular, we provide a formal definition of AKKT2, as well as its stronger variant, called Complementary AKKT2 (CAKKT2), and prove that these conditions are necessary for local minima without any assumption. Moreover, under Robinson's CQ and the weak constant rank property, we show that AKKT2 implies the so-called weak second-order necessary condition. Finally, we propose a penalty-based algorithm that generates sequences whose accumulation points satisfy the AKKT2 and the CAKKT2 conditions.
△ Less
Submitted 29 July, 2025;
originally announced July 2025.
-
Local convergence analysis of stabilized sequential quadratic programming methods for optimization problems in Banach spaces
Authors:
Yuya Yamakawa
Abstract:
This paper presents a stabilized sequential quadratic programming (SQP) method for solving optimization problems in Banach spaces. The optimization problem considered in this study has a general form that enables us to represent various types of optimization problems and is particularly applicable to optimal control, obstacle, and shape optimization problems. Several SQP methods have been proposed…
▽ More
This paper presents a stabilized sequential quadratic programming (SQP) method for solving optimization problems in Banach spaces. The optimization problem considered in this study has a general form that enables us to represent various types of optimization problems and is particularly applicable to optimal control, obstacle, and shape optimization problems. Several SQP methods have been proposed for optimization problems in Banach spaces with specific structures; however, research on the local analysis of SQP-type methods for general problems, such as those considered in this study, is limited. We focus on the local behavior of the proposed stabilized SQP method and prove its local quadratic convergence under reasonable assumptions.
△ Less
Submitted 2 June, 2025; v1 submitted 15 March, 2025;
originally announced March 2025.
-
A strong second-order sequential optimality condition for nonlinear programming problems
Authors:
Huimin Li,
Yuya Yamakawa,
Ellen H. Fukuda,
Nobuo Yamashita
Abstract:
Most numerical methods developed for solving nonlinear programming problems are designed to find points that satisfy certain optimality conditions. While the Karush-Kuhn-Tucker conditions are well-known, they become invalid when constraint qualifications (CQ) are not met. Recent advances in sequential optimality conditions address this limitation in both first- and second-order cases, providing ge…
▽ More
Most numerical methods developed for solving nonlinear programming problems are designed to find points that satisfy certain optimality conditions. While the Karush-Kuhn-Tucker conditions are well-known, they become invalid when constraint qualifications (CQ) are not met. Recent advances in sequential optimality conditions address this limitation in both first- and second-order cases, providing genuine optimality guarantees at local optima, even when CQs do not hold. However, some second-order sequential optimality conditions still require some restrictive conditions on constraints in the recent literature. In this paper, we propose a new strong second-order sequential optimality condition without CQs. We also show that a penalty-type method and an augmented Lagrangian method generate points satisfying these new optimality conditions.
△ Less
Submitted 3 March, 2025;
originally announced March 2025.
-
Convergence analysis of a regularized Newton method with generalized regularization terms for convex optimization problems
Authors:
Yuya Yamakawa,
Nobuo Yamashita
Abstract:
This paper presents a regularized Newton method (RNM) with generalized regularization terms for unconstrained convex optimization problems. The generalized regularization includes quadratic, cubic, and elastic net regularizations as special cases. Therefore, the proposed method serves as a general framework that includes not only the classical and cubic RNMs but also a novel RNM with elastic net r…
▽ More
This paper presents a regularized Newton method (RNM) with generalized regularization terms for unconstrained convex optimization problems. The generalized regularization includes quadratic, cubic, and elastic net regularizations as special cases. Therefore, the proposed method serves as a general framework that includes not only the classical and cubic RNMs but also a novel RNM with elastic net regularization. We show that the proposed RNM has the global $\mathcal{O}(k^{-2})$ and local superlinear convergence, which are the same as those of the cubic RNM.
△ Less
Submitted 10 July, 2024; v1 submitted 14 June, 2024;
originally announced June 2024.
-
Modified Armijo line search in optimization on Riemannian submanifolds with reduced computational cost
Authors:
Hiroyuki Sato,
Yuya Yamakawa,
Kensuke Aihara
Abstract:
For optimization problems on Riemannian manifolds, many types of globally convergent algorithms have been proposed, and they are often equipped with the Riemannian version of the Armijo line search for global convergence. Such existing methods need to compute the value of a retraction mapping regarding the search direction several times at each iteration; this may result in high computational cost…
▽ More
For optimization problems on Riemannian manifolds, many types of globally convergent algorithms have been proposed, and they are often equipped with the Riemannian version of the Armijo line search for global convergence. Such existing methods need to compute the value of a retraction mapping regarding the search direction several times at each iteration; this may result in high computational costs, particularly if computing the value of the retraction is expensive. To address this issue, this study focuses on Riemannian submanifolds of the Euclidean spaces and proposes a novel Riemannian line search that achieves lower computational cost by incorporating a new strategy that computes the retraction only when inevitable. A class of Riemannian optimization algorithms, including the steepest descent and Newton methods, with the new line search strategy is proposed and proved to be globally convergent. Furthermore, numerical experiments on solving optimization problems on several types of Riemannian submanifolds illustrate that the proposed methods are superior to the standard Riemannian Armijo line search-based methods.
△ Less
Submitted 10 April, 2025; v1 submitted 4 April, 2023;
originally announced April 2023.
-
Superlinear and quadratic convergence of a stabilized sequential quadratic semidefinite programming method for nonlinear semidefinite programming problems
Authors:
Yuya Yamakawa
Abstract:
In this paper, we present a stabilized sequential quadratic semidefinite programming (SQSDP) method for nonlinear semidefinite programming (NSDP) problems and prove its local convergence. The stabilized SQSDP method is originally developed to solve degenerate NSDP problems and is based on the stabilized sequential programming (SQP) methods for nonlinear programming (NLP) problems. Although some SQ…
▽ More
In this paper, we present a stabilized sequential quadratic semidefinite programming (SQSDP) method for nonlinear semidefinite programming (NSDP) problems and prove its local convergence. The stabilized SQSDP method is originally developed to solve degenerate NSDP problems and is based on the stabilized sequential programming (SQP) methods for nonlinear programming (NLP) problems. Although some SQP-type methods for NSDP problems have been proposed, most of them are SQSDP methods which are based on the SQP methods for NLP problems, and there are few researches regarding the stabilized SQSDP methods. In particular, there is room for the development of locally fast convergent stabilized SQSDP methods. We prove not only superlinear but also quadratic convergence of the proposed method under some mild assumptions, such as strict Robinson's constraint qualification and second-order sufficient condition.
△ Less
Submitted 18 March, 2024; v1 submitted 31 October, 2022;
originally announced October 2022.
-
A revised sequential quadratic semidefinite programming method for nonlinear semidefinite optimization
Authors:
Kosuke Okabe,
Yuya Yamakawa,
Ellen H. Fukuda
Abstract:
In 2020, Yamakawa and Okuno proposed a stabilized sequential quadratic semidefinite programming (SQSDP) method for solving, in particular, degenerate nonlinear semidefinite optimization problems. The algorithm is shown to converge globally without a constraint qualification, and it has some nice properties, including the feasible subproblems, and their possible inexact computations. In particular,…
▽ More
In 2020, Yamakawa and Okuno proposed a stabilized sequential quadratic semidefinite programming (SQSDP) method for solving, in particular, degenerate nonlinear semidefinite optimization problems. The algorithm is shown to converge globally without a constraint qualification, and it has some nice properties, including the feasible subproblems, and their possible inexact computations. In particular, the convergence was established for approximate-Karush-Kuhn-Tucker (AKKT) and trace-AKKT conditions, which are two sequential optimality conditions for the nonlinear conic contexts. However, recently, complementarity-AKKT (CAKKT) conditions were also consider, as an alternative to the previous mentioned ones, that is more practical. Since few methods are shown to converge to CAKKT points, at least in conic optimization, and to complete the study associated to the SQSDP, here we propose a revised version of the method, maintaining the good properties. We modify the previous algorithm, prove the global convergence in the sense of CAKKT, and show some preliminary numerical experiments.
△ Less
Submitted 1 April, 2022;
originally announced April 2022.
-
Distributionally Robust Expected Residual Minimization for Stochastic Variational Inequality Problems
Authors:
Atsushi Hori,
Yuya Yamakawa,
Nobuo Yamashita
Abstract:
The stochastic variational inequality problem (SVIP) is an equilibrium model that includes random variables and has been widely applied in various fields such as economics and engineering. Expected residual minimization (ERM) is an established model for obtaining a reasonable solution for the SVIP, and its objective function is an expected value of a suitable merit function for the SVIP. However,…
▽ More
The stochastic variational inequality problem (SVIP) is an equilibrium model that includes random variables and has been widely applied in various fields such as economics and engineering. Expected residual minimization (ERM) is an established model for obtaining a reasonable solution for the SVIP, and its objective function is an expected value of a suitable merit function for the SVIP. However, the ERM is restricted to the case where the distribution is known in advance. We extend the ERM to ensure the attainment of robust solutions for the SVIP under the uncertainty distribution (the extended ERM is referred to as distributionally robust expected residual minimization (DRERM), where the worst-case distribution is derived from the set of probability measures in which the expected value and variance take the same sample mean and variance, respectively). Under suitable assumptions, we demonstrate that the DRERM can be reformulated as a deterministic convex nonlinear semidefinite programming to avoid numerical integration.
△ Less
Submitted 24 January, 2023; v1 submitted 14 November, 2021;
originally announced November 2021.
-
An equivalent nonlinear optimization model with triangular low-rank factorization for semidefinite programs
Authors:
Yuya Yamakawa,
Tetsuya Ikegami,
Ellen H. Fukuda,
Nobuo Yamashita
Abstract:
In this paper, we propose a new nonlinear optimization model to solve semidefinite optimization problems (SDPs), providing some properties related to local optimal solutions. The proposed model is based on another nonlinear optimization model given by Burer and Monteiro (2003), but it has several nice properties not seen in the existing one. Firstly, the decision variable of the proposed model is…
▽ More
In this paper, we propose a new nonlinear optimization model to solve semidefinite optimization problems (SDPs), providing some properties related to local optimal solutions. The proposed model is based on another nonlinear optimization model given by Burer and Monteiro (2003), but it has several nice properties not seen in the existing one. Firstly, the decision variable of the proposed model is a triangular low-rank matrix, and hence the dimension of its decision variable space is smaller. Secondly, the existence of a strict local optimum of the proposed model is guaranteed under some conditions, whereas the existing model has no strict local optimum. In other words, it is difficult to construct solution methods equipped with fast convergence using the existing model. Some numerical results are also presented to examine the efficiency of the proposed model.
△ Less
Submitted 29 March, 2021;
originally announced March 2021.
-
A stabilized sequential quadratic semidefinite programming method for degenerate nonlinear semidefinite programs
Authors:
Yuya Yamakawa,
Takayuki Okuno
Abstract:
In this paper, we propose a new sequential quadratic semidefinite programming (SQSDP) method for solving degenerate nonlinear semidefinite programs (NSDPs), in which we produce iteration points by solving a sequence of stabilized quadratic semidefinite programming (QSDP) subproblems, which we derive from the minimax problem associated with the NSDP. Unlike the existing SQSDP methods, the proposed…
▽ More
In this paper, we propose a new sequential quadratic semidefinite programming (SQSDP) method for solving degenerate nonlinear semidefinite programs (NSDPs), in which we produce iteration points by solving a sequence of stabilized quadratic semidefinite programming (QSDP) subproblems, which we derive from the minimax problem associated with the NSDP. Unlike the existing SQSDP methods, the proposed one allows us to solve those QSDP subproblems inexactly, and each QSDP is feasible. One more remarkable point of the proposed method is that constraint qualifications (CQs) or boundedness of Lagrange multiplier sequences are not required in the global convergence analysis. Specifically, without assuming such conditions, we prove the global convergence to a point satisfying any of the following: the stationary conditions for the feasibility problem, the approximate-Karush-Kuhn-Tucker (AKKT) conditions, and the trace-AKKT conditions. Finally, we conduct some numerical experiments to examine the efficiency of the proposed method.
△ Less
Submitted 22 April, 2021; v1 submitted 30 September, 2019;
originally announced September 2019.
