-
A Proximal Variable Smoothing for Minimization of Nonlinearly Composite Nonsmooth Function -- Maxmin Dispersion and MIMO Applications
Authors:
Keita Kume,
Isao Yamada
Abstract:
We propose a proximal variable smoothing algorithm for a nonsmooth optimization problem whose cost function is the sum of three functions including a weakly convex composite function. The proposed algorithm has a single-loop structure inspired by a proximal gradient-type method. More precisely, the proposed algorithm consists of two steps: (i) a gradient descent of a time-varying smoothed surrogat…
▽ More
We propose a proximal variable smoothing algorithm for a nonsmooth optimization problem whose cost function is the sum of three functions including a weakly convex composite function. The proposed algorithm has a single-loop structure inspired by a proximal gradient-type method. More precisely, the proposed algorithm consists of two steps: (i) a gradient descent of a time-varying smoothed surrogate function designed partially with the Moreau envelope of the weakly convex function; (ii) an application of the proximity operator of the remaining function not covered by the smoothed surrogate function. We also present a convergence analysis of the proposed algorithm by exploiting a novel asymptotic approximation of a gradient mapping-type stationarity measure. Numerical experiments demonstrate the effectiveness of the proposed algorithm in two scenarios: (i) maxmin dispersion problem and (ii) multiple-input-multiple-output (MIMO) signal detection.
△ Less
Submitted 6 June, 2025;
originally announced June 2025.
-
Hierarchical Variational Inequality Problem for Noncooperative Game-theoretic Selection of Generalized Nash Equilibrium
Authors:
Shota Matsuo,
Keita Kume,
Isao Yamada
Abstract:
The equilibrium selection problem in the variational Generalized Nash Equilibrium Problem (v-GNEP) has been reported as an optimization problem defined over the solution set of v-GNEP, called in this paper the lower-level v-GNEP. However, to make such a selection fair for every player, we have to rely on an unrealistic assumption, that is, the availability of a trusted center that does not induce…
▽ More
The equilibrium selection problem in the variational Generalized Nash Equilibrium Problem (v-GNEP) has been reported as an optimization problem defined over the solution set of v-GNEP, called in this paper the lower-level v-GNEP. However, to make such a selection fair for every player, we have to rely on an unrealistic assumption, that is, the availability of a trusted center that does not induce any bias among players. In this paper, to ensure fairness for every player even in the process of equilibrium selection, we propose a new equilibrium selection problem, named the upper-level v-GNEP. The proposed upper-level v-GNEP is formulated as a v-GNEP defined over the solution set of the lower-level v-GNEP. We also present an iterative algorithm, of guaranteed convergence to a solution of the upper-level v-GNEP, as an application of the hybrid steepest descent method to a fixed point set characterization of the solution of the lower-level v-GNEP. Numerical experiments illustrate the proposed equilibrium selection and algorithm.
△ Less
Submitted 4 April, 2025;
originally announced April 2025.
-
Variable smoothing algorithm for inner-loop-free DC composite optimizations
Authors:
Kumataro Yazawa,
Keita Kume,
Isao Yamada
Abstract:
We propose a variable smoothing algorithm for minimizing a nonsmooth and nonconvex cost function. The cost function is the sum of a smooth function and a composition of a difference-of-convex (DC) function with a smooth mapping. At each step of our algorithm, we generate a smooth surrogate function by using the Moreau envelope of each weakly convex function in the DC function, and then perform the…
▽ More
We propose a variable smoothing algorithm for minimizing a nonsmooth and nonconvex cost function. The cost function is the sum of a smooth function and a composition of a difference-of-convex (DC) function with a smooth mapping. At each step of our algorithm, we generate a smooth surrogate function by using the Moreau envelope of each weakly convex function in the DC function, and then perform the gradient descent update of the surrogate function. The proposed algorithm does not require any inner loop unlike many existing algorithms for DC problem. We also present a convergence analysis in terms of a DC critical point for the proposed algorithm as well as its application to robust phase retrieval.
△ Less
Submitted 18 March, 2025;
originally announced March 2025.
-
A convexity preserving nonconvex regularization for inverse problems under non-Gaussian noise
Authors:
Wataru Yata,
Keita Kume,
Isao Yamada
Abstract:
We propose a nonconvexly regularized convex model for linear regression problems under non-Gaussian noise. The cost function of the proposed model is designed with a possibly non-quadratic data fidelity term and a nonconvex regularizer via the generalized Moreau enhancement of a seed convex regularizer. We present sufficient conditions (i) for the cost function of the proposed model to be convex o…
▽ More
We propose a nonconvexly regularized convex model for linear regression problems under non-Gaussian noise. The cost function of the proposed model is designed with a possibly non-quadratic data fidelity term and a nonconvex regularizer via the generalized Moreau enhancement of a seed convex regularizer. We present sufficient conditions (i) for the cost function of the proposed model to be convex over the entire space, and (ii) for the existence of a minimizer of the proposed model. Under such conditions, we propose a proximal splitting type algorithm with guaranteed convergence to a global minimizer of the proposed model. As an application, we enhance nonconvexly a convex sparsity-promoting regularizer in a scenario of simultaneous declipping and denoising.
△ Less
Submitted 17 March, 2025;
originally announced March 2025.
-
An LiGME Regularizer of Designated Isolated Minimizers -- An Application to Discrete-Valued Signal Estimation
Authors:
Satoshi Shoji,
Wataru Yata,
Keita Kume,
Isao Yamada
Abstract:
For a regularized least squares estimation of discrete-valued signals, we propose a Linearly involved Generalized Moreau Enhanced (LiGME) regularizer, as a nonconvex regularizer, of designated isolated minimizers. The proposed regularizer is designed as a Generalized Moreau Enhancement (GME) of the so-called sum-of-absolute-values (SOAV) convex regularizer. Every candidate vector in the discrete-v…
▽ More
For a regularized least squares estimation of discrete-valued signals, we propose a Linearly involved Generalized Moreau Enhanced (LiGME) regularizer, as a nonconvex regularizer, of designated isolated minimizers. The proposed regularizer is designed as a Generalized Moreau Enhancement (GME) of the so-called sum-of-absolute-values (SOAV) convex regularizer. Every candidate vector in the discrete-valued set is aimed to be assigned to an isolated local minimizer of the proposed regularizer while the overall convexity of the regularized least squares model is maintained. Moreover, a global minimizer of the proposed model can be approximated iteratively by using a variant of the constrained LiGME (cLiGME) algorithm. To enhance the accuracy of the proposed estimation, we also propose a pair of simple modifications, called respectively an iterative reweighting and a generalized superiorization. Numerical experiments demonstrate the effectiveness of the proposed model and algorithms in a scenario of multiple-input multiple-output (MIMO) signal detection.
△ Less
Submitted 28 May, 2025; v1 submitted 13 March, 2025;
originally announced March 2025.
-
A Variable Smoothing for Weakly Convex Composite Minimization with Nonconvex Constraint
Authors:
Keita Kume,
Isao Yamada
Abstract:
In this paper, we address a nonconvexly constrained nonsmooth optimization problem involving the composition of a weakly convex function and a smooth mapping. To find a stationary point of the target problem, we propose a variable smoothing-type algorithm by combining the ideas of (i) translating the constrained problem into a Euclidean optimization problem with a smooth parametrization of the con…
▽ More
In this paper, we address a nonconvexly constrained nonsmooth optimization problem involving the composition of a weakly convex function and a smooth mapping. To find a stationary point of the target problem, we propose a variable smoothing-type algorithm by combining the ideas of (i) translating the constrained problem into a Euclidean optimization problem with a smooth parametrization of the constraint set; (ii) exploiting a sequence of smoothed surrogate functions, of the cost function, given with the Moreau envelope of a weakly convex function. The proposed algorithm produces a vector sequence by the gradient descent update of a smoothed surrogate function at each iteration. In a case where the proximity operator of the weakly convex function is available, the proposed algorithm does not require any iterative solver for subproblems therein. By leveraging tools in the variational analysis, we show the so-called {\em gradient consistency property}, which is a key ingredient for smoothing-type algorithms, of the smoothed surrogate function used in this paper. Based on the gradient consistency property, we also establish an asymptotic convergence analysis for the proposed algorithm. Numerical experiments demonstrate the efficacy of the proposed algorithm.
△ Less
Submitted 23 January, 2025; v1 submitted 5 December, 2024;
originally announced December 2024.
-
Hierarchical Nash Equilibrium over Variational Equilibria via Fixed-point Set Expression of Quasi-nonexpansive Operator
Authors:
Shota Matsuo,
Keita Kume,
Isao Yamada
Abstract:
The equilibrium selection problem in the generalized Nash equilibrium problem (GNEP) has recently been studied as an optimization problem, defined over the set of all variational equilibria achievable through a lower-level non-cooperative game among players. However, to make such a selection fair for every player, we have to rely on an unrealistic assumption, that is, the availability of a trusted…
▽ More
The equilibrium selection problem in the generalized Nash equilibrium problem (GNEP) has recently been studied as an optimization problem, defined over the set of all variational equilibria achievable through a lower-level non-cooperative game among players. However, to make such a selection fair for every player, we have to rely on an unrealistic assumption, that is, the availability of a trusted center that does not induce any bias for every player. In this paper, we study a new equilibrium selection problem, named the hierarchical Nash equilibrium problem (HNEP), and propose an iterative algorithm for solving the HNEP. The HNEP is designed to ensure a fair selection without assuming any trusted center. More precisely, the HNEP is the GNEP for an upper-level non-cooperative game defined over the set of all variational equilibria of the lower-level non-cooperative game. The proposed algorithm for the HNEP is established by applying the hybrid steepest descent method to a variational inequality defined over the fixed point set of a quasi-nonexpansive operator. Numerical experiments show the effectiveness of the proposed equilibrium selection problem and its algorithmic solution.
△ Less
Submitted 27 January, 2025; v1 submitted 17 September, 2024;
originally announced September 2024.
-
A Proximal Variable Smoothing for Nonsmooth Minimization Involving Weakly Convex Composite with MIMO Application
Authors:
Keita Kume,
Isao Yamada
Abstract:
We propose a proximal variable smoothing algorithm for nonsmooth optimization problem with sum of three functions involving weakly convex composite function. The proposed algorithm is designed as a time-varying forward-backward splitting algorithm with two steps: (i) a time-varying forward step with the gradient of a smoothed surrogate function, designed with the Moreau envelope, of the sum of two…
▽ More
We propose a proximal variable smoothing algorithm for nonsmooth optimization problem with sum of three functions involving weakly convex composite function. The proposed algorithm is designed as a time-varying forward-backward splitting algorithm with two steps: (i) a time-varying forward step with the gradient of a smoothed surrogate function, designed with the Moreau envelope, of the sum of two functions; (ii) the backward step with a proximity operator of the remaining function. For the proposed algorithm, we present a convergence analysis in terms of a stationary point by using a newly smoothed surrogate stationarity measure. As an application of the target problem, we also present a formulation of multiple-input-multiple-output (MIMO) signal detection with phase-shift keying. Numerical experiments demonstrate the efficacy of the proposed formulation and algorithm.
△ Less
Submitted 23 January, 2025; v1 submitted 17 September, 2024;
originally announced September 2024.
-
Generalized Left-Localized Cayley Parametrization for Optimization with Orthogonality Constraints
Authors:
Kieta Kume,
Isao Yamada
Abstract:
We present a reformulation of optimization problems over the Stiefel manifold by using a Cayley-type transform, named the generalized left-localized Cayley transform, for the Stiefel manifold. The reformulated optimization problem is defined over a vector space, whereby we can apply directly powerful computational arts designed for optimization over a vector space. The proposed Cayley-type transfo…
▽ More
We present a reformulation of optimization problems over the Stiefel manifold by using a Cayley-type transform, named the generalized left-localized Cayley transform, for the Stiefel manifold. The reformulated optimization problem is defined over a vector space, whereby we can apply directly powerful computational arts designed for optimization over a vector space. The proposed Cayley-type transform enjoys several key properties which are useful to (i) study relations between the original problem and the proposed problem; (ii) check the conditions to guarantee the global convergence of optimization algorithms. Numerical experiments demonstrate that the proposed algorithm outperforms the standard algorithms designed with a retraction on the Stiefel manifold.
△ Less
Submitted 1 December, 2023;
originally announced December 2023.
-
A variable smoothing for Nonconvexly constrained nonsmooth optimization with application to sparse spectral clustering
Authors:
Keita Kume,
Isao Yamada
Abstract:
We propose a variable smoothing algorithm for solving nonconvexly constrained nonsmooth optimization problems. The target problem has two issues that need to be addressed: (i) the nonconvex constraint and (ii) the nonsmooth term. To handle the nonconvex constraint, we translate the target problem into an unconstrained problem by parameterizing the nonconvex constraint in terms of a Euclidean space…
▽ More
We propose a variable smoothing algorithm for solving nonconvexly constrained nonsmooth optimization problems. The target problem has two issues that need to be addressed: (i) the nonconvex constraint and (ii) the nonsmooth term. To handle the nonconvex constraint, we translate the target problem into an unconstrained problem by parameterizing the nonconvex constraint in terms of a Euclidean space. We show that under a certain condition, these problems are equivalent in view of finding a stationary point. To find a stationary point of the parameterized problem, the proposed algorithm performs the gradient descent update for the smoothed version of the parameterized problem with replacement of the nonsmooth function by the Moreau envelope, inspired by a variable smoothing algorithm [Böhm-Wright, J. Optim. Theory Appl., 2021] specialized for unconstrained nonsmooth optimization. We also present a convergence analysis of the proposed algorithm as well as its application to a nonconvex reformulation of the sparse spectral clustering.
△ Less
Submitted 2 April, 2024; v1 submitted 21 September, 2023;
originally announced September 2023.
-
Adaptive Localized Cayley Parametrization for Optimization over Stiefel Manifold
Authors:
Keita Kume,
Isao Yamada
Abstract:
We present an adaptive parametrization strategy for optimization problems over the Stiefel manifold by using generalized Cayley transforms to utilize powerful Euclidean optimization algorithms efficiently. The generalized Cayley transform can translate an open dense subset of the Stiefel manifold into a vector space, and the open dense subset is determined according to a tunable parameter called a…
▽ More
We present an adaptive parametrization strategy for optimization problems over the Stiefel manifold by using generalized Cayley transforms to utilize powerful Euclidean optimization algorithms efficiently. The generalized Cayley transform can translate an open dense subset of the Stiefel manifold into a vector space, and the open dense subset is determined according to a tunable parameter called a center point. With the generalized Cayley transform, we recently proposed the naive Cayley parametrization, which reformulates the optimization problem over the Stiefel manifold as that over the vector space. Although this reformulation enables us to transplant powerful Euclidean optimization algorithms, their convergences may become slow by a poor choice of center points. To avoid such a slow convergence, in this paper, we propose to estimate adaptively 'good' center points so that the reformulated problem can be solved faster. We also present a unified convergence analysis, regarding the gradient, in cases where fairly standard Euclidean optimization algorithms are employed in the proposed adaptive parametrization strategy. Numerical experiments demonstrate that (i) the proposed strategy succeeds in escaping from the slow convergence observed in the naive Cayley parametrization strategy; (ii) the proposed strategy outperforms the standard strategy which employs a retraction.
△ Less
Submitted 29 May, 2023;
originally announced May 2023.
-
Filter characteristics in image decomposition with singular spectrum analysis
Authors:
Kenji Kume,
Naoko Nose-Togawa
Abstract:
Singular spectrum analysis is developed as a nonparametric spectral decomposition of a time series. It can be easily extended to the decomposition of multidimensional lattice-like data through the filtering interpretation. In this viewpoint, the singular spectrum analysis can be understood as the adaptive and optimal generation of the filters and their two-step point-symmetric operation to the ori…
▽ More
Singular spectrum analysis is developed as a nonparametric spectral decomposition of a time series. It can be easily extended to the decomposition of multidimensional lattice-like data through the filtering interpretation. In this viewpoint, the singular spectrum analysis can be understood as the adaptive and optimal generation of the filters and their two-step point-symmetric operation to the original data. In this paper, we point out that, when applied to the multidimensional data, the adaptively generated filters exhibit symmetry properties resulting from the bisymmetric nature of the lag-covariance matrices. The eigenvectors of the lag-covariance matrix are either symmetric or antisymmetric, and for the 2D image data, these lead to the differential-type filters with even- or odd-order derivatives. The dominant filter is a smoothing filter, reflecting the dominance of low-frequency components of the photo images. The others are the edge-enhancement or the noise filters corresponding to the band-pass or the high-pass filters. The implication of the decomposition to the image denoising is briefly discussed.
△ Less
Submitted 7 May, 2015;
originally announced May 2015.
