-
Uncertain standard quadratic optimization under distributional assumptions: a chance-constrained epigraphic approach
Authors:
Immanuel M. Bomze,
Daniel de Vicente
Abstract:
The standard quadratic optimization problem (StQP) consists of minimizing a quadratic form over the standard simplex. Without convexity or concavity of the quadratic form, the StQP is NP-hard. This problem has many relevant real-life applications ranging portfolio optimization to pairwise clustering and replicator dynamics. Sometimes, the data matrix is uncertain. We investigate models where the d…
▽ More
The standard quadratic optimization problem (StQP) consists of minimizing a quadratic form over the standard simplex. Without convexity or concavity of the quadratic form, the StQP is NP-hard. This problem has many relevant real-life applications ranging portfolio optimization to pairwise clustering and replicator dynamics. Sometimes, the data matrix is uncertain. We investigate models where the distribution of the data matrix is known but where both the StQP after realization of the data matrix and the here-and-now problem are indefinite. We test the performance of a chance-constrained epigraphic StQP to the uncertain StQP.
△ Less
Submitted 9 April, 2025; v1 submitted 22 November, 2024;
originally announced November 2024.
-
Finding quadratic underestimators for optimal value functions of nonconvex all-quadratic problems via copositive optimization
Authors:
Markus Gabl,
Immanuel Bomze
Abstract:
Modeling parts of an optimization problem as an optimal value function that depends on a top-level decision variable is a regular occurrence in optimization and an essential ingredient for methods such as Benders Decomposition. It often allows for the disentanglement of computational complexity and exploitation of special structures in the lower-level problem that define the optimal value function…
▽ More
Modeling parts of an optimization problem as an optimal value function that depends on a top-level decision variable is a regular occurrence in optimization and an essential ingredient for methods such as Benders Decomposition. It often allows for the disentanglement of computational complexity and exploitation of special structures in the lower-level problem that define the optimal value functions. If this problem is convex, duality theory can be used to build piecewise affine models of the optimal value function over which the top-level problem can be optimized efficiently. In this text, we are interested in the optimal value function of an all-quadratic problem (also called quadratically constrained quadratic problem, QCQP) which is not necessarily convex, so that duality theory can not be applied without introducing a generally unquantifiable relaxation error. This issue can be bypassed by employing copositive reformulations of the underlying QCQP. We investigate two ways to parametrize these by the top-level variable. The first one leads to a copositive characterization of an underestimator that is sandwiched between the convex envelope of the optimal value function and that envelope's lower-semicontinuous hull. The dual of that characterization allows us to derive affine underestimators. The second parametrization yields an alternative characterization of the optimal value function itself, which other than the original version has an exact dual counterpart. From the latter, we can derive convex and nonconvex quadratic underestimators of the optimal value function. In fact, we can show that any quadratic underestimator is associated with a dual feasible solution in a certain sense.
△ Less
Submitted 30 September, 2024;
originally announced September 2024.
-
Tighter yet more tractable relaxations and nontrivial instance generation for sparse standard quadratic optimization
Authors:
Immanuel Bomze,
Bo Peng,
Yuzhou Qiu,
E. Alper Yildirim
Abstract:
The Standard Quadratic optimization Problem (StQP), arguably the simplest among all classes of NP-hard optimization problems, consists of extremizing a quadratic form (the simplest nonlinear polynomial) over the standard simplex (the simplest polytope/compact feasible set). As a problem class, StQPs may be nonconvex with an exponential number of inefficient local solutions. StQPs arise in a multit…
▽ More
The Standard Quadratic optimization Problem (StQP), arguably the simplest among all classes of NP-hard optimization problems, consists of extremizing a quadratic form (the simplest nonlinear polynomial) over the standard simplex (the simplest polytope/compact feasible set). As a problem class, StQPs may be nonconvex with an exponential number of inefficient local solutions. StQPs arise in a multitude of applications, among them mathematical finance, machine learning (clustering), and modeling in biosciences (e.g., selection and ecology). This paper deals with such StQPs under an additional sparsity or cardinality constraint, which, even for convex objectives, renders NP-hard problems. One motivation to study StQPs under such sparsity restrictions is the high-dimensional portfolio selection problem with too many assets to handle, in particular, in the presence of transaction costs. Here, relying on modern conic optimization techniques, we present tractable convex relaxations for this relevant but difficult problem. We propose novel equivalent reformulations of these relaxations with significant dimensional reduction, which is essential for the tractability of these relaxations when the problem size grows. Moreover, we propose an instance generation procedure which systematically avoids too easy instances. Our extensive computational results illustrate the high quality of the relaxation bounds in a significant number of instances. Furthermore, in contrast with exact mixed-integer quadratic programming models, the solution time of the relaxations is very robust to the choices of the problem parameters. In particular, the reduced formulations achieve significant improvements in terms of the solution time over their counterparts.
△ Less
Submitted 3 June, 2024;
originally announced June 2024.
-
Feature selection in linear SVMs via a hard cardinality constraint: a scalable SDP decomposition approach
Authors:
Immanuel Bomze,
Federico D'Onofrio,
Laura Palagi,
Bo Peng
Abstract:
In this paper, we study the embedded feature selection problem in linear Support Vector Machines (SVMs), in which a cardinality constraint is employed, leading to an interpretable classification model. The problem is NP-hard due to the presence of the cardinality constraint, even though the original linear SVM amounts to a problem solvable in polynomial time. To handle the hard problem, we first i…
▽ More
In this paper, we study the embedded feature selection problem in linear Support Vector Machines (SVMs), in which a cardinality constraint is employed, leading to an interpretable classification model. The problem is NP-hard due to the presence of the cardinality constraint, even though the original linear SVM amounts to a problem solvable in polynomial time. To handle the hard problem, we first introduce two mixed-integer formulations for which novel semidefinite relaxations are proposed. Exploiting the sparsity pattern of the relaxations, we decompose the problems and obtain equivalent relaxations in a much smaller cone, making the conic approaches scalable. To make the best usage of the decomposed relaxations, we propose heuristics using the information of its optimal solution. Moreover, an exact procedure is proposed by solving a sequence of mixed-integer decomposed semidefinite optimization problems. Numerical results on classical benchmarking datasets are reported, showing the efficiency and effectiveness of our approach.
△ Less
Submitted 19 December, 2024; v1 submitted 15 April, 2024;
originally announced April 2024.
-
On Tractable Convex Relaxations of Standard Quadratic Optimization Problems under Sparsity Constraints
Authors:
Immanuel Bomze,
Bo Peng,
Yuzhou Qiu,
E. Alper Yıldırım
Abstract:
Standard quadratic optimization problems (StQPs) provide a versatile modelling tool in various applications. In this paper, we consider StQPs with a hard sparsity constraint, referred to as sparse StQPs. We focus on various tractable convex relaxations of sparse StQPs arising from a mixed-binary quadratic formulation, namely, the linear optimization relaxation given by the reformulation-linearizat…
▽ More
Standard quadratic optimization problems (StQPs) provide a versatile modelling tool in various applications. In this paper, we consider StQPs with a hard sparsity constraint, referred to as sparse StQPs. We focus on various tractable convex relaxations of sparse StQPs arising from a mixed-binary quadratic formulation, namely, the linear optimization relaxation given by the reformulation-linearization technique, the Shor relaxation, and the relaxation resulting from their combination. We establish several structural properties of these relaxations in relation to the corresponding relaxations of StQPs without any sparsity constraints, and pay particular attention to the rank-one feasible solutions retained by these relaxations. We then utilize these relations to establish several results about the quality of the lower bounds arising from different relaxations. We also present several conditions that ensure the exactness of each relaxation.
△ Less
Submitted 6 October, 2023;
originally announced October 2023.
-
Projection free methods on product domains
Authors:
Immanuel Bomze,
Francesco Rinaldi,
Damiano Zeffiro
Abstract:
Projection-free block-coordinate methods avoid high computational cost per iteration and at the same time exploit the particular problem structure of product domains. Frank-Wolfe-like approaches rank among the most popular ones of this type. However, as observed in the literature, there was a gap between the classical Frank-Wolfe theory and the block-coordinate case. Moreover, most of previous res…
▽ More
Projection-free block-coordinate methods avoid high computational cost per iteration and at the same time exploit the particular problem structure of product domains. Frank-Wolfe-like approaches rank among the most popular ones of this type. However, as observed in the literature, there was a gap between the classical Frank-Wolfe theory and the block-coordinate case. Moreover, most of previous research concentrated on convex objectives. This study now deals also with the non-convex case and reduces above-mentioned theory gap, in combining a new, fully developed convergence theory with novel active set identification results which ensure that inherent sparsity of solutions can be exploited in an efficient way. Preliminary numerical experiments seem to justify our approach and also show promising results for obtaining global solutions in the non-convex case.
△ Less
Submitted 6 December, 2023; v1 submitted 9 February, 2023;
originally announced February 2023.
-
Frank-Wolfe and friends: a journey into projection-free first-order optimization methods
Authors:
Immanuel. M. Bomze,
Francesco Rinaldi,
Damiano Zeffiro
Abstract:
Invented some 65 years ago in a seminal paper by Marguerite Straus-Frank and Philip Wolfe, the Frank-Wolfe method recently enjoys a remarkable revival, fuelled by the need of fast and reliable first-order optimization methods in Data Science and other relevant application areas. This review tries to explain the success of this approach by illustrating versatility and applicability in a wide range…
▽ More
Invented some 65 years ago in a seminal paper by Marguerite Straus-Frank and Philip Wolfe, the Frank-Wolfe method recently enjoys a remarkable revival, fuelled by the need of fast and reliable first-order optimization methods in Data Science and other relevant application areas. This review tries to explain the success of this approach by illustrating versatility and applicability in a wide range of contexts, combined with an account on recent progress in variants, both improving on the speed and efficiency of this surprisingly simple principle of first-order optimization.
△ Less
Submitted 18 June, 2021;
originally announced June 2021.
-
Fast cluster detection in networks by first-order optimization
Authors:
Immanuel M. Bomze,
Francesco Rinaldi,
Damiano Zeffiro
Abstract:
Cluster detection plays a fundamental role in the analysis of data. In this paper, we focus on the use of s-defective clique models for network-based cluster detection and propose a nonlinear optimization approach that efficiently handles those models in practice. In particular, we introduce an equivalent continuous formulation for the problem under analysis, and we analyze some tailored variants…
▽ More
Cluster detection plays a fundamental role in the analysis of data. In this paper, we focus on the use of s-defective clique models for network-based cluster detection and propose a nonlinear optimization approach that efficiently handles those models in practice. In particular, we introduce an equivalent continuous formulation for the problem under analysis, and we analyze some tailored variants of the Frank-Wolfe algorithm that enable us to quickly find maximal s-defective cliques. The good practical behavior of those algorithmic tools, which is closely connected to their support identification properties, makes them very appealing in practical applications. The reported numerical results clearly show the effectiveness of the proposed approach.
△ Less
Submitted 29 March, 2021;
originally announced March 2021.
-
Uncertainty Preferences in Robust Mixed-Integer Linear Optimization with Endogenous Uncertainty
Authors:
Immanuel Bomze,
Markus Gabl
Abstract:
In robust optimization one seeks to make a decision under uncertainty, where the goal is to find the solution with the best worst-case performance. The set of possible realizations of the uncertain data is described by a so-called uncertainty set. In many scenarios, a decision maker may influence the uncertainty regime she is facing, for example, by investing in market research, or in machines whi…
▽ More
In robust optimization one seeks to make a decision under uncertainty, where the goal is to find the solution with the best worst-case performance. The set of possible realizations of the uncertain data is described by a so-called uncertainty set. In many scenarios, a decision maker may influence the uncertainty regime she is facing, for example, by investing in market research, or in machines which work with higher precision. Recently, this situation was addressed in the literature by introducing decision dependent uncertainty sets (endogenous uncertainty), i.e., uncertainty sets whose structure depends on (typically discrete) decision variables. In this way, one can model the trade-off between reducing the cost of robustness versus the cost of the investment necessary for influencing the uncertainty. However, there is another trade-off to be made here. With different uncertainty regimes, not only do the worst-case optimal solutions vary, but also other aspects of that solutions such as max-regret, best-case performance or predictability of the performance. A decision maker may still be interested in having a performance guarantee, but at the same time be willing to forgo superior worst-case performance if those other aspects can be enhanced by switching to a suitable uncertainty regime. We introduce the notion of uncertainty preference in order to capture such stances. We present three ways to formalize uncertainty preferences and study the resulting mathematical models. The goal is to have reformulations/approximations of these models which can be solved with standard methods. The workhorse is mixed-integer linear and conic optimization. We apply our framework to the uncertain shortest path problem and conduct numerical experiments for the resulting models. We can demonstrate that our models can be handled very well by standard mixed-integer linear solvers.
△ Less
Submitted 24 January, 2022; v1 submitted 30 November, 2020;
originally announced November 2020.
-
Active set complexity of the Away-step Frank-Wolfe Algorithm
Authors:
Immanuel M. Bomze,
Francesco Rinaldi,
Damiano Zeffiro
Abstract:
In this paper, we study active set identification results for the away-step Frank-Wolfe algorithm in different settings. We first prove a local identification property that we apply, in combination with a convergence hypothesis, to get an active set identification result. We then prove, in the nonconvex case, a novel $O(1/\sqrt{k})$ convergence rate result and active set identification for differe…
▽ More
In this paper, we study active set identification results for the away-step Frank-Wolfe algorithm in different settings. We first prove a local identification property that we apply, in combination with a convergence hypothesis, to get an active set identification result. We then prove, in the nonconvex case, a novel $O(1/\sqrt{k})$ convergence rate result and active set identification for different stepsizes (under suitable assumptions on the set of stationary points). By exploiting those results, we also give explicit active set complexity bounds for both strongly convex and nonconvex objectives. While we initially consider the probability simplex as feasible set, in the appendix we show how to adapt some of our results to generic polytopes.
△ Less
Submitted 24 December, 2019;
originally announced December 2019.
-
Hessian barrier algorithms for linearly constrained optimization problems
Authors:
Immanuel M. Bomze,
Panayotis Mertikopoulos,
Werner Schachinger,
Mathias Staudigl
Abstract:
In this paper, we propose an interior-point method for linearly constrained optimization problems (possibly nonconvex). The method - which we call the Hessian barrier algorithm (HBA) - combines a forward Euler discretization of Hessian Riemannian gradient flows with an Armijo backtracking step-size policy. In this way, HBA can be seen as an alternative to mirror descent (MD), and contains as speci…
▽ More
In this paper, we propose an interior-point method for linearly constrained optimization problems (possibly nonconvex). The method - which we call the Hessian barrier algorithm (HBA) - combines a forward Euler discretization of Hessian Riemannian gradient flows with an Armijo backtracking step-size policy. In this way, HBA can be seen as an alternative to mirror descent (MD), and contains as special cases the affine scaling algorithm, regularized Newton processes, and several other iterative solution methods. Our main result is that, modulo a non-degeneracy condition, the algorithm converges to the problem's set of critical points; hence, in the convex case, the algorithm converges globally to the problem's minimum set. In the case of linearly constrained quadratic programs (not necessarily convex), we also show that the method's convergence rate is $\mathcal{O}(1/k^ρ)$ for some $ρ\in(0,1]$ that depends only on the choice of kernel function (i.e., not on the problem's primitives). These theoretical results are validated by numerical experiments in standard non-convex test functions and large-scale traffic assignment problems.
△ Less
Submitted 8 May, 2019; v1 submitted 25 September, 2018;
originally announced September 2018.
-
Extended Trust-Region Problems with One or Two Balls: Exact Copositive and Lagrangian Relaxations
Authors:
I. M. Bomze,
V. Jeyakumar,
G. Li
Abstract:
We establish a geometric condition guaranteeing exact copositive relaxation for the nonconvex quadratic optimization problem under two quadratic and several linear constraints, and present sufficient conditions for global optimality in terms of generalized Karush-Kuhn-Tucker multipliers. The copositive relaxation is tighter than the usual Lagrangian relaxation. We illustrate this by providing a wh…
▽ More
We establish a geometric condition guaranteeing exact copositive relaxation for the nonconvex quadratic optimization problem under two quadratic and several linear constraints, and present sufficient conditions for global optimality in terms of generalized Karush-Kuhn-Tucker multipliers. The copositive relaxation is tighter than the usual Lagrangian relaxation. We illustrate this by providing a whole class of quadratic optimization problems that enjoys exactness of copositive relaxation while the usual Lagrangian duality gap is infinite. Finally, we also provide verifiable conditions under which both the usual Lagrangian relaxation and the copositive relaxation are exact for an extended CDT (two-ball trust-region) problem. Importantly, the sufficient conditions can be verified by solving linear optimization problems.
△ Less
Submitted 2 October, 2017; v1 submitted 26 February, 2017;
originally announced February 2017.
-
New results on the cp rank and related properties of co(mpletely)positive matrices
Authors:
Naomi Shaked-Monderer,
Abraham Berman,
Immanuel M. Bomze,
Florian Jarre,
Werner Schachinger
Abstract:
Copositive and completely positive matrices play an increasingly important role in Applied Mathematics, namely as a key concept for approximating NP-hard optimization problems. The cone of copositive matrices of a given order and the cone of completely positive matrices of the same order are dual to each other with respect to the standard scalar product on the space of symmetric matrices. This pap…
▽ More
Copositive and completely positive matrices play an increasingly important role in Applied Mathematics, namely as a key concept for approximating NP-hard optimization problems. The cone of copositive matrices of a given order and the cone of completely positive matrices of the same order are dual to each other with respect to the standard scalar product on the space of symmetric matrices. This paper establishes some new relations between orthogonal pairs of such matrices lying on the boundary of either cone. As a consequence, we can establish an improvement on the upper bound of the cp-rank of completely positive matrices of general order, and a further improvement for such matrices of order six.
△ Less
Submitted 23 November, 2013; v1 submitted 27 April, 2013;
originally announced May 2013.
-
Improving SDP bounds for minimizing quadratic functions over the l1-ball
Authors:
Immanuel M. Bomze,
Florian Frommlet,
Martin Rubey
Abstract:
In this note, we establish superiority of the so-called copositive bound over a bound suggested by Nesterov for the quadratic problem to minimize a quadratic form over the l1-ball. We illustrate the improvement by simulation results. The copositive bound has the additional advantage that it can be easily extended to the inhomogeneous case of quadratic objectives including a linear term. We also…
▽ More
In this note, we establish superiority of the so-called copositive bound over a bound suggested by Nesterov for the quadratic problem to minimize a quadratic form over the l1-ball. We illustrate the improvement by simulation results. The copositive bound has the additional advantage that it can be easily extended to the inhomogeneous case of quadratic objectives including a linear term. We also indicate some improvements of the eigenvalue bound for the quadratic optimization over the lp-ball with 1<p<2, at least for p close to one.
△ Less
Submitted 22 March, 2005; v1 submitted 9 March, 2005;
originally announced March 2005.
