Showing 1–28 of 28 results for author: Kilinc-Karzan, F

On the strength of Burer's lifted convex relaxation to quadratic programming with ball constraints

Authors: Fatma Kılınç-Karzan, Shengding Sun

Abstract: We study quadratic programs with $m$ ball constraints, and the strength of a lifted convex relaxation for it recently proposed by Burer (2024). Burer shows this relaxation is exact when $m=2$. For general $m$, Burer (2024) provides numerical evidence that this lifted relaxation is tighter than the Kronecker product based Reformulation Linearization Technique (RLT) inequalities introduced by Anstre… ▽ More We study quadratic programs with $m$ ball constraints, and the strength of a lifted convex relaxation for it recently proposed by Burer (2024). Burer shows this relaxation is exact when $m=2$. For general $m$, Burer (2024) provides numerical evidence that this lifted relaxation is tighter than the Kronecker product based Reformulation Linearization Technique (RLT) inequalities introduced by Anstreicher (2017), and conjectures that this must be theoretically true as well. In this note, we provide an affirmative answer to this question and formally prove that this lifted relaxation indeed implies the Kronecker inequalities. Our proof is based on a decomposition of non-rank-one extreme rays of the lifted relaxation for each pair of ball constraints. Burer (2024) also numerically observes that for this lifted relaxation, an RLT-based inequality proposed by Zhen et al. (2021) is redundant, and conjectures this to be theoretically true as well. We also provide a formal proof that Zhen et al. (2021) inequalities are redundant for this lifted relaxation. In addition, we establish that Burer's lifted relaxation is a particular case of the moment-sum-of-squares hierarchy. △ Less

Submitted 20 July, 2024; originally announced July 2024.

Comments: 14 pages

Mistake, Manipulation and Margin Guarantees in Online Strategic Classification

Authors: Lingqing Shen, Nam Ho-Nguyen, Khanh-Hung Giang-Tran, Fatma Kılınç-Karzan

Abstract: We consider an online strategic classification problem where each arriving agent can manipulate their true feature vector to obtain a positive predicted label, while incurring a cost that depends on the amount of manipulation. The learner seeks to predict the agent's true label given access to only the manipulated features. After the learner releases their prediction, the agent's true label is rev… ▽ More We consider an online strategic classification problem where each arriving agent can manipulate their true feature vector to obtain a positive predicted label, while incurring a cost that depends on the amount of manipulation. The learner seeks to predict the agent's true label given access to only the manipulated features. After the learner releases their prediction, the agent's true label is revealed. Previous algorithms such as the strategic perceptron guarantee finitely many mistakes under a margin assumption on agents' true feature vectors. However, these are not guaranteed to encourage agents to be truthful. Promoting truthfulness is intimately linked to obtaining adequate margin on the predictions, thus we provide two new algorithms aimed at recovering the maximum margin classifier in the presence of strategic agent behavior. We prove convergence, finite mistake and finite manipulation guarantees for a variety of agent cost structures. We also provide generalized versions of the strategic perceptron with mistake guarantees for different costs. Our numerical study on real and synthetic data demonstrates that the new algorithms outperform previous ones in terms of margin, number of manipulation and number of mistakes. △ Less

Submitted 26 March, 2024; originally announced March 2024.

On semidefinite descriptions for convex hulls of quadratic programs

Authors: Alex L. Wang, Fatma Kilinc-Karzan

Abstract: Quadratically constrained quadratic programs (QCQPs) are a highly expressive class of nonconvex optimization problems. While QCQPs are NP-hard in general, they admit a natural convex relaxation via the standard semidefinite program (SDP) relaxation. In this paper we study when the convex hull of the epigraph of a QCQP coincides with the projected epigraph of the SDP relaxation. We present a suffic… ▽ More Quadratically constrained quadratic programs (QCQPs) are a highly expressive class of nonconvex optimization problems. While QCQPs are NP-hard in general, they admit a natural convex relaxation via the standard semidefinite program (SDP) relaxation. In this paper we study when the convex hull of the epigraph of a QCQP coincides with the projected epigraph of the SDP relaxation. We present a sufficient condition for convex hull exactness and show that this condition is further necessary under an additional geometric assumption. The sufficient condition is based on geometric properties of $Γ$, the cone of convex Lagrange multipliers, and its relatives $Γ_1$ and $Γ^\circ$. △ Less

Submitted 20 March, 2024; v1 submitted 7 March, 2024; originally announced March 2024.

Comments: This paper is a significant rewrite of arXiv:2011.07155 [math.OC] and contains both new content and rewritten content

A Field Test of Bandit Algorithms for Recommendations: Understanding the Validity of Assumptions on Human Preferences in Multi-armed Bandits

Authors: Liu Leqi, Giulio Zhou, Fatma Kılınç-Karzan, Zachary C. Lipton, Alan L. Montgomery

Abstract: Personalized recommender systems suffuse modern life, shaping what media we read and what products we consume. Algorithms powering such systems tend to consist of supervised learning-based heuristics, such as latent factor models with a variety of heuristically chosen prediction targets. Meanwhile, theoretical treatments of recommendation frequently address the decision-theoretic nature of the pro… ▽ More Personalized recommender systems suffuse modern life, shaping what media we read and what products we consume. Algorithms powering such systems tend to consist of supervised learning-based heuristics, such as latent factor models with a variety of heuristically chosen prediction targets. Meanwhile, theoretical treatments of recommendation frequently address the decision-theoretic nature of the problem, including the need to balance exploration and exploitation, via the multi-armed bandits (MABs) framework. However, MAB-based approaches rely heavily on assumptions about human preferences. These preference assumptions are seldom tested using human subject studies, partly due to the lack of publicly available toolkits to conduct such studies. In this work, we conduct a study with crowdworkers in a comics recommendation MABs setting. Each arm represents a comic category, and users provide feedback after each recommendation. We check the validity of core MABs assumptions-that human preferences (reward distributions) are fixed over time-and find that they do not hold. This finding suggests that any MAB algorithm used for recommender systems should account for human preference dynamics. While answering these questions, we provide a flexible experimental framework for understanding human preference dynamics and testing MABs algorithms with human users. The code for our experimental framework and the collected data can be found at https://github.com/HumainLab/human-bandit-evaluation. △ Less

Submitted 16 April, 2023; originally announced April 2023.

Comments: Accepted to CHI. 16 pages, 6 figures

Constrained Optimization of Rank-One Functions with Indicator Variables

Authors: Soroosh Shafiee, Fatma Kılınç-Karzan

Abstract: Optimization problems involving minimization of a rank-one convex function over constraints modeling restrictions on the support of the decision variables emerge in various machine learning applications. These problems are often modeled with indicator variables for identifying the support of the continuous variables. In this paper we investigate compact extended formulations for such problems thro… ▽ More Optimization problems involving minimization of a rank-one convex function over constraints modeling restrictions on the support of the decision variables emerge in various machine learning applications. These problems are often modeled with indicator variables for identifying the support of the continuous variables. In this paper we investigate compact extended formulations for such problems through perspective reformulation techniques. In contrast to the majority of previous work that relies on support function arguments and disjunctive programming techniques to provide convex hull results, we propose a constructive approach that exploits a hidden conic structure induced by perspective functions. To this end, we first establish a convex hull result for a general conic mixed-binary set in which each conic constraint involves a linear function of independent continuous variables and a set of binary variables. We then demonstrate that extended representations of sets associated with epigraphs of rank-one convex functions over constraints modeling indicator relations naturally admit such a conic representation. This enables us to systematically give perspective formulations for the convex hull descriptions of these sets with nonlinear separable or non-separable objective functions, sign constraints on continuous variables, and combinatorial constraints on indicator variables. We illustrate the efficacy of our results on sparse nonnegative logistic regression problems. △ Less

Submitted 27 November, 2023; v1 submitted 31 March, 2023; originally announced March 2023.

Accelerated first-order methods for a class of semidefinite programs

Authors: Alex L. Wang, Fatma Kilinc-Karzan

Abstract: This paper introduces a new storage-optimal first-order method (FOM), CertSDP, for solving a special class of semidefinite programs (SDPs) to high accuracy. The class of SDPs that we consider, the exact QMP-like SDPs, is characterized by low-rank solutions, a priori knowledge of the restriction of the SDP solution to a small subspace, and standard regularity assumptions such as strict complementar… ▽ More This paper introduces a new storage-optimal first-order method (FOM), CertSDP, for solving a special class of semidefinite programs (SDPs) to high accuracy. The class of SDPs that we consider, the exact QMP-like SDPs, is characterized by low-rank solutions, a priori knowledge of the restriction of the SDP solution to a small subspace, and standard regularity assumptions such as strict complementarity. Crucially, we show how to use a certificate of strict complementarity to construct a low-dimensional strongly convex minimax problem whose optimizer coincides with a factorization of the SDP optimizer. From an algorithmic standpoint, we show how to construct the necessary certificate and how to solve the minimax problem efficiently. We accompany our theoretical results with preliminary numerical experiments suggesting that CertSDP significantly outperforms current state-of-the-art methods on large sparse exact QMP-like SDPs. △ Less

Submitted 3 March, 2024; v1 submitted 1 June, 2022; originally announced June 2022.

Implicit regularity and linear convergence rates for the generalized trust-region subproblem

Authors: Alex L. Wang, Yunlei Lu, Fatma Kilinc-Karzan

Abstract: In this paper we develop efficient first-order algorithms for the generalized trust-region subproblem (GTRS), which has applications in signal processing, compressed sensing, and engineering. Although the GTRS, as stated, is nonlinear and nonconvex, it is well-known that objective value exactness holds for its SDP relaxation under a Slater condition. While polynomial-time SDP-based algorithms exis… ▽ More In this paper we develop efficient first-order algorithms for the generalized trust-region subproblem (GTRS), which has applications in signal processing, compressed sensing, and engineering. Although the GTRS, as stated, is nonlinear and nonconvex, it is well-known that objective value exactness holds for its SDP relaxation under a Slater condition. While polynomial-time SDP-based algorithms exist for the GTRS, their relatively large computational complexity has motivated and spurred the development of custom approaches for solving the GTRS. In particular, recent work in this direction has developed first-order methods for the GTRS whose running times are linear in the sparsity (the number of nonzero entries) of the input data. In contrast to these algorithms, in this paper we develop algorithms for computing $ε$-approximate solutions to the GTRS whose running times are linear in both the input sparsity and the precision $\log(1/ε)$ whenever a regularity parameter is positive. We complement our theoretical guarantees with numerical experiments comparing our approach against algorithms from the literature. Our numerical experiments highlight that our new algorithms significantly outperform prior state-of-the-art algorithms on sparse large-scale instances. △ Less

Submitted 27 December, 2021; originally announced December 2021.

Exactness in SDP relaxations of QCQPs: Theory and applications

Authors: Fatma Kılınç-Karzan, Alex L. Wang

Abstract: Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems. In a QCQP, we are asked to minimize a (possibly nonconvex) quadratic function subject to a number of (possibly nonconvex) quadratic constraints. Such problems arise naturally in many areas of operations research, computer science, and engineering. Although QCQPs are NP-hard to solve in general, t… ▽ More Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems. In a QCQP, we are asked to minimize a (possibly nonconvex) quadratic function subject to a number of (possibly nonconvex) quadratic constraints. Such problems arise naturally in many areas of operations research, computer science, and engineering. Although QCQPs are NP-hard to solve in general, they admit a natural convex relaxation via the standard (Shor) semidefinite program (SDP) relaxation. In this tutorial, we will study the SDP relaxation for general QCQPs, present various exactness concepts related to this relaxation and discuss conditions guaranteeing such SDP exactness. In particular, we will define and examine three notions of SDP exactness: (i) objective value exactness -- the condition that the optimal value of the QCQP and the optimal value of its SDP relaxation coincide, (ii) convex hull exactness -- the condition that the convex hull of the QCQP epigraph coincides with the (projected) SDP epigraph, and (iii) the rank-one generated (ROG) property -- the condition that a particular conic subset of the positive semidefinite matrices related to a given QCQP is generated by its rank-one matrices. Our analysis for objective value exactness and convex hull exactness stems from a geometric treatment of the projected SDP relaxation and crucially considers how the objective function interacts with the constraints. The ROG property complements these results by offering a sufficient condition for both objective value exactness and convex hull exactness which is oblivious to the objective function. We will give a variety of sufficient conditions for these exactness conditions and discuss settings where these sufficient conditions are additionally necessary. Throughout, we will highlight implications of our results for a number of example applications. △ Less

Submitted 14 July, 2021; originally announced July 2021.

Risk Guarantees for End-to-End Prediction and Optimization Processes

Authors: Nam Ho-Nguyen, Fatma Kılınç-Karzan

Abstract: Prediction models are often employed in estimating parameters of optimization models. Despite the fact that in an end-to-end view, the real goal is to achieve good optimization performance, the prediction performance is measured on its own. While it is usually believed that good prediction performance in estimating the parameters will result in good subsequent optimization performance, formal theo… ▽ More Prediction models are often employed in estimating parameters of optimization models. Despite the fact that in an end-to-end view, the real goal is to achieve good optimization performance, the prediction performance is measured on its own. While it is usually believed that good prediction performance in estimating the parameters will result in good subsequent optimization performance, formal theoretical guarantees on this are notably lacking. In this paper, we explore conditions that allow us to explicitly describe how the prediction performance governs the optimization performance. Our weaker condition allows for an asymptotic convergence result, while our stronger condition allows for exact quantification of the optimization performance in terms of the prediction performance. In general, verification of these conditions is a non-trivial task. Nevertheless, we show that our weaker condition is equivalent to the well-known Fisher consistency concept from the learning theory literature. This then allows us to easily check our weaker condition for several loss functions. We also establish that the squared error loss function satisfies our stronger condition. Consequently, we derive the exact theoretical relationship between prediction performance measured with the squared loss, as well as a class of symmetric loss functions, and the subsequent optimization performance. In a computational study on portfolio optimization, fractional knapsack and multiclass classification problems, we compare the optimization performance of using of several prediction loss functions (some that are Fisher consistent and some that are not) and demonstrate that lack of consistency of the loss function can indeed have a detrimental effect on performance. △ Less

Submitted 30 December, 2020; originally announced December 2020.

Conic Mixed-Binary Sets: Convex Hull Characterizations and Applications

Authors: Fatma Kılınç-Karzan, Simge Küçükyavuz, Dabeen Lee, Soroosh Shafieezadeh-Abadeh

Abstract: We consider a general conic mixed-binary set where each homogeneous conic constraint $j$ involves an affine function of independent continuous variables and an epigraph variable associated with a nonnegative function, $f_j$, of common binary variables. Sets of this form naturally arise as substructures in a number of applications including mean-risk optimization, chance-constrained problems, portf… ▽ More We consider a general conic mixed-binary set where each homogeneous conic constraint $j$ involves an affine function of independent continuous variables and an epigraph variable associated with a nonnegative function, $f_j$, of common binary variables. Sets of this form naturally arise as substructures in a number of applications including mean-risk optimization, chance-constrained problems, portfolio optimization, lot-sizing and scheduling, fractional programming, variants of the best subset selection problem, a class of sparse semidefinite programs, and distributionally robust chance-constrained programs. We give a convex hull description of this set that relies on simultaneous characterization of the epigraphs of $f_j$'s, which is easy to do when all functions $f_j$'s are submodular. Our result unifies and generalizes an existing result in two important directions. First, it considers \emph{multiple general convex cone} constraints instead of a single second-order cone type constraint. Second, it takes \emph{arbitrary nonnegative functions} instead of a specific submodular function obtained from the square root of an affine function. We close by demonstrating the applicability of our results in the context of a number of problem classes. △ Less

Submitted 28 December, 2023; v1 submitted 29 December, 2020; originally announced December 2020.

Comments: 21 pages

MSC Class: 90C10; 90C11; 90C20; 90C27; 90C30

A Geometric View of SDP Exactness in QCQPs and its Applications

Authors: Alex L. Wang, Fatma Kilinc-Karzan

Abstract: Quadratically constrained quadratic programs (QCQPs) are a highly expressive class of nonconvex optimization problems. While QCQPs are NP-hard in general, they admit a natural convex relaxation via the standard (Shor) semidefinite program (SDP) relaxation. Towards understanding when this relaxation is exact, we study general QCQPs and their (projected) SDP relaxations. We present sufficient (and i… ▽ More Quadratically constrained quadratic programs (QCQPs) are a highly expressive class of nonconvex optimization problems. While QCQPs are NP-hard in general, they admit a natural convex relaxation via the standard (Shor) semidefinite program (SDP) relaxation. Towards understanding when this relaxation is exact, we study general QCQPs and their (projected) SDP relaxations. We present sufficient (and in some cases, also necessary) conditions for objective value exactness (the condition that the objective values of the QCQP and its SDP relaxation coincide) and convex hull exactness (the condition that the convex hull of the QCQP epigraph coincides with the epigraph of its SDP relaxation). Our conditions for exactness are based on geometric properties of $Γ$, the cone of convex Lagrange multipliers, and its relatives $Γ_P$ and $Γ^\circ$. These tools form the basis of our main message: questions of exactness can be treated systematically whenever $Γ$, $Γ_P$, or $Γ^\circ$ is well-understood. As further evidence of this message, we apply our tools to address questions of exactness for a prototypical QCQP involving a binary on-off constraint, quadratic matrix programs, the QCQP formulation of the partition problem, and random and semi-random QCQPs. △ Less

Submitted 26 November, 2021; v1 submitted 13 November, 2020; originally announced November 2020.

Rebounding Bandits for Modeling Satiation Effects

Authors: Liu Leqi, Fatma Kilinc-Karzan, Zachary C. Lipton, Alan L. Montgomery

Abstract: Psychological research shows that enjoyment of many goods is subject to satiation, with short-term satisfaction declining after repeated exposures to the same item. Nevertheless, proposed algorithms for powering recommender systems seldom model these dynamics, instead proceeding as though user preferences were fixed in time. In this work, we introduce rebounding bandits, a multi-armed bandit setup… ▽ More Psychological research shows that enjoyment of many goods is subject to satiation, with short-term satisfaction declining after repeated exposures to the same item. Nevertheless, proposed algorithms for powering recommender systems seldom model these dynamics, instead proceeding as though user preferences were fixed in time. In this work, we introduce rebounding bandits, a multi-armed bandit setup, where satiation dynamics are modeled as time-invariant linear dynamical systems. Expected rewards for each arm decline monotonically with consecutive exposures to it and rebound towards the initial reward whenever that arm is not pulled. Unlike classical bandit settings, methods for tackling rebounding bandits must plan ahead and model-based methods rely on estimating the parameters of the satiation dynamics. We characterize the planning problem, showing that the greedy policy is optimal when the arms exhibit identical deterministic dynamics. To address stochastic satiation dynamics with unknown parameters, we propose Explore-Estimate-Plan (EEP), an algorithm that pulls arms methodically, estimates the system dynamics, and then plans accordingly. △ Less

Submitted 27 October, 2021; v1 submitted 12 November, 2020; originally announced November 2020.

Online Convex Optimization Perspective for Learning from Dynamically Revealed Preferences

Authors: Violet Xinying Chen, Fatma Kılınç-Karzan

Abstract: We study the problem of online learning (OL) from revealed preferences: a learner wishes to learn a non-strategic agent's private utility function through observing the agent's utility-maximizing actions in a changing environment. We adopt an online inverse optimization setup, where the learner observes a stream of agent's actions in an online fashion and the learning performance is measured by re… ▽ More We study the problem of online learning (OL) from revealed preferences: a learner wishes to learn a non-strategic agent's private utility function through observing the agent's utility-maximizing actions in a changing environment. We adopt an online inverse optimization setup, where the learner observes a stream of agent's actions in an online fashion and the learning performance is measured by regret associated with a loss function. We first characterize a special but broad class of agent's utility functions, then utilize this structure in designing a new convex loss function. We establish that the regret with respect to our new loss function also bounds the regret with respect to all other usual loss functions in the literature. This allows us to design a flexible OL framework that enables a unified treatment of loss functions and supports a variety of online convex optimization algorithms. We demonstrate with theoretical and empirical evidence that our framework based on the new loss function (in particular online Mirror Descent) has significant advantages in terms of regret performance and solution time over other OL algorithms from the literature and bypasses the previous technical assumptions as well. △ Less

Submitted 4 June, 2021; v1 submitted 24 August, 2020; originally announced August 2020.

Comments: 34 pages, 9 figures

Necessary and sufficient conditions for rank-one generated cones

Authors: C. J. Argue, Fatma Kılınç-Karzan, Alex L. Wang

Abstract: A closed convex conic subset $\mathcal{S}$ of the positive semidefinite (PSD) cone is rank-one generated (ROG) if all of its extreme rays are generated by rank-one matrices. The ROG property of $\mathcal{S}$ is closely related to the exactness of SDP relaxations of nonconvex quadratically constrained quadratic programs (QCQPs) related to $\mathcal{S}$. We consider the case where $\mathcal{S}$ is o… ▽ More A closed convex conic subset $\mathcal{S}$ of the positive semidefinite (PSD) cone is rank-one generated (ROG) if all of its extreme rays are generated by rank-one matrices. The ROG property of $\mathcal{S}$ is closely related to the exactness of SDP relaxations of nonconvex quadratically constrained quadratic programs (QCQPs) related to $\mathcal{S}$. We consider the case where $\mathcal{S}$ is obtained as the intersection of the PSD cone with finitely many homogeneous linear matrix inequalities and conic constraints and identify sufficient conditions that guarantee that $\mathcal{S}$ is ROG. Our general framework allows us to recover a number of well-known results from the literature. In the case of two linear matrix inequalities, we also establish the necessity of our sufficient conditions. This extends one of the few settings from the literature -- the case of one linear matrix inequality and the S-lemma -- where an explicit characterization for the ROG property exists. Finally, we show how our ROG results on cones can be translated into inhomogeneous SDP exactness results and convex hull descriptions in the original space of a QCQP. We close with a few applications of these results; specifically, we recover the well-known perspective reformulation of a simple mixed-binary set via the ROG toolkit. △ Less

Submitted 25 May, 2021; v1 submitted 14 July, 2020; originally announced July 2020.

Strong Formulations for Distributionally Robust Chance-Constrained Programs with Left-Hand Side Uncertainty under Wasserstein Ambiguity

Authors: Nam Ho-Nguyen, Fatma Kılınç-Karzan, Simge Küçükyavuz, Dabeen Lee

Abstract: Distributionally robust chance-constrained programs (DR-CCP) over Wasserstein ambiguity sets exhibit attractive out-of-sample performance and admit big-$M$-based mixed-integer programming (MIP) reformulations with conic constraints. However, the resulting formulations often suffer from scalability issues as sample size increases. To address this shortcoming, we derive stronger formulations that sc… ▽ More Distributionally robust chance-constrained programs (DR-CCP) over Wasserstein ambiguity sets exhibit attractive out-of-sample performance and admit big-$M$-based mixed-integer programming (MIP) reformulations with conic constraints. However, the resulting formulations often suffer from scalability issues as sample size increases. To address this shortcoming, we derive stronger formulations that scale well with respect to the sample size. Our focus is on ambiguity sets under the so-called left-hand side (LHS) uncertainty, where the uncertain parameters affect the coefficients of the decision variables in the linear inequalities defining the safety sets. The interaction between the uncertain parameters and the variable coefficients in the safety set definition causes challenges in strengthening the original big-$M$ formulations. By exploiting the connection between nominal chance-constrained programs and DR-CCP, we obtain strong formulations with significant enhancements. In particular, through this connection, we derive a linear number of valid inequalities, which can be immediately added to the formulations to obtain improved formulations in the original space of variables. In addition, we suggest a quantile-based strengthening procedure that allows us to reduce the big-$M$ coefficients drastically. Furthermore, based on this procedure, we propose an exponential class of inequalities that can be separated efficiently within a branch-and-cut framework. The quantile-based strengthening procedure can be expensive. Therefore, for the special case of covering and packing type problems, we identify an efficient scheme to carry out this procedure. We demonstrate the computational efficacy of our proposed formulations on two classes of problems, namely stochastic portfolio optimization and resource planning. △ Less

Submitted 13 January, 2021; v1 submitted 13 July, 2020; originally announced July 2020.

MSC Class: 90C17; 90C15; 90C11; 90C57

Distributionally Robust Chance-Constrained Programs with Right-Hand Side Uncertainty under Wasserstein Ambiguity

Authors: Nam Ho-Nguyen, Fatma Kılınç-Karzan, Simge Küçükyavuz, Dabeen Lee

Abstract: We consider exact deterministic mixed-integer programming (MIP) reformulations of distributionally robust chance-constrained programs (DR-CCP) with random right-hand sides over Wasserstein ambiguity sets. The existing MIP formulations are known to have weak continuous relaxation bounds, and, consequently, for hard instances with small radius, or with large problem sizes, the branch-and-bound based… ▽ More We consider exact deterministic mixed-integer programming (MIP) reformulations of distributionally robust chance-constrained programs (DR-CCP) with random right-hand sides over Wasserstein ambiguity sets. The existing MIP formulations are known to have weak continuous relaxation bounds, and, consequently, for hard instances with small radius, or with large problem sizes, the branch-and-bound based solution processes suffer from large optimality gaps even after hours of computation time. This significantly hinders the practical application of the DR-CCP paradigm. Motivated by these challenges, we conduct a polyhedral study to strengthen these formulations. We reveal several hidden connections between DR-CCP and its nominal counterpart (the sample average approximation), mixing sets, and robust 0-1 programming. By exploiting these connections in combination, we provide an improved formulation and two classes of valid inequalities for DR-CCP. We test the impact of our results on a stochastic transportation problem numerically. Our experiments demonstrate the effectiveness of our approach; in particular our improved formulation and proposed valid inequalities reduce the overall solution times remarkably. Moreover, this allows us to significantly scale up the problem sizes that can be handled in such DR-CCP formulations by reducing the solution times from hours to seconds. △ Less

Submitted 7 December, 2020; v1 submitted 27 March, 2020; originally announced March 2020.

Comments: 27 pages

MSC Class: 90C11; 90C15

On convex hulls of epigraphs of QCQPs

Authors: Alex L. Wang, Fatma Kilinc-Karzan

Abstract: Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems well-known to be NP-hard in general. In this paper we study sufficient conditions for a convex hull result that immediately implies that the standard semidefinite program (SDP) relaxation of a QCQP is tight. We begin by outlining a general framework for proving such sufficient conditions. Then usi… ▽ More Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems well-known to be NP-hard in general. In this paper we study sufficient conditions for a convex hull result that immediately implies that the standard semidefinite program (SDP) relaxation of a QCQP is tight. We begin by outlining a general framework for proving such sufficient conditions. Then using this framework, we show that the convex hull result holds whenever the quadratic eigenvalue multiplicity, a parameter capturing the amount of symmetry present in a given problem, is large enough. Our results also imply new sufficient conditions for the tightness (as well as convex hull exactness) of a second order cone program relaxation of simultaneously diagonalizable QCQPs. △ Less

Submitted 4 February, 2020; originally announced February 2020.

Comments: To appear at IPCO 2020. This conference extended abstract covers material already online at arXiv:1911.09195

On the tightness of SDP relaxations of QCQPs

Authors: Alex L. Wang, Fatma Kilinc-Karzan

Abstract: Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems well-known to be NP-hard in general. In this paper we study conditions under which the standard semidefinite program (SDP) relaxation of a QCQP is tight. We begin by outlining a general framework for proving such sufficient conditions. Then using this framework, we show that the SDP relaxation is… ▽ More Quadratically constrained quadratic programs (QCQPs) are a fundamental class of optimization problems well-known to be NP-hard in general. In this paper we study conditions under which the standard semidefinite program (SDP) relaxation of a QCQP is tight. We begin by outlining a general framework for proving such sufficient conditions. Then using this framework, we show that the SDP relaxation is tight whenever the quadratic eigenvalue multiplicity, a parameter capturing the amount of symmetry present in a given problem, is large enough. We present similar sufficient conditions under which the projected epigraph of the SDP gives the convex hull of the epigraph in the original QCQP. Our results also imply new sufficient conditions for the tightness (as well as convex hull exactness) of a second order cone program relaxation of simultaneously diagonalizable QCQPs. △ Less

Submitted 13 November, 2020; v1 submitted 20 November, 2019; originally announced November 2019.

Joint chance-constrained programs and the intersection of mixing sets through a submodularity lens

Authors: Fatma Kılınç-Karzan, Simge Küçükyavuz, Dabeen Lee

Abstract: A particularly important substructure in modeling joint linear chance-constrained programs with random right-hand sides and finite sample space is the intersection of mixing sets with common binary variables (and possibly a knapsack constraint). In this paper, we first revisit basic mixing sets by establishing a strong and previously unrecognized connection to submodularity. In particular, we show… ▽ More A particularly important substructure in modeling joint linear chance-constrained programs with random right-hand sides and finite sample space is the intersection of mixing sets with common binary variables (and possibly a knapsack constraint). In this paper, we first revisit basic mixing sets by establishing a strong and previously unrecognized connection to submodularity. In particular, we show that mixing inequalities with binary variables are nothing but the polymatroid inequalities associated with a specific submodular function. This submodularity viewpoint enables us to unify and extend existing results on valid inequalities and convex hulls of the intersection of multiple mixing sets with common binary variables. Then, we study such intersections under an additional linking constraint lower bounding a linear function of the continuous variables. This is motivated from the desire to exploit the information encoded in the knapsack constraint arising in joint linear CCPs via the quantile cuts. We propose a new class of valid inequalities and characterize when this new class along with the mixing inequalities are sufficient to describe the convex hull. △ Less

Submitted 28 June, 2021; v1 submitted 3 October, 2019; originally announced October 2019.

MSC Class: 90C11; 90C15

The Generalized Trust Region Subproblem: solution complexity and convex hull results

Authors: Alex L. Wang, Fatma Kilinc-Karzan

Abstract: We consider the Generalized Trust Region Subproblem (GTRS) of minimizing a nonconvex quadratic objective over a nonconvex quadratic constraint. A lifting of this problem recasts the GTRS as minimizing a linear objective subject to two nonconvex quadratic constraints. Our first main contribution is structural: we give an explicit description of the convex hull of this nonconvex set in terms of the… ▽ More We consider the Generalized Trust Region Subproblem (GTRS) of minimizing a nonconvex quadratic objective over a nonconvex quadratic constraint. A lifting of this problem recasts the GTRS as minimizing a linear objective subject to two nonconvex quadratic constraints. Our first main contribution is structural: we give an explicit description of the convex hull of this nonconvex set in terms of the generalized eigenvalues of an associated matrix pencil. This result may be of interest in building relaxations for nonconvex quadratic programs. Moreover, this result allows us to reformulate the GTRS as the minimization of two convex quadratic functions in the original space. Our next set of contributions is algorithmic: we present an algorithm for solving the GTRS up to an epsilon additive error based on this reformulation. We carefully handle numerical issues that arise from inexact generalized eigenvalue and eigenvector computations and establish explicit running time guarantees for these algorithms. Notably, our algorithms run in linear (in the size of the input) time. Furthermore, our algorithm for computing an epsilon-optimal solution has a slightly-improved running time dependence on epsilon over the state-of-the-art algorithm. Our analysis shows that the dominant cost in solving the GTRS lies in solving a generalized eigenvalue problem -- establishing a natural connection between these problems. Finally, generalizations of our convex hull results allow us to apply our algorithms and their theoretical guarantees directly to equality-, interval-, and hollow- constrained variants of the GTRS. This gives the first linear-time algorithm in the literature for these variants of the GTRS. △ Less

Submitted 13 November, 2020; v1 submitted 20 July, 2019; originally announced July 2019.

Exploiting Problem Structure in Optimization under Uncertainty via Online Convex Optimization

Authors: Nam Ho-Nguyen, Fatma Kilinc-Karzan

Abstract: In this paper, we consider two paradigms that are developed to account for uncertainty in optimization models: robust optimization (RO) and joint estimation-optimization (JEO). We examine recent developments on efficient and scalable iterative first-order methods for these problems, and show that these iterative methods can be viewed through the lens of online convex optimization (OCO). The standa… ▽ More In this paper, we consider two paradigms that are developed to account for uncertainty in optimization models: robust optimization (RO) and joint estimation-optimization (JEO). We examine recent developments on efficient and scalable iterative first-order methods for these problems, and show that these iterative methods can be viewed through the lens of online convex optimization (OCO). The standard OCO framework has seen much success for its ability to handle decision-making in dynamic, uncertain, and even adversarial environments. Nevertheless, our applications of interest present further flexibility in OCO via three simple modifications to standard OCO assumptions: we introduce two new concepts of weighted regret and online saddle point problems and study the possibility of making lookahead (anticipatory) decisions. Our analyses demonstrate that these flexibilities introduced into the OCO framework have significant consequences whenever they are applicable. For example, in the strongly convex case, minimizing unweighted regret has a proven optimal bound of $O(\log(T)/T)$, whereas we show that a bound of $O(1/T)$ is possible when we consider weighted regret. Similarly, for the smooth case, considering $1$-lookahead decisions results in a $O(1/T)$ bound, compared to $O(1/\sqrt{T})$ in the standard OCO setting. Consequently, these OCO tools are instrumental in exploiting structural properties of functions and resulting in improved convergence rates for RO and JEO. In certain cases, our results for RO and JEO match the best known or optimal rates in the corresponding problem classes without data uncertainty. △ Less

Submitted 12 April, 2018; v1 submitted 7 September, 2017; originally announced September 2017.

MSC Class: 90C06; 90C25

Dynamic Data-Driven Estimation of Non-Parametric Choice Models

Authors: Nam Ho-Nguyen, Fatma Kilinc-Karzan

Abstract: We study non-parametric estimation of choice models, which were introduced to alleviate unreasonable assumptions in traditional parametric models, and are prevalent in several application areas. Existing literature focuses only on the static observational setting where all of the observations are given upfront, they are not equipped with explicit convergence rate guarantees, and consequently they… ▽ More We study non-parametric estimation of choice models, which were introduced to alleviate unreasonable assumptions in traditional parametric models, and are prevalent in several application areas. Existing literature focuses only on the static observational setting where all of the observations are given upfront, they are not equipped with explicit convergence rate guarantees, and consequently they cannot provide an a priori analysis for the model accuracy vs sparsity trade-off on the actual estimated model returned by their algorithms. As opposed to this, we focus on estimating a non-parametric choice model from observational data in a \emph{dynamic} setting, where observations are obtained over time. We show that choice model estimation can be cast as a convex-concave saddle-point (SP) joint estimation and optimization (JEO) problem, and we provide a primal-dual framework for deriving algorithms to solve this based on online convex optimization. By tailoring our framework carefully to the choice model estimation problem, we obtain tractable algorithms with provable convergence guarantees and explicit bounds on the sparsity of the estimated model. Our numerical experiments confirm the effectiveness of the algorithms derived from our framework. △ Less

Submitted 6 August, 2020; v1 submitted 19 February, 2017; originally announced February 2017.

MSC Class: 90B60; 90C25; 90C47

Theoretical and Practical Advances on Smoothing for Extensive-Form Games

Authors: Christian Kroer, Kevin Waugh, Fatma Kilinc-Karzan, Tuomas Sandholm

Abstract: Sparse iterative methods, in particular first-order methods, are known to be among the most effective in solving large-scale two-player zero-sum extensive-form games. The convergence rates of these methods depend heavily on the properties of the distance-generating function that they are based on. We investigate the acceleration of first-order methods for solving extensive-form games through bette… ▽ More Sparse iterative methods, in particular first-order methods, are known to be among the most effective in solving large-scale two-player zero-sum extensive-form games. The convergence rates of these methods depend heavily on the properties of the distance-generating function that they are based on. We investigate the acceleration of first-order methods for solving extensive-form games through better design of the dilated entropy function---a class of distance-generating functions related to the domains associated with the extensive-form games. By introducing a new weighting scheme for the dilated entropy function, we develop the first distance-generating function for the strategy spaces of sequential games that has no dependence on the branching factor of the player. This result improves the convergence rate of several first-order methods by a factor of $Ω(b^dd)$, where $b$ is the branching factor of the player, and $d$ is the depth of the game tree. Thus far, counterfactual regret minimization methods have been faster in practice, and more popular, than first-order methods despite their theoretically inferior convergence rates. Using our new weighting scheme and practical tuning we show that, for the first time, the excessive gap technique can be made faster than the fastest counterfactual regret minimization algorithm, CFR+, in practice. △ Less

Submitted 8 May, 2017; v1 submitted 15 February, 2017; originally announced February 2017.

Online First-Order Framework for Robust Convex Optimization

Authors: Nam Ho-Nguyen, Fatma Kilinc-Karzan

Abstract: Robust optimization (RO) has emerged as one of the leading paradigms to efficiently model parameter uncertainty. The recent connections between RO and problems in statistics and machine learning domains demand for solving RO problems in ever more larger scale. However, the traditional approaches for solving RO formulations based on building and solving robust counterparts or the iterative approach… ▽ More Robust optimization (RO) has emerged as one of the leading paradigms to efficiently model parameter uncertainty. The recent connections between RO and problems in statistics and machine learning domains demand for solving RO problems in ever more larger scale. However, the traditional approaches for solving RO formulations based on building and solving robust counterparts or the iterative approaches utilizing nominal feasibility oracles can be prohibitively expensive and thus significantly hinder the scalability of RO paradigm. In this paper, we present a general and flexible iterative framework to approximately solve robust convex optimization problems that is built on a fully online first-order paradigm. In comparison to the existing literature, a key distinguishing feature of our approach is that it only requires access to first-order oracles that are remarkably cheaper than pessimization or nominal feasibility oracles, while maintaining the same convergence rates. This, in particular, makes our approach much more scalable and hence preferable in large-scale applications, specifically those from machine learning and statistics domains. We also provide new interpretations of existing iterative approaches in our framework and illustrate our framework on robust quadratic programming. △ Less

Submitted 17 November, 2017; v1 submitted 21 July, 2016; originally announced July 2016.

MSC Class: 90C25; 90C26; 90C30; 90C47

A Second-Order Cone Based Approach for Solving the Trust Region Subproblem and Its Variants

Authors: Nam Ho-Nguyen, Fatma Kilinc-Karzan

Abstract: We study the trust-region subproblem (TRS) of minimizing a nonconvex quadratic function over the unit ball with additional conic constraints. Despite having a nonconvex objective, it is known that the classical TRS and a number of its variants are polynomial-time solvable. In this paper, we follow a second-order cone (SOC) based approach to derive an exact convex reformulation of the TRS under a s… ▽ More We study the trust-region subproblem (TRS) of minimizing a nonconvex quadratic function over the unit ball with additional conic constraints. Despite having a nonconvex objective, it is known that the classical TRS and a number of its variants are polynomial-time solvable. In this paper, we follow a second-order cone (SOC) based approach to derive an exact convex reformulation of the TRS under a structural condition on the conic constraint. Our structural condition is immediately satisfied when there is no additional conic constraints, and it generalizes several such conditions studied in the literature. As a result, our study highlights an explicit connection between the classical nonconvex TRS and smooth convex quadratic minimization, which allows for the application of cheap iterative methods such as Nesterov's accelerated gradient descent, to the TRS. Furthermore, under slightly stronger conditions, we give a low-complexity characterization of the convex hull of the epigraph of the nonconvex quadratic function intersected with the constraints defining the domain without any additional variables. We also explore the inclusion of additional hollow constraints to the domain of the TRS, and convexification of the associated epigraph. △ Less

Submitted 17 November, 2016; v1 submitted 10 March, 2016; originally announced March 2016.

MSC Class: 90C20; 90C25; 90C26; 90C30

Journal ref: SIAM Journal on Optimization 27.3 (2017): 1485-1512

On Minimal Valid Inequalities for Mixed Integer Conic Programs

Authors: Fatma Kılınç-Karzan

Abstract: We study disjunctive conic sets involving a general regular (closed, convex, full dimensional, and pointed) cone K such as the nonnegative orthant, the Lorentz cone or the positive semidefinite cone. In a unified framework, we introduce K-minimal inequalities and show that under mild assumptions, these inequalities together with the trivial cone-implied inequalities are sufficient to describe the… ▽ More We study disjunctive conic sets involving a general regular (closed, convex, full dimensional, and pointed) cone K such as the nonnegative orthant, the Lorentz cone or the positive semidefinite cone. In a unified framework, we introduce K-minimal inequalities and show that under mild assumptions, these inequalities together with the trivial cone-implied inequalities are sufficient to describe the convex hull. We study the properties of K-minimal inequalities by establishing algebraic necessary conditions for an inequality to be K-minimal. This characterization leads to a broader algebraically defined class of K- sublinear inequalities. We establish a close connection between K-sublinear inequalities and the support functions of sets with a particular structure. This connection results in practical ways of showing that a given inequality is K-sublinear and K-minimal. Our framework generalizes some of the results from the mixed integer linear case. It is well known that the minimal inequalities for mixed integer linear programs are generated by sublinear (positively homogeneous, subadditive and convex) functions that are also piecewise linear. This result is easily recovered by our analysis. Whenever possible we highlight the connections to the existing literature. However, our study unveils that such a cut generating function view treating the data associated with each individual variable independently is not possible in the case of general cones other than nonnegative orthant, even when the cone involved is the Lorentz cone. △ Less

Submitted 1 April, 2015; v1 submitted 29 August, 2014; originally announced August 2014.

Report number: GSIA Working Paper Number: 2013-E20

How to Convexify the Intersection of a Second Order Cone and a Nonconvex Quadratic

Authors: Sam Burer, Fatma Kilinc-Karzan

Abstract: A recent series of papers has examined the extension of disjunctive-programming techniques to mixed-integer second-order-cone programming. For example, it has been shown---by several authors using different techniques---that the convex hull of the intersection of an ellipsoid, $E$, and a split disjunction, $(l - x_j)(x_j - u) \le 0$ with $l < u$, equals the intersection of $E$ with an additional s… ▽ More A recent series of papers has examined the extension of disjunctive-programming techniques to mixed-integer second-order-cone programming. For example, it has been shown---by several authors using different techniques---that the convex hull of the intersection of an ellipsoid, $E$, and a split disjunction, $(l - x_j)(x_j - u) \le 0$ with $l < u$, equals the intersection of $E$ with an additional second-order-cone representable (SOCr) set. In this paper, we study more general intersections of the form $K \cap Q$ and $K \cap Q \cap H$, where $K$ is a SOCr cone, $Q$ is a nonconvex cone defined by a single homogeneous quadratic, and $H$ is an affine hyperplane. Under several easy-to-verify conditions, we derive simple, computable convex relaxations $K \cap S$ and $K \cap S \cap H$, where $S$ is a SOCr cone. Under further conditions, we prove that these two sets capture precisely the corresponding conic/convex hulls. Our approach unifies and extends previous results, and we illustrate its applicability and generality with many examples. △ Less

Submitted 24 May, 2016; v1 submitted 4 June, 2014; originally announced June 2014.

Two-Term Disjunctions on the Second-Order Cone

Authors: Fatma Kilinc-Karzan, Sercan Yildiz

Abstract: Balas introduced disjunctive cuts in the 1970s for mixed-integer linear programs. Several recent papers have attempted to extend this work to mixed-integer conic programs. In this paper we study the structure of the convex hull of a two-term disjunction applied to the second-order cone, and develop a methodology to derive closed-form expressions for convex inequalities describing the resulting con… ▽ More Balas introduced disjunctive cuts in the 1970s for mixed-integer linear programs. Several recent papers have attempted to extend this work to mixed-integer conic programs. In this paper we study the structure of the convex hull of a two-term disjunction applied to the second-order cone, and develop a methodology to derive closed-form expressions for convex inequalities describing the resulting convex hull. Our approach is based on first characterizing the structure of undominated valid linear inequalities for the disjunction and then using conic duality to derive a family of convex, possibly nonlinear, valid inequalities that correspond to these linear inequalities. We identify and study the cases where these valid inequalities can equivalently be expressed in conic quadratic form and where a single inequality from this family is sufficient to describe the convex hull. In particular, our results on two-term disjunctions on the second-order cone generalize related results on split cuts by Modaresi, Kilinc, and Vielma, and by Andersen and Jensen. △ Less

Submitted 30 April, 2014; originally announced April 2014.