-
Extended formulations for the multilinear polytope of acyclic hypergraphs
Authors:
Alberto Del Pia,
Aida Khajavirad
Abstract:
This article provides an overview of our joint work on binary polynomial optimization over the past decade. We define the multilinear polytope as the convex hull of the feasible region of a linearized binary polynomial optimization problem. By representing the multilinear polytope with hypergraphs, we investigate the connections between hypergraph acyclicity and the complexity of the facial struct…
▽ More
This article provides an overview of our joint work on binary polynomial optimization over the past decade. We define the multilinear polytope as the convex hull of the feasible region of a linearized binary polynomial optimization problem. By representing the multilinear polytope with hypergraphs, we investigate the connections between hypergraph acyclicity and the complexity of the facial structure of the multilinear polytope. We characterize the acyclic hypergraphs for which a polynomial-size extended formulation for the multilinear polytope can be constructed in polynomial time.
△ Less
Submitted 8 January, 2025;
originally announced January 2025.
-
Beyond hypergraph acyclicity: limits of tractability for pseudo-Boolean optimization
Authors:
Alberto Del Pia,
Aida Khajavirad
Abstract:
In this paper, we study the problem of minimizing a polynomial function with literals over all binary points, often referred to as pseudo-Boolean optimization. We investigate the fundamental limits of computation for this problem by providing new necessary conditions and sufficient conditions for tractability. On the one hand, we obtain the first intractability results, in the best-case sense, for…
▽ More
In this paper, we study the problem of minimizing a polynomial function with literals over all binary points, often referred to as pseudo-Boolean optimization. We investigate the fundamental limits of computation for this problem by providing new necessary conditions and sufficient conditions for tractability. On the one hand, we obtain the first intractability results, in the best-case sense, for pseudo-Boolean optimization problems on signed hypergraphs with bounded rank, in terms of the treewidth of the intersection graph. Namely, first, under some mild assumptions, we show that for every sequence of hypergraphs indexed by the treewidth and with bounded rank, the complexity of solving the associated pseudo-Boolean optimization problem grows super-polynomially in the treewidth. Second, we show that any hypergraph of bounded rank is the underlying hypergraph of some signed hypergraph for which the corresponding pseudo-Boolean polytope has an exponential extension complexity in the treewidth. On the other hand, we introduce the nest-set gap, a new hypergraph-theoretic notion that enables us to define a notion of "distance" from the hypergaph acyclicity. We prove that if this distance is bounded, the pseudo-Boolean polytope admits a polynomial-size extended formulation. This in turn enables us to obtain a polynomial-time algorithm for a large class of pseudo-Boolean optimization problems whose underlying hypergraphs contain beta-cycles.
△ Less
Submitted 31 January, 2025; v1 submitted 30 October, 2024;
originally announced October 2024.
-
Inference in higher-order undirected graphical models and binary polynomial optimization
Authors:
Aida Khajavirad,
Yakun Wang
Abstract:
We consider the problem of inference in higher-order undirected graphical models with binary labels. We formulate this problem as a binary polynomial optimization problem and propose several linear programming relaxations for it. We compare the strength of the proposed linear programming relaxations theoretically. Finally, we demonstrate the effectiveness of these relaxations by performing a compu…
▽ More
We consider the problem of inference in higher-order undirected graphical models with binary labels. We formulate this problem as a binary polynomial optimization problem and propose several linear programming relaxations for it. We compare the strength of the proposed linear programming relaxations theoretically. Finally, we demonstrate the effectiveness of these relaxations by performing a computational study for two important applications, namely, image restoration and decoding error-correcting codes.
△ Less
Submitted 15 December, 2024; v1 submitted 15 May, 2024;
originally announced May 2024.
-
The circle packing problem: a theoretical comparison of various convexification techniques
Authors:
Aida Khajavirad
Abstract:
We consider the problem of packing congruent circles with the maximum radius in a unit square as a mathematical optimization problem. Due to the presence of non-overlapping constraints, this problem is a notoriously difficult nonconvex quadratically constrained optimization problem, which possesses many local optima. We consider several popular convexification techniques, giving rise to linear pro…
▽ More
We consider the problem of packing congruent circles with the maximum radius in a unit square as a mathematical optimization problem. Due to the presence of non-overlapping constraints, this problem is a notoriously difficult nonconvex quadratically constrained optimization problem, which possesses many local optima. We consider several popular convexification techniques, giving rise to linear programming relaxations and semidefinite programming relaxations for the circle packing problem. We compare the strength of these relaxations theoretically, thereby proving the conjectures by Anstreicher (JOGO, 2009). Our results serve as a theoretical justification for the ineffectiveness of existing machinery for convexification of non-overlapping constraints.
△ Less
Submitted 3 April, 2024;
originally announced April 2024.
-
On the power of linear programming for K-means clustering
Authors:
Antonio De Rosa,
Aida Khajavirad,
Yakun Wang
Abstract:
In [SIAM J. Optim., 2022], the authors introduced a new linear programming (LP) relaxation for K-means clustering. In this paper, we further investigate both theoretical and computational properties of this relaxation. As evident from our numerical experiments with both synthetic real-world data sets, the proposed LP relaxation is almost always tight; i.e. its optimal solution is feasible for the…
▽ More
In [SIAM J. Optim., 2022], the authors introduced a new linear programming (LP) relaxation for K-means clustering. In this paper, we further investigate both theoretical and computational properties of this relaxation. As evident from our numerical experiments with both synthetic real-world data sets, the proposed LP relaxation is almost always tight; i.e. its optimal solution is feasible for the original nonconvex problem. To better understand this unexpected behaviour, on the theoretical side, we focus on K-means clustering with two clusters, and we obtain sufficient conditions under which the LP relaxation is tight. We further analyze the sufficient conditions when the input is generated according to a popular stochastic model and obtain recovery guarantees for the LP relaxation. We conclude our theoretical study by constructing a family of inputs for which the LP relaxation is never tight. Denoting by $n$ the number of data points to be clustered, the LP relaxation contains $Ω(n^3)$ inequalities making it impractical for large data sets. To address the scalability issue, by building upon a cutting-plane algorithm together with the GPU implementation of PDLP, a first-order method LP solver, we develop an efficient algorithm that solves the proposed LP and hence the K-means clustering problem, for up to $n \leq 4000$ data points.
△ Less
Submitted 15 August, 2024; v1 submitted 1 February, 2024;
originally announced February 2024.
-
The pseudo-Boolean polytope and polynomial-size extended formulations for binary polynomial optimization
Authors:
Alberto Del Pia,
Aida Khajavirad
Abstract:
With the goal of obtaining strong relaxations for binary polynomial optimization problems, we introduce the pseudo-Boolean polytope defined as the convex hull of the set of binary points satisfying a collection of equations containing pseudo-Boolean functions. By representing the pseudo-Boolean polytope via a signed hypergraph, we obtain sufficient conditions under which this polytope has a polyno…
▽ More
With the goal of obtaining strong relaxations for binary polynomial optimization problems, we introduce the pseudo-Boolean polytope defined as the convex hull of the set of binary points satisfying a collection of equations containing pseudo-Boolean functions. By representing the pseudo-Boolean polytope via a signed hypergraph, we obtain sufficient conditions under which this polytope has a polynomial-size extended formulation. Our new framework unifies and extends all prior results on the existence of polynomial-size extended formulations for the convex hull of the feasible region of binary polynomial optimization problems of degree at least three.
△ Less
Submitted 1 July, 2024; v1 submitted 15 September, 2023;
originally announced September 2023.
-
A polynomial-size extended formulation for the multilinear polytope of beta-acyclic hypergraphs
Authors:
Alberto Del Pia,
Aida Khajavirad
Abstract:
We consider the multilinear polytope defined as the convex hull of the set of binary points satisfying a collection of multilinear equations. The complexity of the facial structure of the multilinear polytope is closely related to the acyclicity degree of the underlying hypergraph. We obtain a polynomial-size extended formulation for the multilinear polytope of beta-acyclic hypergraphs, hence char…
▽ More
We consider the multilinear polytope defined as the convex hull of the set of binary points satisfying a collection of multilinear equations. The complexity of the facial structure of the multilinear polytope is closely related to the acyclicity degree of the underlying hypergraph. We obtain a polynomial-size extended formulation for the multilinear polytope of beta-acyclic hypergraphs, hence characterizing the acyclic hypergraphs for which such a formulation can be constructed.
△ Less
Submitted 29 August, 2023; v1 submitted 15 December, 2022;
originally announced December 2022.
-
On the strength of recursive McCormick relaxations for binary polynomial optimization
Authors:
Aida Khajavirad
Abstract:
Recursive McCormick relaxations have been among the most popular convexification techniques for binary polynomial optimization problems. It is well-understood that both the quality and the size of these relaxations depend on the recursive sequence, and finding an optimal recursive sequence amounts to solving a difficult combinatorial optimization problem. In this paper, we prove that any recursive…
▽ More
Recursive McCormick relaxations have been among the most popular convexification techniques for binary polynomial optimization problems. It is well-understood that both the quality and the size of these relaxations depend on the recursive sequence, and finding an optimal recursive sequence amounts to solving a difficult combinatorial optimization problem. In this paper, we prove that any recursive McCormick relaxation is implied by the extended flower relaxation, a linear programming relaxation that is a natural generalization of the flower relaxation introduced by Del Pia and Khajavirad 2018, which for binary polynomial optimization problems with fixed degree can be solved in strongly polynomial time.
△ Less
Submitted 17 January, 2023; v1 submitted 26 September, 2022;
originally announced September 2022.
-
Explicit convex hull description of bivariate quadratic sets with indicator variables
Authors:
Antonio De Rosa,
Aida Khajavirad
Abstract:
We consider the nonconvex set $\mathcal S_n = \{(x,X,z): X = x x^T, \; x (1-z) =0,\; x \geq 0,\; z \in \{0,1\}^n\}$, which is closely related to the feasible region of several difficult nonconvex optimization problems such as the best subset selection and constrained portfolio optimization. Utilizing ideas from convex analysis and disjunctive programming, we obtain an explicit description for the…
▽ More
We consider the nonconvex set $\mathcal S_n = \{(x,X,z): X = x x^T, \; x (1-z) =0,\; x \geq 0,\; z \in \{0,1\}^n\}$, which is closely related to the feasible region of several difficult nonconvex optimization problems such as the best subset selection and constrained portfolio optimization. Utilizing ideas from convex analysis and disjunctive programming, we obtain an explicit description for the closure of the convex hull of $\mathcal S_2$ in the space of original variables. In order to generate valid inequalities corresponding to supporting hyperplanes of the convex hull of $\mathcal S_2$, we present a simple separation algorithm that can be incorporated in branch-and-cut based solvers to enhance the quality of existing relaxations.
△ Less
Submitted 24 February, 2023; v1 submitted 18 August, 2022;
originally announced August 2022.
-
Rank-one Boolean tensor factorization and the multilinear polytope
Authors:
Alberto Del Pia,
Aida Khajavirad
Abstract:
We consider the NP-hard problem of finding the closest rank-one binary tensor to a given binary tensor, which we refer to as the rank-one Boolean tensor factorization (BTF) problem. This optimization problem can be used to recover a planted rank-one tensor from noisy observations. We formulate rank-one BTF as the problem of minimizing a linear function over a highly structured multilinear set. Lev…
▽ More
We consider the NP-hard problem of finding the closest rank-one binary tensor to a given binary tensor, which we refer to as the rank-one Boolean tensor factorization (BTF) problem. This optimization problem can be used to recover a planted rank-one tensor from noisy observations. We formulate rank-one BTF as the problem of minimizing a linear function over a highly structured multilinear set. Leveraging on our prior results regarding the facial structure of multilinear polytopes, we propose novel linear programming relaxations for rank-one BTF. We then establish deterministic sufficient conditions under which our proposed linear programs recover a planted rank-one tensor. To analyze the effectiveness of these deterministic conditions, we consider a semi-random model for the noisy tensor, and obtain high probability recovery guarantees for the linear programs. Our theoretical results as well as numerical simulations indicate that certain facets of the multilinear polytope significantly improve the recovery properties of linear programming relaxations for rank-one BTF.
△ Less
Submitted 2 May, 2024; v1 submitted 14 February, 2022;
originally announced February 2022.
-
Efficient Joint Object Matching via Linear Programming
Authors:
Antonio De Rosa,
Aida Khajavirad
Abstract:
Joint object matching, also known as multi-image matching, namely, the problem of finding consistent partial maps among all pairs of objects within a collection, is a crucial task in many areas of computer vision. This problem subsumes bipartite graph matching and graph partitioning as special cases and is NP-hard, in general. We develop scalable linear programming (LP) relaxations with theoretica…
▽ More
Joint object matching, also known as multi-image matching, namely, the problem of finding consistent partial maps among all pairs of objects within a collection, is a crucial task in many areas of computer vision. This problem subsumes bipartite graph matching and graph partitioning as special cases and is NP-hard, in general. We develop scalable linear programming (LP) relaxations with theoretical performance guarantees for joint object matching. We start by proposing a new characterization of consistent partial maps; this in turn enables us to formulate joint object matching as an integer linear programming (ILP) problem. To construct strong LP relaxations, we study the facial structure of the convex hull of the feasible region of this ILP, which we refer to as the joint matching polytope. We present an exponential family of facet-defining inequalities that can be separated in strongly polynomial time, hence obtaining a partial characterization of the joint matching polytope that is both tight and cheap to compute. To analyze the theoretical performance of the proposed LP relaxations, we focus on permutation group synchronization, an important special case of joint object matching. We show that under the random corruption model for the input maps, a simple LP relaxation, that is, an LP containing only a very small fraction of the proposed facet-defining inequalities, recovers the ground truth with high probability if the corruption level is below $40\%$. Finally, via a preliminary computational study on synthetic data, we show that the proposed LP relaxations outperform a popular SDP relaxation both in terms of recovery and tightness.
△ Less
Submitted 25 November, 2022; v1 submitted 26 August, 2021;
originally announced August 2021.
-
The ratio-cut polytope and K-means clustering
Authors:
Antonio De Rosa,
Aida Khajavirad
Abstract:
We introduce the ratio-cut polytope defined as the convex hull of ratio-cut vectors corresponding to all partitions of $n$ points in $\mathbb R^m$ into at most $K$ clusters. This polytope is closely related to the convex hull of the feasible region of a number of clustering problems such as K-means clustering and spectral clustering. We study the facial structure of the ratio-cut polytope and deri…
▽ More
We introduce the ratio-cut polytope defined as the convex hull of ratio-cut vectors corresponding to all partitions of $n$ points in $\mathbb R^m$ into at most $K$ clusters. This polytope is closely related to the convex hull of the feasible region of a number of clustering problems such as K-means clustering and spectral clustering. We study the facial structure of the ratio-cut polytope and derive several types of facet-defining inequalities. We then consider the problem of K-means clustering and introduce a novel linear programming (LP) relaxation for it. Subsequently, we focus on the case of two clusters and derive sufficient conditions under which the proposed LP relaxation recovers the underlying clusters exactly. Namely, we consider the stochastic ball model, a popular generative model for K-means clustering, and we show that if the separation distance between cluster centers satisfies $Δ> 1+\sqrt 3$, then the LP relaxation recovers the planted clusters with high probability. This is a major improvement over the only existing recovery guarantee for an LP relaxation of K-means clustering stating that recovery is possible with high probability if and only if $Δ> 4$. Our numerical experiments indicate that the proposed LP relaxation significantly outperforms a popular semidefinite programming relaxation in recovering the planted clusters.
△ Less
Submitted 1 February, 2024; v1 submitted 26 June, 2020;
originally announced June 2020.
-
Linear Programming and Community Detection
Authors:
Alberto Del Pia,
Aida Khajavirad,
Dmitriy Kunisky
Abstract:
The problem of community detection with two equal-sized communities is closely related to the minimum graph bisection problem over certain random graph models. In the stochastic block model distribution over networks with community structure, a well-known semidefinite programming (SDP) relaxation of the minimum bisection problem recovers the underlying communities whenever possible. Motivated by t…
▽ More
The problem of community detection with two equal-sized communities is closely related to the minimum graph bisection problem over certain random graph models. In the stochastic block model distribution over networks with community structure, a well-known semidefinite programming (SDP) relaxation of the minimum bisection problem recovers the underlying communities whenever possible. Motivated by their superior scalability, we study the theoretical performance of linear programming (LP) relaxations of the minimum bisection problem for the same random models. We show that unlike the SDP relaxation that undergoes a phase transition in the logarithmic average-degree regime, the LP relaxation exhibits a transition from recovery to non-recovery in the linear average-degree regime. We show that in the logarithmic average-degree regime, the LP relaxation fails in recovering the planted bisection with high probability.
△ Less
Submitted 11 May, 2022; v1 submitted 4 June, 2020;
originally announced June 2020.
