-
Modern graph neural networks do worse than classical greedy algorithms in solving combinatorial optimization problems like maximum independent set
Authors:
Maria Chiara Angelini,
Federico Ricci-Tersenghi
Abstract:
The recent work ``Combinatorial Optimization with Physics-Inspired Graph Neural Networks'' [Nat Mach Intell 4 (2022) 367] introduces a physics-inspired unsupervised Graph Neural Network (GNN) to solve combinatorial optimization problems on sparse graphs. To test the performances of these GNNs, the authors of the work show numerical results for two fundamental problems: maximum cut and maximum inde…
▽ More
The recent work ``Combinatorial Optimization with Physics-Inspired Graph Neural Networks'' [Nat Mach Intell 4 (2022) 367] introduces a physics-inspired unsupervised Graph Neural Network (GNN) to solve combinatorial optimization problems on sparse graphs. To test the performances of these GNNs, the authors of the work show numerical results for two fundamental problems: maximum cut and maximum independent set (MIS). They conclude that "the graph neural network optimizer performs on par or outperforms existing solvers, with the ability to scale beyond the state of the art to problems with millions of variables."
In this comment, we show that a simple greedy algorithm, running in almost linear time, can find solutions for the MIS problem of much better quality than the GNN. The greedy algorithm is faster by a factor of $10^4$ with respect to the GNN for problems with a million variables. We do not see any good reason for solving the MIS with these GNN, as well as for using a sledgehammer to crack nuts.
In general, many claims of superiority of neural networks in solving combinatorial problems are at risk of being not solid enough, since we lack standard benchmarks based on really hard problems. We propose one of such hard benchmarks, and we hope to see future neural network optimizers tested on these problems before any claim of superiority is made.
△ Less
Submitted 2 January, 2023; v1 submitted 27 June, 2022;
originally announced June 2022.
-
Biased landscapes for random Constraint Satisfaction Problems
Authors:
Louise Budzynski,
Federico Ricci-Tersenghi,
Guilhem Semerjian
Abstract:
The typical complexity of Constraint Satisfaction Problems (CSPs) can be investigated by means of random ensembles of instances. The latter exhibit many threshold phenomena besides their satisfiability phase transition, in particular a clustering or dynamic phase transition (related to the tree reconstruction problem) at which their typical solutions shatter into disconnected components. In this p…
▽ More
The typical complexity of Constraint Satisfaction Problems (CSPs) can be investigated by means of random ensembles of instances. The latter exhibit many threshold phenomena besides their satisfiability phase transition, in particular a clustering or dynamic phase transition (related to the tree reconstruction problem) at which their typical solutions shatter into disconnected components. In this paper we study the evolution of this phenomenon under a bias that breaks the uniformity among solutions of one CSP instance, concentrating on the bicoloring of k-uniform random hypergraphs. We show that for small k the clustering transition can be delayed in this way to higher density of constraints, and that this strategy has a positive impact on the performances of Simulated Annealing algorithms. We characterize the modest gain that can be expected in the large k limit from the simple implementation of the biasing idea studied here. This paper contains also a contribution of a more methodological nature, made of a review and extension of the methods to determine numerically the discontinuous dynamic transition threshold.
△ Less
Submitted 8 March, 2019; v1 submitted 5 November, 2018;
originally announced November 2018.
-
Typology of phase transitions in Bayesian inference problems
Authors:
Federico Ricci-Tersenghi,
Guilhem Semerjian,
Lenka Zdeborova
Abstract:
Many inference problems, notably the stochastic block model (SBM) that generates a random graph with a hidden community structure, undergo phase transitions as a function of the signal-to-noise ratio, and can exhibit hard phases in which optimal inference is information-theoretically possible but computationally challenging. In this paper we refine this description by emphasizing the existence of…
▽ More
Many inference problems, notably the stochastic block model (SBM) that generates a random graph with a hidden community structure, undergo phase transitions as a function of the signal-to-noise ratio, and can exhibit hard phases in which optimal inference is information-theoretically possible but computationally challenging. In this paper we refine this description by emphasizing the existence of more generic phase diagrams with a hybrid-hard phase in which it is computationally easy to reach a non-trivial inference accuracy, but computationally hard to match the information theoretically optimal one. We support this discussion by quantitative expansions of the functional cavity equations that describe inference problems on sparse graphs. These expansions shed light on the existence of hybrid-hard phases, for a large class of planted constraint satisfaction problems, and on the question of the tightness of the Kesten-Stigum (KS) bound for the associated tree reconstruction problem. Our results show that the instability of the trivial fixed point is not a generic evidence for the Bayes-optimality of the message passing algorithms. We clarify in particular the status of the symmetric SBM with 4 communities and of the tree reconstruction of the associated Potts model: in the assortative (ferromagnetic) case the KS bound is always tight, whereas in the disassortative (antiferromagnetic) case we exhibit an explicit criterion involving the degree distribution that separates a large degree regime where the KS bound is tight and a low degree regime where it is not. We also investigate the SBM with 2 communities of different sizes, a.k.a. the asymmetric Ising model, and describe quantitatively its computational gap as a function of its asymmetry, and a version of the SBM with 2 groups of communities. We complement this study with numerical simulations of the Belief Propagation algorithm.
△ Less
Submitted 20 March, 2019; v1 submitted 28 June, 2018;
originally announced June 2018.
-
Egalitarianism in the rank aggregation problem: a new dimension for democracy
Authors:
Pierluigi Contucci,
Emanuele Panizzi,
Federico Ricci-Tersenghi,
Alina Sîrbu
Abstract:
Winner selection by majority, in an election between two candidates, is the only rule compatible with democratic principles. Instead, when the candidates are three or more and the voters rank candidates in order of preference, there are no univocal criteria for the selection of the winning (consensus) ranking and the outcome is known to depend sensibly on the adopted rule. Building upon XVIII cent…
▽ More
Winner selection by majority, in an election between two candidates, is the only rule compatible with democratic principles. Instead, when the candidates are three or more and the voters rank candidates in order of preference, there are no univocal criteria for the selection of the winning (consensus) ranking and the outcome is known to depend sensibly on the adopted rule. Building upon XVIII century Condorcet theory, whose idea was to maximize total voter satisfaction, we propose here the addition of a new basic principle (dimension) to guide the selection: satisfaction should be distributed among voters as equally as possible. With this new criterion we identify an optimal set of rankings. They range from the Condorcet solution to the one which is the most egalitarian with respect to the voters. We show that highly egalitarian rankings have the important property to be more stable with respect to fluctuations and that classical consensus rankings (Copeland, Tideman, Schulze) often turn out to be non optimal. The new dimension we have introduced provides, when used together with that of Condorcet, a clear classification of all the possible rankings. By increasing awareness in selecting a consensus ranking our method may lead to social choices which are more egalitarian compared to those achieved by presently available voting systems.
△ Less
Submitted 9 June, 2015; v1 submitted 30 June, 2014;
originally announced June 2014.
