Showing 1–41 of 41 results for author: Bulatov, A

Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity

Authors: Yuri Kuratov, Mikhail Arkhipov, Aydar Bulatov, Mikhail Burtsev

Abstract: A range of recent works addresses the problem of compression of sequence of tokens into a shorter sequence of real-valued vectors to be used as inputs instead of token embeddings or key-value cache. These approaches are focused on reduction of the amount of compute in existing language models rather than minimization of number of bits needed to store text. Despite relying on powerful models as enc… ▽ More A range of recent works addresses the problem of compression of sequence of tokens into a shorter sequence of real-valued vectors to be used as inputs instead of token embeddings or key-value cache. These approaches are focused on reduction of the amount of compute in existing language models rather than minimization of number of bits needed to store text. Despite relying on powerful models as encoders, the maximum attainable lossless compression ratio is typically not higher than x10. This fact is highly intriguing because, in theory, the maximum information capacity of large real-valued vectors is far beyond the presented rates even for 16-bit precision and a modest vector size. In this work, we explore the limits of compression by replacing the encoder with a per-sample optimization procedure. We show that vectors with compression ratios up to x1500 exist, which highlights two orders of magnitude gap between existing and practically attainable solutions. Furthermore, we empirically show that the compression limits are determined not by the length of the input but by the amount of uncertainty to be reduced, namely, the cross-entropy loss on this sequence without any conditioning. The obtained limits highlight the substantial gap between the theoretical capacity of input embeddings and their practical utilization, suggesting significant room for optimization in model design. △ Less

Submitted 22 June, 2025; v1 submitted 18 February, 2025; originally announced February 2025.

Comments: ACL 2025 (main conference)

Modular Counting CSP: Reductions and Algorithms

Authors: Amirhossein Kazeminia, Andrei A. Bulatov

Abstract: The Constraint Satisfaction Problem (CSP) is ubiquitous in various areas of mathematics and computer science. Many of its variations have been studied including the Counting CSP, where the goal is to find the number of solutions to a CSP instance. The complexity of finding the exact number of solutions of a CSP is well understood (Bulatov, 2013, and Dyer and Richerby, 2013) and the focus has shift… ▽ More The Constraint Satisfaction Problem (CSP) is ubiquitous in various areas of mathematics and computer science. Many of its variations have been studied including the Counting CSP, where the goal is to find the number of solutions to a CSP instance. The complexity of finding the exact number of solutions of a CSP is well understood (Bulatov, 2013, and Dyer and Richerby, 2013) and the focus has shifted to other variations of the Counting CSP such as counting the number of solutions modulo an integer. This problem has attracted considerable attention recently. In the case of CSPs based on undirected graphs Bulatov and Kazeminia (STOC 2022) obtained a complexity classification for the problem of counting solutions modulo p for arbitrary prime p. In this paper we report on the progress made towards a similar classification for the general CSP, not necessarily based on graphs. We identify several features that make the general case very different from the graph case such as a stronger form of rigidity and the structure of automorphisms of powers of relational structures. We provide a solution algorithm in the case p=2 that works under some additional conditions and prove the hardness of the problem under some assumptions about automorphisms of the powers of the relational structure. We also reduce the general CSP to the case that only uses binary relations satisfying strong additional conditions. △ Less

Submitted 23 January, 2025; v1 submitted 7 January, 2025; originally announced January 2025.

Long Input Benchmark for Russian Analysis

Authors: Igor Churin, Murat Apishev, Maria Tikhonova, Denis Shevelev, Aydar Bulatov, Yuri Kuratov, Sergej Averkiev, Alena Fenogenova

Abstract: Recent advancements in Natural Language Processing (NLP) have fostered the development of Large Language Models (LLMs) that can solve an immense variety of tasks. One of the key aspects of their application is their ability to work with long text documents and to process long sequences of tokens. This has created a demand for proper evaluation of long-context understanding. To address this need fo… ▽ More Recent advancements in Natural Language Processing (NLP) have fostered the development of Large Language Models (LLMs) that can solve an immense variety of tasks. One of the key aspects of their application is their ability to work with long text documents and to process long sequences of tokens. This has created a demand for proper evaluation of long-context understanding. To address this need for the Russian language, we propose LIBRA (Long Input Benchmark for Russian Analysis), which comprises 21 adapted datasets to study the LLM's abilities to understand long texts thoroughly. The tests are divided into four complexity groups and allow the evaluation of models across various context lengths ranging from 4k up to 128k tokens. We provide the open-source datasets, codebase, and public leaderboard for LIBRA to guide forthcoming research. △ Less

Submitted 5 August, 2024; originally announced August 2024.

Associative Recurrent Memory Transformer

Authors: Ivan Rodkin, Yuri Kuratov, Aydar Bulatov, Mikhail Burtsev

Abstract: This paper addresses the challenge of creating a neural architecture for very long sequences that requires constant time for processing new information at each time step. Our approach, Associative Recurrent Memory Transformer (ARMT), is based on transformer self-attention for local context and segment-level recurrence for storage of task specific information distributed over a long context. We dem… ▽ More This paper addresses the challenge of creating a neural architecture for very long sequences that requires constant time for processing new information at each time step. Our approach, Associative Recurrent Memory Transformer (ARMT), is based on transformer self-attention for local context and segment-level recurrence for storage of task specific information distributed over a long context. We demonstrate that ARMT outperfors existing alternatives in associative retrieval tasks and sets a new performance record in the recent BABILong multi-task long-context benchmark by answering single-fact questions over 50 million tokens with an accuracy of 79.9%. The source code for training and evaluation is available on github. △ Less

Submitted 13 February, 2025; v1 submitted 5 July, 2024; originally announced July 2024.

Comments: ICML 2024 Next Generation of Sequence Modeling Architectures Workshop

ACM Class: I.2.7

BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack

Authors: Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Ivan Rodkin, Dmitry Sorokin, Artyom Sorokin, Mikhail Burtsev

Abstract: In recent years, the input context sizes of large language models (LLMs) have increased dramatically. However, existing evaluation methods have not kept pace, failing to comprehensively assess the efficiency of models in handling long contexts. To bridge this gap, we introduce the BABILong benchmark, designed to test language models' ability to reason across facts distributed in extremely long doc… ▽ More In recent years, the input context sizes of large language models (LLMs) have increased dramatically. However, existing evaluation methods have not kept pace, failing to comprehensively assess the efficiency of models in handling long contexts. To bridge this gap, we introduce the BABILong benchmark, designed to test language models' ability to reason across facts distributed in extremely long documents. BABILong includes a diverse set of 20 reasoning tasks, including fact chaining, simple induction, deduction, counting, and handling lists/sets. These tasks are challenging on their own, and even more demanding when the required facts are scattered across long natural text. Our evaluations show that popular LLMs effectively utilize only 10-20\% of the context and their performance declines sharply with increased reasoning complexity. Among alternatives to in-context reasoning, Retrieval-Augmented Generation methods achieve a modest 60\% accuracy on single-fact question answering, independent of context length. Among context extension methods, the highest performance is demonstrated by recurrent memory transformers after fine-tuning, enabling the processing of lengths up to 50 million tokens. The BABILong benchmark is extendable to any length to support the evaluation of new upcoming models with increased capabilities, and we provide splits up to 10 million token lengths. △ Less

Submitted 6 November, 2024; v1 submitted 14 June, 2024; originally announced June 2024.

Comments: NeurIPS 2024 Datasets and Benchmarks Track

Satisfiability of commutative vs. non-commutative CSPs

Authors: Andrei A. Bulatov, Stanislav Živný

Abstract: The Mermin-Peres magic square is a celebrated example of a system of Boolean linear equations that is not (classically) satisfiable but is satisfiable via linear operators on a Hilbert space of dimension four. A natural question is then, for what kind of problems such a phenomenon occurs? Atserias, Kolaitis, and Severini answered this question for all Boolean Constraint Satisfaction Problems (CSPs… ▽ More The Mermin-Peres magic square is a celebrated example of a system of Boolean linear equations that is not (classically) satisfiable but is satisfiable via linear operators on a Hilbert space of dimension four. A natural question is then, for what kind of problems such a phenomenon occurs? Atserias, Kolaitis, and Severini answered this question for all Boolean Constraint Satisfaction Problems (CSPs): For 0-Valid-SAT, 1-Valid-SAT, 2-SAT, Horn-SAT, and Dual Horn-SAT, classical satisfiability and operator satisfiability is the same and thus there is no gap; for all other Boolean CSPs, these notions differ as there are gaps, i.e., there are unsatisfiable instances that are satisfiable via operators on Hilbert spaces. We generalize their result to CSPs on arbitrary finite domains and give an almost complete classification: First, we show that NP-hard CSPs admit a separation between classical satisfiability and satisfiability via operators on finite- and infinite-dimensional Hilbert spaces. Second, we show that tractable CSPs of bounded width have no satisfiability gaps of any kind. Finally, we show that tractable CSPs of unbounded width can simulate, in a satisfiability-gap-preserving fashion, linear equations over an Abelian group of prime order $p$; for such CSPs, we obtain a separation of classical satisfiability and satisfiability via operators on infinite-dimensional Hilbert spaces. Furthermore, if $p=2$, such CSPs also have gaps separating classical satisfiability and satisfiability via operators on finite- and infinite-dimensional Hilbert spaces. △ Less

Submitted 4 November, 2024; v1 submitted 17 April, 2024; originally announced April 2024.

Comments: v2: the main result now omits one case, but also includes infinite-dimensional operators v3: more discussion and comments on related work. arXiv admin note: text overlap with arXiv:1704.01736 by other authors

In Search of Needles in a 11M Haystack: Recurrent Memory Finds What LLMs Miss

Authors: Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Dmitry Sorokin, Artyom Sorokin, Mikhail Burtsev

Abstract: This paper addresses the challenge of processing long documents using generative transformer models. To evaluate different approaches, we introduce BABILong, a new benchmark designed to assess model capabilities in extracting and processing distributed facts within extensive texts. Our evaluation, which includes benchmarks for GPT-4 and RAG, reveals that common methods are effective only for seque… ▽ More This paper addresses the challenge of processing long documents using generative transformer models. To evaluate different approaches, we introduce BABILong, a new benchmark designed to assess model capabilities in extracting and processing distributed facts within extensive texts. Our evaluation, which includes benchmarks for GPT-4 and RAG, reveals that common methods are effective only for sequences up to $10^4$ elements. In contrast, fine-tuning GPT-2 with recurrent memory augmentations enables it to handle tasks involving up to $11\times 10^6$ elements. This achievement marks a substantial leap, as it is by far the longest input processed by any neural network model to date, demonstrating a significant improvement in the processing capabilities for long sequences. △ Less

Submitted 20 February, 2024; v1 submitted 16 February, 2024; originally announced February 2024.

Comments: 11M tokens, fix qa3 min facts per task in Table 1

Better Together: Enhancing Generative Knowledge Graph Completion with Language Models and Neighborhood Information

Authors: Alla Chepurova, Aydar Bulatov, Yuri Kuratov, Mikhail Burtsev

Abstract: Real-world Knowledge Graphs (KGs) often suffer from incompleteness, which limits their potential performance. Knowledge Graph Completion (KGC) techniques aim to address this issue. However, traditional KGC methods are computationally intensive and impractical for large-scale KGs, necessitating the learning of dense node embeddings and computing pairwise distances. Generative transformer-based lang… ▽ More Real-world Knowledge Graphs (KGs) often suffer from incompleteness, which limits their potential performance. Knowledge Graph Completion (KGC) techniques aim to address this issue. However, traditional KGC methods are computationally intensive and impractical for large-scale KGs, necessitating the learning of dense node embeddings and computing pairwise distances. Generative transformer-based language models (e.g., T5 and recent KGT5) offer a promising solution as they can predict the tail nodes directly. In this study, we propose to include node neighborhoods as additional information to improve KGC methods based on language models. We examine the effects of this imputation and show that, on both inductive and transductive Wikidata subsets, our method outperforms KGT5 and conventional KGC approaches. We also provide an extensive analysis of the impact of neighborhood on model prediction and show its importance. Furthermore, we point the way to significantly improve KGC through more effective neighborhood selection. △ Less

Submitted 2 November, 2023; originally announced November 2023.

Comments: Accepted to Findings of the Association for Computational Linguistics: EMNLP 2023

Scaling Transformer to 1M tokens and beyond with RMT

Authors: Aydar Bulatov, Yuri Kuratov, Yermek Kapushev, Mikhail S. Burtsev

Abstract: A major limitation for the broader scope of problems solvable by transformers is the quadratic scaling of computational complexity with input size. In this study, we investigate the recurrent memory augmentation of pre-trained transformer models to extend input context length while linearly scaling compute. Our approach demonstrates the capability to store information in memory for sequences of up… ▽ More A major limitation for the broader scope of problems solvable by transformers is the quadratic scaling of computational complexity with input size. In this study, we investigate the recurrent memory augmentation of pre-trained transformer models to extend input context length while linearly scaling compute. Our approach demonstrates the capability to store information in memory for sequences of up to an unprecedented two million tokens while maintaining high retrieval accuracy. Experiments with language modeling tasks show perplexity improvement as the number of processed input segments increases. These results underscore the effectiveness of our method, which has significant potential to enhance long-term dependency handling in natural language understanding and generation tasks, as well as enable large-scale context processing for memory-intensive applications. △ Less

Submitted 6 February, 2024; v1 submitted 19 April, 2023; originally announced April 2023.

Recurrent Memory Transformer

Authors: Aydar Bulatov, Yuri Kuratov, Mikhail S. Burtsev

Abstract: Transformer-based models show their effectiveness across multiple domains and tasks. The self-attention allows to combine information from all sequence elements into context-aware representations. However, global and local information has to be stored mostly in the same element-wise representations. Moreover, the length of an input sequence is limited by quadratic computational complexity of self-… ▽ More Transformer-based models show their effectiveness across multiple domains and tasks. The self-attention allows to combine information from all sequence elements into context-aware representations. However, global and local information has to be stored mostly in the same element-wise representations. Moreover, the length of an input sequence is limited by quadratic computational complexity of self-attention. In this work, we propose and study a memory-augmented segment-level recurrent Transformer (RMT). Memory allows to store and process local and global information as well as to pass information between segments of the long sequence with the help of recurrence. We implement a memory mechanism with no changes to Transformer model by adding special memory tokens to the input or output sequence. Then the model is trained to control both memory operations and sequence representations processing. Results of experiments show that RMT performs on par with the Transformer-XL on language modeling for smaller memory sizes and outperforms it for tasks that require longer sequence processing. We show that adding memory tokens to Tr-XL is able to improve its performance. This makes Recurrent Memory Transformer a promising architecture for applications that require learning of long-term dependencies and general purpose in memory processing, such as algorithmic tasks and reasoning. △ Less

Submitted 8 December, 2022; v1 submitted 14 July, 2022; originally announced July 2022.

Comments: 36th Conference on Neural Information Processing Systems (NeurIPS 2022)

The Ideal Membership Problem and Abelian Groups

Authors: Andrei A. Bulatov, Akbar Rafiey

Abstract: Given polynomials $f_0,\dots, f_k$ the Ideal Membership Problem, IMP for short, asks if $f_0$ belongs to the ideal generated by $f_1,\dots, f_k$. In the search version of this problem the task is to find a proof of this fact. The IMP is a well-known fundamental problem with numerous applications, for instance, it underlies many proof systems based on polynomials such as Nullstellensatz, Polynomial… ▽ More Given polynomials $f_0,\dots, f_k$ the Ideal Membership Problem, IMP for short, asks if $f_0$ belongs to the ideal generated by $f_1,\dots, f_k$. In the search version of this problem the task is to find a proof of this fact. The IMP is a well-known fundamental problem with numerous applications, for instance, it underlies many proof systems based on polynomials such as Nullstellensatz, Polynomial Calculus, and Sum-of-Squares. Although the IMP is in general intractable, in many important cases it can be efficiently solved. Mastrolilli [SODA'19] initiated a systematic study of IMPs for ideals arising from Constraint Satisfaction Problems (CSPs), parameterized by constraint languages, denoted IMP($Γ$). The ultimate goal of this line of research is to classify all such IMPs accordingly to their complexity. Mastrolilli achieved this goal for IMPs arising from CSP($Γ$) where $Γ$ is a Boolean constraint language, while Bulatov and Rafiey [ArXiv'21] advanced these results to several cases of CSPs over finite domains. In this paper we consider IMPs arising from CSPs over `affine' constraint languages, in which constraints are subgroups (or their cosets) of direct products of Abelian groups. This kind of CSPs include systems of linear equations and are considered one of the most important types of tractable CSPs. Some special cases of the problem have been considered before by Bharathi and Mastrolilli [MFCS'21] for linear equation modulo 2, and by Bulatov and Rafiey [ArXiv'21] to systems of linear equations over $GF(p)$, $p$ prime. Here we prove that if $Γ$ is an affine constraint language then IMP($Γ$) is solvable in polynomial time assuming the input polynomial has bounded degree. △ Less

Submitted 13 January, 2022; originally announced January 2022.

Complexity classification of counting graph homomorphisms modulo a prime number

Authors: Andrei A. Bulatov, Amirhossein Kazeminia

Abstract: Counting graph homomorphisms and its generalizations such as the Counting Constraint Satisfaction Problem (CSP), its variations, and counting problems in general have been intensively studied since the pioneering work of Valiant. While the complexity of exact counting of graph homomorphisms (Dyer and Greenhill, 2000) and the counting CSP (Bulatov, 2013, and Dyer and Richerby, 2013) is well underst… ▽ More Counting graph homomorphisms and its generalizations such as the Counting Constraint Satisfaction Problem (CSP), its variations, and counting problems in general have been intensively studied since the pioneering work of Valiant. While the complexity of exact counting of graph homomorphisms (Dyer and Greenhill, 2000) and the counting CSP (Bulatov, 2013, and Dyer and Richerby, 2013) is well understood, counting modulo some natural number has attracted considerable interest as well. In their 2015 paper Faben and Jerrum suggested a conjecture stating that counting homomorphisms to a fixed graph H modulo a prime number is hard whenever it is hard to count exactly, unless H has automorphisms of certain kind. In this paper we confirm this conjecture. As a part of this investigation we develop techniques that widen the spectrum of reductions available for modular counting and apply to the general CSP rather than being limited to graph homomorphisms. △ Less

Submitted 7 June, 2021; originally announced June 2021.

Unifying the Three Algebraic Approaches to the CSP via Minimal Taylor Algebras

Authors: Libor Barto, Zarathustra Brady, Andrei Bulatov, Marcin Kozik, Dmitriy Zhuk

Abstract: This paper focuses on the algebraic theory underlying the study of the complexity and the algorithms for the Constraint Satisfaction Problem (CSP). We unify, simplify, and extend parts of the three approaches that have been developed to study the CSP over finite templates -- absorption theory that was used to characterize CSPs solvable by local consistency methods (JACM'14), and Bulatov's and Zhuk… ▽ More This paper focuses on the algebraic theory underlying the study of the complexity and the algorithms for the Constraint Satisfaction Problem (CSP). We unify, simplify, and extend parts of the three approaches that have been developed to study the CSP over finite templates -- absorption theory that was used to characterize CSPs solvable by local consistency methods (JACM'14), and Bulatov's and Zhuk's theories that were used for two independent proofs of the CSP Dichotomy Theorem (FOCS'17, JACM'20). As the first contribution we present an elementary theorem about primitive positive definability and use it to obtain the starting points of Bulatov's and Zhuk's proofs as corollaries. As the second contribution we propose and initiate a systematic study of minimal Taylor algebras. This class of algebras is broad enough that it suffices to verify the CSP Dichotomy Theorem on this class only, but still is unusually well behaved. In particular, many concepts from the three approaches coincide in this class, which is in striking contrast with the general setting. We believe that the theory initiated in this paper will eventually result in a simple and more natural proof of the Dichotomy Theorem that employs a simpler and more efficient algorithm, and will help in attacking complexity questions in other CSP-related problems. △ Less

Submitted 14 May, 2024; v1 submitted 23 April, 2021; originally announced April 2021.

Journal ref: TheoretiCS, Volume 3 (May 15, 2024) theoretics:11361

On the Complexity of CSP-based Ideal Membership Problems

Authors: Andrei A. Bulatov, Akbar Rafiey

Abstract: In this paper we consider the Ideal Membership Problem (IMP for short), in which we are given real polynomials $f_0,f_1,\dots, f_k$ and the question is to decide whether $f_0$ belongs to the ideal generated by $f_1,\dots,f_k$. In the more stringent version the task is also to find a proof of this fact. The IMP underlies many proof systems based on polynomials such as Nullstellensatz, Polynomial Ca… ▽ More In this paper we consider the Ideal Membership Problem (IMP for short), in which we are given real polynomials $f_0,f_1,\dots, f_k$ and the question is to decide whether $f_0$ belongs to the ideal generated by $f_1,\dots,f_k$. In the more stringent version the task is also to find a proof of this fact. The IMP underlies many proof systems based on polynomials such as Nullstellensatz, Polynomial Calculus, and Sum-of-Squares. In the majority of such applications the IMP involves so called combinatorial ideals that arise from a variety of discrete combinatorial problems. This restriction makes the IMP significantly easier and in some cases allows for an efficient algorithm to solve it. The first part of this paper follows the work of Mastrolilli [SODA'19] who initiated a systematic study of IMPs arising from Constraint Satisfaction Problems (CSP) of the form $CSP(Γ)$, that is, CSPs in which the type of constraints is limited to relations from a set $Γ$. We show that many CSP techniques can be translated to IMPs thus allowing us to significantly improve the methods of studying the complexity of the IMP. We also develop universal algebraic techniques for the IMP that have been so useful in the study of the CSP. This allows us to prove a general necessary condition for the tractability of the IMP, and three sufficient ones. The sufficient conditions include IMPs arising from systems of linear equations over $GF(p)$, $p$ prime, and also some conditions defined through special kinds of polymorphisms. Our work has several consequences and applications in terms of bit complexity of sum-of-squares (SOS) proofs and their automatizability, and studying (construction of) theta bodies of combinatorial problems. △ Less

Submitted 8 June, 2021; v1 submitted 7 November, 2020; originally announced November 2020.

A dichotomy theorem for nonuniform CSPs simplified

Authors: Andrei A. Bulatov

Abstract: In a non-uniform Constraint Satisfaction problem CSP(G), where G is a set of relations on a finite set A, the goal is to find an assignment of values to variables subject to constraints imposed on specified sets of variables using the relations from G. The Dichotomy Conjecture for the non-uniform CSP states that for every constraint language G the problem CSP(G) is either solvable in polynomial ti… ▽ More In a non-uniform Constraint Satisfaction problem CSP(G), where G is a set of relations on a finite set A, the goal is to find an assignment of values to variables subject to constraints imposed on specified sets of variables using the relations from G. The Dichotomy Conjecture for the non-uniform CSP states that for every constraint language G the problem CSP(G) is either solvable in polynomial time or is NP-complete. It was proposed by Feder and Vardi in their seminal 1993 paper. In this paper we confirm the Dichotomy Conjecture. △ Less

Submitted 15 July, 2020; originally announced July 2020.

Comments: This is an updated and improved version of the proof of the CSP dichotomy from CoRR abs/1703.03021, 2017. arXiv admin note: text overlap with arXiv:1703.03021

Graphs of relational structures: restricted types

Authors: Andrei A. Bulatov

Abstract: The algebraic approach to the Constraint Satisfaction Problem (CSP) uses high order symmetries of relational structures -- polymorphisms -- to study the complexity of the CSP. In this paper we further develop one of the methods the algebraic approach can be implemented, and apply it to some kinds of the CSP. This method was introduced in our LICS 2004 paper and involves the study of the local stru… ▽ More The algebraic approach to the Constraint Satisfaction Problem (CSP) uses high order symmetries of relational structures -- polymorphisms -- to study the complexity of the CSP. In this paper we further develop one of the methods the algebraic approach can be implemented, and apply it to some kinds of the CSP. This method was introduced in our LICS 2004 paper and involves the study of the local structure of finite algebras and relational structures. It associates with an algebra A or a relational structure S a graph, whose vertices are the elements of A (or S), the edges represent subsets of A such that the restriction of some term operation of A is `good' on the subset, that is, act as an operation of one of the 3 types: semilattice, majority, or affine. In this paper we use this theory and consider algebras with edges from a restricted set of types. We prove type restrictions are preserved under the standard algebraic constructions. Then we show that if the types edges in a relational structure are restricted, then the corresponding CSP can be solved in polynomial time by specific algorithms. In particular, we give a new, somewhat more intuitive proof of the Bounded Width Theorem: the CSP over algebra A has bounded width if and only if A does not contain affine edges. Actually, this result shows that bounded width implies width (2,3). Finally, we prove that algebras without semilattice edges have few subalgebras of powers, that is, the CSP over such algebras is also polynomial time. The methods and results obtained in this paper are important ingredients of the 2017 proof of the Dichotomy Conjecture by the author. The Dichotomy Conjecture was also proved independently by Zhuk. △ Less

Submitted 20 July, 2020; v1 submitted 21 June, 2020; originally announced June 2020.

Comments: The conference version of the paper appeared in LICS 2016. In this version a typo in the definition of thin affine edges is corrected. No statements or proofs changed, as they use the correct definition

Local structure of idempotent algebras II

Authors: Andrei A. Bulatov

Abstract: In this paper we continue the study of edge-colored graphs associated with finite idempotent algebras initiated in arXiv:2006.09599. We prove stronger connectivity properties of such graphs that will allows us to demonstrate several useful structural features of subdirect products of idempotent algebras such as rectangularity and 2-decomposition. In this paper we continue the study of edge-colored graphs associated with finite idempotent algebras initiated in arXiv:2006.09599. We prove stronger connectivity properties of such graphs that will allows us to demonstrate several useful structural features of subdirect products of idempotent algebras such as rectangularity and 2-decomposition. △ Less

Submitted 14 January, 2025; v1 submitted 17 June, 2020; originally announced June 2020.

Local structure of idempotent algebras I

Authors: Andrei A. Bulatov

Abstract: We refine and advance the study of the local structure of idempotent finite algebras started in [A.Bulatov, The Graph of a Relational Structure and Constraint Satisfaction Problems, LICS, 2004]. We introduce a graph-like structure on an arbitrary finite idempotent algebra including those admitting type 1. We show that this graph is connected, its edges can be classified into 4 types corresponding… ▽ More We refine and advance the study of the local structure of idempotent finite algebras started in [A.Bulatov, The Graph of a Relational Structure and Constraint Satisfaction Problems, LICS, 2004]. We introduce a graph-like structure on an arbitrary finite idempotent algebra including those admitting type 1. We show that this graph is connected, its edges can be classified into 4 types corresponding to the local behavior (set, semilattice, majority, or affine) of certain term operations. We also show that if the variety generated by the algebra omits type 1, then the structure of the algebra can be `improved' without introducing type 1 by choosing an appropriate reduct of the original algebra. Taylor minimal idempotent algebras introduced recently is a special case of such reducts. Then we refine this structure demonstrating that the edges of the graph of an algebra omitting type 1 can be made `thin', that is, there are term operations that behave very similar to semilattice, majority, or affine operations on 2-element subsets of the algebra. Finally, we prove certain connectivity properties of the refined structures. This research is motivated by the study of the Constraint Satisfaction Problem, although the problem itself does not really show up in this paper. △ Less

Submitted 14 January, 2025; v1 submitted 16 June, 2020; originally announced June 2020.

Comments: arXiv admin note: text overlap with arXiv:1601.07403

Approximate counting CSP seen from the other side

Authors: Andrei A. Bulatov, Stanislav Zivny

Abstract: In this paper we study the complexity of counting Constraint Satisfaction Problems (CSPs) of the form #CSP($\mathcal{C}$,-), in which the goal is, given a relational structure $\mathbf{A}$ from a class $\mathcal{C}$ of structures and an arbitrary structure $\mathbf{B}$, to find the number of homomorphisms from $\mathbf{A}$ to $\mathbf{B}$. Flum and Grohe showed that #CSP($\mathcal{C}$,-) is solvab… ▽ More In this paper we study the complexity of counting Constraint Satisfaction Problems (CSPs) of the form #CSP($\mathcal{C}$,-), in which the goal is, given a relational structure $\mathbf{A}$ from a class $\mathcal{C}$ of structures and an arbitrary structure $\mathbf{B}$, to find the number of homomorphisms from $\mathbf{A}$ to $\mathbf{B}$. Flum and Grohe showed that #CSP($\mathcal{C}$,-) is solvable in polynomial time if $\mathcal{C}$ has bounded treewidth [FOCS'02]. Building on the work of Grohe [JACM'07] on decision CSPs, Dalmau and Jonsson then showed that, if $\mathcal{C}$ is a recursively enumerable class of relational structures of bounded arity, then assuming FPT $\neq$ #W[1], there are no other cases of #CSP($\mathcal{C}$,-) solvable exactly in polynomial time (or even fixed-parameter time) [TCS'04]. We show that, assuming FPT $\neq$ W[1] (under randomised parametrised reductions) and for $\mathcal{C}$ satisfying certain general conditions, #CSP($\mathcal{C}$,-) is not solvable even approximately for $\mathcal{C}$ of unbounded treewidth; that is, there is no fixed parameter tractable (and thus also not fully polynomial) randomised approximation scheme for #CSP($\mathcal{C}$,-). In particular, our condition generalises the case when $\mathcal{C}$ is closed under taking minors. △ Less

Submitted 11 January, 2020; v1 submitted 18 July, 2019; originally announced July 2019.

Comments: Full version of an MFCS'19 paper

Journal ref: ACM Transactions on Computation Theory 12(2) Article No. 11 (2020)

Counting Homomorphisms Modulo a Prime Number

Authors: Amirhossein Kazeminia, Andrei A. Bulatov

Abstract: Counting problems in general and counting graph homomorphisms in particular have numerous applications in combinatorics, computer science, statistical physics, and elsewhere. One of the most well studied problems in this area is #GraphHom(H) --- the problem of finding the number of homomorphisms from a given graph G to the graph H. Not only the complexity of this basic problem is known, but also o… ▽ More Counting problems in general and counting graph homomorphisms in particular have numerous applications in combinatorics, computer science, statistical physics, and elsewhere. One of the most well studied problems in this area is #GraphHom(H) --- the problem of finding the number of homomorphisms from a given graph G to the graph H. Not only the complexity of this basic problem is known, but also of its many variants for digraphs, more general relational structures, graphs with weights, and others. In this paper we consider a modification of #GraphHom(H), the #_p GraphHom(H) problem, p a prime number: Given a graph G, find the number of homomorphisms from G to H modulo p. In a series of papers Faben and Jerrum, and Goebel et al. determined the complexity of #_2 GraphHom(H) in the case H (or, in fact, a certain graph derived from H) is square-free, that is, does not contain a 4-cycle. Also, Goebel et al. found the complexity of #_p GraphHom(H) for an arbitrary prime p when H is a tree. Here we extend the above result to show that the #_p GraphHom(H) problem is #_p P-hard whenever the derived graph associated with H is square-free and is not a star, which completely classifies the complexity of #_p GraphHom(H) for square-free graphs H. △ Less

Submitted 25 May, 2019; originally announced May 2019.

Satisfiability Threshold for Power Law Random 2-SAT in Configuration Model

Authors: Oleksii Omelchenko, Andrei A. Bulatov

Abstract: The Random Satisfiability problem has been intensively studied for decades. For a number of reasons the focus of this study has mostly been on the model, in which instances are sampled uniformly at random from a set of formulas satisfying some clear conditions, such as fixed density or the probability of a clause to occur. However, some non-uniform distributions are also of considerable interest.… ▽ More The Random Satisfiability problem has been intensively studied for decades. For a number of reasons the focus of this study has mostly been on the model, in which instances are sampled uniformly at random from a set of formulas satisfying some clear conditions, such as fixed density or the probability of a clause to occur. However, some non-uniform distributions are also of considerable interest. In this paper we consider Random 2-SAT problems, in which instances are sampled from a wide range of non-uniform distributions. The model of random SAT we choose is the so-called configuration model, given by a distribution $ξ$ for the degree (or the number of occurrences) of each variable. Then to generate a formula the degree of each variable is sampled from $ξ$, generating several \emph{clones} of the variable. Then 2-clauses are created by choosing a random paritioning into 2-element sets on the set of clones and assigning the polarity of literals at random. Here we consider the random 2-SAT problem in the configuration model for power-law-like distributions $ξ$. More precisely, we assume that $ξ$ is such that its right tail $F_ξ(x)$ satisfies the conditions $W\ell^{-α}\le F_ξ(\ell)\le V\ell^{-α}$ for some constants $V,W$. The main goal is to study the satisfiability threshold phenomenon depending on the parameters $α,V,W$. We show that a satisfiability threshold exists and is determined by a simple relation between the first and second moments of $ξ$. △ Less

Submitted 12 May, 2019; originally announced May 2019.

Dismantlability, connectedness, and mixing in relational structures

Authors: Raimundo Briceño, Andrei Bulatov, Victor Dalmau, Benoit Larose

Abstract: The Constraint Satisfaction Problem (CSP) and its counting counterpart appears under different guises in many areas of mathematics, computer science, and elsewhere. Its structural and algorithmic properties have demonstrated to play a crucial role in many of those applications. For instance, in the decision CSPs, structural properties of the relational structures involved---like, for example, dism… ▽ More The Constraint Satisfaction Problem (CSP) and its counting counterpart appears under different guises in many areas of mathematics, computer science, and elsewhere. Its structural and algorithmic properties have demonstrated to play a crucial role in many of those applications. For instance, in the decision CSPs, structural properties of the relational structures involved---like, for example, dismantlability---and their logical characterizations have been instrumental for determining the complexity and other properties of the problem. Topological properties of the solution set such as connectedness are related to the hardness of CSPs over random structures. Additionally, in approximate counting and statistical physics, where CSPs emerge in the form of spin systems, mixing properties and the uniqueness of Gibbs measures have been heavily exploited for approximating partition functions and free energy. In spite of the great diversity of those features, there are some eerie similarities between them. These were observed and made more precise in the case of graph homomorphisms by Brightwell and Winkler, who showed that dismantlability of the target graph, connectedness of the set of homomorphisms, and good mixing properties of the corresponding spin system are all equivalent. In this paper we go a step further and demonstrate similar connections for arbitrary CSPs. This requires much deeper understanding of dismantling and the structure of the solution space in the case of relational structures, and new refined concepts of mixing introduced by Briceño. In addition, we develop properties related to the study of valid extensions of a given partially defined homomorphism, an approach that turns out to be novel even in the graph case. We also add to the mix the combinatorial property of finite duality and its logic counterpart, FO-definability, studied by Larose, Loten, and Tardif. △ Less

Submitted 14 July, 2020; v1 submitted 14 January, 2019; originally announced January 2019.

Comments: 27 pages, full version of the paper accepted to ICALP 2019

MSC Class: 08A70; 68Q87; 68R01; 82B20; 68R10; 05C15

Counting homomorphisms in plain exponential time

Authors: Amineh Dadsetan, Andrei A. Bulatov

Abstract: In the counting Graph Homomorphism problem (#GraphHom) the question is: Given graphs G,H, find the number of homomorphisms from G to H. This problem is generally #P-complete, moreover, Cygan et al. proved that unless the ETH is false there is no algorithm that solves this problem in time O(|V(H)|^{o(|V(G)|)}. This, however, does not rule out the possibility that faster algorithms exist for restric… ▽ More In the counting Graph Homomorphism problem (#GraphHom) the question is: Given graphs G,H, find the number of homomorphisms from G to H. This problem is generally #P-complete, moreover, Cygan et al. proved that unless the ETH is false there is no algorithm that solves this problem in time O(|V(H)|^{o(|V(G)|)}. This, however, does not rule out the possibility that faster algorithms exist for restricted problems of this kind. Wahlstrom proved that #GraphHom can be solved in plain exponential time, that is, in time k^{|V(G)|+V(H)|}\poly(|V(H)|,|V(G)|) provided H has clique width k. We generalize this result to a larger class of graphs, and also identify several other graph classes that admit a plain exponential algorithm for #GraphHom. △ Less

Submitted 7 October, 2018; originally announced October 2018.

Boolean approximate counting CSPs with weak conservativity, and implications for ferromagnetic two-spin

Authors: Miriam Backens, Andrei Bulatov, Leslie Ann Goldberg, Colin McQuillan, Stanislav Živný

Abstract: We analyse the complexity of approximate counting constraint satisfactions problems $\mathrm{\#CSP}(\mathcal{F})$, where $\mathcal{F}$ is a set of nonnegative rational-valued functions of Boolean variables. A complete classification is known in the conservative case, where $\mathcal{F}$ is assumed to contain arbitrary unary functions. We strengthen this result by fixing any permissive strictly inc… ▽ More We analyse the complexity of approximate counting constraint satisfactions problems $\mathrm{\#CSP}(\mathcal{F})$, where $\mathcal{F}$ is a set of nonnegative rational-valued functions of Boolean variables. A complete classification is known in the conservative case, where $\mathcal{F}$ is assumed to contain arbitrary unary functions. We strengthen this result by fixing any permissive strictly increasing unary function and any permissive strictly decreasing unary function, and adding only those to $\mathcal{F}$: this is weak conservativity. The resulting classification is employed to characterise the complexity of a wide range of two-spin problems, fully classifying the ferromagnetic case. In a further weakening of conservativity, we also consider what happens if only the pinning functions are assumed to be in $\mathcal{F}$ (instead of the two permissive unaries). We show that any set of functions for which pinning is not sufficient to recover the two kinds of permissive unaries must either have a very simple range, or must satisfy a certain monotonicity condition. We exhibit a non-trivial example of a set of functions satisfying the monotonicity condition. △ Less

Submitted 15 December, 2019; v1 submitted 13 April, 2018; originally announced April 2018.

Comments: 37 pages

Journal ref: Journal of Computer and System Sciences 109 95-125 (2020)

The Subpower Membership Problem for Finite Algebras with Cube Terms

Authors: Andrei Bulatov, Peter Mayr, Ágnes Szendrei

Abstract: The subalgebra membership problem is the problem of deciding if a given element belongs to an algebra given by a set of generators. This is one of the best established computational problems in algebra. We consider a variant of this problem, which is motivated by recent progress in the Constraint Satisfaction Problem, and is often referred to as the Subpower Membership Problem (SMP). In the SMP we… ▽ More The subalgebra membership problem is the problem of deciding if a given element belongs to an algebra given by a set of generators. This is one of the best established computational problems in algebra. We consider a variant of this problem, which is motivated by recent progress in the Constraint Satisfaction Problem, and is often referred to as the Subpower Membership Problem (SMP). In the SMP we are given a set of tuples in a direct product of algebras from a fixed finite set $\mathcal{K}$ of finite algebras, and are asked whether or not a given tuple belongs to the subalgebra of the direct product generated by a given set. Our main result is that the subpower membership problem SMP($\mathcal{K}$) is in P if $\mathcal{K}$ is a finite set of finite algebras with a cube term, provided $\mathcal{K}$ is contained in a residually small variety. We also prove that for any finite set of finite algebras $\mathcal{K}$ in a variety with a cube term, each one of the problems SMP($\mathcal{K}$), SMP($\mathbb{HS} \mathcal{K}$), and finding compact representations for subpowers in $\mathcal{K}$, is polynomial time reducible to any of the others, and the first two lie in NP. △ Less

Submitted 12 February, 2019; v1 submitted 21 March, 2018; originally announced March 2018.

MSC Class: Primary: 68Q25; Secondary 08A30; 08A70

Journal ref: Logical Methods in Computer Science, Volume 15, Issue 1 (February 13, 2019) lmcs:4396

A dichotomy theorem for nonuniform CSPs

Authors: Andrei A. Bulatov

Abstract: In this paper we prove the Dichotomy Conjecture on the complexity of nonuniform constraint satisfaction problems posed by Feder and Vardi. In this paper we prove the Dichotomy Conjecture on the complexity of nonuniform constraint satisfaction problems posed by Feder and Vardi. △ Less

Submitted 6 April, 2017; v1 submitted 8 March, 2017; originally announced March 2017.

Comments: Apart from correcting typos and inaccuracies, this version of the paper is different from the first arXiv version in the following ways: A self-contained high level presentation of the main results is added; the Preliminaries section is extended, several proofs are reorganized and expanded for improved readability

Constraint Satisfaction Problems over semilattice block Mal'tsev algebras

Authors: Andrei A. Bulatov

Abstract: There are two well known types of algorithms for solving CSPs: local propagation and generating a basis of the solution space. For several years the focus of the CSP research has been on `hybrid' algorithms that somehow combine the two approaches. In this paper we present a new method of such hybridization that allows us to solve certain CSPs that has been out of reach for a quite a while. We cons… ▽ More There are two well known types of algorithms for solving CSPs: local propagation and generating a basis of the solution space. For several years the focus of the CSP research has been on `hybrid' algorithms that somehow combine the two approaches. In this paper we present a new method of such hybridization that allows us to solve certain CSPs that has been out of reach for a quite a while. We consider these method on a fairly restricted class of CSPs given by algebras we will call semilattice block Mal'tsev. An algebra A is called semilattice block Mal'tsev if it has a binary operation f, a ternary operation m, and a congruence s such that the quotient A/s with operation $f$ is a semilattice, $f$ is a projection on every block of s, and every block of s is a Mal'tsev algebra with Mal'tsev operation m. We show that the constraint satisfaction problem over a semilattice block Mal'tsev algebra is solvable in polynomial time. △ Less

Submitted 14 May, 2017; v1 submitted 10 January, 2017; originally announced January 2017.

Comments: This version features a different proof of the main result, which uses an approach closer to that in [Andrei A. Bulatov: A dichotomy theorem for nonuniform CSPs. CoRR abs/1703.03021 (2017)], and is much simplified

Functional Clones and Expressibility of Partition Functions

Authors: Andrei Bulatov, Leslie Ann Goldberg, Mark Jerrum, David Richerby, Stanislav Živný

Abstract: We study functional clones, which are sets of non-negative pseudo-Boolean functions (functions $\{0,1\}^k\to\mathbb{R}_{\geq 0}$) closed under (essentially) multiplication, summation and limits. Functional clones naturally form a lattice under set inclusion and are closely related to counting Constraint Satisfaction Problems (CSPs). We identify a sublattice of interesting functional clones and inv… ▽ More We study functional clones, which are sets of non-negative pseudo-Boolean functions (functions $\{0,1\}^k\to\mathbb{R}_{\geq 0}$) closed under (essentially) multiplication, summation and limits. Functional clones naturally form a lattice under set inclusion and are closely related to counting Constraint Satisfaction Problems (CSPs). We identify a sublattice of interesting functional clones and investigate the relationships and properties of the functional clones in this sublattice. △ Less

Submitted 28 April, 2017; v1 submitted 23 September, 2016; originally announced September 2016.

Comments: 42 pages, 6 figures; minor corrections

Journal ref: Theoretical Computer Science 687 (2017) 11-39

Lower Bounds on Words Separation: Are There Short Identities in Transformation Semigroups?

Authors: Andrei A. Bulatov, Olga Karpova, Arseny M. Shur, Konstantin Startsev

Abstract: The words separation problem, originally formulated by Goralcik and Koubek (1986), is stated as follows. Let $Sep(n)$ be the minimum number such that for any two words of length $\le n$ there is a deterministic finite automaton with $Sep(n)$ states, accepting exactly one of them. The problem is to find the asymptotics of the function $Sep$. This problem is inverse to finding the asymptotics of the… ▽ More The words separation problem, originally formulated by Goralcik and Koubek (1986), is stated as follows. Let $Sep(n)$ be the minimum number such that for any two words of length $\le n$ there is a deterministic finite automaton with $Sep(n)$ states, accepting exactly one of them. The problem is to find the asymptotics of the function $Sep$. This problem is inverse to finding the asymptotics of the length of the shortest identity in full transformation semigroups $T_k$. The known lower bound on $Sep$ stems from the unary identity in $T_k$. We find the first series of identities in $T_k$ which are shorter than the corresponding unary identity for infinitely many values of $k$, and thus slightly improve the lower bound on $Sep(n)$. Then we present some short positive identities in symmetric groups, improving the lower bound on separating words by permutational automata by a multiplicative constant. Finally, we present the results of computer search for short identities for small $k$. △ Less

Submitted 11 September, 2016; originally announced September 2016.

Comments: 12 pages, 2 figures, 2 tables

MSC Class: 68R15; 68Q70; 20B30; 20M20

The subpower membership problem for semigroups

Authors: Andrei Bulatov, Marcin Kozik, Peter Mayr, Markus Steindl

Abstract: Fix a finite semigroup $S$ and let $a_1,\ldots,a_k, b$ be tuples in a direct power $S^n$. The subpower membership problem (SMP) asks whether $b$ can be generated by $a_1,\ldots,a_k$. If $S$ is a finite group, then there is a folklore algorithm that decides this problem in time polynomial in $nk$. For semigroups this problem always lies in PSPACE. We show that the SMP for a full transformation semi… ▽ More Fix a finite semigroup $S$ and let $a_1,\ldots,a_k, b$ be tuples in a direct power $S^n$. The subpower membership problem (SMP) asks whether $b$ can be generated by $a_1,\ldots,a_k$. If $S$ is a finite group, then there is a folklore algorithm that decides this problem in time polynomial in $nk$. For semigroups this problem always lies in PSPACE. We show that the SMP for a full transformation semigroup on 3 letters or more is actually PSPACE-complete, while on 2 letters it is in P. For commutative semigroups, we provide a dichotomy result: if a commutative semigroup $S$ embeds into a direct product of a Clifford semigroup and a nilpotent semigroup, then SMP(S) is in P; otherwise it is NP-complete. △ Less

Submitted 29 August, 2016; v1 submitted 29 March, 2016; originally announced March 2016.

Graphs of finite algebras, edges, and connectivity

Authors: Andrei A. Bulatov

Abstract: We refine and advance the study of the local structure of idempotent finite algebras started in [A.Bulatov, The Graph of a Relational Structure and Constraint Satisfaction Problems, LICS, 2004]. We introduce a graph-like structure on an arbitrary finite idempotent algebra omitting type 1. We show that this graph is connected, its edges can be classified into 3 types corresponding to the local beha… ▽ More We refine and advance the study of the local structure of idempotent finite algebras started in [A.Bulatov, The Graph of a Relational Structure and Constraint Satisfaction Problems, LICS, 2004]. We introduce a graph-like structure on an arbitrary finite idempotent algebra omitting type 1. We show that this graph is connected, its edges can be classified into 3 types corresponding to the local behavior (semilattice, majority, or affine) of certain term operations, and that the structure of the algebra can be `improved' without introducing type 1 by choosing an appropriate reduct of the original algebra. Then we refine this structure demonstrating that the edges of the graph of an algebra can be made `thin', that is, there are term operations that behave very similar to semilattice, majority, or affine operations on 2-element subsets of the algebra. Finally, we prove certain connectivity properties of the refined structures. This research is motivated by the study of the Constraint Satisfaction Problem, although the problem itself does not really show up in this paper. △ Less

Submitted 8 January, 2016; originally announced January 2016.

Conservative constraint satisfaction re-revisited

Authors: Andrei A. Bulatov

Abstract: Conservative constraint satisfaction problems (CSPs) constitute an important particular case of the general CSP, in which the allowed values of each variable can be restricted in an arbitrary way. Problems of this type are well studied for graph homomorphisms. A dichotomy theorem characterizing conservative CSPs solvable in polynomial time and proving that the remaining ones are NP-complete was pr… ▽ More Conservative constraint satisfaction problems (CSPs) constitute an important particular case of the general CSP, in which the allowed values of each variable can be restricted in an arbitrary way. Problems of this type are well studied for graph homomorphisms. A dichotomy theorem characterizing conservative CSPs solvable in polynomial time and proving that the remaining ones are NP-complete was proved by Bulatov in 2003. Its proof, however, is quite long and technical. A shorter proof of this result based on the absorbing subuniverses technique was suggested by Barto in 2011. In this paper we give a short elementary prove of the dichotomy theorem for the conservative CSP. △ Less

Submitted 15 August, 2014; originally announced August 2014.

Inferring Attitude in Online Social Networks Based On Quadratic Correlation

Authors: Cong Wang, Andrei A. Bulatov

Abstract: The structure of an online social network in most cases cannot be described just by links between its members. We study online social networks, in which members may have certain attitude, positive or negative toward each other, and so the network consists of a mixture of both positive and negative relationships. Our goal is to predict the sign of a given relationship based on the evidences provide… ▽ More The structure of an online social network in most cases cannot be described just by links between its members. We study online social networks, in which members may have certain attitude, positive or negative toward each other, and so the network consists of a mixture of both positive and negative relationships. Our goal is to predict the sign of a given relationship based on the evidences provided in the current snapshot of the network. More precisely, using machine learning techniques we develop a model that after being trained on a particular network predicts the sign of an unknown or hidden link. The model uses relationships and influences from peers as evidences for the guess, however, the set of peers used is not predefined but rather learned during the training process. We use quadratic correlation between peer members to train the predictor. The model is tested on popular online datasets such as Epinions, Slashdot, and Wikipedia. In many cases it shows almost perfect prediction accuracy. Moreover, our model can also be efficiently updated as the underlaying social network evolves. △ Less

Submitted 7 December, 2012; originally announced December 2012.

Comments: 18 pages, 3 figures

ACM Class: H.2.8

Galois correspondence for counting quantifiers

Authors: Andrei A. Bulatov, Amir Hedayaty

Abstract: We introduce a new type of closure operator on the set of relations, max-implementation, and its weaker analog max-quantification. Then we show that approximation preserving reductions between counting constraint satisfaction problems (#CSPs) are preserved by these two types of closure operators. Together with some previous results this means that the approximation complexity of counting CSPs is d… ▽ More We introduce a new type of closure operator on the set of relations, max-implementation, and its weaker analog max-quantification. Then we show that approximation preserving reductions between counting constraint satisfaction problems (#CSPs) are preserved by these two types of closure operators. Together with some previous results this means that the approximation complexity of counting CSPs is determined by partial clones of relations that additionally closed under these new types of closure operators. Galois correspondence of various kind have proved to be quite helpful in the study of the complexity of the CSP. While we were unable to identify a Galois correspondence for partial clones closed under max-implementation and max-quantification, we obtain such results for slightly different type of closure operators, k-existential quantification. This type of quantifiers are known as counting quantifiers in model theory, and often used to enhance first order logic languages. We characterize partial clones of relations closed under k-existential quantification as sets of relations invariant under a set of partial functions that satisfy the condition of k-subset surjectivity. Finally, we give a description of Boolean max-co-clones, that is, sets of relations on {0,1} closed under max-implementations. △ Less

Submitted 11 October, 2012; originally announced October 2012.

Comments: 28 pages, 2 figures

ACM Class: F.2.2; F.4.1; G.2.1

Constraint satisfaction parameterized by solution size

Authors: Andrei A. Bulatov, Dániel Marx

Abstract: In the constraint satisfaction problem (CSP) corresponding to a constraint language (i.e., a set of relations) $Γ$, the goal is to find an assignment of values to variables so that a given set of constraints specified by relations from $Γ$ is satisfied. The complexity of this problem has received substantial amount of attention in the past decade. In this paper we study the fixed-parameter tractab… ▽ More In the constraint satisfaction problem (CSP) corresponding to a constraint language (i.e., a set of relations) $Γ$, the goal is to find an assignment of values to variables so that a given set of constraints specified by relations from $Γ$ is satisfied. The complexity of this problem has received substantial amount of attention in the past decade. In this paper we study the fixed-parameter tractability of constraint satisfaction problems parameterized by the size of the solution in the following sense: one of the possible values, say 0, is "free," and the number of variables allowed to take other, "expensive," values is restricted. A size constraint requires that exactly $k$ variables take nonzero values. We also study a more refined version of this restriction: a global cardinality constraint prescribes how many variables have to be assigned each particular value. We study the parameterized complexity of these types of CSPs where the parameter is the required number $k$ of nonzero variables. As special cases, we can obtain natural and well-studied parameterized problems such as Independent Set, Vertex Cover, d-Hitting Set, Biclique, etc. In the case of constraint languages closed under substitution of constants, we give a complete characterization of the fixed-parameter tractable cases of CSPs with size constraints, and we show that all the remaining problems are W[1]-hard. For CSPs with cardinality constraints, we obtain a similar classification, but for some of the problems we are only able to show that they are Biclique-hard. The exact parameterized complexity of the Biclique problem is a notorious open problem, although it is believed to be W[1]-hard. △ Less

Submitted 18 January, 2014; v1 submitted 21 June, 2012; originally announced June 2012.

Comments: To appear in SICOMP. Conference version in ICALP 2011

The expressibility of functions on the Boolean domain, with applications to Counting CSPs

Authors: Andrei A. Bulatov, Martin Dyer, Leslie Ann Goldberg, Mark Jerrum, Colin McQuillan

Abstract: An important tool in the study of the complexity of Constraint Satisfaction Problems (CSPs) is the notion of a relational clone, which is the set of all relations expressible using primitive positive formulas over a particular set of base relations. Post's lattice gives a complete classification of all Boolean relational clones, and this has been used to classify the computational difficulty of CS… ▽ More An important tool in the study of the complexity of Constraint Satisfaction Problems (CSPs) is the notion of a relational clone, which is the set of all relations expressible using primitive positive formulas over a particular set of base relations. Post's lattice gives a complete classification of all Boolean relational clones, and this has been used to classify the computational difficulty of CSPs. Motivated by a desire to understand the computational complexity of (weighted) counting CSPs, we develop an analogous notion of functional clones and study the landscape of these clones. One of these clones is the collection of log-supermodular (lsm) functions, which turns out to play a significant role in classifying counting CSPs. In the conservative case (where all nonnegative unary functions are available), we show that there are no functional clones lying strictly between the clone of lsm functions and the total clone (containing all functions). Thus, any counting CSP that contains a single nontrivial non-lsm function is computationally as hard to approximate as any problem in #P. Furthermore, we show that any non-trivial functional clone (in a sense that will be made precise) contains the binary function "implies". As a consequence, in the conservative case, all non-trivial counting CSPs are as hard as #BIS, the problem of counting independent sets in a bipartite graph. Given the complexity-theoretic results, it is natural to ask whether the "implies" clone is equivalent to the clone of lsm functions. We use the Mobius transform and the Fourier transform to show that these clones coincide precisely up to arity 3. It is an intriguing open question whether the lsm clone is finitely generated. Finally, we investigate functional clones in which only restricted classes of unary functions are available. △ Less

Submitted 13 June, 2012; v1 submitted 26 August, 2011; originally announced August 2011.

Comments: corrected typo in title :-)

MSC Class: 68Q15; 68Q17

Journal ref: JACM Vol 60 Issue 5 Oct 2013 Article 32

The complexity of global cardinality constraints

Authors: Andrei A. Bulatov, Daniel Marx

Abstract: In a constraint satisfaction problem (CSP) the goal is to find an assignment of a given set of variables subject to specified constraints. A global cardinality constraint is an additional requirement that prescribes how many variables must be assigned a certain value. We study the complexity of the problem CCSP(G), the constraint satisfaction problem with global cardinality constraints that allow… ▽ More In a constraint satisfaction problem (CSP) the goal is to find an assignment of a given set of variables subject to specified constraints. A global cardinality constraint is an additional requirement that prescribes how many variables must be assigned a certain value. We study the complexity of the problem CCSP(G), the constraint satisfaction problem with global cardinality constraints that allows only relations from the set G. The main result of this paper characterizes sets G that give rise to problems solvable in polynomial time, and states that the remaining such problems are NP-complete. △ Less

Submitted 27 October, 2010; v1 submitted 1 October, 2010; originally announced October 2010.

ACM Class: F.2.2, F.4.1

Journal ref: Logical Methods in Computer Science, Volume 6, Issue 4 (October 27, 2010) lmcs:1025

The complexity of weighted and unweighted #CSP

Authors: Andrei Bulatov, Martin Dyer, Leslie Ann Goldberg, Markus Jalsenius, Mark Jerrum, David Richerby

Abstract: We give some reductions among problems in (nonnegative) weighted #CSP which restrict the class of functions that needs to be considered in computational complexity studies. Our reductions can be applied to both exact and approximate computation. In particular, we show that a recent dichotomy for unweighted #CSP can be extended to rational-weighted #CSP. We give some reductions among problems in (nonnegative) weighted #CSP which restrict the class of functions that needs to be considered in computational complexity studies. Our reductions can be applied to both exact and approximate computation. In particular, we show that a recent dichotomy for unweighted #CSP can be extended to rational-weighted #CSP. △ Less

Submitted 24 October, 2011; v1 submitted 15 May, 2010; originally announced May 2010.

Comments: 11 pages

ACM Class: F.2.2; F.4.1; G.2.1

Journal ref: JCSS 2012

Enumerating Homomorphisms

Authors: Andrei A. Bulatov, Victor Dalmau, Martin Grohe, Daniel Marx

Abstract: The homomorphism problem for relational structures is an abstract way of formulating constraint satisfaction problems (CSP) and various problems in database theory. The decision version of the homomorphism problem received a lot of attention in literature; in particular, the way the graph-theoretical structure of the variables and constraints influences the complexity of the problem is intensive… ▽ More The homomorphism problem for relational structures is an abstract way of formulating constraint satisfaction problems (CSP) and various problems in database theory. The decision version of the homomorphism problem received a lot of attention in literature; in particular, the way the graph-theoretical structure of the variables and constraints influences the complexity of the problem is intensively studied. Here we study the problem of enumerating all the solutions with polynomial delay from a similar point of view. It turns out that the enumeration problem behaves very differently from the decision version. We give evidence that it is unlikely that a characterization result similar to the decision version can be obtained. Nevertheless, we show nontrivial cases where enumeration can be done with polynomial delay. △ Less

Submitted 7 February, 2009; originally announced February 2009.

Journal ref: 26th International Symposium on Theoretical Aspects of Computer Science STACS 2009 (2009) 231-242

The Complexity of Weighted Boolean #CSP with Mixed Signs

Authors: Andrei Bulatov, Martin Dyer, Leslie Ann Goldberg, Markus Jalsenius, David Richerby

Abstract: We give a complexity dichotomy for the problem of computing the partition function of a weighted Boolean constraint satisfaction problem. Such a problem is parameterized by a set of rational-valued functions, which generalize constraints. Each function assigns a weight to every assignment to a set of Boolean variables. Our dichotomy extends previous work in which the weight functions were restri… ▽ More We give a complexity dichotomy for the problem of computing the partition function of a weighted Boolean constraint satisfaction problem. Such a problem is parameterized by a set of rational-valued functions, which generalize constraints. Each function assigns a weight to every assignment to a set of Boolean variables. Our dichotomy extends previous work in which the weight functions were restricted to being non-negative. We represent a weight function as a product of the form (-1)^s g, where the polynomial s determines the sign of the weight and the non-negative function g determines its magnitude. We show that the problem of computing the partition function (the sum of the weights of all possible variable assignments) is in polynomial time if either every weight function can be defined by a "pure affine" magnitude with a quadratic sign polynomial or every function can be defined by a magnitude of "product type" with a linear sign polynomial. In all other cases, computing the partition function is FP^#P-complete. △ Less

Submitted 3 June, 2009; v1 submitted 22 December, 2008; originally announced December 2008.

Comments: 24 pages

ACM Class: F.2.2; F.4.1; G.2.1

Phase transition for Local Search on planted SAT

Authors: Andrei A. Bulatov, Evgeny S. Skvortsov

Abstract: The Local Search algorithm (or Hill Climbing, or Iterative Improvement) is one of the simplest heuristics to solve the Satisfiability and Max-Satisfiability problems. It is a part of many satisfiability and max-satisfiability solvers, where it is used to find a good starting point for a more sophisticated heuristics, and to improve a candidate solution. In this paper we give an analysis of Local… ▽ More The Local Search algorithm (or Hill Climbing, or Iterative Improvement) is one of the simplest heuristics to solve the Satisfiability and Max-Satisfiability problems. It is a part of many satisfiability and max-satisfiability solvers, where it is used to find a good starting point for a more sophisticated heuristics, and to improve a candidate solution. In this paper we give an analysis of Local Search on random planted 3-CNF formulas. We show that if there is k<7/6 such that the clause-to-variable ratio is less than k ln(n) (n is the number of variables in a CNF) then Local Search whp does not find a satisfying assignment, and if there is k>7/6 such that the clause-to-variable ratio is greater than k ln(n)$ then the local search whp finds a satisfying assignment. As a byproduct we also show that for any constant r there is g such that Local Search applied to a random (not necessarily planted) 3-CNF with clause-to-variable ratio r produces an assignment that satisfies at least gn clauses less than the maximal number of satisfiable clauses. △ Less

Submitted 15 November, 2008; originally announced November 2008.

Comments: 20 pages, 3 figures, submitted to a conference