-
The Aldous--Hoover Theorem in Categorical Probability
Authors:
Leihao Chen,
Tobias Fritz,
Tomáš Gonda,
Andreas Klingler,
Antonio Lorenzin
Abstract:
The Aldous-Hoover Theorem concerns an infinite matrix of random variables whose distribution is invariant under finite permutations of rows and columns. It states that, up to equality in distribution, each random variable in the matrix can be expressed as a function only depending on four key variables: one common to the entire matrix, one that encodes information about its row, one that encodes i…
▽ More
The Aldous-Hoover Theorem concerns an infinite matrix of random variables whose distribution is invariant under finite permutations of rows and columns. It states that, up to equality in distribution, each random variable in the matrix can be expressed as a function only depending on four key variables: one common to the entire matrix, one that encodes information about its row, one that encodes information about its column, and a fourth one specific to the matrix entry.
We state and prove the theorem within a category-theoretic approach to probability, namely the theory of Markov categories. This makes the proof more transparent and intuitive when compared to measure-theoretic ones. A key role is played by a newly identified categorical property, the Cauchy--Schwarz axiom, which also facilitates a new synthetic de Finetti Theorem.
We further provide a variant of our proof using the ordered Markov property and the d-separation criterion, both generalized from Bayesian networks to Markov categories. We expect that this approach will facilitate a systematic development of more complex results in the future, such as categorical approaches to hierarchical exchangeability.
△ Less
Submitted 19 November, 2024;
originally announced November 2024.
-
Positive Moments Forever: Undecidable and Decidable Cases
Authors:
Gemma De les Coves,
Joshua Graf,
Andreas Klingler,
Tim Netzer
Abstract:
We investigate the generalized moment membership problem for matrices, a formulation equivalent to Skolem's problem for linear recurrence sequences. We show decidability for orthogonal, unitary, and real eigenvalue matrices, and undecidability for matrices over certain commutative and non-commutative polynomial rings. As consequences, we deduce that positivity is decidable for simple unitary linea…
▽ More
We investigate the generalized moment membership problem for matrices, a formulation equivalent to Skolem's problem for linear recurrence sequences. We show decidability for orthogonal, unitary, and real eigenvalue matrices, and undecidability for matrices over certain commutative and non-commutative polynomial rings. As consequences, we deduce that positivity is decidable for simple unitary linear recurrence sequences and undecidable for linear recurrence sequences over commutative polynomial rings. As a byproduct, we also prove a free version of Polya's theorem.
△ Less
Submitted 21 May, 2025; v1 submitted 23 April, 2024;
originally announced April 2024.
-
Homotopy Methods for Convex Optimization
Authors:
Andreas Klingler,
Tim Netzer
Abstract:
Convex optimization encompasses a wide range of optimization problems that contain many efficiently solvable subclasses. Interior point methods are currently the state-of-the-art approach for solving such problems, particularly effective for classes like semidefinite programming, quadratic programming, and geometric programming. However, their success hinges on the construction of self-concordant…
▽ More
Convex optimization encompasses a wide range of optimization problems that contain many efficiently solvable subclasses. Interior point methods are currently the state-of-the-art approach for solving such problems, particularly effective for classes like semidefinite programming, quadratic programming, and geometric programming. However, their success hinges on the construction of self-concordant barrier functions for feasible sets. In this work, we investigate and develop a homotopy-based approach to solve convex optimization problems. While homotopy methods have been considered in optimization before, their potential for general convex programs remains underexplored. This approach gradually transforms the feasible set of a trivial optimization problem into the target one while tracking solutions by solving a differential equation, in contrast to traditional central path methods. We establish a criterion that ensures that the homotopy method correctly solves the optimization problem and prove the existence of such homotopies for several important classes, including semidefinite and hyperbolic programs. Furthermore, we demonstrate that our approach numerically outperforms state-of-the-art methods in hyperbolic programming, highlighting its practical advantages.
△ Less
Submitted 27 March, 2025; v1 submitted 4 March, 2024;
originally announced March 2024.
-
Hidden Markov Models and the Bayes Filter in Categorical Probability
Authors:
Tobias Fritz,
Andreas Klingler,
Drew McNeely,
Areeb Shah-Mohammed,
Yuwen Wang
Abstract:
We use Markov categories to generalize the basic theory of Markov chains and hidden Markov models to an abstract setting. This comprises characterizations of hidden Markov models in terms of conditional independences and algorithms for Bayesian filtering and smoothing applicable in all Markov categories with conditionals. When instantiated in appropriate Markov categories, these algorithms special…
▽ More
We use Markov categories to generalize the basic theory of Markov chains and hidden Markov models to an abstract setting. This comprises characterizations of hidden Markov models in terms of conditional independences and algorithms for Bayesian filtering and smoothing applicable in all Markov categories with conditionals. When instantiated in appropriate Markov categories, these algorithms specialize to existing ones such as the Kalman filter, forward-backward algorithm, and the Rauch-Tung-Striebel smoother. We also prove that the sequence of outputs of our abstract Bayes filter is itself a Markov chain with a concrete formula for its transition maps.
There are two main features of this categorical framework. The first is its abstract generality, as manifested in our unified account of hidden Markov models and algorithms for filtering and smoothing in discrete probability, Gaussian probability, measure-theoretic probability, possibilistic nondeterminism and others at the same time. The second feature is the intuitive visual representation of information flow in terms of string diagrams.
△ Less
Submitted 24 January, 2025; v1 submitted 26 January, 2024;
originally announced January 2024.
-
Border Ranks of Positive and Invariant Tensor Decompositions: Applications to Correlations
Authors:
Andreas Klingler,
Tim Netzer,
Gemma De les Coves
Abstract:
The matrix rank and its positive versions are robust for small approximations, i.e. they do not decrease under small perturbations. In contrast, the multipartite tensor rank can collapse for arbitrarily small errors, i.e. there may be a gap between rank and border rank, leading to instabilities in the optimization over sets with fixed tensor rank. Can multipartite positive ranks also collapse for…
▽ More
The matrix rank and its positive versions are robust for small approximations, i.e. they do not decrease under small perturbations. In contrast, the multipartite tensor rank can collapse for arbitrarily small errors, i.e. there may be a gap between rank and border rank, leading to instabilities in the optimization over sets with fixed tensor rank. Can multipartite positive ranks also collapse for small perturbations? In this work, we prove that multipartite positive and invariant tensor decompositions exhibit gaps between rank and border rank, including tensor rank purifications and cyclic separable decompositions. We also prove a correspondence between positive decompositions and membership in certain sets of multipartite probability distributions, and leverage the gaps between rank and border rank to prove that these correlation sets are not closed. It follows that testing membership of probability distributions arising from resources like translational invariant Matrix Product States is impossible in finite time. Overall, this work sheds light on the instability of ranks and the unique behavior of bipartite systems.
△ Less
Submitted 19 February, 2025; v1 submitted 26 April, 2023;
originally announced April 2023.
-
The d-separation criterion in Categorical Probability
Authors:
Tobias Fritz,
Andreas Klingler
Abstract:
The d-separation criterion detects the compatibility of a joint probability distribution with a directed acyclic graph through certain conditional independences. In this work, we study this problem in the context of categorical probability theory by introducing a categorical definition of causal models, a categorical notion of d-separation, and proving an abstract version of the d-separation crite…
▽ More
The d-separation criterion detects the compatibility of a joint probability distribution with a directed acyclic graph through certain conditional independences. In this work, we study this problem in the context of categorical probability theory by introducing a categorical definition of causal models, a categorical notion of d-separation, and proving an abstract version of the d-separation criterion. This approach has two main benefits. First, categorical d-separation is a very intuitive criterion based on topological connectedness. Second, our results apply both to measure-theoretic probability (with standard Borel spaces) and beyond probability theory, including to deterministic and possibilistic networks. It therefore provides a clean proof of the equivalence of local and global Markov properties with causal compatibility for continuous and mixed random variables as well as deterministic and possibilistic variables.
△ Less
Submitted 20 February, 2023; v1 submitted 12 July, 2022;
originally announced July 2022.
-
Approximate Pythagoras Numbers on $*$-algebras over $\mathbb{C}$
Authors:
Paria Abbasi,
Sander Gribling,
Andreas Klingler,
Tim Netzer
Abstract:
The Pythagoras number of a sum of squares is the shortest length among its sums of squares representations. In many algebras, for example real polynomial algebras in two or more variables, there exists no upper bound on the Pythagoras number for all sums of squares. In this paper, we study how Pythagoras numbers in $*$-algebras over $\mathbb{C}$ behave with respect to small perturbations of elemen…
▽ More
The Pythagoras number of a sum of squares is the shortest length among its sums of squares representations. In many algebras, for example real polynomial algebras in two or more variables, there exists no upper bound on the Pythagoras number for all sums of squares. In this paper, we study how Pythagoras numbers in $*$-algebras over $\mathbb{C}$ behave with respect to small perturbations of elements. More precisely, the approximate Pythagoras number of an element is the smallest Pythagoras number among all elements in its $\varepsilon$-ball. We show that these approximate Pythagoras numbers are often significantly smaller than their exact versions, and allow for (almost) dimension-independent upper bounds. Our results use low-rank approximations for Gram matrices of sums of squares and estimates for the operator norm of the Gram map.
△ Less
Submitted 26 July, 2022; v1 submitted 10 September, 2021;
originally announced September 2021.
-
Approximate Completely Positive Semidefinite Factorizations and their Ranks
Authors:
Paria Abbasi,
Andreas Klingler,
Tim Netzer
Abstract:
In this paper we show the existence of approximate completely positive semidefinite (cpsd) factorizations with a cpsd-rank bounded above (almost) independently from the cpsd-rank of the initial matrix. This is particularly relevant since the cpsd-rank of a matrix cannot, in general, be upper bounded by a function only depending on its size. For this purpose, we make use of the Approximate Caratheo…
▽ More
In this paper we show the existence of approximate completely positive semidefinite (cpsd) factorizations with a cpsd-rank bounded above (almost) independently from the cpsd-rank of the initial matrix. This is particularly relevant since the cpsd-rank of a matrix cannot, in general, be upper bounded by a function only depending on its size. For this purpose, we make use of the Approximate Caratheodory Theorem in order to construct an approximate matrix with a low-rank Gram representation. We then employ the Johnson-Lindenstrauss Lemma to improve to a logarithmic dependence of the cpsd-rank on the size.
△ Less
Submitted 4 September, 2023; v1 submitted 11 December, 2020;
originally announced December 2020.
-
Cats climb entails mammals move: preserving hyponymy in compositional distributional semantics
Authors:
Gemma De las Cuevas,
Andreas Klingler,
Martha Lewis,
Tim Netzer
Abstract:
To give vector-based representations of meaning more structure, one approach is to use positive semidefinite (psd) matrices. These allow us to model similarity of words as well as the hyponymy or is-a relationship. Psd matrices can be learnt relatively easily in a given vector space $M\otimes M^*$, but to compose words to form phrases and sentences, we need representations in larger spaces. In thi…
▽ More
To give vector-based representations of meaning more structure, one approach is to use positive semidefinite (psd) matrices. These allow us to model similarity of words as well as the hyponymy or is-a relationship. Psd matrices can be learnt relatively easily in a given vector space $M\otimes M^*$, but to compose words to form phrases and sentences, we need representations in larger spaces. In this paper, we introduce a generic way of composing the psd matrices corresponding to words. We propose that psd matrices for verbs, adjectives, and other functional words be lifted to completely positive (CP) maps that match their grammatical type. This lifting is carried out by our composition rule called Compression, Compr. In contrast to previous composition rules like Fuzz and Phaser (a.k.a. KMult and BMult), Compr preserves hyponymy. Mathematically, Compr is itself a CP map, and is therefore linear and generally non-commutative. We give a number of proposals for the structure of Compr, based on spiders, cups and caps, and generate a range of composition rules. We test these rules on a small sentence entailment dataset, and see some improvements over the performance of Fuzz and Phaser.
△ Less
Submitted 29 May, 2020; v1 submitted 28 May, 2020;
originally announced May 2020.
-
Approximate tensor decompositions: disappearance of many separations
Authors:
Gemma De las Cuevas,
Andreas Klingler,
Tim Netzer
Abstract:
It is well-known that tensor decompositions show separations, that is, that constraints on local terms (such as positivity) may entail an arbitrarily high cost in their representation. Here we show that many of these separations disappear in the approximate case. Specifically, for every approximation error $\varepsilon$ and norm, we define the approximate rank as the minimum rank of an element in…
▽ More
It is well-known that tensor decompositions show separations, that is, that constraints on local terms (such as positivity) may entail an arbitrarily high cost in their representation. Here we show that many of these separations disappear in the approximate case. Specifically, for every approximation error $\varepsilon$ and norm, we define the approximate rank as the minimum rank of an element in the $\varepsilon$-ball with respect to that norm. For positive semidefinite matrices, we show that the separations between rank, purification rank, and separable rank disappear for a large class of Schatten $p$-norms. For nonnegative tensors, we show that the separations between rank, positive semidefinite rank, and nonnegative rank disappear for all $\ell_p$-norms with $p>1$. For the trace norm ($p = 1$), we obtain upper bounds that depend on the ambient dimension. We also provide a deterministic algorithm to obtain the approximate decomposition attaining our bounds. Our main tool is an approximate version of Carathéodory's Theorem. Our results imply that many separations are not robust under small perturbations of the tensor, with implications in quantum many-body systems and communication complexity.
△ Less
Submitted 14 August, 2021; v1 submitted 21 April, 2020;
originally announced April 2020.
