-
Embarrassingly Parallel GFlowNets
Authors:
Tiago da Silva,
Luiz Max Carvalho,
Amauri Souza,
Samuel Kaski,
Diego Mesquita
Abstract:
GFlowNets are a promising alternative to MCMC sampling for discrete compositional random variables. Training GFlowNets requires repeated evaluations of the unnormalized target distribution or reward function. However, for large-scale posterior sampling, this may be prohibitive since it incurs traversing the data several times. Moreover, if the data are distributed across clients, employing standar…
▽ More
GFlowNets are a promising alternative to MCMC sampling for discrete compositional random variables. Training GFlowNets requires repeated evaluations of the unnormalized target distribution or reward function. However, for large-scale posterior sampling, this may be prohibitive since it incurs traversing the data several times. Moreover, if the data are distributed across clients, employing standard GFlowNets leads to intensive client-server communication. To alleviate both these issues, we propose embarrassingly parallel GFlowNet (EP-GFlowNet). EP-GFlowNet is a provably correct divide-and-conquer method to sample from product distributions of the form $R(\cdot) \propto R_1(\cdot) ... R_N(\cdot)$ -- e.g., in parallel or federated Bayes, where each $R_n$ is a local posterior defined on a data partition. First, in parallel, we train a local GFlowNet targeting each $R_n$ and send the resulting models to the server. Then, the server learns a global GFlowNet by enforcing our newly proposed \emph{aggregating balance} condition, requiring a single communication step. Importantly, EP-GFlowNets can also be applied to multi-objective optimization and model reuse. Our experiments illustrate the EP-GFlowNets's effectiveness on many tasks, including parallel Bayesian phylogenetics, multi-objective multiset, sequence generation, and federated Bayesian structure learning.
△ Less
Submitted 5 June, 2024;
originally announced June 2024.
-
Locking and Quacking: Stacking Bayesian model predictions by log-pooling and superposition
Authors:
Yuling Yao,
Luiz Max Carvalho,
Diego Mesquita,
Yann McLatchie
Abstract:
Combining predictions from different models is a central problem in Bayesian inference and machine learning more broadly. Currently, these predictive distributions are almost exclusively combined using linear mixtures such as Bayesian model averaging, Bayesian stacking, and mixture of experts. Such linear mixtures impose idiosyncrasies that might be undesirable for some applications, such as multi…
▽ More
Combining predictions from different models is a central problem in Bayesian inference and machine learning more broadly. Currently, these predictive distributions are almost exclusively combined using linear mixtures such as Bayesian model averaging, Bayesian stacking, and mixture of experts. Such linear mixtures impose idiosyncrasies that might be undesirable for some applications, such as multi-modality. While there exist alternative strategies (e.g. geometric bridge or superposition), optimising their parameters usually involves computing an intractable normalising constant repeatedly. We present two novel Bayesian model combination tools. These are generalisations of model stacking, but combine posterior densities by log-linear pooling (locking) and quantum superposition (quacking). To optimise model weights while avoiding the burden of normalising constants, we investigate the Hyvarinen score of the combined posterior predictions. We demonstrate locking with an illustrative example and discuss its practical application with importance sampling.
△ Less
Submitted 12 May, 2023;
originally announced May 2023.
-
Beyond the shortest path: the path length index as a distribution
Authors:
Leonardo B. L. Santos,
Luiz Max Carvalho,
Giovanni G. Soares,
Leonardo N. Ferreira,
Igor M. Sokolov
Abstract:
The traditional complex network approach considers only the shortest paths from one node to another, not taking into account several other possible paths. This limitation is significant, for example, in urban mobility studies. In this short report, as the first steps, we present an exhaustive approach to address that problem and show we can go beyond the shortest path, but we do not need to go so…
▽ More
The traditional complex network approach considers only the shortest paths from one node to another, not taking into account several other possible paths. This limitation is significant, for example, in urban mobility studies. In this short report, as the first steps, we present an exhaustive approach to address that problem and show we can go beyond the shortest path, but we do not need to go so far: we present an interactive procedure and an early stop possibility. After presenting some fundamental concepts in graph theory, we presented an analytical solution for the problem of counting the number of possible paths between two nodes in complete graphs, and a depth-limited approach to get all possible paths between each pair of nodes in a general graph (an NP-hard problem). We do not collapse the distribution of path lengths between a pair of nodes into a scalar number, we look at the distribution itself - taking all paths up to a pre-defined path length (considering a truncated distribution), and show the impact of that approach on the most straightforward distance-based graph index: the walk/path length.
△ Less
Submitted 6 October, 2022;
originally announced October 2022.
