-
Scalable Bernoulli factories for Bayesian inference with intractable likelihoods
Authors:
Timothée Stumpf-Fétizon,
Flávio B. Gonçalves
Abstract:
Bernoulli factory MCMC algorithms implement accept-reject Markov chains without explicit computation of acceptance probabilities, and are used to target posterior distributions associated with intractable likelihood models. These algorithms often mix better than alternatives based on data augmentation or acceptance probability estimation. However, we show that their computational performance typic…
▽ More
Bernoulli factory MCMC algorithms implement accept-reject Markov chains without explicit computation of acceptance probabilities, and are used to target posterior distributions associated with intractable likelihood models. These algorithms often mix better than alternatives based on data augmentation or acceptance probability estimation. However, we show that their computational performance typically deteriorates exponentially with data size. To address this, we propose a simple divide-and-conquer Bernoulli factory MCMC algorithm and prove that it has polynomial complexity of degree between 1 and 2, with the exact degree depending on the existence of efficient unbiased estimators of the intractable likelihood ratio. We demonstrate the effectiveness of our approach with applications to Bayesian inference in two intractable likelihood models, and observe respective polynomial cost of degree 1.2 and 1 in the data size.
△ Less
Submitted 8 May, 2025;
originally announced May 2025.
-
Exact Bayesian inference for Markov switching diffusions
Authors:
Timothée Stumpf-Fétizon,
Krzysztof Łatuszyński,
Jan Palczewski,
Gareth Roberts
Abstract:
We give the first exact Bayesian methodology for the problem of inference in discretely observed regime switching diffusions. We design an MCMC and an MCEM algorithm that target the exact posterior of diffusion parameters and the latent regime process. The algorithms are exact in the sense that they target the correct posterior distribution of the continuous model, so that the errors are due to Mo…
▽ More
We give the first exact Bayesian methodology for the problem of inference in discretely observed regime switching diffusions. We design an MCMC and an MCEM algorithm that target the exact posterior of diffusion parameters and the latent regime process. The algorithms are exact in the sense that they target the correct posterior distribution of the continuous model, so that the errors are due to Monte Carlo only. Switching diffusion models extend ordinary diffusions by allowing for jumps in instantaneous drift and volatility. The jumps are driven by a latent, continuous time Markov switching process. We illustrate the method on numerical examples, including an empirical analysis of the method's scalability in the length of the time series, and find that it is comparable in computational cost with discrete approximations while avoiding their shortcomings.
△ Less
Submitted 13 February, 2025;
originally announced February 2025.
-
MCMC for multi-modal distributions
Authors:
Krzysztof Łatuszyński,
Matthew T. Moores,
Timothée Stumpf-Fétizon
Abstract:
We explain the fundamental challenges of sampling from multimodal distributions, particularly for high-dimensional problems. We present the major types of MCMC algorithms that are designed for this purpose, including parallel tempering, mode jumping and Wang-Landau, as well as several state-of-the-art approaches that have recently been proposed. We demonstrate these methods using both synthetic an…
▽ More
We explain the fundamental challenges of sampling from multimodal distributions, particularly for high-dimensional problems. We present the major types of MCMC algorithms that are designed for this purpose, including parallel tempering, mode jumping and Wang-Landau, as well as several state-of-the-art approaches that have recently been proposed. We demonstrate these methods using both synthetic and real-world examples of multimodal distributions with discrete or continuous state spaces.
△ Less
Submitted 10 January, 2025;
originally announced January 2025.
-
Scalable Bayesian computation for crossed and nested hierarchical models
Authors:
Omiros Papaspiliopoulos,
Timothée Stumpf-Fétizon,
Giacomo Zanella
Abstract:
We develop sampling algorithms to fit Bayesian hierarchical models, the computational complexity of which scales linearly with the number of observations and the number of parameters in the model. We focus on crossed random effect and nested multilevel models, which are used ubiquitously in applied sciences. The posterior dependence in both classes is sparse: in crossed random effects models it re…
▽ More
We develop sampling algorithms to fit Bayesian hierarchical models, the computational complexity of which scales linearly with the number of observations and the number of parameters in the model. We focus on crossed random effect and nested multilevel models, which are used ubiquitously in applied sciences. The posterior dependence in both classes is sparse: in crossed random effects models it resembles a random graph, whereas in nested multilevel models it is tree-structured. For each class we identify a framework for scalable computation, building on previous work. Methods for crossed models are based on extensions of appropriately designed collapsed Gibbs samplers, where we introduce the idea of local centering; while methods for nested models are based on sparse linear algebra and data augmentation. We provide a theoretical analysis of the proposed algorithms in some simplified settings, including a comparison with previously proposed methodologies and an average-case analysis based on random graph theory. Numerical experiments, including two challenging real data analyses on predicting electoral results and real estate prices, compare with off-the-shelf Hamiltonian Monte Carlo, displaying drastic improvement in performance.
△ Less
Submitted 5 October, 2023; v1 submitted 19 March, 2021;
originally announced March 2021.
-
Bayesian forecasting of electoral outcomes with new parties' competition
Authors:
José García Montalvo,
Omiros Papaspiliopoulos,
Timothée Stumpf-Fétizon
Abstract:
This paper proposed a methodology to forecast electoral outcomes using the result of the combination of a fundamental model and a model-based aggregation of polls. We propose a Bayesian hierarchical structure for the fundamental model that synthesises data at the provincial, regional and national level. We use a Bayesian strategy to combine the fundamental model with the information coming for rec…
▽ More
This paper proposed a methodology to forecast electoral outcomes using the result of the combination of a fundamental model and a model-based aggregation of polls. We propose a Bayesian hierarchical structure for the fundamental model that synthesises data at the provincial, regional and national level. We use a Bayesian strategy to combine the fundamental model with the information coming for recent polls. This model can naturally be updated every time new information, for instance a new poll, becomes available. This methodology is well suited to deal with increasingly frequent situations in which new political parties enter an electoral competition, although our approach is general enough to accommodate any other electoral situation. We illustrate the advantages of our method using the 2015 Spanish Congressional Election in which two new parties ended up receiving 30\% of the votes. We compare the predictive performance of our model versus alternative models. In general the predictions of our model outperform the alternative specifications, including hybrid models that combine fundamental and polls models. Our predictions are, in relative terms, particularly accurate in predicting the seats obtained by each political party.
△ Less
Submitted 1 February, 2019; v1 submitted 9 December, 2016;
originally announced December 2016.
