Showing 1–50 of 70 results for author: Drovandi, C

Beyond data: leveraging non-empirical information and expert knowledge in Bayesian model calibration

Authors: Sarah A. Vollert, Christopher Drovandi, Cailan Jeynes-Smith, Luz V. Pascal, Matthew P. Adams

Abstract: Mathematical models connect theory with the real world through data, enabling us to interpret, understand, and predict complex phenomena. However, scientific knowledge often extends beyond what can be empirically measured, offering valuable insights into complex and uncertain systems. Here, we introduce a statistical framework for calibrating mathematical models using non-empirical information. Th… ▽ More Mathematical models connect theory with the real world through data, enabling us to interpret, understand, and predict complex phenomena. However, scientific knowledge often extends beyond what can be empirically measured, offering valuable insights into complex and uncertain systems. Here, we introduce a statistical framework for calibrating mathematical models using non-empirical information. Through examples in ecology, biology, and medicine, we demonstrate how expert knowledge, scientific theory, and qualitative observations can meaningfully constrain models. In each case, these non-empirical insights guide models toward more realistic dynamics and more informed predictions than empirical data alone could achieve. Now, our understanding of the systems represented by mathematical models is not limited by the data that can be obtained; they instead sit at the edge of scientific understanding. △ Less

Submitted 27 May, 2025; originally announced May 2025.

A Unified Framework for Variable Selection in Model-Based Clustering with Missing Not at Random

Authors: Binh H. Ho, Long Nguyen Chi, TrungTin Nguyen, Binh T. Nguyen, Van Ha Hoang, Christopher Drovandi

Abstract: Model-based clustering integrated with variable selection is a powerful tool for uncovering latent structures within complex data. However, its effectiveness is often hindered by challenges such as identifying relevant variables that define heterogeneous subgroups and handling data that are missing not at random, a prevalent issue in fields like transcriptomics. While several notable methods have… ▽ More Model-based clustering integrated with variable selection is a powerful tool for uncovering latent structures within complex data. However, its effectiveness is often hindered by challenges such as identifying relevant variables that define heterogeneous subgroups and handling data that are missing not at random, a prevalent issue in fields like transcriptomics. While several notable methods have been proposed to address these problems, they typically tackle each issue in isolation, thereby limiting their flexibility and adaptability. This paper introduces a unified framework designed to address these challenges simultaneously. Our approach incorporates a data-driven penalty matrix into penalized clustering to enable more flexible variable selection, along with a mechanism that explicitly models the relationship between missingness and latent class membership. We demonstrate that, under certain regularity conditions, the proposed framework achieves both asymptotic consistency and selection consistency, even in the presence of missing data. This unified strategy significantly enhances the capability and efficiency of model-based clustering, advancing methodologies for identifying informative variables that define homogeneous subgroups in the presence of complex missing data patterns. The performance of the framework, including its computational efficiency, is evaluated through simulations and demonstrated using both synthetic and real-world transcriptomic datasets. △ Less

Submitted 25 May, 2025; originally announced May 2025.

Model Selection for Gaussian-gated Gaussian Mixture of Experts Using Dendrograms of Mixing Measures

Authors: Tuan Thai, TrungTin Nguyen, Dat Do, Nhat Ho, Christopher Drovandi

Abstract: Mixture of Experts (MoE) models constitute a widely utilized class of ensemble learning approaches in statistics and machine learning, known for their flexibility and computational efficiency. They have become integral components in numerous state-of-the-art deep neural network architectures, particularly for analyzing heterogeneous data across diverse domains. Despite their practical success, the… ▽ More Mixture of Experts (MoE) models constitute a widely utilized class of ensemble learning approaches in statistics and machine learning, known for their flexibility and computational efficiency. They have become integral components in numerous state-of-the-art deep neural network architectures, particularly for analyzing heterogeneous data across diverse domains. Despite their practical success, the theoretical understanding of model selection, especially concerning the optimal number of mixture components or experts, remains limited and poses significant challenges. These challenges primarily stem from the inclusion of covariates in both the Gaussian gating functions and expert networks, which introduces intrinsic interactions governed by partial differential equations with respect to their parameters. In this paper, we revisit the concept of dendrograms of mixing measures and introduce a novel extension to Gaussian-gated Gaussian MoE models that enables consistent estimation of the true number of mixture components and achieves the pointwise optimal convergence rate for parameter estimation in overfitted scenarios. Notably, this approach circumvents the need to train and compare a range of models with varying numbers of components, thereby alleviating the computational burden, particularly in high-dimensional or deep neural network settings. Experimental results on synthetic data demonstrate the effectiveness of the proposed method in accurately recovering the number of experts. It outperforms common criteria such as the Akaike information criterion, the Bayesian information criterion, and the integrated completed likelihood, while achieving optimal convergence rates for parameter estimation and accurately approximating the regression function. △ Less

Submitted 23 May, 2025; v1 submitted 19 May, 2025; originally announced May 2025.

Comments: Correct typos and update the numerical experiments. Tuan Thai and TrungTin Nguyen are co-first authors

Robustifying Approximate Bayesian Computation

Authors: Chaya Weerasinghe, David T. Frazier, Ruben Loaiza-Maya, Christopher Drovandi

Abstract: Approximate Bayesian computation (ABC) is one of the most popular "likelihood-free" methods. These methods have been applied in a wide range of fields by providing solutions to intractable likelihood problems in which exact Bayesian approaches are either infeasible or computationally costly. However, the performance of ABC can be unreliable when dealing with model misspecification. To circumvent t… ▽ More Approximate Bayesian computation (ABC) is one of the most popular "likelihood-free" methods. These methods have been applied in a wide range of fields by providing solutions to intractable likelihood problems in which exact Bayesian approaches are either infeasible or computationally costly. However, the performance of ABC can be unreliable when dealing with model misspecification. To circumvent the poor behavior of ABC in these settings, we propose a novel ABC approach that is robust to model misspecification. This new method can deliver more accurate statistical inference under model misspecification than alternatives and also enables the detection of summary statistics that are incompatible with the assumed data-generating process. We demonstrate the effectiveness of our approach through several simulated examples, where it delivers more accurate point estimates and uncertainty quantification over standard ABC approaches when the model is misspecified. Additionally, we apply our approach to an empirical example, further showcasing its advantages over alternative methods. △ Less

Submitted 7 April, 2025; originally announced April 2025.

Simulation-based Bayesian inference under model misspecification

Authors: Ryan P. Kelly, David J. Warne, David T. Frazier, David J. Nott, Michael U. Gutmann, Christopher Drovandi

Abstract: Simulation-based Bayesian inference (SBI) methods are widely used for parameter estimation in complex models where evaluating the likelihood is challenging but generating simulations is relatively straightforward. However, these methods commonly assume that the simulation model accurately reflects the true data-generating process, an assumption that is frequently violated in realistic scenarios. I… ▽ More Simulation-based Bayesian inference (SBI) methods are widely used for parameter estimation in complex models where evaluating the likelihood is challenging but generating simulations is relatively straightforward. However, these methods commonly assume that the simulation model accurately reflects the true data-generating process, an assumption that is frequently violated in realistic scenarios. In this paper, we focus on the challenges faced by SBI methods under model misspecification. We consolidate recent research aimed at mitigating the effects of misspecification, highlighting three key strategies: i) robust summary statistics, ii) generalised Bayesian inference, and iii) error modelling and adjustment parameters. To illustrate both the vulnerabilities of popular SBI methods and the effectiveness of misspecification-robust alternatives, we present empirical results on an illustrative example. △ Less

Submitted 15 March, 2025; originally announced March 2025.

Comments: 46 pages, 8 figures

A Principled Approach to Bayesian Transfer Learning

Authors: Adam Bretherton, Joshua J. Bon, David J. Warne, Kerrie Mengersen, Christopher Drovandi

Abstract: Updating $\textit{a priori}$ information given some observed data is the core tenet of Bayesian inference. Bayesian transfer learning extends this idea by incorporating information from a related dataset to improve the inference on the observed data which may have been collected under slightly different settings. The use of related information can be useful when the observed data is scarce, for ex… ▽ More Updating $\textit{a priori}$ information given some observed data is the core tenet of Bayesian inference. Bayesian transfer learning extends this idea by incorporating information from a related dataset to improve the inference on the observed data which may have been collected under slightly different settings. The use of related information can be useful when the observed data is scarce, for example. Current Bayesian transfer learning methods that are based on the so-called $\textit{power prior}$ can adaptively transfer information from related data. Unfortunately, it is not always clear under which scenario Bayesian transfer learning performs best or even if it will improve Bayesian inference. Additionally, current power prior methods rely on conjugacy to evaluate the posterior of interest. We propose using leave-one-out cross validation on the target dataset as a means of evaluating Bayesian transfer learning methods. Further, we introduce a new framework, $\textit{transfer sequential Monte Carlo}$, for power prior approaches that efficiently chooses the transfer parameter while avoiding the need for conjugate priors. We assess the performance of our proposed methods in two comprehensive simulation studies. △ Less

Submitted 19 March, 2025; v1 submitted 27 February, 2025; originally announced February 2025.

Comments: 17 pages, 2 tables, 2 figures

MSC Class: 62F15; 62F07 (Primary) 62-08 (Secondary)

The Polynomial Stein Discrepancy for Assessing Moment Convergence

Authors: Narayan Srinivasan, Matthew Sutton, Christopher Drovandi, Leah F South

Abstract: We propose a novel method for measuring the discrepancy between a set of samples and a desired posterior distribution for Bayesian inference. Classical methods for assessing sample quality like the effective sample size are not appropriate for scalable Bayesian sampling algorithms, such as stochastic gradient Langevin dynamics, that are asymptotically biased. Instead, the gold standard is to use t… ▽ More We propose a novel method for measuring the discrepancy between a set of samples and a desired posterior distribution for Bayesian inference. Classical methods for assessing sample quality like the effective sample size are not appropriate for scalable Bayesian sampling algorithms, such as stochastic gradient Langevin dynamics, that are asymptotically biased. Instead, the gold standard is to use the kernel Stein Discrepancy (KSD), which is itself not scalable given its quadratic cost in the number of samples. The KSD and its faster extensions also typically suffer from the curse-of-dimensionality and can require extensive tuning. To address these limitations, we develop the polynomial Stein discrepancy (PSD) and an associated goodness-of-fit test. While the new test is not fully convergence-determining, we prove that it detects differences in the first r moments in the Bernstein-von Mises limit. We empirically show that the test has higher power than its competitors in several examples, and at a lower computational cost. Finally, we demonstrate that the PSD can assist practitioners to select hyper-parameters of Bayesian sampling algorithms more efficiently than competitors. △ Less

Submitted 6 December, 2024; originally announced December 2024.

Comments: 17 Pages, 14 Figs

The Statistical Accuracy of Neural Posterior and Likelihood Estimation

Authors: David T. Frazier, Ryan Kelly, Christopher Drovandi, David J. Warne

Abstract: Neural posterior estimation (NPE) and neural likelihood estimation (NLE) are machine learning approaches that provide accurate posterior, and likelihood, approximations in complex modeling scenarios, and in situations where conducting amortized inference is a necessity. While such methods have shown significant promise across a range of diverse scientific applications, the statistical accuracy of… ▽ More Neural posterior estimation (NPE) and neural likelihood estimation (NLE) are machine learning approaches that provide accurate posterior, and likelihood, approximations in complex modeling scenarios, and in situations where conducting amortized inference is a necessity. While such methods have shown significant promise across a range of diverse scientific applications, the statistical accuracy of these methods is so far unexplored. In this manuscript, we give, for the first time, an in-depth exploration on the statistical behavior of NPE and NLE. We prove that these methods have similar theoretical guarantees to common statistical methods like approximate Bayesian computation (ABC) and Bayesian synthetic likelihood (BSL). While NPE and NLE methods are just as accurate as ABC and BSL, we prove that this accuracy can often be achieved at a vastly reduced computational cost, and will therefore deliver more attractive approximations than ABC and BSL in certain problems. We verify our results theoretically and in several examples from the literature. △ Less

Submitted 18 November, 2024; originally announced November 2024.

A Comprehensive Guide to Simulation-based Inference in Computational Biology

Authors: Xiaoyu Wang, Ryan P. Kelly, Adrianne L. Jenner, David J. Warne, Christopher Drovandi

Abstract: Computational models are invaluable in capturing the complexities of real-world biological processes. Yet, the selection of appropriate algorithms for inference tasks, especially when dealing with real-world observational data, remains a challenging and underexplored area. This gap has spurred the development of various parameter estimation algorithms, particularly within the realm of Simulation-B… ▽ More Computational models are invaluable in capturing the complexities of real-world biological processes. Yet, the selection of appropriate algorithms for inference tasks, especially when dealing with real-world observational data, remains a challenging and underexplored area. This gap has spurred the development of various parameter estimation algorithms, particularly within the realm of Simulation-Based Inference (SBI), such as neural and statistical SBI methods. Limited research exists on how to make informed choices on SBI methods when faced with real-world data, which often results in some form of model misspecification. In this paper, we provide comprehensive guidelines for deciding between SBI approaches for complex biological models. We apply the guidelines to two agent-based models that describe cellular dynamics using real-world data. Our study unveils a critical insight: while neural SBI methods demand significantly fewer simulations for inference results, they tend to yield biased estimations, a trend persistent even with robust variants of these algorithms. On the other hand, the accuracy of statistical SBI methods enhances substantially as the number of simulations increases. This finding suggests that, given a sufficient computational budget, statistical SBI can surpass neural SBI in performance. Our results not only shed light on the efficacy of different SBI methodologies in real-world scenarios but also suggest potential avenues for enhancing neural SBI approaches. This study is poised to be a useful resource for computational biologists navigating the intricate landscape of SBI in biological modeling. △ Less

Submitted 29 September, 2024; originally announced September 2024.

Ecosystem knowledge should replace coexistence and stability assumptions in ecological network modelling

Authors: Sarah A. Vollert, Christopher Drovandi, Matthew P. Adams

Abstract: Quantitative population modelling is an invaluable tool for identifying the cascading effects of ecosystem management and interventions. Ecosystem models are often constructed by assuming stability and coexistence in ecological communities as a proxy for abundance data when monitoring programs are not available. However, a growing body of literature suggests that these assumptions are inappropriat… ▽ More Quantitative population modelling is an invaluable tool for identifying the cascading effects of ecosystem management and interventions. Ecosystem models are often constructed by assuming stability and coexistence in ecological communities as a proxy for abundance data when monitoring programs are not available. However, a growing body of literature suggests that these assumptions are inappropriate for modelling conservation outcomes. In this work, we develop an alternative for dataless population modelling that instead relies on expert-elicited knowledge of species abundances. While time series abundance data is often not available for ecosystems of interest, these systems may still be highly studied or observed in an informal capacity. In particular, limits on population sizes and their capacity to rapidly change during an observation period can be reasonably elicited for many species. We propose a robust framework for generating an ensemble of ecosystem models whose population predictions match the expected population dynamics, as defined by experts. Our new Bayesian algorithm systematically removes model parameters that lead to unreasonable population predictions without incurring excessive computational costs. Our results demonstrate that models constructed using expert-elicited information, rather than stability and coexistence assumptions, can dramatically impact population predictions, expected responses to management, conservation decision-making, and long-term ecosystem behaviour. In the absence of data, we argue that field observations and expert knowledge are preferred for representing ecosystems observed in nature instead of theoretical assumptions of coexistence and stability. △ Less

Submitted 5 September, 2024; v1 submitted 1 May, 2024; originally announced May 2024.

Exact Sampling of Gibbs Measures with Estimated Losses

Authors: David T. Frazier, Jeremias Knoblauch, Jack Jewson, Christopher Drovandi

Abstract: In recent years, the shortcomings of Bayesian posteriors as inferential devices have received increased attention. A popular strategy for fixing them has been to instead target a Gibbs measure based on losses that connect a parameter of interest to observed data. However, existing theory for such inference procedures assumes these losses are analytically available, while in many situations these l… ▽ More In recent years, the shortcomings of Bayesian posteriors as inferential devices have received increased attention. A popular strategy for fixing them has been to instead target a Gibbs measure based on losses that connect a parameter of interest to observed data. However, existing theory for such inference procedures assumes these losses are analytically available, while in many situations these losses must be stochastically estimated using pseudo-observations. In such cases, we show that when standard Markov Chain Monte Carlo algorithms are used to produce posterior samples, the resulting posterior exhibits strong dependence on the number of pseudo-observations: unless the number of pseudo-observations diverge sufficiently fast the resulting posterior will concentrate very slowly. However, we show that in many situations it is feasible to alleviate this dependence entirely using a modified piecewise deterministic Markov process (PDMP) sampler, and we formally and empirically show that these samplers produce posterior draws that have no dependence on the number of pseudo-observations used to estimate the loss within the Gibbs Measure. We apply our results to three examples that feature intractable likelihoods and model misspecification. △ Less

Submitted 22 April, 2025; v1 submitted 24 April, 2024; originally announced April 2024.

Comments: This is a substantially updated and expanded version of the previous submission, "The Impact of Loss Estimation on Gibbs Measures". This version contains a new algorithm that delivers unbiased samples from the intractable Gibbs measure and was not considered in the previous version

Preconditioned Neural Posterior Estimation for Likelihood-free Inference

Authors: Xiaoyu Wang, Ryan P. Kelly, David J. Warne, Christopher Drovandi

Abstract: Simulation based inference (SBI) methods enable the estimation of posterior distributions when the likelihood function is intractable, but where model simulation is feasible. Popular neural approaches to SBI are the neural posterior estimator (NPE) and its sequential version (SNPE). These methods can outperform statistical SBI approaches such as approximate Bayesian computation (ABC), particularly… ▽ More Simulation based inference (SBI) methods enable the estimation of posterior distributions when the likelihood function is intractable, but where model simulation is feasible. Popular neural approaches to SBI are the neural posterior estimator (NPE) and its sequential version (SNPE). These methods can outperform statistical SBI approaches such as approximate Bayesian computation (ABC), particularly for relatively small numbers of model simulations. However, we show in this paper that the NPE methods are not guaranteed to be highly accurate, even on problems with low dimension. In such settings the posterior cannot be accurately trained over the prior predictive space, and even the sequential extension remains sub-optimal. To overcome this, we propose preconditioned NPE (PNPE) and its sequential version (PSNPE), which uses a short run of ABC to effectively eliminate regions of parameter space that produce large discrepancy between simulations and data and allow the posterior emulator to be more accurately trained. We present comprehensive empirical evidence that this melding of neural and statistical SBI methods improves performance over a range of examples, including a motivating example involving a complex agent-based model applied to real tumour growth data. △ Less

Submitted 21 April, 2024; originally announced April 2024.

Comments: 31 pages, 11 figures

Calibrated Generalized Bayesian Inference

Authors: David T. Frazier, Christopher Drovandi, Robert Kohn

Abstract: We provide a simple and general solution for accurate uncertainty quantification of Bayesian inference in misspecified or approximate models, and for generalized posteriors more generally. While existing solutions are based on explicit Gaussian posterior approximations, or post-processing procedures, we demonstrate that correct uncertainty quantification can be achieved by substituting the usual p… ▽ More We provide a simple and general solution for accurate uncertainty quantification of Bayesian inference in misspecified or approximate models, and for generalized posteriors more generally. While existing solutions are based on explicit Gaussian posterior approximations, or post-processing procedures, we demonstrate that correct uncertainty quantification can be achieved by substituting the usual posterior with an intuitively appealing alternative posterior that conveys the same information. This solution applies to both likelihood-based and loss-based posteriors, and we formally demonstrate the reliable uncertainty quantification of this approach. The new approach is demonstrated through a range of examples, including linear models, and doubly intractable models. △ Less

Submitted 19 November, 2024; v1 submitted 26 November, 2023; originally announced November 2023.

Comments: This paper is a substantially revised version of arXiv:2302.06031v1. This revised version has a slightly different focus, additional examples, and theoretical results, as well as different authors

Adaptively switching between a particle marginal Metropolis-Hastings and a particle Gibbs kernel in SMC$^2$

Authors: Imke Botha, Robert Kohn, Leah South, Christopher Drovandi

Abstract: Sequential Monte Carlo squared (SMC$^2$; Chopin et al., 2012) methods can be used to sample from the exact posterior distribution of intractable likelihood state space models. These methods are the SMC analogue to particle Markov chain Monte Carlo (MCMC; Andrieu et al., 2010) and rely on particle MCMC kernels to mutate the particles at each iteration. Two options for the particle MCMC kernels are… ▽ More Sequential Monte Carlo squared (SMC$^2$; Chopin et al., 2012) methods can be used to sample from the exact posterior distribution of intractable likelihood state space models. These methods are the SMC analogue to particle Markov chain Monte Carlo (MCMC; Andrieu et al., 2010) and rely on particle MCMC kernels to mutate the particles at each iteration. Two options for the particle MCMC kernels are particle marginal Metropolis-Hastings (PMMH) and particle Gibbs (PG). We introduce a method to adaptively select the particle MCMC kernel at each iteration of SMC$^2$, with a particular focus on switching between a PMMH and PG kernel. The resulting method can significantly improve the efficiency of SMC$^2$ compared to using a fixed particle MCMC kernel throughout the algorithm. Code for our methods is available at https://github.com/imkebotha/kernel_switching_smc2. △ Less

Submitted 21 July, 2023; originally announced July 2023.

Unlocking ensemble ecosystem modelling for large and complex networks

Authors: Sarah A. Vollert, Christopher Drovandi, Matthew P. Adams

Abstract: The potential effects of conservation actions on threatened species can be predicted using ensemble ecosystem models by forecasting populations with and without intervention. These model ensembles commonly assume stable coexistence of species in the absence of available data. However, existing ensemble-generation methods become computationally inefficient as the size of the ecosystem network incre… ▽ More The potential effects of conservation actions on threatened species can be predicted using ensemble ecosystem models by forecasting populations with and without intervention. These model ensembles commonly assume stable coexistence of species in the absence of available data. However, existing ensemble-generation methods become computationally inefficient as the size of the ecosystem network increases, preventing larger networks from being studied. We present a novel sequential Monte Carlo sampling approach for ensemble generation that is orders of magnitude faster than existing approaches. We demonstrate that the methods produce equivalent parameter inferences, model predictions, and tightly constrained parameter combinations using a novel sensitivity analysis method. For one case study, we demonstrate a speed-up from 108 days to 6 hours, while maintaining equivalent ensembles. Additionally, we demonstrate how to identify the parameter combinations that strongly drive feasibility and stability, drawing ecological insight from the ensembles. Now, for the first time, larger and more realistic networks can be practically simulated and analysed. △ Less

Submitted 24 January, 2024; v1 submitted 21 July, 2023; originally announced July 2023.

Journal ref: PLoS Comput Biol 20(3): e1011976

Wasserstein Gaussianization and Efficient Variational Bayes for Robust Bayesian Synthetic Likelihood

Authors: Nhat-Minh Nguyen, Minh-Ngoc Tran, Christopher Drovandi, David Nott

Abstract: The Bayesian Synthetic Likelihood (BSL) method is a widely-used tool for likelihood-free Bayesian inference. This method assumes that some summary statistics are normally distributed, which can be incorrect in many applications. We propose a transformation, called the Wasserstein Gaussianization transformation, that uses a Wasserstein gradient flow to approximately transform the distribution of th… ▽ More The Bayesian Synthetic Likelihood (BSL) method is a widely-used tool for likelihood-free Bayesian inference. This method assumes that some summary statistics are normally distributed, which can be incorrect in many applications. We propose a transformation, called the Wasserstein Gaussianization transformation, that uses a Wasserstein gradient flow to approximately transform the distribution of the summary statistics into a Gaussian distribution. BSL also implicitly requires compatibility between simulated summary statistics under the working model and the observed summary statistics. A robust BSL variant which achieves this has been developed in the recent literature. We combine the Wasserstein Gaussianization transformation with robust BSL, and an efficient Variational Bayes procedure for posterior approximation, to develop a highly efficient and reliable approximate Bayesian inference method for likelihood-free problems. △ Less

Submitted 14 August, 2024; v1 submitted 24 May, 2023; originally announced May 2023.

Generalised likelihood profiles for models with intractable likelihoods

Authors: David J. Warne, Oliver J. Maclaren, Elliot J. Carr, Matthew J. Simpson, Christopher Drovandi

Abstract: Likelihood profiling is an efficient and powerful frequentist approach for parameter estimation, uncertainty quantification and practical identifiablity analysis. Unfortunately, these methods cannot be easily applied for stochastic models without a tractable likelihood function. Such models are typical in many fields of science, rendering these classical approaches impractical in these settings. T… ▽ More Likelihood profiling is an efficient and powerful frequentist approach for parameter estimation, uncertainty quantification and practical identifiablity analysis. Unfortunately, these methods cannot be easily applied for stochastic models without a tractable likelihood function. Such models are typical in many fields of science, rendering these classical approaches impractical in these settings. To address this limitation, we develop a new approach to generalising the methods of likelihood profiling for situations when the likelihood cannot be evaluated but stochastic simulations of the assumed data generating process are possible. Our approach is based upon recasting developments from generalised Bayesian inference into a frequentist setting. We derive a method for constructing generalised likelihood profiles and calibrating these profiles to achieve desired frequentist coverage for a given coverage level. We demonstrate the performance of our method on realistic examples from the literature and highlight the capability of our approach for the purpose of practical identifability analysis for models with intractable likelihoods. △ Less

Submitted 19 May, 2023; v1 submitted 18 May, 2023; originally announced May 2023.

MSC Class: 62M20 (Primary) 62-08; 62F40 (Secondary)

Bayesian inference for misspecified generative models

Authors: David J. Nott, Christopher Drovandi, David T. Frazier

Abstract: Bayesian inference is a powerful tool for combining information in complex settings, a task of increasing importance in modern applications. However, Bayesian inference with a flawed model can produce unreliable conclusions. This review discusses approaches to performing Bayesian inference when the model is misspecified, where by misspecified we mean that the analyst is unwilling to act as if the… ▽ More Bayesian inference is a powerful tool for combining information in complex settings, a task of increasing importance in modern applications. However, Bayesian inference with a flawed model can produce unreliable conclusions. This review discusses approaches to performing Bayesian inference when the model is misspecified, where by misspecified we mean that the analyst is unwilling to act as if the model is correct. Much has been written about this topic, and in most cases we do not believe that a conventional Bayesian analysis is meaningful when there is serious model misspecification. Nevertheless, in some cases it is possible to use a well-specified model to give meaning to a Bayesian analysis of a misspecified model and we will focus on such cases. Three main classes of methods are discussed - restricted likelihood methods, which use a model based on a non-sufficient summary of the original data; modular inference methods which use a model constructed from coupled submodels and some of the submodels are correctly specified; and the use of a reference model to construct a projected posterior or predictive distribution for a simplified model considered to be useful for prediction or interpretation. △ Less

Submitted 18 May, 2023; v1 submitted 15 May, 2023; originally announced May 2023.

Comments: Review paper under submission for Annual Review of Statistics and its Application

Bayesian Synthetic Likelihood

Authors: David T. Frazier, Christopher Drovandi, David J. Nott

Abstract: Bayesian statistics is concerned with conducting posterior inference for the unknown quantities in a given statistical model. Conventional Bayesian inference requires the specification of a probabilistic model for the observed data, and the construction of the resulting likelihood function. However, sometimes the model is so complicated that evaluation of the likelihood is infeasible, which render… ▽ More Bayesian statistics is concerned with conducting posterior inference for the unknown quantities in a given statistical model. Conventional Bayesian inference requires the specification of a probabilistic model for the observed data, and the construction of the resulting likelihood function. However, sometimes the model is so complicated that evaluation of the likelihood is infeasible, which renders exact Bayesian inference impossible. Bayesian synthetic likelihood (BSL) is a posterior approximation procedure that can be used to conduct inference in situations where the likelihood is intractable, but where simulation from the model is straightforward. In this entry, we give a high-level presentation of BSL, and its extensions aimed at delivering scalable and robust posterior inferences. △ Less

Submitted 10 May, 2023; v1 submitted 8 May, 2023; originally announced May 2023.

Comments: This manuscript will eventually appear in Wiley StatsRef-Statistics Reference Online, and should not be confused with the original article on Bayesian Synthetic Likelihood by Price, Drovandi, Nott and Lee (2018)

Reliable Bayesian Inference in Misspecified Models

Authors: David T. Frazier, Robert Kohn, Christopher Drovandi, David Gunawan

Abstract: We provide a general solution to a fundamental open problem in Bayesian inference, namely poor uncertainty quantification, from a frequency standpoint, of Bayesian methods in misspecified models. While existing solutions are based on explicit Gaussian approximations of the posterior, or computationally onerous post-processing procedures, we demonstrate that correct uncertainty quantification can b… ▽ More We provide a general solution to a fundamental open problem in Bayesian inference, namely poor uncertainty quantification, from a frequency standpoint, of Bayesian methods in misspecified models. While existing solutions are based on explicit Gaussian approximations of the posterior, or computationally onerous post-processing procedures, we demonstrate that correct uncertainty quantification can be achieved by replacing the usual posterior with an intuitive approximate posterior. Critically, our solution is applicable to likelihood-based, and generalized, posteriors as well as cases where the likelihood is intractable and must be estimated. We formally demonstrate the reliable uncertainty quantification of our proposed approach, and show that valid uncertainty quantification is not an asymptotic result but occurs even in small samples. We illustrate this approach through a range of examples, including linear, and generalized, mixed effects models. △ Less

Submitted 12 February, 2023; originally announced February 2023.

Misspecification-robust Sequential Neural Likelihood for Simulation-based Inference

Authors: Ryan P. Kelly, David J. Nott, David T. Frazier, David J. Warne, Chris Drovandi

Abstract: Simulation-based inference techniques are indispensable for parameter estimation of mechanistic and simulable models with intractable likelihoods. While traditional statistical approaches like approximate Bayesian computation and Bayesian synthetic likelihood have been studied under well-specified and misspecified settings, they often suffer from inefficiencies due to wasted model simulations. Neu… ▽ More Simulation-based inference techniques are indispensable for parameter estimation of mechanistic and simulable models with intractable likelihoods. While traditional statistical approaches like approximate Bayesian computation and Bayesian synthetic likelihood have been studied under well-specified and misspecified settings, they often suffer from inefficiencies due to wasted model simulations. Neural approaches, such as sequential neural likelihood (SNL) avoid this wastage by utilising all model simulations to train a neural surrogate for the likelihood function. However, the performance of SNL under model misspecification is unreliable and can result in overconfident posteriors centred around an inaccurate parameter estimate. In this paper, we propose a novel SNL method, which through the incorporation of additional adjustment parameters, is robust to model misspecification and capable of identifying features of the data that the model is not able to recover. We demonstrate the efficacy of our approach through several illustrative examples, where our method gives more accurate point estimates and uncertainty quantification than SNL. △ Less

Submitted 7 March, 2024; v1 submitted 30 January, 2023; originally announced January 2023.

Pooling information in likelihood-free inference

Authors: David T. Frazier, Christopher Drovandi, Lucas Kock, David J. Nott

Abstract: Likelihood-free inference (LFI) methods, such as approximate Bayesian computation, have become commonplace for conducting inference in complex models. Many approaches are based on summary statistics or discrepancies derived from synthetic data. However, determining which summary statistics or discrepancies to use for constructing the posterior remains a challenging question, both practically and t… ▽ More Likelihood-free inference (LFI) methods, such as approximate Bayesian computation, have become commonplace for conducting inference in complex models. Many approaches are based on summary statistics or discrepancies derived from synthetic data. However, determining which summary statistics or discrepancies to use for constructing the posterior remains a challenging question, both practically and theoretically. Instead of relying on a single vector of summaries for inference, we propose a new pooled posterior that optimally combines inferences from multiple LFI posteriors. This pooled approach eliminates the need to select a single vector of summaries or even a specific LFI algorithm. Our approach is straightforward to implement and avoids performing a high-dimensional LFI analysis involving all summary statistics. We give theoretical guarantees for the improved performance of the pooled posterior mean in terms of asymptotic frequentist risk and demonstrate the effectiveness of the approach in a number of benchmark examples. △ Less

Submitted 5 June, 2025; v1 submitted 5 December, 2022; originally announced December 2022.

Being Bayesian in the 2020s: opportunities and challenges in the practice of modern applied Bayesian statistics

Authors: Joshua J. Bon, Adam Bretherton, Katie Buchhorn, Susanna Cramb, Christopher Drovandi, Conor Hassan, Adrianne L. Jenner, Helen J. Mayfield, James M. McGree, Kerrie Mengersen, Aiden Price, Robert Salomone, Edgar Santos-Fernandez, Julie Vercelloni, Xiaoyu Wang

Abstract: Building on a strong foundation of philosophy, theory, methods and computation over the past three decades, Bayesian approaches are now an integral part of the toolkit for most statisticians and data scientists. Whether they are dedicated Bayesians or opportunistic users, applied professionals can now reap many of the benefits afforded by the Bayesian paradigm. In this paper, we touch on six moder… ▽ More Building on a strong foundation of philosophy, theory, methods and computation over the past three decades, Bayesian approaches are now an integral part of the toolkit for most statisticians and data scientists. Whether they are dedicated Bayesians or opportunistic users, applied professionals can now reap many of the benefits afforded by the Bayesian paradigm. In this paper, we touch on six modern opportunities and challenges in applied Bayesian statistics: intelligent data collection, new data sources, federated analysis, inference for implicit models, model transfer and purposeful software products. △ Less

Submitted 17 January, 2023; v1 submitted 17 November, 2022; originally announced November 2022.

Comments: 27 pages, 8 figures

Bayesian score calibration for approximate models

Authors: Joshua J Bon, David J Warne, David J Nott, Christopher Drovandi

Abstract: Scientists continue to develop increasingly complex mechanistic models to reflect their knowledge more realistically. Statistical inference using these models can be challenging since the corresponding likelihood function is often intractable and model simulation may be computationally burdensome. Fortunately, in many of these situations, it is possible to adopt a surrogate model or approximate li… ▽ More Scientists continue to develop increasingly complex mechanistic models to reflect their knowledge more realistically. Statistical inference using these models can be challenging since the corresponding likelihood function is often intractable and model simulation may be computationally burdensome. Fortunately, in many of these situations, it is possible to adopt a surrogate model or approximate likelihood function. It may be convenient to conduct Bayesian inference directly with the surrogate, but this can result in bias and poor uncertainty quantification. In this paper we propose a new method for adjusting approximate posterior samples to reduce bias and produce more accurate uncertainty quantification. We do this by optimizing a transform of the approximate posterior that maximizes a scoring rule. Our approach requires only a (fixed) small number of complex model simulations and is numerically stable. We demonstrate good performance of the new method on several examples of increasing complexity. △ Less

Submitted 27 October, 2023; v1 submitted 10 November, 2022; originally announced November 2022.

Comments: 27 pages, 8 figures, 5 tables

Transport Reversible Jump Proposals

Authors: Laurence Davies, Robert Salomone, Matthew Sutton, Christopher Drovandi

Abstract: Reversible jump Markov chain Monte Carlo (RJMCMC) proposals that achieve reasonable acceptance rates and mixing are notoriously difficult to design in most applications. Inspired by recent advances in deep neural network-based normalizing flows and density estimation, we demonstrate an approach to enhance the efficiency of RJMCMC sampling by performing transdimensional jumps involving reference di… ▽ More Reversible jump Markov chain Monte Carlo (RJMCMC) proposals that achieve reasonable acceptance rates and mixing are notoriously difficult to design in most applications. Inspired by recent advances in deep neural network-based normalizing flows and density estimation, we demonstrate an approach to enhance the efficiency of RJMCMC sampling by performing transdimensional jumps involving reference distributions. In contrast to other RJMCMC proposals, the proposed method is the first to apply a non-linear transport-based approach to construct efficient proposals between models with complicated dependency structures. It is shown that, in the setting where exact transports are used, our RJMCMC proposals have the desirable property that the acceptance probability depends only on the model probabilities. Numerical experiments demonstrate the efficacy of the approach. △ Less

Submitted 24 February, 2023; v1 submitted 22 October, 2022; originally announced October 2022.

Comments: 14 pages, 9 figures

Monte Carlo twisting for particle filters

Authors: Joshua J Bon, Christopher Drovandi, Anthony Lee

Abstract: We consider the problem of designing efficient particle filters for twisted Feynman--Kac models. Particle filters using twisted models can deliver low error approximations of statistical quantities and such twisting functions can be learnt iteratively. Practical implementations of these algorithms are complicated by the need to (i) sample from the twisted transition dynamics, and (ii) calculate th… ▽ More We consider the problem of designing efficient particle filters for twisted Feynman--Kac models. Particle filters using twisted models can deliver low error approximations of statistical quantities and such twisting functions can be learnt iteratively. Practical implementations of these algorithms are complicated by the need to (i) sample from the twisted transition dynamics, and (ii) calculate the twisted potential functions. We expand the class of applicable models using rejection sampling for (i) and unbiased approximations for (ii) using a random weight particle filter. We characterise the average acceptance rates within the particle filter in order to control the computational cost, and analyse the asymptotic variance. Empirical results show the mean squared error of the normalising constant estimate in our method is smaller than a memory-equivalent particle filter but not a computation-equivalent filter. Both comparisons are improved when more efficient sampling is possible which we demonstrate on a stochastic volatility model. △ Less

Submitted 8 August, 2022; originally announced August 2022.

Comments: Main: 7 pages, 1 figure. Supplementary: 9 pages, 1 figure

Efficient inference and identifiability analysis for differential equation models with random parameters

Authors: Alexander P. Browning, Christopher Drovandi, Ian W. Turner, Adrianne L. Jenner, Matthew J. Simpson

Abstract: Heterogeneity is a dominant factor in the behaviour of many biological processes. Despite this, it is common for mathematical and statistical analyses to ignore biological heterogeneity as a source of variability in experimental data. Therefore, methods for exploring the identifiability of models that explicitly incorporate heterogeneity through variability in model parameters are relatively under… ▽ More Heterogeneity is a dominant factor in the behaviour of many biological processes. Despite this, it is common for mathematical and statistical analyses to ignore biological heterogeneity as a source of variability in experimental data. Therefore, methods for exploring the identifiability of models that explicitly incorporate heterogeneity through variability in model parameters are relatively underdeveloped. We develop a new likelihood-based framework, based on moment matching, for inference and identifiability analysis of differential equation models that capture biological heterogeneity through parameters that vary according to probability distributions. As our novel method is based on an approximate likelihood function, it is highly flexible; we demonstrate identifiability analysis using both a frequentist approach based on profile likelihood, and a Bayesian approach based on Markov-chain Monte Carlo. Through three case studies, we demonstrate our method by providing a didactic guide to inference and identifiability analysis of hyperparameters that relate to the statistical moments of model parameters from independent observed data. Our approach has a computational cost comparable to analysis of models that neglect heterogeneity, a significant improvement over many existing alternatives. We demonstrate how analysis of random parameter models can aid better understanding of the sources of heterogeneity from biological data. △ Less

Submitted 27 October, 2022; v1 submitted 20 July, 2022; originally announced July 2022.

Comments: Minor changes to text. Additional results in supplementary material. Additional statistics regarding results given in main and supplementary material

MSC Class: 92B99; 62F99

Improving the Accuracy of Marginal Approximations in Likelihood-Free Inference via Localisation

Authors: Christopher Drovandi, David J Nott, David T Frazier

Abstract: Likelihood-free methods are an essential tool for performing inference for implicit models which can be simulated from, but for which the corresponding likelihood is intractable. However, common likelihood-free methods do not scale well to a large number of model parameters. A promising approach to high-dimensional likelihood-free inference involves estimating low-dimensional marginal posteriors b… ▽ More Likelihood-free methods are an essential tool for performing inference for implicit models which can be simulated from, but for which the corresponding likelihood is intractable. However, common likelihood-free methods do not scale well to a large number of model parameters. A promising approach to high-dimensional likelihood-free inference involves estimating low-dimensional marginal posteriors by conditioning only on summary statistics believed to be informative for the low-dimensional component, and then combining the low-dimensional approximations in some way. In this paper, we demonstrate that such low-dimensional approximations can be surprisingly poor in practice for seemingly intuitive summary statistic choices. We describe an idealized low-dimensional summary statistic that is, in principle, suitable for marginal estimation. However, a direct approximation of the idealized choice is difficult in practice. We thus suggest an alternative approach to marginal estimation which is easier to implement and automate. Given an initial choice of low-dimensional summary statistic that might only be informative about a marginal posterior location, the new method improves performance by first crudely localising the posterior approximation using all the summary statistics to ensure global identifiability, followed by a second step that hones in on an accurate low-dimensional approximation using the low-dimensional summary statistic. We show that the posterior this approach targets can be represented as a logarithmic pool of posterior distributions based on the low-dimensional and full summary statistics, respectively. The good performance of our method is illustrated in several examples. △ Less

Submitted 14 July, 2022; originally announced July 2022.

Comments: 30 pages, 9 figures

Component-wise iterative ensemble Kalman inversion for static Bayesian models with unknown measurement error covariance

Authors: Imke Botha, Matthew P. Adams, Dang Khuong Tran, Frederick R. Bennett, Christopher Drovandi

Abstract: The ensemble Kalman filter (EnKF) is a Monte Carlo approximation of the Kalman filter for high dimensional linear Gaussian state space models. EnKF methods have also been developed for parameter inference of static Bayesian models with a Gaussian likelihood, in a way that is analogous to likelihood tempering sequential Monte Carlo (SMC). These methods are commonly referred to as ensemble Kalman in… ▽ More The ensemble Kalman filter (EnKF) is a Monte Carlo approximation of the Kalman filter for high dimensional linear Gaussian state space models. EnKF methods have also been developed for parameter inference of static Bayesian models with a Gaussian likelihood, in a way that is analogous to likelihood tempering sequential Monte Carlo (SMC). These methods are commonly referred to as ensemble Kalman inversion (EKI). Unlike SMC, the inference from EKI is only asymptotically unbiased if the likelihood is linear Gaussian and the priors are Gaussian. However, EKI is significantly faster to run. Currently, a large limitation of EKI methods is that the covariance of the measurement error is assumed to be fully known. We develop a new method, which we call component-wise iterative ensemble Kalman inversion (CW-IEKI), that allows elements of the covariance matrix to be inferred alongside the model parameters at negligible extra cost. This novel method is compared to SMC on three different application examples: a model of nitrogen mineralisation in soil that is based on the Agricultural Production Systems Simulator (APSIM), a model predicting seagrass decline due to stress from water temperature and light, and a model predicting coral calcification rates. On all of these examples, we find that CW-IEKI has relatively similar predictive performance to SMC, albeit with greater uncertainty, and it has a significantly faster run time. △ Less

Submitted 6 June, 2022; originally announced June 2022.

Many-levelled continuation ratio models for frequency of alcohol and drug use data

Authors: Mark Chambers, Christopher Drovandi

Abstract: Studies of alcohol and drug use are often interested in the number of days that people use the substance of interest over an interval, such as 28 days before a survey date. Although count models are often used for this purpose, they are not strictly appropriate for this type of data because the response variable is bounded above. Furthermore, if some peoples' substance use behaviors are characteri… ▽ More Studies of alcohol and drug use are often interested in the number of days that people use the substance of interest over an interval, such as 28 days before a survey date. Although count models are often used for this purpose, they are not strictly appropriate for this type of data because the response variable is bounded above. Furthermore, if some peoples' substance use behaviors are characterized by various weekly patterns of use, summaries of substance days-of-use used over longer periods can exhibit multiple modes. These characteristics of substance days-of-use data are not easily fitted with conventional parametric model families. We propose a continuation ratio ordinal model for substance days-of-use data. Instead of grouping the set of possible response values into a small set of ordinal categories, each possible value is assigned its own category. This allows the exact numeric distribution implied by the predicted ordinal response to be recovered. We demonstrate the proposed model using survey data reporting days of alcohol use over 28-day intervals. We show the continuation ratio model is better able to capture the complexity in the drinking days dataset compared to binomial, hurdle-negative binomial and beta-binomial models. △ Less

Submitted 22 May, 2023; v1 submitted 24 May, 2022; originally announced May 2022.

Comments: 13 pages, 6 figures, 2 tables

Strategic model reduction by analysing model sloppiness: a case study in coral calcification

Authors: Sarah A. Vollert, Christopher Drovandi, Gloria M. Monsalve-Bravo, Matthew P. Adams

Abstract: It can be difficult to identify ways to reduce the complexity of large models whilst maintaining predictive power, particularly where there are hidden parameter interdependencies. Here, we demonstrate that the analysis of model sloppiness can be a new invaluable tool for strategically simplifying complex models. Such an analysis identifies parameter combinations which strongly and/or weakly inform… ▽ More It can be difficult to identify ways to reduce the complexity of large models whilst maintaining predictive power, particularly where there are hidden parameter interdependencies. Here, we demonstrate that the analysis of model sloppiness can be a new invaluable tool for strategically simplifying complex models. Such an analysis identifies parameter combinations which strongly and/or weakly inform model behaviours, yet the approach has not previously been used to inform model reduction. Using a case study on a coral calcification model calibrated to experimental data, we show how the analysis of model sloppiness can strategically inform model simplifications which maintain predictive power. Additionally, when comparing various approaches to analysing sloppiness, we find that Bayesian methods can be advantageous when unambiguous identification of the best-fit model parameters is a challenge for standard optimisation procedures. △ Less

Submitted 12 December, 2022; v1 submitted 12 April, 2022; originally announced April 2022.

Journal ref: Environmental Modelling and Software 159 (2023) 105578

Analysis of sloppiness in model simulations: unveiling parameter uncertainty when mathematical models are fitted to data

Authors: Gloria M. Monsalve-Bravo, Brodie A. J. Lawson, Christopher Drovandi, Kevin Burrage, Kevin S. Brown, Christopher M. Baker, Sarah A. Vollert, Kerrie Mengersen, Eve McDonald-Madden, Matthew P. Adams

Abstract: This work introduces a comprehensive approach to assess the sensitivity of model outputs to changes in parameter values, constrained by the combination of prior beliefs and data. This novel approach identifies stiff parameter combinations strongly affecting the quality of the model-data fit while simultaneously revealing which of these key parameter combinations are informed primarily by the data… ▽ More This work introduces a comprehensive approach to assess the sensitivity of model outputs to changes in parameter values, constrained by the combination of prior beliefs and data. This novel approach identifies stiff parameter combinations strongly affecting the quality of the model-data fit while simultaneously revealing which of these key parameter combinations are informed primarily by the data or are also substantively influenced by the priors. We focus on the very common context in complex systems where the amount and quality of data are low compared to the number of model parameters to be collectively estimated, and showcase the benefits of this technique for applications in biochemistry, ecology, and cardiac electrophysiology. We also show how stiff parameter combinations, once identified, uncover controlling mechanisms underlying the system being modeled and inform which of the model parameters need to be prioritized in future experiments for improved parameter inference from collective model-data fitting. △ Less

Submitted 21 September, 2022; v1 submitted 28 March, 2022; originally announced March 2022.

Journal ref: Sci.Adv. 8(38) eabm5952 (2022)

Modularized Bayesian analyses and cutting feedback in likelihood-free inference

Authors: Atlanta Chakraborty, David J. Nott, Christopher Drovandi, David T. Frazier, Scott A. Sisson

Abstract: There has been much recent interest in modifying Bayesian inference for misspecified models so that it is useful for specific purposes. One popular modified Bayesian inference method is "cutting feedback" which can be used when the model consists of a number of coupled modules, with only some of the modules being misspecified. Cutting feedback methods represent the full posterior distribution in t… ▽ More There has been much recent interest in modifying Bayesian inference for misspecified models so that it is useful for specific purposes. One popular modified Bayesian inference method is "cutting feedback" which can be used when the model consists of a number of coupled modules, with only some of the modules being misspecified. Cutting feedback methods represent the full posterior distribution in terms of conditional and sequential components, and then modify some terms in such a representation based on the modular structure for specification or computation of a modified posterior distribution. The main goal of this is to avoid contamination of inferences for parameters of interest by misspecified modules. Computation for cut posterior distributions is challenging, and here we consider cutting feedback for likelihood-free inference based on Gaussian mixture approximations to the joint distribution of parameters and data summary statistics. We exploit the fact that marginal and conditional distributions of a Gaussian mixture are Gaussian mixtures to give explicit approximations to marginal or conditional posterior distributions so that we can easily approximate cut posterior analyses. The mixture approach allows repeated approximation of posterior distributions for different data based on a single mixture fit, which is important for model checks which aid in the decision of whether to "cut". A semi-modular approach to likelihood-free inference where feedback is partially cut is also developed. The benefits of the method are illustrated in two challenging examples, a collective cell spreading model and a continuous time model for asset returns with jumps. △ Less

Submitted 18 March, 2022; originally announced March 2022.

Population Calibration using Likelihood-Free Bayesian Inference

Authors: Christopher Drovandi, Brodie Lawson, Adrianne L Jenner, Alexander P Browning

Abstract: In this paper we develop a likelihood-free approach for population calibration, which involves finding distributions of model parameters when fed through the model produces a set of outputs that matches available population data. Unlike most other approaches to population calibration, our method produces uncertainty quantification on the estimated distribution. Furthermore, the method can be appli… ▽ More In this paper we develop a likelihood-free approach for population calibration, which involves finding distributions of model parameters when fed through the model produces a set of outputs that matches available population data. Unlike most other approaches to population calibration, our method produces uncertainty quantification on the estimated distribution. Furthermore, the method can be applied to any population calibration problem, regardless of whether the model of interest is deterministic or stochastic, or whether the population data is observed with or without measurement error. We demonstrate the method on several examples, including one with real data. We also discuss the computational limitations of the approach. Immediate applications for the methodology developed here exist in many areas of medical research including cancer, COVID-19, drug development and cardiology. △ Less

Submitted 3 February, 2022; originally announced February 2022.

Comments: 20 pages, 6 figures

Automatically adapting the number of state particles in SMC$^2$

Authors: Imke Botha, Robert Kohn, Leah South, Christopher Drovandi

Abstract: Sequential Monte Carlo squared (SMC$^2$) methods can be used for parameter inference of intractable likelihood state-space models. These methods replace the likelihood with an unbiased particle filter estimator, similarly to particle Markov chain Monte Carlo (MCMC). As with particle MCMC, the efficiency of SMC$^2$ greatly depends on the variance of the likelihood estimator, and therefore on the nu… ▽ More Sequential Monte Carlo squared (SMC$^2$) methods can be used for parameter inference of intractable likelihood state-space models. These methods replace the likelihood with an unbiased particle filter estimator, similarly to particle Markov chain Monte Carlo (MCMC). As with particle MCMC, the efficiency of SMC$^2$ greatly depends on the variance of the likelihood estimator, and therefore on the number of state particles used within the particle filter. We introduce novel methods to adaptively select the number of state particles within SMC$^2$ using the expected squared jumping distance to trigger the adaptation, and modifying the exchange importance sampling method of \citet{Chopin2012a} to replace the current set of state particles with the new set of state particles. The resulting algorithm is fully automatic, and can significantly improve current methods. Code for our methods is available at https://github.com/imkebotha/adaptive-exact-approximate-smc. △ Less

Submitted 21 October, 2022; v1 submitted 27 January, 2022; originally announced January 2022.

The effect of biologically mediated decay rates on modelling soil carbon sequestration in agricultural settings

Authors: Mohammad Javad Davoudabadi, Daniel Pagendam, Christopher Drovandi, Jeff Baldock, Gentry White

Abstract: Microbial biomass carbon (MBC), a crucial soil labile carbon fraction, is the most active component of the soil organic carbon (SOC) that regulates bio-geochemical processes in terrestrial ecosystems. Some studies in the literature ignore the effect of microbial population growth on carbon decomposition rates. In reality, we might expect that the decomposition rate should be related to the populat… ▽ More Microbial biomass carbon (MBC), a crucial soil labile carbon fraction, is the most active component of the soil organic carbon (SOC) that regulates bio-geochemical processes in terrestrial ecosystems. Some studies in the literature ignore the effect of microbial population growth on carbon decomposition rates. In reality, we might expect that the decomposition rate should be related to the population of microbes in the soil and have a positive relationship with the size of the microbial biomass pool. In this study, we explore the effect of microbial population growth on the accuracy of modelling soil carbon sequestration by developing and comparing two soil carbon models that consider a carrying capacity and limit to the growth of the microbial pool. We apply our models to three datasets, two small and one large datasets, and we select the best model in terms of having the best predictive performance through two model selection methods. Through this analysis we reveal that commonly used complex soil carbon models can over-fit in the presence of both small and large time-series datasets, and our simpler model can produce more accurate predictions. We conclude that considering the microbial population growth in a soil carbon model improves the accuracy of a model in the presence of a large dataset. △ Less

Submitted 5 January, 2022; originally announced January 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2105.04789

Bayesian Detectability of Induced Polarisation in Airborne Electromagnetic Data using Reversible Jump Sequential Monte Carlo

Authors: Laurence Davies, Alan Yusen Ley-Cooper, Matthew Sutton, Christopher Drovandi

Abstract: Detection of induced polarisation (IP) effects in airborne electromagnetic (AEM) measurements does not yet have an established methodology. This contribution develops a Bayesian approach to the IP-detectability problem using decoupled transdimensional layered models, and applies an approach novel to geophysics whereby transdimensional proposals are used within the embarrassingly parallelisable and… ▽ More Detection of induced polarisation (IP) effects in airborne electromagnetic (AEM) measurements does not yet have an established methodology. This contribution develops a Bayesian approach to the IP-detectability problem using decoupled transdimensional layered models, and applies an approach novel to geophysics whereby transdimensional proposals are used within the embarrassingly parallelisable and robust static Sequential Monte Carlo (SMC) class of algorithms for the simultaneous inference of parameters and models. Henceforth referring to this algorithm as Reversible Jump Sequential Monte Carlo (RJSMC), the statistical methodological contributions to the algorithm account for adaptivity considerations for multiple models and proposal types, especially surrounding particle impoverishment in unlikely models. Methodological contributions to solid Earth geophysics include the decoupled model approach and proposal of a statistic that use posterior model odds for IP detectability. A case study is included investigating detectability of IP effects in AEM data at a broad scale. △ Less

Submitted 1 September, 2021; originally announced September 2021.

Comments: 28 pages, 4 figures

MSC Class: 62-08; 62P99

Innovative Approaches in Soil Carbon Sequestration Modelling for Better Prediction with Limited Data

Authors: Mohammad Javad Davoudabadi, Daniel Pagendam, Christopher Drovandi, Jeff Baldock, Gentry White

Abstract: Soil carbon accounting and prediction play a key role in building decision support systems for land managers selling carbon credits, in the spirit of the Paris and Kyoto protocol agreements. Land managers typically rely on computationally complex models fit using sparse datasets to make these accounts and predictions. The model complexity and sparsity of the data can lead to over-fitting, leading… ▽ More Soil carbon accounting and prediction play a key role in building decision support systems for land managers selling carbon credits, in the spirit of the Paris and Kyoto protocol agreements. Land managers typically rely on computationally complex models fit using sparse datasets to make these accounts and predictions. The model complexity and sparsity of the data can lead to over-fitting, leading to inaccurate results when making predictions with new data. Modellers address over-fitting by simplifying their models and reducing the number of parameters, and in the current context this could involve neglecting some soil organic carbon (SOC) components. In this study, we introduce two novel SOC models and a new RothC-like model and investigate how the SOC components and complexity of the SOC models affect the SOC prediction in the presence of small and sparse time series data. We develop model selection methods that can identify the soil carbon model with the best predictive performance, in light of the available data. Through this analysis we reveal that commonly used complex soil carbon models can over-fit in the presence of sparse time series data, and our simpler models can produce more accurate predictions. The published version of this study is available in Scientific Reports (https://www.nature.com/articles/s41598-024-53516-z/<10.1038/s41598-024-53516-z>) △ Less

Submitted 10 February, 2024; v1 submitted 11 May, 2021; originally announced May 2021.

Comments: 28 pages, 7 figures

Journal ref: Sci Rep 14, 3191 (2024)

Synthetic Likelihood in Misspecified Models: Consequences and Corrections

Authors: David T. Frazier, Christopher Drovandi, David J. Nott

Abstract: We analyse the behaviour of the synthetic likelihood (SL) method when the model generating the simulated data differs from the actual data generating process. One of the most common methods to obtain SL-based inferences is via the Bayesian posterior distribution, with this method often referred to as Bayesian synthetic likelihood (BSL). We demonstrate that when the model is misspecified, the BSL p… ▽ More We analyse the behaviour of the synthetic likelihood (SL) method when the model generating the simulated data differs from the actual data generating process. One of the most common methods to obtain SL-based inferences is via the Bayesian posterior distribution, with this method often referred to as Bayesian synthetic likelihood (BSL). We demonstrate that when the model is misspecified, the BSL posterior can be poorly behaved, placing significant posterior mass on values of the model parameters that do not represent the true features observed in the data. Theoretical results demonstrate that in misspecified models the BSL posterior can display a wide range of behaviours depending on the level of model misspecification, including being asymptotically non-Gaussian. Our results suggest that a recently proposed robust BSL approach can ameliorate this behavior and deliver reliable posterior inference under model misspecification. We document all theoretical results using a simple running example. △ Less

Submitted 7 April, 2021; originally announced April 2021.

A Comparison of Likelihood-Free Methods With and Without Summary Statistics

Authors: Christopher Drovandi, David T Frazier

Abstract: Likelihood-free methods are useful for parameter estimation of complex models with intractable likelihood functions for which it is easy to simulate data. Such models are prevalent in many disciplines including genetics, biology, ecology and cosmology. Likelihood-free methods avoid explicit likelihood evaluation by finding parameter values of the model that generate data close to the observed data… ▽ More Likelihood-free methods are useful for parameter estimation of complex models with intractable likelihood functions for which it is easy to simulate data. Such models are prevalent in many disciplines including genetics, biology, ecology and cosmology. Likelihood-free methods avoid explicit likelihood evaluation by finding parameter values of the model that generate data close to the observed data. The general consensus has been that it is most efficient to compare datasets on the basis of a low dimensional informative summary statistic, incurring information loss in favour of reduced dimensionality. More recently, researchers have explored various approaches for efficiently comparing empirical distributions in the likelihood-free context in an effort to avoid data summarisation. This article provides a review of these full data distance based approaches, and conducts the first comprehensive comparison of such methods, both qualitatively and empirically. We also conduct a substantive empirical comparison with summary statistic based likelihood-free methods. The discussion and results offer guidance to practitioners considering a likelihood-free approach. Whilst we find the best approach to be problem dependent, we also find that the full data distance based approaches are promising and warrant further development. We discuss some opportunities for future research in this space. Computer code to implement the methods discussed in this paper can be found at https://github.com/cdrovandi/ABC-dist-compare. △ Less

Submitted 25 March, 2022; v1 submitted 3 March, 2021; originally announced March 2021.

Comments: To appear in Statistics and Computing

Accelerating sequential Monte Carlo with surrogate likelihoods

Authors: Joshua J Bon, Anthony Lee, Christopher Drovandi

Abstract: Delayed-acceptance is a technique for reducing computational effort for Bayesian models with expensive likelihoods. Using a delayed-acceptance kernel for Markov chain Monte Carlo can reduce the number of expensive likelihoods evaluations required to approximate a posterior expectation. Delayed-acceptance uses a surrogate, or approximate, likelihood to avoid evaluation of the expensive likelihood w… ▽ More Delayed-acceptance is a technique for reducing computational effort for Bayesian models with expensive likelihoods. Using a delayed-acceptance kernel for Markov chain Monte Carlo can reduce the number of expensive likelihoods evaluations required to approximate a posterior expectation. Delayed-acceptance uses a surrogate, or approximate, likelihood to avoid evaluation of the expensive likelihood when possible. Within the sequential Monte Carlo framework, we utilise the history of the sampler to adaptively tune the surrogate likelihood to yield better approximations of the expensive likelihood, and use a surrogate first annealing schedule to further increase computational efficiency. Moreover, we propose a framework for optimising computation time whilst avoiding particle degeneracy, which encapsulates existing strategies in the literature. Overall, we develop a novel algorithm for computationally efficient SMC with expensive likelihood functions. The method is applied to static Bayesian models, which we demonstrate on toy and real examples, code for which is available at https://github.com/bonStats/smcdar. △ Less

Submitted 20 July, 2021; v1 submitted 8 September, 2020; originally announced September 2020.

Comments: 35 pages, 10 figures

Robust Approximate Bayesian Computation: An Adjustment Approach

Authors: David T. Frazier, Christopher Drovandi, Ruben Loaiza-Maya

Abstract: We propose a novel approach to approximate Bayesian computation (ABC) that seeks to cater for possible misspecification of the assumed model. This new approach can be equally applied to rejection-based ABC and to popular regression adjustment ABC. We demonstrate that this new approach mitigates the poor performance of regression adjusted ABC that can eventuate when the model is misspecified. In ad… ▽ More We propose a novel approach to approximate Bayesian computation (ABC) that seeks to cater for possible misspecification of the assumed model. This new approach can be equally applied to rejection-based ABC and to popular regression adjustment ABC. We demonstrate that this new approach mitigates the poor performance of regression adjusted ABC that can eventuate when the model is misspecified. In addition, this new adjustment approach allows us to detect which features of the observed data can not be reliably reproduced by the assumed model. A series of simulated and empirical examples illustrate this new approach. △ Less

Submitted 7 August, 2020; originally announced August 2020.

Comments: arXiv admin note: text overlap with arXiv:1904.04551

Transformations in Semi-Parametric Bayesian Synthetic Likelihood

Authors: Jacob W. Priddle, Christopher Drovandi

Abstract: Bayesian synthetic likelihood (BSL) is a popular method for performing approximate Bayesian inference when the likelihood function is intractable. In synthetic likelihood methods, the likelihood function is approximated parametrically via model simulations, and then standard likelihood-based techniques are used to perform inference. The Gaussian synthetic likelihood estimator has become ubiquitous… ▽ More Bayesian synthetic likelihood (BSL) is a popular method for performing approximate Bayesian inference when the likelihood function is intractable. In synthetic likelihood methods, the likelihood function is approximated parametrically via model simulations, and then standard likelihood-based techniques are used to perform inference. The Gaussian synthetic likelihood estimator has become ubiquitous in BSL literature, primarily for its simplicity and ease of implementation. However, it is often too restrictive and may lead to poor posterior approximations. Recently, a more flexible semi-parametric Bayesian synthetic likelihood (semiBSL) estimator has been introduced, which is significantly more robust to irregularly distributed summary statistics. In this work, we propose a number of extensions to semiBSL. First, we consider even more flexible estimators of the marginal distributions using transformation kernel density estimation. Second, we propose whitening semiBSL (wsemiBSL) -- a method to significantly improve the computational efficiency of semiBSL. wsemiBSL uses an approximate whitening transformation to decorrelate summary statistics at each algorithm iteration. The methods developed herein significantly improve the versatility and efficiency of BSL algorithms. △ Less

Submitted 2 July, 2020; originally announced July 2020.

Sequential Bayesian Experimental Design for Implicit Models via Mutual Information

Authors: Steven Kleinegesse, Christopher Drovandi, Michael U. Gutmann

Abstract: Bayesian experimental design (BED) is a framework that uses statistical models and decision making under uncertainty to optimise the cost and performance of a scientific experiment. Sequential BED, as opposed to static BED, considers the scenario where we can sequentially update our beliefs about the model parameters through data gathered in the experiment. A class of models of particular interest… ▽ More Bayesian experimental design (BED) is a framework that uses statistical models and decision making under uncertainty to optimise the cost and performance of a scientific experiment. Sequential BED, as opposed to static BED, considers the scenario where we can sequentially update our beliefs about the model parameters through data gathered in the experiment. A class of models of particular interest for the natural and medical sciences are implicit models, where the data generating distribution is intractable, but sampling from it is possible. Even though there has been a lot of work on static BED for implicit models in the past few years, the notoriously difficult problem of sequential BED for implicit models has barely been touched upon. We address this gap in the literature by devising a novel sequential design framework for parameter estimation that uses the Mutual Information (MI) between model parameters and simulated data as a utility function to find optimal experimental designs, which has not been done before for implicit models. Our approach uses likelihood-free inference by ratio estimation to simultaneously estimate posterior distributions and the MI. During the sequential BED procedure we utilise Bayesian optimisation to help us optimise the MI utility. We find that our framework is efficient for the various implicit models tested, yielding accurate parameter estimates after only a few iterations. △ Less

Submitted 20 March, 2020; originally announced March 2020.

MSC Class: 62K05; 62L05

Combined parameter and state inference with automatically calibrated ABC

Authors: Anthony Ebert, Pierre Pudlo, Kerrie Mengersen, Paul Wu, Christopher Drovandi

Abstract: State space models contain time-indexed parameters, termed states, as well as static parameters, simply termed parameters. The problem of inferring both static parameters as well as states simultaneously, based on time-indexed observations, is the subject of much recent literature. This problem is compounded once we consider models with intractable likelihoods. In these situations, some emerging a… ▽ More State space models contain time-indexed parameters, termed states, as well as static parameters, simply termed parameters. The problem of inferring both static parameters as well as states simultaneously, based on time-indexed observations, is the subject of much recent literature. This problem is compounded once we consider models with intractable likelihoods. In these situations, some emerging approaches have incorporated existing likelihood-free techniques for static parameters, such as approximate Bayesian computation (ABC) into likelihood-based algorithms for combined inference of parameters and states. These emerging approaches currently require extensive manual calibration of a time-indexed tuning parameter: the acceptance threshold. We design an SMC$^2$ algorithm (Chopin et al., 2013, JRSS B) for likelihood-free approximation with automatically tuned thresholds. We prove consistency of the algorithm and discuss the proposed calibration. We demonstrate this algorithm's performance with three examples. We begin with two examples of state space models. The first example is a toy example, with an emission distribution that is a skew normal distribution. The second example is a stochastic volatility model involving an intractable stable distribution. The last example is the most challenging; it deals with an inhomogeneous Hawkes process. △ Less

Submitted 26 May, 2021; v1 submitted 30 October, 2019; originally announced October 2019.

Efficient Bayesian synthetic likelihood with whitening transformations

Authors: Jacob W. Priddle, Scott A. Sisson, David T. Frazier, Christopher Drovandi

Abstract: Likelihood-free methods are an established approach for performing approximate Bayesian inference for models with intractable likelihood functions. However, they can be computationally demanding. Bayesian synthetic likelihood (BSL) is a popular such method that approximates the likelihood function of the summary statistic with a known, tractable distribution -- typically Gaussian -- and then perfo… ▽ More Likelihood-free methods are an established approach for performing approximate Bayesian inference for models with intractable likelihood functions. However, they can be computationally demanding. Bayesian synthetic likelihood (BSL) is a popular such method that approximates the likelihood function of the summary statistic with a known, tractable distribution -- typically Gaussian -- and then performs statistical inference using standard likelihood-based techniques. However, as the number of summary statistics grows, the number of model simulations required to accurately estimate the covariance matrix for this likelihood rapidly increases. This poses significant challenge for the application of BSL, especially in cases where model simulation is expensive. In this article we propose whitening BSL (wBSL) -- an efficient BSL method that uses approximate whitening transformations to decorrelate the summary statistics at each algorithm iteration. We show empirically that this can reduce the number of model simulations required to implement BSL by more than an order of magnitude, without much loss of accuracy. We explore a range of whitening procedures and demonstrate the performance of wBSL on a range of simulated and real modelling scenarios from ecology and biology. △ Less

Submitted 31 January, 2020; v1 submitted 11 September, 2019; originally announced September 2019.

Estimating a novel stochastic model for within-field disease dynamics of banana bunchy top virus via approximate Bayesian computation

Authors: Abhishek Varghese, Christopher Drovandi, Kerrie Mengersen, Antonietta Mira

Abstract: The Banana Bunchy Top Virus (BBTV) is one of the most economically important vector-borne banana diseases throughout the Asia-Pacific Basin and presents a significant challenge to the agricultural sector. Current models of BBTV are largely deterministic, limited by an incomplete understanding of interactions in complex natural systems, and the appropriate identification of parameters. A stochastic… ▽ More The Banana Bunchy Top Virus (BBTV) is one of the most economically important vector-borne banana diseases throughout the Asia-Pacific Basin and presents a significant challenge to the agricultural sector. Current models of BBTV are largely deterministic, limited by an incomplete understanding of interactions in complex natural systems, and the appropriate identification of parameters. A stochastic network-based Susceptible-Infected model has been created which simulates the spread of BBTV across the subsections of a banana plantation, parameterising nodal recovery, neighbouring and distant infectivity across summer and winter. Findings from posterior results achieved through Markov Chain Monte Carlo approach to approximate Bayesian computation suggest seasonality in all parameters, which are influenced by correlated changes in inspection accuracy, temperatures and aphid activity. This paper demonstrates how the model may be used for monitoring and forecasting of various disease management strategies to support policy-level decision making. △ Less

Submitted 16 March, 2020; v1 submitted 4 September, 2019; originally announced September 2019.

Comments: 40 pages, 16 figures

MSC Class: 62P12

Particle Methods for Stochastic Differential Equation Mixed Effects Models

Authors: Imke Botha, Robert Kohn, Christopher Drovandi

Abstract: Parameter inference for stochastic differential equation mixed effects models (SDEMEMs) is a challenging problem. Analytical solutions for these models are rarely available, which means that the likelihood is also intractable. In this case, exact inference is possible using the pseudo-marginal method, where the intractable likelihood is replaced by its nonnegative unbiased estimate. A useful appli… ▽ More Parameter inference for stochastic differential equation mixed effects models (SDEMEMs) is a challenging problem. Analytical solutions for these models are rarely available, which means that the likelihood is also intractable. In this case, exact inference is possible using the pseudo-marginal method, where the intractable likelihood is replaced by its nonnegative unbiased estimate. A useful application of this idea is particle MCMC, which uses a particle filter estimate of the likelihood. While the exact posterior is targeted by these methods, a naive implementation for SDEMEMs can be highly inefficient. We develop three extensions to the naive approach which exploits specific aspects of SDEMEMs and other advances such as correlated pseudo-marginal methods. We compare these methods on real and simulated data from a tumour xenography study on mice. △ Less

Submitted 26 September, 2019; v1 submitted 25 July, 2019; originally announced July 2019.

Comments: Minor revisions throughout, added a simple running example, some updates to example and results (Sections 5-7), link to code on GitHub is provided

BSL: An R Package for Efficient Parameter Estimation for Simulation-Based Models via Bayesian Synthetic Likelihood

Authors: Ziwen An, Leah F South, Christopher Drovandi

Abstract: Bayesian synthetic likelihood (BSL) is a popular method for estimating the parameter posterior distribution for complex statistical models and stochastic processes that possess a computationally intractable likelihood function. Instead of evaluating the likelihood, BSL approximates the likelihood of a judiciously chosen summary statistic of the data via model simulation and density estimation. Com… ▽ More Bayesian synthetic likelihood (BSL) is a popular method for estimating the parameter posterior distribution for complex statistical models and stochastic processes that possess a computationally intractable likelihood function. Instead of evaluating the likelihood, BSL approximates the likelihood of a judiciously chosen summary statistic of the data via model simulation and density estimation. Compared to alternative methods such as approximate Bayesian computation (ABC), BSL requires little tuning and requires less model simulations than ABC when the chosen summary statistic is high-dimensional. The original synthetic likelihood relies on a multivariate normal approximation of the intractable likelihood, where the mean and covariance are estimated by simulation. An extension of BSL considers replacing the sample covariance with a penalised covariance estimator to reduce the number of required model simulations. Further, a semi-parametric approach has been developed to relax the normality assumption. In this paper, we present an R package called BSL that amalgamates the aforementioned methods and more into a single, easy-to-use and coherent piece of software. The R package also includes several examples to illustrate how to use the package and demonstrate the utility of the methods. △ Less

Submitted 25 July, 2019; originally announced July 2019.

Sequential Experimental Design for Predator-Prey Functional Response Experiments

Authors: Hayden Moffat, Markus Hainy, Nikos E. Papanikolaou, Christopher Drovandi

Abstract: Understanding functional response within a predator-prey dynamic is a cornerstone for many quantitative ecological studies. Over the past 60 years, the methodology for modelling functional response has gradually transitioned from the classic mechanistic models to more statistically oriented models. To obtain inferences on these statistical models, a substantial number of experiments need to be con… ▽ More Understanding functional response within a predator-prey dynamic is a cornerstone for many quantitative ecological studies. Over the past 60 years, the methodology for modelling functional response has gradually transitioned from the classic mechanistic models to more statistically oriented models. To obtain inferences on these statistical models, a substantial number of experiments need to be conducted. The obvious disadvantages of collecting this volume of data include cost, time and the sacrificing of animals. Therefore, optimally designed experiments are useful as they may reduce the total number of experimental runs required to attain the same statistical results. In this paper, we develop the first sequential experimental design method for predator-prey functional response experiments. To make inferences on the parameters in each of the statistical models we consider, we use sequential Monte Carlo, which is computationally efficient and facilitates convenient estimation of important utility functions. It provides coverage of experimental goals including parameter estimation, model discrimination as well as a combination of these. The results of our simulation study illustrate that for predator-prey functional response experiments sequential design outperforms static design for our experimental goals. R code for implementing the methodology is available via https://github.com/haydenmoffat/sequential_design_for_predator_prey_experiments. △ Less

Submitted 28 April, 2020; v1 submitted 3 July, 2019; originally announced July 2019.

Comments: Main Text: 23 pages, 7 Figures - Supplementary Text: 11 pages, 5 Figures