-
Multi-fidelity Hamiltonian Monte Carlo
Authors:
Dhruv V. Patel,
Jonghyun Lee,
Matthew W. Farthing,
Peter K. Kitanidis,
Eric F. Darve
Abstract:
Numerous applications in biology, statistics, science, and engineering require generating samples from high-dimensional probability distributions. In recent years, the Hamiltonian Monte Carlo (HMC) method has emerged as a state-of-the-art Markov chain Monte Carlo technique, exploiting the shape of such high-dimensional target distributions to efficiently generate samples. Despite its impressive em…
▽ More
Numerous applications in biology, statistics, science, and engineering require generating samples from high-dimensional probability distributions. In recent years, the Hamiltonian Monte Carlo (HMC) method has emerged as a state-of-the-art Markov chain Monte Carlo technique, exploiting the shape of such high-dimensional target distributions to efficiently generate samples. Despite its impressive empirical success and increasing popularity, its wide-scale adoption remains limited due to the high computational cost of gradient calculation. Moreover, applying this method is impossible when the gradient of the posterior cannot be computed (for example, with black-box simulators). To overcome these challenges, we propose a novel two-stage Hamiltonian Monte Carlo algorithm with a surrogate model. In this multi-fidelity algorithm, the acceptance probability is computed in the first stage via a standard HMC proposal using an inexpensive differentiable surrogate model, and if the proposal is accepted, the posterior is evaluated in the second stage using the high-fidelity (HF) numerical solver. Splitting the standard HMC algorithm into these two stages allows for approximating the gradient of the posterior efficiently, while producing accurate posterior samples by using HF numerical solvers in the second stage. We demonstrate the effectiveness of this algorithm for a range of problems, including linear and nonlinear Bayesian inverse problems with in-silico data and experimental data. The proposed algorithm is shown to seamlessly integrate with various low-fidelity and HF models, priors, and datasets. Remarkably, our proposed method outperforms the traditional HMC algorithm in both computational and statistical efficiency by several orders of magnitude, all while retaining or improving the accuracy in computed posterior statistics.
△ Less
Submitted 8 May, 2024;
originally announced May 2024.
-
A Kalman filter powered by $\mathcal{H}^2$-matrices for quasi-continuous data assimilation problems
Authors:
Judith Y. Li,
Sivaram Ambikasaran,
Eric F. Darve,
Peter K. Kitanidis
Abstract:
Continuously tracking the movement of a fluid or a plume in the subsurface is a challenge that is often encountered in applications, such as tracking a plume of injected CO$_2$ or of a hazardous substance. Advances in monitoring techniques have made it possible to collect measurements at a high frequency while the plume moves, which has the potential advantage of providing continuous high-resoluti…
▽ More
Continuously tracking the movement of a fluid or a plume in the subsurface is a challenge that is often encountered in applications, such as tracking a plume of injected CO$_2$ or of a hazardous substance. Advances in monitoring techniques have made it possible to collect measurements at a high frequency while the plume moves, which has the potential advantage of providing continuous high-resolution images of fluid flow with the aid of data processing. However, the applicability of this approach is limited by the high computational cost associated with having to analyze large data sets within the time constraints imposed by real-time monitoring. Existing data assimilation methods have computational requirements that increase super-linearly with the size of the unknowns $m$. In this paper, we present the HiKF, a new Kalman filter (KF) variant powered by the hierarchical matrix approach that dramatically reduces the computational and storage cost of the standard KF from $\mathcal{O}(m^2)$ to $\mathcal{O}(m)$, while producing practically the same results. The version of HiKF that is presented here takes advantage of the so-called random walk dynamical model, which is tailored to a class of data assimilation problems in which measurements are collected quasi-continuously. The proposed method has been applied to a realistic CO$_2$ injection model and compared with the ensemble Kalman filter (EnKF). Numerical results show that HiKF can provide estimates that are more accurate than EnKF, and also demonstrate the usefulness of modeling the system dynamics as a random walk in this context.
△ Less
Submitted 15 April, 2014;
originally announced April 2014.
-
An Improved Data Assimilation Scheme for High Dimensional Nonlinear Systems
Authors:
Hatef Monajemi,
Peter K. Kitanidis
Abstract:
Nonlinear/non-Gaussian filtering has broad applications in many areas of life sciences where either the dynamic is nonlinear and/or the probability density function of uncertain state is non-Gaussian. In such problems, the accuracy of the estimated quantities depends highly upon how accurately their posterior pdf can be approximated. In low dimensional state spaces, methods based on Sequential Imp…
▽ More
Nonlinear/non-Gaussian filtering has broad applications in many areas of life sciences where either the dynamic is nonlinear and/or the probability density function of uncertain state is non-Gaussian. In such problems, the accuracy of the estimated quantities depends highly upon how accurately their posterior pdf can be approximated. In low dimensional state spaces, methods based on Sequential Importance Sampling (SIS) can suitably approximate the posterior pdf. For higher dimensional problems, however, these techniques are usually inappropriate since the required number of particles to achieve satisfactory estimates grows exponentially with the dimension of state space. On the other hand, ensemble Kalman filter (EnKF) and its variants are more suitable for large-scale problems due to transformation of particles in the Bayesian update step. It has been shown that the latter class of methods may lead to suboptimal solutions for strongly nonlinear problems due to the Gaussian assumption in the update step. In this paper, we introduce a new technique based on the Gaussian sum expansion which captures the non-Gaussian features more accurately while the required computational effort remains within reason for high dimensional problems. We demonstrate the performance of the method for non-Gaussian processes through several examples including the strongly nonlinear Lorenz models. Results show a remarkable improvement in the mean square error compared to EnKF, and a desirable convergence behavior as the number of particles increases.
△ Less
Submitted 31 July, 2012;
originally announced August 2012.
