-
A localized consensus-based sampling algorithm
Authors:
Arne Bouillon,
Alexander Bodard,
Panagiotis Patrinos,
Dirk Nuyens,
Giovanni Samaey
Abstract:
We develop a novel interacting-particle method for sampling from non-Gaussian distributions. As a first step, we propose a new way to derive the consensus-based sampling (CBS) algorithm, starting from ensemble-preconditioned Langevin diffusions. We approximate the target potential by its Moreau envelope, such that the gradient in the Langevin equation can be replaced by a proximal operator. We the…
▽ More
We develop a novel interacting-particle method for sampling from non-Gaussian distributions. As a first step, we propose a new way to derive the consensus-based sampling (CBS) algorithm, starting from ensemble-preconditioned Langevin diffusions. We approximate the target potential by its Moreau envelope, such that the gradient in the Langevin equation can be replaced by a proximal operator. We then approximate the proximal operator by a weighted mean, and finally assume that the initial and target distributions are Gaussian, resulting in the CBS dynamics. If we keep only those approximations that can be justified in the non-Gaussian setting, the result is a new interacting-particle method for sampling, which we call localized consensus-based sampling. We prove that our algorithm is affine-invariant and exact for Gaussian distributions in the mean-field setting. Numerical tests illustrate that localized CBS compares favorably to alternative methods in terms of affine-invariance and performance on non-Gaussian distributions.
△ Less
Submitted 8 July, 2025; v1 submitted 30 May, 2025;
originally announced May 2025.
-
Efficient parallel inversion of ParaOpt preconditioners
Authors:
Corentin Bonte,
Arne Bouillon,
Giovanni Samaey,
Karl Meerbergen
Abstract:
Recently, the ParaOpt algorithm was proposed as an extension of the time-parallel Parareal method to optimal control. ParaOpt uses quasi-Newton steps that each require solving a system of matching conditions iteratively. The state-of-the-art parallel preconditioner for linear problems leads to a set of independent smaller systems that are currently hard to solve. We generalize the preconditioner t…
▽ More
Recently, the ParaOpt algorithm was proposed as an extension of the time-parallel Parareal method to optimal control. ParaOpt uses quasi-Newton steps that each require solving a system of matching conditions iteratively. The state-of-the-art parallel preconditioner for linear problems leads to a set of independent smaller systems that are currently hard to solve. We generalize the preconditioner to the nonlinear case and propose a new, fast inversion method for these smaller systems, avoiding disadvantages of the current options with adjusted boundary conditions in the subproblems.
△ Less
Submitted 3 December, 2024;
originally announced December 2024.
-
Single-ensemble multilevel Monte Carlo for discrete ensemble Kalman methods
Authors:
Arne Bouillon,
Toon Ingelaere,
Giovanni Samaey
Abstract:
Ensemble Kalman methods solve problems in domains such as filtering and inverse problems with interacting particles that evolve over time. For computationally expensive problems, the cost of attaining a high accuracy quickly becomes prohibitive. We exploit a hierarchy of approximations to the underlying forward model and apply multilevel Monte Carlo (MLMC) techniques, improving the asymptotic cost…
▽ More
Ensemble Kalman methods solve problems in domains such as filtering and inverse problems with interacting particles that evolve over time. For computationally expensive problems, the cost of attaining a high accuracy quickly becomes prohibitive. We exploit a hierarchy of approximations to the underlying forward model and apply multilevel Monte Carlo (MLMC) techniques, improving the asymptotic cost-to-error relation. More specifically, we use MLMC at each time step to estimate the interaction term in a single, globally-coupled ensemble. This technique was proposed by Hoel et al. for the ensemble Kalman filter; our goal is to study its applicability to a broader family of ensemble Kalman methods.
△ Less
Submitted 17 February, 2025; v1 submitted 16 May, 2024;
originally announced May 2024.
-
Diagonalization-based preconditioners and generalized convergence bounds for ParaOpt
Authors:
Arne Bouillon,
Giovanni Samaey,
Karl Meerbergen
Abstract:
The ParaOpt algorithm was recently introduced as a time-parallel solver for optimal-control problems with a terminal-cost objective, and convergence results have been presented for the linear diffusive case with implicit-Euler time integrators. We reformulate ParaOpt for tracking problems and provide generalized convergence analyses for both objectives. We focus on linear diffusive equations and p…
▽ More
The ParaOpt algorithm was recently introduced as a time-parallel solver for optimal-control problems with a terminal-cost objective, and convergence results have been presented for the linear diffusive case with implicit-Euler time integrators. We reformulate ParaOpt for tracking problems and provide generalized convergence analyses for both objectives. We focus on linear diffusive equations and prove convergence bounds that are generic in the time integrators used. For large problem dimensions, ParaOpt's performance depends crucially on having a good preconditioner to solve the arising linear systems. For the case where ParaOpt's cheap, coarse-grained propagator is linear, we introduce diagonalization-based preconditioners inspired by recent advances in the ParaDiag family of methods. These preconditioners not only lead to a weakly-scalable ParaOpt version, but are themselves invertible in parallel, making maximal use of available concurrency. They have proven convergence properties in the linear diffusive case that are generic in the time discretization used, similarly to our ParaOpt results. Numerical results confirm that the iteration count of the iterative solvers used for ParaOpt's linear systems becomes constant in the limit of an increasing processor count. The paper is accompanied by a sequential MATLAB implementation.
△ Less
Submitted 8 May, 2023; v1 submitted 18 April, 2023;
originally announced April 2023.
-
On generalized preconditioners for time-parallel parabolic optimal control
Authors:
Arne Bouillon,
Giovanni Samaey,
Karl Meerbergen
Abstract:
The ParaDiag family of algorithms solves differential equations by using preconditioners that can be inverted in parallel through diagonalization. In the context of optimal control of linear parabolic PDEs, the state-of-the-art ParaDiag method is limited to solving self-adjoint problems with a tracking objective. We propose three improvements to the ParaDiag method: the use of alpha-circulant matr…
▽ More
The ParaDiag family of algorithms solves differential equations by using preconditioners that can be inverted in parallel through diagonalization. In the context of optimal control of linear parabolic PDEs, the state-of-the-art ParaDiag method is limited to solving self-adjoint problems with a tracking objective. We propose three improvements to the ParaDiag method: the use of alpha-circulant matrices to construct an alternative preconditioner, a generalization of the algorithm for solving non-self-adjoint equations, and the formulation of an algorithm for terminal-cost objectives. We present novel analytic results about the eigenvalues of the preconditioned systems for all discussed ParaDiag algorithms in the case of self-adjoint equations, which proves the favorable properties the alpha-circulant preconditioner. We use these results to perform a theoretical parallel-scaling analysis of ParaDiag for self-adjoint problems. Numerical tests confirm our findings and suggest that the self-adjoint behavior, which is backed by theory, generalizes to the non-self-adjoint case. We provide a sequential, open-source reference solver in Matlab for all discussed algorithms.
△ Less
Submitted 14 March, 2024; v1 submitted 13 February, 2023;
originally announced February 2023.
