-
Adaptive Reduced Multilevel Splitting
Authors:
Frédéric Cérou,
Patrick Héas,
Mathias Rousset
Abstract:
This paper considers the classical problem of sampling with Monte Carlo methods a target rare event distribution defined by a score function that is very expensive to compute. We assume we can build using evaluations of the true score, an approximate surrogate score certified with error bounds. This work proposes a fully adaptive algorithm to sequentially sample surrogate rare event distributions…
▽ More
This paper considers the classical problem of sampling with Monte Carlo methods a target rare event distribution defined by a score function that is very expensive to compute. We assume we can build using evaluations of the true score, an approximate surrogate score certified with error bounds. This work proposes a fully adaptive algorithm to sequentially sample surrogate rare event distributions with increasing target levels. An essential contribution consists in sampling at each iteration the surrogate rare event at a critical level corresponding to a specific cost. This cost is related to importance sampling for a target for a given budget. The critical level is calculated solely from the reduced score and its error bound From a practical point of view, sampling the proposal sequence is performed by extending the framework of the popular adaptive multilevel splitting algorithm to the use of score approximations. Numerical experiments evaluate the proposed importance sampling algorithm in terms of computational complexity versus squared error. In particular, we investigate the performance of the algorithm when simulating rare events related to the solution of a parametric PDE, which is approximated by a reduced basis.
△ Less
Submitted 24 October, 2024; v1 submitted 23 December, 2023;
originally announced December 2023.
-
Entropy minimizing distributions are worst-case optimal importance proposals
Authors:
Frédéric Cérou,
Patrick Héas,
Mathias Rousset
Abstract:
Importance sampling of target probability distributions belonging to a given convex class is considered. Motivated by previous results, the cost of importance sampling is quantified using the relative entropy of the target with respect to proposal distributions. Using a reference measure as a reference for cost, we prove under some general conditions that the worst-case optimal proposal is precise…
▽ More
Importance sampling of target probability distributions belonging to a given convex class is considered. Motivated by previous results, the cost of importance sampling is quantified using the relative entropy of the target with respect to proposal distributions. Using a reference measure as a reference for cost, we prove under some general conditions that the worst-case optimal proposal is precisely given by the distribution minimizing entropy with respect to the reference within the considered convex class of distributions. The latter conditions are in particular satisfied when the convex class is defined using a push-forward map defining atomless conditional measures. Applications in which the optimal proposal is Gibbsian and can be practically sampled using Monte Carlo methods are discussed.
△ Less
Submitted 8 December, 2022;
originally announced December 2022.
-
Chilled Sampling for Uncertainty Quantification: A Motivation From A Meteorological Inverse Problem
Authors:
Patrick Héas,
Frédéric Cérou,
Mathias Rousset
Abstract:
Atmospheric motion vectors (AMVs) extracted from satellite imagery are the only wind observations with good global coverage. They are important features for feeding numerical weather prediction (NWP) models. Several Bayesian models have been proposed to estimate AMVs. Although critical for correct assimilation into NWP models, very few methods provide a thorough characterization of the estimation…
▽ More
Atmospheric motion vectors (AMVs) extracted from satellite imagery are the only wind observations with good global coverage. They are important features for feeding numerical weather prediction (NWP) models. Several Bayesian models have been proposed to estimate AMVs. Although critical for correct assimilation into NWP models, very few methods provide a thorough characterization of the estimation errors. The difficulty of estimating errors stems from the specificity of the posterior distribution, which is both very high dimensional, and highly ill-conditioned due to a singular likelihood. Motivated by this difficult inverse problem, this work studies the evaluation of the (expected) estimation errors using gradient-based Markov Chain Monte Carlo (MCMC) algorithms. The main contribution is to propose a general strategy, called here chilling, which amounts to sampling a local approximation of the posterior distribution in the neighborhood of a point estimate. From a theoretical point of view, we show that under regularity assumptions, the family of chilled posterior distributions converges in distribution as temperature decreases to an optimal Gaussian approximation at a point estimate given by the Maximum A Posteriori, also known as the Laplace approximation. Chilled sampling therefore provides access to this approximation generally out of reach in such high-dimensional nonlinear contexts. From an empirical perspective, we evaluate the proposed approach based on some quantitative Bayesian criteria. Our numerical simulations are performed on synthetic and real meteorological data. They reveal that not only the proposed chilling exhibits a significant gain in terms of accuracy of the point estimates and of their associated expected errors, but also a substantial acceleration in the convergence speed of the MCMC algorithms.
△ Less
Submitted 25 October, 2023; v1 submitted 7 July, 2022;
originally announced July 2022.
-
Generalized Kernel-Based Dynamic Mode Decomposition
Authors:
Patrick Heas,
Cedric Herzet,
Benoit Combes
Abstract:
Reduced modeling in high-dimensional reproducing kernel Hilbert spaces offers the opportunity to approximate efficiently non-linear dynamics. In this work, we devise an algorithm based on low rank constraint optimization and kernel-based computation that generalizes a recent approach called "kernel-based dynamic mode decomposition". This new algorithm is characterized by a gain in approximation ac…
▽ More
Reduced modeling in high-dimensional reproducing kernel Hilbert spaces offers the opportunity to approximate efficiently non-linear dynamics. In this work, we devise an algorithm based on low rank constraint optimization and kernel-based computation that generalizes a recent approach called "kernel-based dynamic mode decomposition". This new algorithm is characterized by a gain in approximation accuracy, as evidenced by numerical simulations, and in computational complexity.
△ Less
Submitted 11 February, 2020;
originally announced February 2020.
-
Selecting Reduced Models in the Cross-Entropy Method
Authors:
Patrick Héas
Abstract:
This paper deals with the estimation of rare event probabilities using importance sampling (IS), where an optimal proposal distribution is computed with the cross-entropy (CE) method. Although, IS optimized with the CE method leads to an efficient reduction of the estimator variance, this approach remains unaffordable for problems where the repeated evaluation of the score function represents a to…
▽ More
This paper deals with the estimation of rare event probabilities using importance sampling (IS), where an optimal proposal distribution is computed with the cross-entropy (CE) method. Although, IS optimized with the CE method leads to an efficient reduction of the estimator variance, this approach remains unaffordable for problems where the repeated evaluation of the score function represents a too intensive computational effort. This is often the case for score functions related to the solution of a partial differential equation (PDE) with random inputs. This work proposes to alleviate computation by the parsimonious use of a hierarchy of score function approximations in the CE optimization process. The score function approximation is obtained by selecting the surrogate of lowest dimensionality, whose accuracy guarantees to pass the current CE optimization stage. The selection relies on certified upper bounds on the error norm. An asymptotic analysis provides some theoretical guarantees on the efficiency and convergence of the proposed algorithm. Numerical results demonstrate the gain brought by the method in the context of pollution alerts and a system modeled by a PDE.
△ Less
Submitted 4 February, 2020; v1 submitted 5 June, 2018;
originally announced June 2018.
-
A Mathematical Characterization of the Performance of the "Multi-Slice" Projector
Authors:
C. Herzet,
M. Diallo,
P. Héas
Abstract:
We consider an enhanced version of the well-kwown "Petrov-Galerkin" projection in Hilbert spaces. The proposed procedure, dubbed "multi-slice" projector, exploits the fact that the sought solution belongs to the intersection of several high-dimensional slices. This setup is for example of interest in model-order reduction where this type of prior may be computed off-line. In this note, we provide…
▽ More
We consider an enhanced version of the well-kwown "Petrov-Galerkin" projection in Hilbert spaces. The proposed procedure, dubbed "multi-slice" projector, exploits the fact that the sought solution belongs to the intersection of several high-dimensional slices. This setup is for example of interest in model-order reduction where this type of prior may be computed off-line. In this note, we provide a mathematical characterization of the performance achievable by the multi-slice projector and compare the latter with the results holding in the Petrov-Galerkin setup. In particular, we illustrate the superiority of the multi-slice approach in certain situations.
△ Less
Submitted 10 May, 2018;
originally announced May 2018.
-
Low-Rank Dynamic Mode Decomposition: An Exact and Tractable Solution
Authors:
Patrick Héas,
Cédric Herzet
Abstract:
This work studies the linear approximation of high-dimensional dynamical systems using low-rank dynamic mode decomposition (DMD). Searching this approximation in a data-driven approach is formalised as attempting to solve a low-rank constrained optimisation problem. This problem is non-convex and state-of-the-art algorithms are all sub-optimal. This paper shows that there exists a closed-form solu…
▽ More
This work studies the linear approximation of high-dimensional dynamical systems using low-rank dynamic mode decomposition (DMD). Searching this approximation in a data-driven approach is formalised as attempting to solve a low-rank constrained optimisation problem. This problem is non-convex and state-of-the-art algorithms are all sub-optimal. This paper shows that there exists a closed-form solution, which is computed in polynomial time, and characterises the l2-norm of the optimal approximation error. The paper also proposes low-complexity algorithms building reduced models from this optimal solution, based on singular value decomposition or eigen value decomposition. The algorithms are evaluated by numerical simulations using synthetic and physical data benchmarks.
△ Less
Submitted 20 August, 2021; v1 submitted 10 October, 2016;
originally announced October 2016.
-
Model Reduction from Partial Observations
Authors:
C. Herzet,
P. Héas,
A. Drémeau
Abstract:
This paper deals with model-order reduction of parametric partial differential equations (PPDE). More specifically, we consider the problem of finding a good approximation subspace of the solution manifold of the PPDE when only partial information on the latter is available. We assume that two sources of information are available: i) a "rough" prior knowledge, taking the form of a manifold contain…
▽ More
This paper deals with model-order reduction of parametric partial differential equations (PPDE). More specifically, we consider the problem of finding a good approximation subspace of the solution manifold of the PPDE when only partial information on the latter is available. We assume that two sources of information are available: i) a "rough" prior knowledge, taking the form of a manifold containing the target solution manifold, ii) partial linear measurements of the solutions of the PPDE (the term partial refers to the fact that observation operator cannot be inverted). We provide and study several tools to derive good approximation subspaces from these two sources of information. We first identify the best worst-case performance achievable in this setup and propose simple procedures to approximate the corresponding optimal approximation subspace. We then provide, in a simplified setup, a theoretical analysis relating the achievable reduction performance to the choice of the observation operator and the prior knowledge available on the solution manifold.
△ Less
Submitted 3 July, 2017; v1 submitted 28 September, 2016;
originally announced September 2016.
-
Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)
Authors:
L. Jacques,
C. De Vleeschouwer,
Y. Boursier,
P. Sudhakar,
C. De Mol,
A. Pizurica,
S. Anthoine,
P. Vandergheynst,
P. Frossard,
C. Bilen,
S. Kitic,
N. Bertin,
R. Gribonval,
N. Boumal,
B. Mishra,
P. -A. Absil,
R. Sepulchre,
S. Bundervoet,
C. Schretter,
A. Dooms,
P. Schelkens,
O. Chabiron,
F. Malgouyres,
J. -Y. Tourneret,
N. Dobigeon
, et al. (42 additional authors not shown)
Abstract:
The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in…
▽ More
The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in Belgium, from Wednesday August 27th till Friday August 29th, 2014. The workshop was conveniently located in "The Arsenal" building within walking distance of both hotels and town center. iTWIST'14 has gathered about 70 international participants and has featured 9 invited talks, 10 oral presentations, and 14 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing; Union of low dimensional subspaces; Beyond linear and convex inverse problem; Matrix/manifold/graph sensing/processing; Blind inverse problems and dictionary learning; Sparsity and computational neuroscience; Information theory, geometry and randomness; Complexity/accuracy tradeoffs in numerical methods; Sparsity? What's next?; Sparse machine learning and inference.
△ Less
Submitted 9 October, 2014; v1 submitted 2 October, 2014;
originally announced October 2014.
-
Self-similar prior and wavelet bases for hidden incompressible turbulent motion
Authors:
Patrick Héas,
Frédéric Lavancier,
Souleymane Kadri-Harouna
Abstract:
This work is concerned with the ill-posed inverse problem of estimating turbulent flows from the observation of an image sequence. From a Bayesian perspective, a divergence-free isotropic fractional Brownian motion (fBm) is chosen as a prior model for instantaneous turbulent velocity fields. This self-similar prior characterizes accurately second-order statistics of velocity fields in incompressib…
▽ More
This work is concerned with the ill-posed inverse problem of estimating turbulent flows from the observation of an image sequence. From a Bayesian perspective, a divergence-free isotropic fractional Brownian motion (fBm) is chosen as a prior model for instantaneous turbulent velocity fields. This self-similar prior characterizes accurately second-order statistics of velocity fields in incompressible isotropic turbulence. Nevertheless, the associated maximum a posteriori involves a fractional Laplacian operator which is delicate to implement in practice. To deal with this issue, we propose to decompose the divergent-free fBm on well-chosen wavelet bases. As a first alternative, we propose to design wavelets as whitening filters. We show that these filters are fractional Laplacian wavelets composed with the Leray projector. As a second alternative, we use a divergence-free wavelet basis, which takes implicitly into account the incompressibility constraint arising from physics. Although the latter decomposition involves correlated wavelet coefficients, we are able to handle this dependence in practice. Based on these two wavelet decompositions, we finally provide effective and efficient algorithms to approach the maximum a posteriori. An intensive numerical evaluation proves the relevance of the proposed wavelet-based self-similar priors.
△ Less
Submitted 13 March, 2014; v1 submitted 22 February, 2013;
originally announced February 2013.
