-
Non-asymptotic confidence regions on RKHS. The Paley-Wiener and standard Sobolev space cases
Authors:
Fabrice Gamboa,
Olivier Roustant
Abstract:
We consider the problem of constructing a global, probabilistic, and non-asymptotic confidence region for an unknown function observed on a random design. The unknown function is assumed to lie in a reproducing kernel Hilbert space (RKHS). We show that this construction can be reduced to accurately estimating the RKHS norm of the unknown function. Our analysis primarily focuses both on the Paley-W…
▽ More
We consider the problem of constructing a global, probabilistic, and non-asymptotic confidence region for an unknown function observed on a random design. The unknown function is assumed to lie in a reproducing kernel Hilbert space (RKHS). We show that this construction can be reduced to accurately estimating the RKHS norm of the unknown function. Our analysis primarily focuses both on the Paley-Wiener and on the standard Sobolev space settings.
△ Less
Submitted 9 July, 2025;
originally announced July 2025.
-
General reproducing properties in RKHS with application to derivative and integral operators
Authors:
Fatima-Zahrae El-Boukkouri,
Josselin Garnier,
Olivier Roustant
Abstract:
In this paper, we consider the reproducing property in Reproducing Kernel Hilbert Spaces (RKHS). We establish a reproducing property for the closure of the class of combinations of composition operators under minimal conditions. This allows to revisit the sufficient conditions for the reproducing property to hold for the derivative operator, as well as for the existence of the mean embedding funct…
▽ More
In this paper, we consider the reproducing property in Reproducing Kernel Hilbert Spaces (RKHS). We establish a reproducing property for the closure of the class of combinations of composition operators under minimal conditions. This allows to revisit the sufficient conditions for the reproducing property to hold for the derivative operator, as well as for the existence of the mean embedding function. These results provide a framework of application of the representer theorem for regularized learning algorithms that involve data for function values, gradients, or any other operator from the considered class.
△ Less
Submitted 31 March, 2025; v1 submitted 20 March, 2025;
originally announced March 2025.
-
Fast pick-freeze estimation of Sobol' sensitivity maps using basis expansions
Authors:
Yuri Sao,
Olivier Roustant,
Geraldo de Freitas Maciel
Abstract:
Global sensitivity analysis (GSA) aims at quantifying the contribution of input variables over the variability of model outputs. In the frame of functional outputs, a common goal is to compute sensitivity maps (SM), i.e sensitivity indices at each output dimension (e.g. time step for time series, or pixels for spatial outputs). In specific settings, some works have shown that the computation of So…
▽ More
Global sensitivity analysis (GSA) aims at quantifying the contribution of input variables over the variability of model outputs. In the frame of functional outputs, a common goal is to compute sensitivity maps (SM), i.e sensitivity indices at each output dimension (e.g. time step for time series, or pixels for spatial outputs). In specific settings, some works have shown that the computation of Sobol' SM can be speeded up by using basis expansions employed for dimension reduction. However, how to efficiently compute such SM in a general setting has not received too much attention in the GSA literature.In this work, we propose fast computations of Sobol' SM using a general basis expansion, with a focus on statistical estimation. First, we write a closed-form expression of SM in function of the matrix-valued Sobol' index of the vector of basis coefficients. Secondly, we consider pick-freeze (PF) estimators, which have nice statistical properties (in terms of asymptotical efficiency) for Sobol' indices of any order. We provide similar basis-derived formulas for the PF estimator of Sobol' SM in function of the matrix-valued PF estimator of the vector of basis coefficients. We give the computational cost, and show that, compared to a dimension-wise approach, the computational gain is substantial and allows to calculate both SM and their associated bootstrap confidence bounds in a reasonable time. Finally, we illustrate the whole methodology on an analytical test case and on an application in non-Newtonian hydraulics, modelling an idealized dam-break flow.
△ Less
Submitted 11 December, 2024;
originally announced December 2024.
-
On one dimensional weighted Poincare inequalities for Global Sensitivity Analysis
Authors:
David Heredia,
Aldéric Joulin,
Olivier Roustant
Abstract:
One-dimensional Poincare inequalities are used in Global Sensitivity Analysis (GSA) to provide derivative-based upper bounds and approximations of Sobol indices. We add new perspectives by investigating weighted Poincare inequalities. Our contributions are twofold. In a first part, we provide new theoretical results for weighted Poincare inequalities, guided by GSA needs. We revisit the constructi…
▽ More
One-dimensional Poincare inequalities are used in Global Sensitivity Analysis (GSA) to provide derivative-based upper bounds and approximations of Sobol indices. We add new perspectives by investigating weighted Poincare inequalities. Our contributions are twofold. In a first part, we provide new theoretical results for weighted Poincare inequalities, guided by GSA needs. We revisit the construction of weights from monotonic functions, providing a new proof from a spectral point of view. In this approach, given a monotonic function g, the weight is built such that g is the first non-trivial eigenfunction of a convenient diffusion operator. This allows us to reconsider the linear standard, i.e. the weight associated to a linear g. In particular, we construct weights that guarantee the existence of an orthonormal basis of eigenfunctions, leading to approximation of Sobol indices with Parseval formulas. In a second part, we develop specific methods for GSA. We study the equality case of the upper bound of a total Sobol index, and link the sharpness of the inequality to the proximity of the main effect to the eigenfunction. This leads us to theoretically investigate the construction of data-driven weights from estimators of the main effects when they are monotonic, another extension of the linear standard. Finally, we illustrate the benefits of using weights on a GSA study of two toy models and a real flooding application, involving the Poincare constant and/or the whole eigenbasis.
△ Less
Submitted 6 December, 2024;
originally announced December 2024.
-
Covariance models and Gaussian process regression for the wave equation. Application to related inverse problems
Authors:
Iain Henderson,
Pascal Noble,
Olivier Roustant
Abstract:
In this article, we consider the general task of performing Gaussian process regression (GPR) on pointwise observations of solutions of the 3 dimensional homogeneous free space wave equation.In a recent article, we obtained promising covariance expressions tailored to this equation: we now explore the potential applications of these formulas.We first study the particular cases of stationarity and…
▽ More
In this article, we consider the general task of performing Gaussian process regression (GPR) on pointwise observations of solutions of the 3 dimensional homogeneous free space wave equation.In a recent article, we obtained promising covariance expressions tailored to this equation: we now explore the potential applications of these formulas.We first study the particular cases of stationarity and radial symmetry, for which significant simplifications arise. We next show that the true-angle multilateration method for point source localization, as used in GPS systems, is naturally recovered by our GPR formulas in the limit of the small source radius. Additionally, we show that this GPR framework provides a new answer to the ill-posed inverse problem of reconstructing initial conditions for the wave equation from a limited number of sensors, and simultaneously enables the inference of physical parameters from these data. We finish by illustrating this ``physics informed'' GPR on a number of practical examples.
△ Less
Submitted 9 November, 2023;
originally announced November 2023.
-
A scalable problem to benchmark robust multidisciplinary design optimization techniques
Authors:
A Aziz-Alaoui,
O Roustant,
M de Lozzo
Abstract:
A scalable problem to benchmark robust multidisciplinary design optimization algorithms (RMDO) is proposed. This allows the user to choose the number of disciplines, the dimensions of the coupling and design variables and the extent of the feasible domain. After a description of the mathematical background, a deterministic version of the scalable problem is defined and the conditions on the existe…
▽ More
A scalable problem to benchmark robust multidisciplinary design optimization algorithms (RMDO) is proposed. This allows the user to choose the number of disciplines, the dimensions of the coupling and design variables and the extent of the feasible domain. After a description of the mathematical background, a deterministic version of the scalable problem is defined and the conditions on the existence and uniqueness of the solution are given. Then, this deterministic scalable problem is made uncertain by adding random variables to the coupling equations. Under classical assumptions, the existence and uniqueness of the solution of this RMDO problem is guaranteed. This solution can be easily computed with a quadratic programming algorithm and serves as a reference to assess the performances of RMDO algorithms. This scalable problem has been implemented in the open source software GEMSEO and tested with two techniques of statistics estimation: Monte-Carlo sampling and Taylor polynomials.
△ Less
Submitted 27 February, 2023;
originally announced March 2023.
-
Characterization of the second order random fields subject to linear distributional PDE constraints
Authors:
Iain Henderson,
Pascal Noble,
Olivier Roustant
Abstract:
Let $L$ be a linear differential operator acting on functions defined over an open set $\mathcal{D}\subset \mathbb{R}^d$. In this article, we characterize the measurable second order random fields $U = (U(x))_{x\in\mathcal{D}}$ whose sample paths all verify the partial differential equation (PDE) $L(u) = 0$, solely in terms of their first two moments. When compared to previous similar results, the…
▽ More
Let $L$ be a linear differential operator acting on functions defined over an open set $\mathcal{D}\subset \mathbb{R}^d$. In this article, we characterize the measurable second order random fields $U = (U(x))_{x\in\mathcal{D}}$ whose sample paths all verify the partial differential equation (PDE) $L(u) = 0$, solely in terms of their first two moments. When compared to previous similar results, the novelty lies in that the equality $L(u) = 0$ is understood in the sense of distributions, which is a powerful functional analysis framework mostly designed to study linear PDEs. This framework enables to reduce to the minimum the required differentiability assumptions over the first two moments of $(U(x))_{x\in\mathcal{D}}$ as well as over its sample paths in order to make sense of the PDE $L(U_ω)=0$. In view of Gaussian process regression (GPR) applications, we show that when $(U(x))_{x\in\mathcal{D}}$ is a Gaussian process (GP), the sample paths of $(U(x))_{x\in\mathcal{D}}$ conditioned on pointwise observations still verify the constraint $L(u)=0$ in the distributional sense. We finish by deriving a simple but instructive example, a GP model for the 3D linear wave equation, for which our theorem is applicable and where the previous results from the literature do not apply in general.
△ Less
Submitted 17 January, 2023;
originally announced January 2023.
-
Bayesian quadrature for $H^1(μ)$ with Poincaré inequality on a compact interval
Authors:
Olivier Roustant,
Nora Lüthen,
Fabrice Gamboa
Abstract:
Motivated by uncertainty quantification of complex systems, we aim at finding quadrature formulas of the form $\int_a^b f(x) dμ(x) = \sum_{i=1}^n w_i f(x_i)$ where $f$ belongs to $H^1(μ)$. Here, $μ$ belongs to a class of continuous probability distributions on $[a, b] \subset \mathbb{R}$ and $\sum_{i=1}^n w_i δ_{x_i}$ is a discrete probability distribution on $[a, b]$. We show that $H^1(μ)$ is a r…
▽ More
Motivated by uncertainty quantification of complex systems, we aim at finding quadrature formulas of the form $\int_a^b f(x) dμ(x) = \sum_{i=1}^n w_i f(x_i)$ where $f$ belongs to $H^1(μ)$. Here, $μ$ belongs to a class of continuous probability distributions on $[a, b] \subset \mathbb{R}$ and $\sum_{i=1}^n w_i δ_{x_i}$ is a discrete probability distribution on $[a, b]$. We show that $H^1(μ)$ is a reproducing kernel Hilbert space with a continuous kernel $K$, which allows to reformulate the quadrature question as a Bayesian (or kernel) quadrature problem. Although $K$ has not an easy closed form in general, we establish a correspondence between its spectral decomposition and the one associated to Poincaré inequalities, whose common eigenfunctions form a $T$-system (Karlin and Studden, 1966). The quadrature problem can then be solved in the finite-dimensional proxy space spanned by the first eigenfunctions. The solution is given by a generalized Gaussian quadrature, which we call Poincaré quadrature. We derive several results for the Poincaré quadrature weights and the associated worst-case error. When $μ$ is the uniform distribution, the results are explicit: the Poincaré quadrature is equivalent to the midpoint (rectangle) quadrature rule. Its nodes coincide with the zeros of an eigenfunction and the worst-case error scales as $\frac{b-a}{2\sqrt{3}}n^{-1}$ for large $n$. By comparison with known results for $H^1(0,1)$, this shows that the Poincaré quadrature is asymptotically optimal. For a general $μ$, we provide an efficient numerical procedure, based on finite elements and linear programming. Numerical experiments provide useful insights: nodes are nearly evenly spaced, weights are close to the probability density at nodes, and the worst-case error is approximately $O(n^{-1})$ for large $n$.
△ Less
Submitted 29 July, 2022;
originally announced July 2022.
-
Stochastic Processes Under Linear Differential Constraints : Application to Gaussian Process Regression for the 3 Dimensional Free Space Wave Equation
Authors:
Iain Henderson,
Pascal Noble,
Olivier Roustant
Abstract:
Let $P$ be a linear differential operator over $\mathcal{D} \subset \mathbb{R}^d$ and $U = (U_x)_{x \in \mathcal{D}}$ a second order stochastic process. In the first part of this article, we prove a new necessary and sufficient condition for all the trajectories of $U$ to verify the partial differential equation (PDE) $T(U) = 0$. This condition is formulated in terms of the covariance kernel of…
▽ More
Let $P$ be a linear differential operator over $\mathcal{D} \subset \mathbb{R}^d$ and $U = (U_x)_{x \in \mathcal{D}}$ a second order stochastic process. In the first part of this article, we prove a new necessary and sufficient condition for all the trajectories of $U$ to verify the partial differential equation (PDE) $T(U) = 0$. This condition is formulated in terms of the covariance kernel of $U$. When compared to previous similar results, the novelty lies in that the equality $T(U) = 0$ is understood in the \textit{sense of distributions}, which is a relevant framework for PDEs. This theorem provides precious insights during the second part of this article, devoted to performing "physically informed" machine learning for the homogeneous 3 dimensional free space wave equation. We perform Gaussian process regression (GPR) on pointwise observations of a solution of this PDE. To do so, we propagate Gaussian processes (GP) priors over its initial conditions through the wave equation. We obtain explicit formulas for the covariance kernel of the propagated GP, which can then be used for GPR. We then explore the particular cases of radial symmetry and point source. For the former, we derive convolution-free GPR formulas; for the latter, we show a direct link between GPR and the classical triangulation method for point source localization used in GPS systems. Additionally, this Bayesian framework provides a new answer for the ill-posed inverse problem of reconstructing initial conditions for the wave equation with a limited number of sensors, and simultaneously enables the inference of physical parameters from these data. Finally, we illustrate this physically informed GPR on a number of practical examples.
△ Less
Submitted 10 February, 2022; v1 submitted 23 November, 2021;
originally announced November 2021.
-
A comparison of mixed-variables Bayesian optimization approaches
Authors:
Jhouben Cuesta-Ramirez,
Rodolphe Le Riche,
Olivier Roustant,
Guillaume Perrin,
Cedric Durantin,
Alain Gliere
Abstract:
Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box simulation.General mixed and costly optimization problems are therefore of a great practical interest, yet their resolution remains in a large part an open scientif…
▽ More
Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box simulation.General mixed and costly optimization problems are therefore of a great practical interest, yet their resolution remains in a large part an open scientific question. In this article, costly mixed problems are approached through Gaussian processes where the discrete variables are relaxed into continuous latent variables. The continuous space is more easily harvested by classical Bayesian optimization techniques than a mixed space would. Discrete variables are recovered either subsequently to the continuous optimization, or simultaneously with an additional continuous-discrete compatibility constraint that is handled with augmented Lagrangians. Several possible implementations of such Bayesian mixed optimizers are compared. In particular, the reformulation of the problem with continuous latent variables is put in competition with searches working directly in the mixed space. Among the algorithms involving latent variables and an augmented Lagrangian, a particular attention is devoted to the Lagrange multipliers for which a local and a global estimation techniques are studied. The comparisons are based on the repeated optimization of three analytical functions and a beam design problem.
△ Less
Submitted 3 May, 2022; v1 submitted 30 October, 2021;
originally announced November 2021.
-
Global sensitivity analysis using derivative-based sparse Poincaré chaos expansions
Authors:
Nora Lüthen,
Olivier Roustant,
Fabrice Gamboa,
Bertrand Iooss,
Stefano Marelli,
Bruno Sudret
Abstract:
Variance-based global sensitivity analysis, in particular Sobol' analysis, is widely used for determining the importance of input variables to a computational model. Sobol' indices can be computed cheaply based on spectral methods like polynomial chaos expansions (PCE). Another choice are the recently developed Poincaré chaos expansions (PoinCE), whose orthonormal tensor-product basis is generated…
▽ More
Variance-based global sensitivity analysis, in particular Sobol' analysis, is widely used for determining the importance of input variables to a computational model. Sobol' indices can be computed cheaply based on spectral methods like polynomial chaos expansions (PCE). Another choice are the recently developed Poincaré chaos expansions (PoinCE), whose orthonormal tensor-product basis is generated from the eigenfunctions of one-dimensional Poincaré differential operators. In this paper, we show that the Poincaré basis is the unique orthonormal basis with the property that partial derivatives of the basis form again an orthogonal basis with respect to the same measure as the original basis. This special property makes PoinCE ideally suited for incorporating derivative information into the surrogate modelling process. Assuming that partial derivative evaluations of the computational model are available, we compute spectral expansions in terms of Poincaré basis functions or basis partial derivatives, respectively, by sparse regression. We show on two numerical examples that the derivative-based expansions provide accurate estimates for Sobol' indices, even outperforming PCE in terms of bias and variance. In addition, we derive an analytical expression based on the PoinCE coefficients for a second popular sensitivity index, the derivative-based sensitivity measure (DGSM), and explore its performance as upper bound to the corresponding total Sobol' indices.
△ Less
Submitted 9 June, 2023; v1 submitted 1 July, 2021;
originally announced July 2021.
-
Sequential construction and dimension reduction of Gaussian processes under constraints
Authors:
François Bachoc,
Andrés F. López Lopera,
Olivier Roustant
Abstract:
Accounting for inequality constraints, such as boundedness, monotonicity or convexity, is challenging when modeling costly-to-evaluate black box functions. In this regard, finite-dimensional Gaussian process (GP) regression models bring a valuable solution, as they guarantee that the inequality constraints are satisfied everywhere. Nevertheless, these models are currently restricted to small dimen…
▽ More
Accounting for inequality constraints, such as boundedness, monotonicity or convexity, is challenging when modeling costly-to-evaluate black box functions. In this regard, finite-dimensional Gaussian process (GP) regression models bring a valuable solution, as they guarantee that the inequality constraints are satisfied everywhere. Nevertheless, these models are currently restricted to small dimensional situations (up to dimension 5). Addressing this issue, we introduce the MaxMod algorithm that sequentially inserts one-dimensional knots or adds active variables, thereby performing at the same time dimension reduction and efficient knot allocation. We prove the convergence of this algorithm. In intermediary steps of the proof, we propose the notion of multi-affine extension and study its properties. We also prove the convergence of finite-dimensional GPs, when the knots are not dense in the input space, extending the recent literature. With simulated and real data, we demonstrate that the MaxMod algorithm remains efficient in higher dimension (at least in dimension 20), and needs fewer knots than other constrained GP models from the state-of-the-art, to reach a given approximation error.
△ Less
Submitted 10 March, 2022; v1 submitted 9 September, 2020;
originally announced September 2020.
-
Sensitivity Analysis and Generalized Chaos Expansions. Lower Bounds for Sobol indices
Authors:
O Roustant,
F. Gamboa,
B Iooss
Abstract:
The so-called polynomial chaos expansion is widely used in computer experiments. For example, it is a powerful tool to estimate Sobol' sensitivity indices. In this paper, we consider generalized chaos expansions built on general tensor Hilbert basis. In this frame, we revisit the computation of the Sobol' indices and give general lower bounds for these indices. The case of the eigenfunctions syste…
▽ More
The so-called polynomial chaos expansion is widely used in computer experiments. For example, it is a powerful tool to estimate Sobol' sensitivity indices. In this paper, we consider generalized chaos expansions built on general tensor Hilbert basis. In this frame, we revisit the computation of the Sobol' indices and give general lower bounds for these indices. The case of the eigenfunctions system associated with a Poincar{é} differential operator leads to lower bounds involving the derivatives of the analyzed function and provides an efficient tool for variable screening. These lower bounds are put in action both on toy and real life models demonstrating their accuracy.
△ Less
Submitted 24 June, 2019;
originally announced June 2019.
-
Group kernels for Gaussian process metamodels with categorical inputs
Authors:
Olivier Roustant,
Esperan Padonou,
Yves Deville,
Aloïs Clément,
Guillaume Perrin,
Jean Giorla,
Henry Wynn
Abstract:
Gaussian processes (GP) are widely used as a metamodel for emulating time-consuming computer codes. We focus on problems involving categorical inputs, with a potentially large number L of levels (typically several tens), partitioned in G << L groups of various sizes. Parsimonious covariance functions, or kernels, can then be defined by block covariance matrices T with constant covariances between…
▽ More
Gaussian processes (GP) are widely used as a metamodel for emulating time-consuming computer codes. We focus on problems involving categorical inputs, with a potentially large number L of levels (typically several tens), partitioned in G << L groups of various sizes. Parsimonious covariance functions, or kernels, can then be defined by block covariance matrices T with constant covariances between pairs of blocks and within blocks. We study the positive definiteness of such matrices to encourage their practical use. The hierarchical group/level structure, equivalent to a nested Bayesian linear model, provides a parameterization of valid block matrices T. The same model can then be used when the assumption within blocks is relaxed, giving a flexible parametric family of valid covariance matrices with constant covariances between pairs of blocks. The positive definiteness of T is equivalent to the positive definiteness of a smaller matrix of size G, obtained by averaging each block. The model is applied to a problem in nuclear waste analysis, where one of the categorical inputs is atomic number, which has more than 90 levels.
△ Less
Submitted 24 July, 2018; v1 submitted 7 February, 2018;
originally announced February 2018.
-
On the validity of parametric block correlation matrices with constant within and between group correlations
Authors:
O Roustant,
Y Deville
Abstract:
We consider the set Bp of parametric block correlation matrices with p blocks of various (and possibly different) sizes, whose diagonal blocks are compound symmetry (CS) correlation matrices and off-diagonal blocks are constant matrices. Such matrices appear in probabilistic models on categorical data, when the levels are partitioned in p groups, assuming a constant correlation within a group and…
▽ More
We consider the set Bp of parametric block correlation matrices with p blocks of various (and possibly different) sizes, whose diagonal blocks are compound symmetry (CS) correlation matrices and off-diagonal blocks are constant matrices. Such matrices appear in probabilistic models on categorical data, when the levels are partitioned in p groups, assuming a constant correlation within a group and a constant correlation for each pair of groups. We obtain two necessary and sufficient conditions for positive definiteness of elements of Bp. Firstly we consider the block average map $φ$, consisting in replacing a block by its mean value. We prove that for any A $\in$ Bp , A is positive definite if and only if $φ$(A) is positive definite. Hence it is equivalent to check the validity of the covariance matrix of group means, which only depends on the number of groups and not on their sizes. This theorem can be extended to a wider set of block matrices. Secondly, we consider the subset of Bp for which the between group correlation is the same for all pairs of groups. Positive definiteness then comes down to find the positive definite interval of a matrix pencil on Sp. We obtain a simple characterization by localizing the roots of the determinant with within group correlation values.
△ Less
Submitted 27 May, 2017;
originally announced May 2017.
-
On the choice of the low-dimensional domain for global optimization via random embeddings
Authors:
Mickaël Binois,
David Ginsbourger,
Olivier Roustant
Abstract:
The challenge of taking many variables into account in optimization problems may be overcome under the hypothesis of low effective dimensionality. Then, the search of solutions can be reduced to the random embedding of a low dimensional space into the original one, resulting in a more manageable optimization problem. Specifically, in the case of time consuming black-box functions and when the budg…
▽ More
The challenge of taking many variables into account in optimization problems may be overcome under the hypothesis of low effective dimensionality. Then, the search of solutions can be reduced to the random embedding of a low dimensional space into the original one, resulting in a more manageable optimization problem. Specifically, in the case of time consuming black-box functions and when the budget of evaluations is severely limited, global optimization with random embeddings appears as a sound alternative to random search. Yet, in the case of box constraints on the native variables, defining suitable bounds on a low dimensional domain appears to be complex. Indeed, a small search domain does not guarantee to find a solution even under restrictive hypotheses about the function, while a larger one may slow down convergence dramatically. Here we tackle the issue of low-dimensional domain selection based on a detailed study of the properties of the random embedding, giving insight on the aforementioned difficulties. In particular, we describe a minimal low-dimensional set in correspondence with the embedded search space. We additionally show that an alternative equivalent embedding procedure yields simultaneously a simpler definition of the low-dimensional minimal set and better properties in practice. Finally, the performance and robustness gains of the proposed enhancements for Bayesian optimization are illustrated on numerical examples.
△ Less
Submitted 22 October, 2018; v1 submitted 18 April, 2017;
originally announced April 2017.
-
Poincaré inequalities on intervals -- application to sensitivity analysis
Authors:
Olivier Roustant,
Franck Barthe,
Bertrand Iooss
Abstract:
The development of global sensitivity analysis of numerical model outputs has recently raised new issues on 1-dimensional Poincaré inequalities. Typically two kind of sensitivity indices are linked by a Poincaré type inequality, which provide upper bounds of the most interpretable index by using the other one, cheaper to compute. This allows performing a low-cost screening of unessential variables…
▽ More
The development of global sensitivity analysis of numerical model outputs has recently raised new issues on 1-dimensional Poincaré inequalities. Typically two kind of sensitivity indices are linked by a Poincaré type inequality, which provide upper bounds of the most interpretable index by using the other one, cheaper to compute. This allows performing a low-cost screening of unessential variables. The efficiency of this screening then highly depends on the accuracy of the upper bounds in Poincaré inequalities. The novelty in the questions concern the wide range of probability distributions involved, which are often truncated on intervals. After providing an overview of the existing knowledge and techniques, we add some theory about Poincaré constants on intervals, with improvements for symmetric intervals. Then we exploit the spectral interpretation for computing exact value of Poincaré constants of any admissible distribution on a given interval. We give semi-analytical results for some frequent distributions (truncated exponential, triangular, truncated normal), and present a numerical method in the general case. Finally, an application is made to a hydrological problem, showing the benefits of the new results in Poincaré inequalities to sensitivity analysis.
△ Less
Submitted 12 December, 2016;
originally announced December 2016.
-
Universal Prediction Distribution for Surrogate Models
Authors:
Malek Ben Salem,
Olivier Roustant,
Fabrice Gamboa,
Lionel Tomaso
Abstract:
The use of surrogate models instead of computationally expensive simulation codes is very convenient in engineering. Roughly speaking, there are two kinds of surrogate models: the deterministic and the probabilistic ones. These last are generally based on Gaussian assumptions. The main advantage of probabilistic approach is that it provides a measure of uncertainty associated with the surrogate…
▽ More
The use of surrogate models instead of computationally expensive simulation codes is very convenient in engineering. Roughly speaking, there are two kinds of surrogate models: the deterministic and the probabilistic ones. These last are generally based on Gaussian assumptions. The main advantage of probabilistic approach is that it provides a measure of uncertainty associated with the surrogate model in the whole space. This uncertainty is an efficient tool to construct strategies for various problems such as prediction enhancement, optimization or inversion.In this paper, we propose a universal method to define a measure of uncertainty suitable for any surrogate model either deterministic or probabilistic. It relies on Cross-Validation (CV) sub-models predictions. This empirical distribution may be computed in much more general frames than the Gaussian one. So that it is called the Universal Prediction distribution (UP distribution).It allows the definition of many sampling criteria. We give and study adaptive sampling techniques for global refinement and an extension of the so-called Efficient Global Optimization (EGO) algorithm. We also discuss the use of the UP distribution for inversion problems. The performances of these new algorithms are studied both on toys models and on an engineering design problem.
△ Less
Submitted 23 December, 2015;
originally announced December 2015.
-
A warped kernel improving robustness in Bayesian optimization via random embeddings
Authors:
Mickaël Binois,
David Ginsbourger,
Olivier Roustant
Abstract:
This works extends the Random Embedding Bayesian Optimization approach by integrating a warping of the high dimensional subspace within the covariance kernel. The proposed warping, that relies on elementary geometric considerations, allows mitigating the drawbacks of the high extrinsic dimensionality while avoiding the algorithm to evaluate points giving redundant information. It also alleviates c…
▽ More
This works extends the Random Embedding Bayesian Optimization approach by integrating a warping of the high dimensional subspace within the covariance kernel. The proposed warping, that relies on elementary geometric considerations, allows mitigating the drawbacks of the high extrinsic dimensionality while avoiding the algorithm to evaluate points giving redundant information. It also alleviates constraints on bound selection for the embedded domain, thus improving the robustness, as illustrated with a test case with 25 variables and intrinsic dimension 6.
△ Less
Submitted 18 March, 2015; v1 submitted 13 November, 2014;
originally announced November 2014.
-
On ANOVA decompositions of kernels and Gaussian random field paths
Authors:
David Ginsbourger,
Olivier Roustant,
Dominic Schuhmacher,
Nicolas Durrande,
Nicolas Lenz
Abstract:
The FANOVA (or "Sobol'-Hoeffding") decomposition of multivariate functions has been used for high-dimensional model representation and global sensitivity analysis. When the objective function f has no simple analytic form and is costly to evaluate, a practical limitation is that computing FANOVA terms may be unaffordable due to numerical integration costs. Several approximate approaches relying on…
▽ More
The FANOVA (or "Sobol'-Hoeffding") decomposition of multivariate functions has been used for high-dimensional model representation and global sensitivity analysis. When the objective function f has no simple analytic form and is costly to evaluate, a practical limitation is that computing FANOVA terms may be unaffordable due to numerical integration costs. Several approximate approaches relying on random field models have been proposed to alleviate these costs, where f is substituted by a (kriging) predictor or by conditional simulations. In the present work, we focus on FANOVA decompositions of Gaussian random field sample paths, and we notably introduce an associated kernel decomposition (into 2^{2d} terms) called KANOVA. An interpretation in terms of tensor product projections is obtained, and it is shown that projected kernels control both the sparsity of Gaussian random field sample paths and the dependence structure between FANOVA effects. Applications on simulated data show the relevance of the approach for designing new classes of covariance kernels dedicated to high-dimensional kriging.
△ Less
Submitted 2 October, 2014; v1 submitted 21 September, 2014;
originally announced September 2014.
-
Invariances of random fields paths, with applications in Gaussian Process Regression
Authors:
David Ginsbourger,
Olivier Roustant,
Nicolas Durrande
Abstract:
We study pathwise invariances of centred random fields that can be controlled through the covariance. A result involving composition operators is obtained in second-order settings, and we show that various path properties including additivity boil down to invariances of the covariance kernel. These results are extended to a broader class of operators in the Gaussian case, via the Loève isometry. S…
▽ More
We study pathwise invariances of centred random fields that can be controlled through the covariance. A result involving composition operators is obtained in second-order settings, and we show that various path properties including additivity boil down to invariances of the covariance kernel. These results are extended to a broader class of operators in the Gaussian case, via the Loève isometry. Several covariance-driven pathwise invariances are illustrated, including fields with symmetric paths, centred paths, harmonic paths, or sparse paths. The proposed approach delivers a number of promising results and perspectives in Gaussian process regression.
△ Less
Submitted 6 August, 2013;
originally announced August 2013.
-
A Radar-Shaped Statistic for Testing and Visualizing Uniformity Properties in Computer Experiments
Authors:
Jessica Franco,
Laurent Carraro,
Olivier Roustant,
Astrid Jourdan
Abstract:
In the study of computer codes, filling space as uniformly as possible is important to describe the complexity of the investigated phenomenon. However, this property is not conserved by reducing the dimension. Some numeric experiment designs are conceived in this sense as Latin hypercubes or orthogonal arrays, but they consider only the projections onto the axes or the coordinate planes. In this…
▽ More
In the study of computer codes, filling space as uniformly as possible is important to describe the complexity of the investigated phenomenon. However, this property is not conserved by reducing the dimension. Some numeric experiment designs are conceived in this sense as Latin hypercubes or orthogonal arrays, but they consider only the projections onto the axes or the coordinate planes. In this article we introduce a statistic which allows studying the good distribution of points according to all 1-dimensional projections. By angularly scanning the domain, we obtain a radar type representation, allowing the uniformity defects of a design to be identified with respect to its projections onto straight lines. The advantages of this new tool are demonstrated on usual examples of space-filling designs (SFD) and a global statistic independent of the angle of rotation is studied.
△ Less
Submitted 15 February, 2008;
originally announced February 2008.
-
Calculations of Sobol indices for the Gaussian process metamodel
Authors:
Amandine Marrel,
Bertrand Iooss,
Beatrice Laurent,
Olivier Roustant
Abstract:
Global sensitivity analysis of complex numerical models can be performed by calculating variance-based importance measures of the input variables, such as the Sobol indices. However, these techniques, requiring a large number of model evaluations, are often unacceptable for time expensive computer codes. A well known and widely used decision consists in replacing the computer code by a metamodel…
▽ More
Global sensitivity analysis of complex numerical models can be performed by calculating variance-based importance measures of the input variables, such as the Sobol indices. However, these techniques, requiring a large number of model evaluations, are often unacceptable for time expensive computer codes. A well known and widely used decision consists in replacing the computer code by a metamodel, predicting the model responses with a negligible computation time and rending straightforward the estimation of Sobol indices. In this paper, we discuss about the Gaussian process model which gives analytical expressions of Sobol indices. Two approaches are studied to compute the Sobol indices: the first based on the predictor of the Gaussian process model and the second based on the global stochastic process model. Comparisons between the two estimates, made on analytical examples, show the superiority of the second approach in terms of convergence and robustness. Moreover, the second approach allows to integrate the modeling error of the Gaussian process model by directly giving some confidence intervals on the Sobol indices. These techniques are finally applied to a real case of hydrogeological modeling.
△ Less
Submitted 7 February, 2008;
originally announced February 2008.
