-
Mutual Information for Explainable Deep Learning of Multiscale Systems
Authors:
Søren Taverniers,
Eric J. Hall,
Markos A. Katsoulakis,
Daniel M. Tartakovsky
Abstract:
Timely completion of design cycles for complex systems ranging from consumer electronics to hypersonic vehicles relies on rapid simulation-based prototyping. The latter typically involves high-dimensional spaces of possibly correlated control variables (CVs) and quantities of interest (QoIs) with non-Gaussian and possibly multimodal distributions. We develop a model-agnostic, moment-independent gl…
▽ More
Timely completion of design cycles for complex systems ranging from consumer electronics to hypersonic vehicles relies on rapid simulation-based prototyping. The latter typically involves high-dimensional spaces of possibly correlated control variables (CVs) and quantities of interest (QoIs) with non-Gaussian and possibly multimodal distributions. We develop a model-agnostic, moment-independent global sensitivity analysis (GSA) that relies on differential mutual information to rank the effects of CVs on QoIs. The data requirements of this information-theoretic approach to GSA are met by replacing computationally intensive components of the physics-based model with a deep neural network surrogate. Subsequently, the GSA is used to explain the network predictions, and the surrogate is deployed to close design loops. Viewed as an uncertainty quantification method for interrogating the surrogate, this framework is compatible with a wide variety of black-box models. We demonstrate that the surrogate-driven mutual information GSA provides useful and distinguishable rankings on two applications of interest in energy storage. Consequently, our information-theoretic GSA provides an "outer loop" for accelerated product design by identifying the most and least sensitive input directions and performing subsequent optimization over appropriately reduced parameter subspaces.
△ Less
Submitted 19 May, 2021; v1 submitted 7 September, 2020;
originally announced September 2020.
-
Weak error rates for option pricing under linear rough volatility
Authors:
Christian Bayer,
Eric Joseph Hall,
Raúl Tempone
Abstract:
In quantitative finance, modeling the volatility structure of underlying assets is vital to pricing options. Rough stochastic volatility models, such as the rough Bergomi model [Bayer, Friz, Gatheral, Quantitative Finance 16(6), 887-904, 2016], seek to fit observed market data based on the observation that the log-realized variance behaves like a fractional Brownian motion with small Hurst paramet…
▽ More
In quantitative finance, modeling the volatility structure of underlying assets is vital to pricing options. Rough stochastic volatility models, such as the rough Bergomi model [Bayer, Friz, Gatheral, Quantitative Finance 16(6), 887-904, 2016], seek to fit observed market data based on the observation that the log-realized variance behaves like a fractional Brownian motion with small Hurst parameter, $H < 1/2$, over reasonable timescales. Both time series of asset prices and option-derived price data indicate that $H$ often takes values close to $0.1$ or less, i.e., rougher than Brownian motion. This change improves the fit to both option prices and time series of underlying asset prices while maintaining parsimoniousness. However, the non-Markovian nature of the driving fractional Brownian motion in rough volatility models poses severe challenges for theoretical and numerical analyses and for computational practice. While the explicit Euler method is known to converge to the solution of the rough Bergomi and similar models, its strong rate of convergence is only $H$. We prove rate $H + 1/2$ for the weak convergence of the Euler method for the rough Stein-Stein model, which treats the volatility as a linear function of the driving fractional Brownian motion, and, surprisingly, we prove rate one for the case of quadratic payoff functions. Our proof uses Talay-Tubaro expansions and an affine Markovian representation of the underlying and is further supported by numerical experiments. These convergence results provide a first step toward deriving weak rates for the rough Bergomi model, which treats the volatility as a nonlinear function of the driving fractional Brownian motion.
△ Less
Submitted 15 December, 2021; v1 submitted 2 September, 2020;
originally announced September 2020.
-
GINNs: Graph-Informed Neural Networks for Multiscale Physics
Authors:
Eric J. Hall,
Søren Taverniers,
Markos A. Katsoulakis,
Daniel M. Tartakovsky
Abstract:
We introduce the concept of a Graph-Informed Neural Network (GINN), a hybrid approach combining deep learning with probabilistic graphical models (PGMs) that acts as a surrogate for physics-based representations of multiscale and multiphysics systems. GINNs address the twin challenges of removing intrinsic computational bottlenecks in physics-based models and generating large data sets for estimat…
▽ More
We introduce the concept of a Graph-Informed Neural Network (GINN), a hybrid approach combining deep learning with probabilistic graphical models (PGMs) that acts as a surrogate for physics-based representations of multiscale and multiphysics systems. GINNs address the twin challenges of removing intrinsic computational bottlenecks in physics-based models and generating large data sets for estimating probability distributions of quantities of interest (QoIs) with a high degree of confidence. Both the selection of the complex physics learned by the NN and its supervised learning/prediction are informed by the PGM, which includes the formulation of structured priors for tunable control variables (CVs) to account for their mutual correlations and ensure physically sound CV and QoI distributions. GINNs accelerate the prediction of QoIs essential for simulation-based decision-making where generating sufficient sample data using physics-based models alone is often prohibitively expensive. Using a real-world application grounded in supercapacitor-based energy storage, we describe the construction of GINNs from a Bayesian network-embedded homogenized model for supercapacitor dynamics, and demonstrate their ability to produce kernel density estimates of relevant non-Gaussian, skewed QoIs with tight confidence intervals.
△ Less
Submitted 26 June, 2020;
originally announced June 2020.
-
Causality and Bayesian network PDEs for multiscale representations of porous media
Authors:
Kimoon Um,
Eric Joseph Hall,
Markos A. Katsoulakis,
Daniel M. Tartakovsky
Abstract:
Microscopic (pore-scale) properties of porous media affect and often determine their macroscopic (continuum- or Darcy-scale) counterparts. Understanding the relationship between processes on these two scales is essential to both the derivation of macroscopic models of, e.g., transport phenomena in natural porous media, and the design of novel materials, e.g., for energy storage. Most microscopic p…
▽ More
Microscopic (pore-scale) properties of porous media affect and often determine their macroscopic (continuum- or Darcy-scale) counterparts. Understanding the relationship between processes on these two scales is essential to both the derivation of macroscopic models of, e.g., transport phenomena in natural porous media, and the design of novel materials, e.g., for energy storage. Most microscopic properties exhibit complex statistical correlations and geometric constraints, which presents challenges for the estimation of macroscopic quantities of interest (QoIs), e.g., in the context of global sensitivity analysis (GSA) of macroscopic QoIs with respect to microscopic material properties. We present a systematic way of building correlations into stochastic multiscale models through Bayesian networks. This allows us to construct the joint probability density function (PDF) of model parameters through causal relationships that emulate engineering processes, e.g., the design of hierarchical nanoporous materials. Such PDFs also serve as input for the forward propagation of parametric uncertainty; our findings indicate that the inclusion of causal relationships impacts predictions of macroscopic QoIs. To assess the impact of correlations and causal relationships between microscopic parameters on macroscopic material properties, we use a moment-independent GSA based on the differential mutual information. Our GSA accounts for the correlated inputs and complex non-Gaussian QoIs. The global sensitivity indices are used to rank the effect of uncertainty in microscopic parameters on macroscopic QoIs, to quantify the impact of causality on the multiscale model's predictions, and to provide physical interpretations of these results for hierarchical nanoporous materials.
△ Less
Submitted 6 January, 2019;
originally announced January 2019.
-
Robust information divergences for model-form uncertainty arising from sparse data in random PDE
Authors:
Eric Joseph Hall,
Markos A. Katsoulakis
Abstract:
We develop a novel application of hybrid information divergences to analyze uncertainty in steady-state subsurface flow problems. These hybrid information divergences are non-intrusive, goal-oriented uncertainty quantification tools that enable robust, data-informed predictions in support of critical decision tasks such as regulatory assessment and risk management. We study the propagation of mode…
▽ More
We develop a novel application of hybrid information divergences to analyze uncertainty in steady-state subsurface flow problems. These hybrid information divergences are non-intrusive, goal-oriented uncertainty quantification tools that enable robust, data-informed predictions in support of critical decision tasks such as regulatory assessment and risk management. We study the propagation of model-form or epistemic uncertainty with numerical experiments that demonstrate uncertainty quantification bounds for (i) parametric sensitivity analysis and (ii) model misspecification due to sparse data. Further, we make connections between the hybrid information divergences and certain concentration inequalities that can be leveraged for efficient computing and account for any available data through suitable statistical quantities.
△ Less
Submitted 16 September, 2018; v1 submitted 11 August, 2017;
originally announced August 2017.
-
Uncertainty quantification for generalized Langevin dynamics
Authors:
Eric Joseph Hall,
Markos A. Katsoulakis,
Luc Rey-Bellet
Abstract:
We present efficient finite difference estimators for goal-oriented sensitivity indices with applications to the generalized Langevin equation (GLE). In particular, we apply these estimators to analyze an extended variable formulation of the GLE where other well known sensitivity analysis techniques such as the likelihood ratio method are not applicable to key parameters of interest. These easily…
▽ More
We present efficient finite difference estimators for goal-oriented sensitivity indices with applications to the generalized Langevin equation (GLE). In particular, we apply these estimators to analyze an extended variable formulation of the GLE where other well known sensitivity analysis techniques such as the likelihood ratio method are not applicable to key parameters of interest. These easily implemented estimators are formed by coupling the nominal and perturbed dynamics appearing in the finite difference through a common driving noise, or common random path. After developing a general framework for variance reduction via coupling, we demonstrate the optimality of the common random path coupling in the sense that it produces a minimal variance surrogate for the difference estimator relative to sampling dynamics driven by independent paths. In order to build intuition for the common random path coupling, we evaluate the efficiency of the proposed estimators for a comprehensive set of examples of interest in particle dynamics. These reduced variance difference estimators are also a useful tool for performing global sensitivity analysis and for investigating non-local perturbations of parameters, such as increasing the number of Prony modes active in an extended variable GLE.
△ Less
Submitted 9 September, 2016;
originally announced September 2016.
-
Computable error estimates for finite element approximations of elliptic partial differential equations with rough stochastic data
Authors:
Eric Joseph Hall,
Håkon Hoel,
Mattias Sandberg,
Anders Szepessy,
Raúl Tempone
Abstract:
We derive computable error estimates for finite element approximations of linear elliptic partial differential equations (PDE) with rough stochastic coefficients. In this setting, the exact solutions contain high frequency content that standard a posteriori error estimates fail to capture. We propose goal-oriented estimates, based on local error indicators, for the pathwise Galerkin and expected q…
▽ More
We derive computable error estimates for finite element approximations of linear elliptic partial differential equations (PDE) with rough stochastic coefficients. In this setting, the exact solutions contain high frequency content that standard a posteriori error estimates fail to capture. We propose goal-oriented estimates, based on local error indicators, for the pathwise Galerkin and expected quadrature errors committed in standard, continuous, piecewise linear finite element approximations. Derived using easily validated assumptions, these novel estimates can be computed at a relatively low cost and have applications to subsurface flow problems in geophysics where the conductivities are assumed to have lognormal distributions with low regularity. Our theory is supported by numerical experiments on test problems in one and two dimensions.
△ Less
Submitted 26 August, 2016; v1 submitted 9 October, 2015;
originally announced October 2015.
-
Higher order spatial approximations for degenerate parabolic stochastic partial differential equations
Authors:
Eric Joseph Hall
Abstract:
We consider an implicit finite difference scheme on uniform grids in time and space for the Cauchy problem for a second order parabolic stochastic partial differential equation where the parabolicity condition is allowed to degenerate. Such equations arise in the nonlinear filtering theory of partially observable diffusion processes. We show that the convergence of the spatial approximation can be…
▽ More
We consider an implicit finite difference scheme on uniform grids in time and space for the Cauchy problem for a second order parabolic stochastic partial differential equation where the parabolicity condition is allowed to degenerate. Such equations arise in the nonlinear filtering theory of partially observable diffusion processes. We show that the convergence of the spatial approximation can be accelerated to an arbitrarily high order, under suitable regularity assumptions, by applying an extrapolation technique.
△ Less
Submitted 4 October, 2012; v1 submitted 3 October, 2012;
originally announced October 2012.
-
Accelerated spatial approximations for time discretized stochastic partial differential equations
Authors:
Eric Joseph Hall
Abstract:
The present article investigates the convergence of a class of space-time discretization schemes for the Cauchy problem for linear parabolic stochastic partial differential equations (SPDEs) defined on the whole space. Sufficient conditions are given for accelerating the convergence of the scheme with respect to the spatial approximation to higher order accuracy by an application of Richardson's m…
▽ More
The present article investigates the convergence of a class of space-time discretization schemes for the Cauchy problem for linear parabolic stochastic partial differential equations (SPDEs) defined on the whole space. Sufficient conditions are given for accelerating the convergence of the scheme with respect to the spatial approximation to higher order accuracy by an application of Richardson's method. This work extends the results of Gyöngy and Krylov [SIAM J. Math. Anal., 42 (2010), pp. 2275--2296] to schemes that discretize in time as well as space.
△ Less
Submitted 27 January, 2012;
originally announced January 2012.
-
Partial choice functions for families of finite sets
Authors:
Eric J. Hall,
Saharon Shelah
Abstract:
Let m>2 be an integer. We show that ZF + "For every integer n, Every countable family of non-empty sets of cardinality at most n has an infinite partial choice function" is not strong enough to prove that every countable set of m-element sets has a choice function. In the case where m=p is prime, to obtain the independence result we make use of a permutation model in which the set of atoms has the…
▽ More
Let m>2 be an integer. We show that ZF + "For every integer n, Every countable family of non-empty sets of cardinality at most n has an infinite partial choice function" is not strong enough to prove that every countable set of m-element sets has a choice function. In the case where m=p is prime, to obtain the independence result we make use of a permutation model in which the set of atoms has the structure of a vector space over the field of p elements. When m is non-prime, a suitable permutation model is built from the models used in the prime cases.
△ Less
Submitted 11 December, 2011; v1 submitted 4 August, 2008;
originally announced August 2008.
