-
"Stochastic Inverse Problems" and Changes-of-Variables
Authors:
Peter W. Marcy,
Rebecca E. Morrison
Abstract:
Over the last decade, a series of applied mathematics papers have explored a type of inverse problem--called by a variety of names including "inverse sensitivity", "pushforward based inference", "consistent Bayesian inference", or "data-consistent inversion"--wherein a solution is a probability density whose pushforward takes a given form. The formulation of such a stochastic inverse problem can b…
▽ More
Over the last decade, a series of applied mathematics papers have explored a type of inverse problem--called by a variety of names including "inverse sensitivity", "pushforward based inference", "consistent Bayesian inference", or "data-consistent inversion"--wherein a solution is a probability density whose pushforward takes a given form. The formulation of such a stochastic inverse problem can be unexpected or confusing to those familiar with traditional Bayesian or otherwise statistical inference. To date, two classes of solutions have been proposed, and these have only been justified through applications of measure theory and its disintegration theorem. In this work we show that, under mild assumptions, the formulation of and solution to all stochastic inverse problems can be more clearly understood using basic probability theory: a stochastic inverse problem is simply a change-of-variables or approximation thereof. For the two existing classes of solutions, we derive the relationship to change(s)-of-variables and illustrate using analytic examples where none had previously existed. Our derivations use neither Bayes' theorem nor the disintegration theorem explicitly. Our final contribution is a careful comparison of changes-of-variables to more traditional statistical inference. While taking stochastic inverse problems at face value for the majority of the paper, our final comparative discussion gives a critique of the framework.
△ Less
Submitted 28 November, 2022;
originally announced November 2022.
-
Diagonal Nonlinear Transformations Preserve Structure in Covariance and Precision Matrices
Authors:
Rebecca E Morrison,
Ricardo Baptista,
Estelle L Basor
Abstract:
For a multivariate normal distribution, the sparsity of the covariance and precision matrices encodes complete information about independence and conditional independence properties. For general distributions, the covariance and precision matrices reveal correlations and so-called partial correlations between variables, but these do not, in general, have any correspondence with respect to independ…
▽ More
For a multivariate normal distribution, the sparsity of the covariance and precision matrices encodes complete information about independence and conditional independence properties. For general distributions, the covariance and precision matrices reveal correlations and so-called partial correlations between variables, but these do not, in general, have any correspondence with respect to independence properties. In this paper, we prove that, for a certain class of non-Gaussian distributions, these correspondences still hold, exactly for the covariance and approximately for the precision. The distributions -- sometimes referred to as "nonparanormal" -- are given by diagonal transformations of multivariate normal random variables. We provide several analytic and numerical examples illustrating these results.
△ Less
Submitted 20 September, 2021; v1 submitted 8 July, 2021;
originally announced July 2021.
-
Learning non-Gaussian graphical models via Hessian scores and triangular transport
Authors:
Ricardo Baptista,
Youssef Marzouk,
Rebecca E. Morrison,
Olivier Zahm
Abstract:
Undirected probabilistic graphical models represent the conditional dependencies, or Markov properties, of a collection of random variables. Knowing the sparsity of such a graphical model is valuable for modeling multivariate distributions and for efficiently performing inference. While the problem of learning graph structure from data has been studied extensively for certain parametric families o…
▽ More
Undirected probabilistic graphical models represent the conditional dependencies, or Markov properties, of a collection of random variables. Knowing the sparsity of such a graphical model is valuable for modeling multivariate distributions and for efficiently performing inference. While the problem of learning graph structure from data has been studied extensively for certain parametric families of distributions, most existing methods fail to consistently recover the graph structure for non-Gaussian data. Here we propose an algorithm for learning the Markov structure of continuous and non-Gaussian distributions. To characterize conditional independence, we introduce a score based on integrated Hessian information from the joint log-density, and we prove that this score upper bounds the conditional mutual information for a general class of distributions. To compute the score, our algorithm SING estimates the density using a deterministic coupling, induced by a triangular transport map, and iteratively exploits sparse structure in the map to reveal sparsity in the graph. For certain non-Gaussian datasets, we show that our algorithm recovers the graph structure even with a biased approximation to the density. Among other examples, we apply SING to learn the dependencies between the states of a chaotic dynamical system with local interactions.
△ Less
Submitted 25 February, 2023; v1 submitted 8 January, 2021;
originally announced January 2021.
-
Embedded model discrepancy: A case study of Zika modeling
Authors:
Rebecca E. Morrison,
Americo Cunha Jr
Abstract:
Mathematical models of epidemiological systems enable investigation of and predictions about potential disease outbreaks. However, commonly used models are often highly simplified representations of incredibly complex systems. Because of these simplifications, the model output, of say new cases of a disease over time, or when an epidemic will occur, may be inconsistent with available data. In this…
▽ More
Mathematical models of epidemiological systems enable investigation of and predictions about potential disease outbreaks. However, commonly used models are often highly simplified representations of incredibly complex systems. Because of these simplifications, the model output, of say new cases of a disease over time, or when an epidemic will occur, may be inconsistent with available data. In this case, we must improve the model, especially if we plan to make decisions based on it that could affect human health and safety, but direct improvements are often beyond our reach. In this work, we explore this problem through a case study of the Zika outbreak in Brazil in 2016. We propose an embedded discrepancy operator---a modification to the model equations that requires modest information about the system and is calibrated by all relevant data. We show that the new enriched model demonstrates greatly increased consistency with real data. Moreover, the method is general enough to easily apply to many other mathematical models in epidemiology.
△ Less
Submitted 13 April, 2020;
originally announced April 2020.
-
Embedded discrepancy operators in reduced models of interacting species
Authors:
Rebecca E Morrison
Abstract:
In many applications of interacting systems, we are only interested in the dynamic behavior of a subset of all possible active species. For example, this is true in combustion models (many transient chemical species are not of interest in a given reaction) and in epidemiological models (only certain critical populations are truly consequential). Thus it is common to use greatly reduced models, in…
▽ More
In many applications of interacting systems, we are only interested in the dynamic behavior of a subset of all possible active species. For example, this is true in combustion models (many transient chemical species are not of interest in a given reaction) and in epidemiological models (only certain critical populations are truly consequential). Thus it is common to use greatly reduced models, in which only the interactions among the species of interest are retained. However, reduction introduces a model error, or discrepancy, which typically is not well characterized. In this work, we explore the use of an embedded and statistically calibrated discrepancy operator to represent model error. The operator is embedded within the differential equations of the model, which allows the action of the operator to be interpretable. Moreover, it is constrained by available physical information, and calibrated over many scenarios. These qualities of the discrepancy model---interpretability, physical-consistency, and robustness to different scenarios---are intended to support reliable predictions under extrapolative conditions.
△ Less
Submitted 17 October, 2019;
originally announced October 2019.
-
Exact Reduction of the Generalized Lotka-Volterra Equations via Integral and Algebraic Substitutions
Authors:
Rebecca E. Morrison
Abstract:
Systems of interacting species, such as biological environments or chemical reactions, are often described mathematically by sets of coupled ordinary differential equations. While a large number $β$ of species may be involved in the coupled dynamics, often only $α< β$ species are of interest or of consequence. In this paper, I explore how to build reduced models that include only those given $α$ s…
▽ More
Systems of interacting species, such as biological environments or chemical reactions, are often described mathematically by sets of coupled ordinary differential equations. While a large number $β$ of species may be involved in the coupled dynamics, often only $α< β$ species are of interest or of consequence. In this paper, I explore how to build reduced models that include only those given $α$ species, but still recreate the dynamics of the original $β$-species model. Under some conditions detailed here, this reduction can be completed exactly, such that the information in the reduced model is exactly the same as the original one, but over fewer equations. Moreover, this reduction process suggests a promising type of approximate model -- no longer exact, but computationally quite simple
△ Less
Submitted 13 January, 2024; v1 submitted 30 September, 2019;
originally announced September 2019.
-
Beyond normality: Learning sparse probabilistic graphical models in the non-Gaussian setting
Authors:
Rebecca E. Morrison,
Ricardo Baptista,
Youssef Marzouk
Abstract:
We present an algorithm to identify sparse dependence structure in continuous and non-Gaussian probability distributions, given a corresponding set of data. The conditional independence structure of an arbitrary distribution can be represented as an undirected graph (or Markov random field), but most algorithms for learning this structure are restricted to the discrete or Gaussian cases. Our new a…
▽ More
We present an algorithm to identify sparse dependence structure in continuous and non-Gaussian probability distributions, given a corresponding set of data. The conditional independence structure of an arbitrary distribution can be represented as an undirected graph (or Markov random field), but most algorithms for learning this structure are restricted to the discrete or Gaussian cases. Our new approach allows for more realistic and accurate descriptions of the distribution in question, and in turn better estimates of its sparse Markov structure. Sparsity in the graph is of interest as it can accelerate inference, improve sampling methods, and reveal important dependencies between variables. The algorithm relies on exploiting the connection between the sparsity of the graph and the sparsity of transport maps, which deterministically couple one probability measure to another.
△ Less
Submitted 6 November, 2017; v1 submitted 2 November, 2017;
originally announced November 2017.
-
Representing model inadequacy: A stochastic operator approach
Authors:
Rebecca E Morrison,
Todd A Oliver,
Robert D Moser
Abstract:
Mathematical models of physical systems are subject to many uncertainties such as measurement errors and uncertain initial and boundary conditions. After accounting for these uncertainties, it is often revealed that discrepancies between the model output and the observations remain; if so, the model is said to be inadequate. In practice, the inadequate model may be the best that is available or tr…
▽ More
Mathematical models of physical systems are subject to many uncertainties such as measurement errors and uncertain initial and boundary conditions. After accounting for these uncertainties, it is often revealed that discrepancies between the model output and the observations remain; if so, the model is said to be inadequate. In practice, the inadequate model may be the best that is available or tractable, and so despite its inadequacy the model may be used to make predictions of unobserved quantities. In this case, a representation of the inadequacy is necessary, so the impact of the observed discrepancy can be determined. We investigate this problem in the context of chemical kinetics and propose a new technique to account for model inadequacy that is both probabilistic and physically meaningful. A stochastic inadequacy operator $\mathcal{S}$ is introduced which is embedded in the ODEs describing the evolution of chemical species concentrations and which respects certain physical constraints such as conservation laws. The parameters of $\mathcal{S}$ are governed by probability distributions, which in turn are characterized by a set of hyperparameters. The model parameters and hyperparameters are calibrated using high-dimensional hierarchical Bayesian inference. We apply the method to a typical problem in chemical kinetics---the reaction mechanism of hydrogen combustion.
△ Less
Submitted 22 May, 2018; v1 submitted 6 April, 2016;
originally announced April 2016.
-
Combinatorial Games with a Pass: A dynamical systems approach
Authors:
Rebecca E. Morrison,
Eric J. Friedman,
Adam S. Landsberg
Abstract:
By treating combinatorial games as dynamical systems, we are able to address a longstanding open question in combinatorial game theory, namely, how the introduction of a "pass" move into a game affects its behavior. We consider two well known combinatorial games, 3-pile Nim and 3-row Chomp. In the case of Nim, we observe that the introduction of the pass dramatically alters the game's underlying s…
▽ More
By treating combinatorial games as dynamical systems, we are able to address a longstanding open question in combinatorial game theory, namely, how the introduction of a "pass" move into a game affects its behavior. We consider two well known combinatorial games, 3-pile Nim and 3-row Chomp. In the case of Nim, we observe that the introduction of the pass dramatically alters the game's underlying structure, rendering it considerably more complex, while for Chomp, the pass move is found to have relatively minimal impact. We show how these results can be understood by recasting these games as dynamical systems describable by dynamical recursion relations. From these recursion relations we are able to identify underlying structural connections between these "games with passes" and a recently introduced class of "generic (perturbed) games." This connection, together with a (non-rigorous) numerical stability analysis, allows one to understand and predict the effect of a pass on a game.
△ Less
Submitted 14 April, 2012;
originally announced April 2012.
