Showing 1–28 of 28 results for author: Stein, M L

Modeling spatial asymmetries in teleconnected extreme temperatures

Authors: Mitchell L. Krock, Julie Bessac, Michael L. Stein

Abstract: Combining strengths from deep learning and extreme value theory can help describe complex relationships between variables where extreme events have significant impacts (e.g., environmental or financial applications). Neural networks learn complicated nonlinear relationships from large datasets under limited parametric assumptions. By definition, the number of occurrences of extreme events is small… ▽ More Combining strengths from deep learning and extreme value theory can help describe complex relationships between variables where extreme events have significant impacts (e.g., environmental or financial applications). Neural networks learn complicated nonlinear relationships from large datasets under limited parametric assumptions. By definition, the number of occurrences of extreme events is small, which limits the ability of the data-hungry, nonparametric neural network to describe rare events. Inspired by recent extreme cold winter weather events in North America caused by atmospheric blocking, we examine several probabilistic generative models for the entire multivariate probability distribution of daily boreal winter surface air temperature. We propose metrics to measure spatial asymmetries, such as long-range anticorrelated patterns that commonly appear in temperature fields during blocking events. Compared to vine copulas, the statistical standard for multivariate copula modeling, deep learning methods show improved ability to reproduce complicated asymmetries in the spatial distribution of ERA5 temperature reanalysis, including the spatial extent of in-sample extreme events. △ Less

Submitted 4 October, 2023; originally announced October 2023.

Teleconnected warm and cold extremes of North American wintertime temperatures

Authors: Mitchell L. Krock, Adam H. Monahan, Michael L. Stein

Abstract: Current models for spatial extremes are concerned with the joint upper (or lower) tail of the distribution at two or more locations. Such models cannot account for teleconnection patterns of two-meter surface air temperature ($T_{2m}$) in North America, where very low temperatures in the contiguous Unites States (CONUS) may coincide with very high temperatures in Alaska in the wintertime. This dep… ▽ More Current models for spatial extremes are concerned with the joint upper (or lower) tail of the distribution at two or more locations. Such models cannot account for teleconnection patterns of two-meter surface air temperature ($T_{2m}$) in North America, where very low temperatures in the contiguous Unites States (CONUS) may coincide with very high temperatures in Alaska in the wintertime. This dependence between warm and cold extremes motivates the need for a model with opposite-tail dependence in spatial extremes. This work develops a statistical modeling framework which has flexible behavior in all four pairings of high and low extremes at pairs of locations. In particular, we use a mixture of rotations of common Archimedean copulas to capture various combinations of four-corner tail dependence. We study teleconnected $T_{2m}$ extremes using ERA5 reanalysis of daily average two-meter temperature during the boreal winter. The estimated mixture model quantifies the strength of opposite-tail dependence between warm temperatures in Alaska and cold temperatures in the midlatitudes of North America, as well as the reverse pattern. These dependence patterns are shown to correspond to blocked and zonal patterns of mid-tropospheric flow. This analysis extends the classical notion of correlation-based teleconnections to considering dependence in higher quantiles. △ Less

Submitted 25 August, 2022; originally announced August 2022.

A Scalable Method to Exploit Screening in Gaussian Process Models with Noise

Authors: Christopher J. Geoga, Michael L. Stein

Abstract: A common approach to approximating Gaussian log-likelihoods at scale exploits the fact that precision matrices can be well-approximated by sparse matrices in some circumstances. This strategy is motivated by the \emph{screening effect}, which refers to the phenomenon in which the linear prediction of a process $Z$ at a point $\mathbf{x}_0$ depends primarily on measurements nearest to… ▽ More A common approach to approximating Gaussian log-likelihoods at scale exploits the fact that precision matrices can be well-approximated by sparse matrices in some circumstances. This strategy is motivated by the \emph{screening effect}, which refers to the phenomenon in which the linear prediction of a process $Z$ at a point $\mathbf{x}_0$ depends primarily on measurements nearest to $\mathbf{x}_0$. But simple perturbations, such as i.i.d. measurement noise, can significantly reduce the degree to which this exploitable phenomenon occurs. While strategies to cope with this issue already exist and are certainly improvements over ignoring the problem, in this work we present a new one based on the EM algorithm that offers several advantages. While in this work we focus on the application to Vecchia's approximation (1988), a particularly popular and powerful framework in which we can demonstrate true second-order optimization of M steps, the method can also be applied using entirely matrix-vector products, making it applicable to a very wide class of precision matrix-based approximation methods. △ Less

Submitted 16 February, 2023; v1 submitted 14 August, 2022; originally announced August 2022.

Scalable Computations for Nonstationary Gaussian Processes

Authors: Paul G. Beckman, Christopher J. Geoga, Michael L. Stein, Mihai Anitescu

Abstract: Nonstationary Gaussian process models can capture complex spatially varying dependence structures in spatial datasets. However, the large number of observations in modern datasets makes fitting such models computationally intractable with conventional dense linear algebra. In addition, derivative-free or even first-order optimization methods can be slow to converge when estimating many spatially v… ▽ More Nonstationary Gaussian process models can capture complex spatially varying dependence structures in spatial datasets. However, the large number of observations in modern datasets makes fitting such models computationally intractable with conventional dense linear algebra. In addition, derivative-free or even first-order optimization methods can be slow to converge when estimating many spatially varying parameters. We present here a computational framework that couples an algebraic block-diagonal plus low-rank covariance matrix approximation with stochastic trace estimation to facilitate the efficient use of second-order solvers for maximum likelihood estimation of Gaussian process models with many parameters. We demonstrate the effectiveness of these methods by simultaneously fitting 192 parameters in the popular nonstationary model of Paciorek and Schervish using 107,600 sea surface temperature anomaly measurements. △ Less

Submitted 10 June, 2022; originally announced June 2022.

Vecchia Likelihood Approximation for Accurate and Fast Inference in Intractable Spatial Extremes Models

Authors: Raphaël Huser, Michael L. Stein, Peng Zhong

Abstract: Max-stable processes are the most popular models for high-impact spatial extreme events, as they arise as the only possible limits of spatially-indexed block maxima. However, likelihood inference for such models suffers severely from the curse of dimensionality, since the likelihood function involves a combinatorially exploding number of terms. In this paper, we propose using the Vecchia approxima… ▽ More Max-stable processes are the most popular models for high-impact spatial extreme events, as they arise as the only possible limits of spatially-indexed block maxima. However, likelihood inference for such models suffers severely from the curse of dimensionality, since the likelihood function involves a combinatorially exploding number of terms. In this paper, we propose using the Vecchia approximation, which conveniently decomposes the full joint density into a linear number of low-dimensional conditional density terms based on well-chosen conditioning sets designed to improve and accelerate inference in high dimensions. Theoretical asymptotic relative efficiencies in the Gaussian setting and simulation experiments in the max-stable setting show significant efficiency gains and computational savings using the Vecchia likelihood approximation method compared to traditional composite likelihoods. Our application to extreme sea surface temperature data at more than a thousand sites across the entire Red Sea further demonstrates the superiority of the Vecchia likelihood approximation for fitting complex models with intractable likelihoods, delivering significantly better results than traditional composite likelihoods, and accurately capturing the extremal dependence structure at lower computational cost. △ Less

Submitted 10 March, 2022; originally announced March 2022.

Fitting Matérn Smoothness Parameters Using Automatic Differentiation

Authors: Christopher J. Geoga, Oana Marin, Michel Schanen, Michael L. Stein

Abstract: The Matérn covariance function is ubiquitous in the application of Gaussian processes to spatial statistics and beyond. Perhaps the most important reason for this is that the smoothness parameter $ν$ gives complete control over the mean-square differentiability of the process, which has significant implications for the behavior of estimated quantities such as interpolants and forecasts. Unfortunat… ▽ More The Matérn covariance function is ubiquitous in the application of Gaussian processes to spatial statistics and beyond. Perhaps the most important reason for this is that the smoothness parameter $ν$ gives complete control over the mean-square differentiability of the process, which has significant implications for the behavior of estimated quantities such as interpolants and forecasts. Unfortunately, derivatives of the Matérn covariance function with respect to $ν$ require derivatives of the modified second-kind Bessel function $\mathcal{K}_ν$ with respect to $ν$. While closed form expressions of these derivatives do exist, they are prohibitively difficult and expensive to compute. For this reason, many software packages require fixing $ν$ as opposed to estimating it, and all existing software packages that attempt to offer the functionality of estimating $ν$ use finite difference estimates for $\partial_ν\mathcal{K}_ν$. In this work, we introduce a new implementation of $\mathcal{K}_ν$ that has been designed to provide derivatives via automatic differentiation (AD), and whose resulting derivatives are significantly faster and more accurate than those computed using finite differences. We provide comprehensive testing for both speed and accuracy and show that our AD solution can be used to build accurate Hessian matrices for second-order maximum likelihood estimation in settings where Hessians built with finite difference approximations completely fail. △ Less

Submitted 9 May, 2022; v1 submitted 31 December, 2021; originally announced January 2022.

Nonstationary seasonal model for daily mean temperature distribution bridging bulk and tails

Authors: Mitchell Krock, Julie Bessac, Michael L. Stein, Adam H. Monahan

Abstract: In traditional extreme value analysis, the bulk of the data is ignored, and only the tails of the distribution are used for inference. Extreme observations are specified as values that exceed a threshold or as maximum values over distinct blocks of time, and subsequent estimation procedures are motivated by asymptotic theory for extremes of random processes. For environmental data, nonstationary b… ▽ More In traditional extreme value analysis, the bulk of the data is ignored, and only the tails of the distribution are used for inference. Extreme observations are specified as values that exceed a threshold or as maximum values over distinct blocks of time, and subsequent estimation procedures are motivated by asymptotic theory for extremes of random processes. For environmental data, nonstationary behavior in the bulk of the distribution, such as seasonality or climate change, will also be observed in the tails. To accurately model such nonstationarity, it seems natural to use the entire dataset rather than just the most extreme values. It is also common to observe different types of nonstationarity in each tail of a distribution. Most work on extremes only focuses on one tail of a distribution, but for temperature, both tails are of interest. This paper builds on a recently proposed parametric model for the entire probability distribution that has flexible behavior in both tails. We apply an extension of this model to historical records of daily mean temperature at several locations across the United States with different climates and local conditions. We highlight the ability of the method to quantify changes in the bulk and tails across the year over the past decades and under different geographic and climatic conditions. The proposed model shows good performance when compared to several benchmark models that are typically used in extreme value analysis of temperature. △ Less

Submitted 19 October, 2021; originally announced October 2021.

Neural Networks for Parameter Estimation in Intractable Models

Authors: Amanda Lenzi, Julie Bessac, Johann Rudi, Michael L. Stein

Abstract: We propose to use deep learning to estimate parameters in statistical models when standard likelihood estimation methods are computationally infeasible. We show how to estimate parameters from max-stable processes, where inference is exceptionally challenging even with small datasets but simulation is straightforward. We use data from model simulations as input and train deep neural networks to le… ▽ More We propose to use deep learning to estimate parameters in statistical models when standard likelihood estimation methods are computationally infeasible. We show how to estimate parameters from max-stable processes, where inference is exceptionally challenging even with small datasets but simulation is straightforward. We use data from model simulations as input and train deep neural networks to learn statistical parameters. Our neural-network-based method provides a competitive alternative to current approaches, as demonstrated by considerable accuracy and computational time improvements. It serves as a proof of concept for deep learning in statistical parameter estimation and can be extended to other estimation problems. △ Less

Submitted 29 July, 2021; originally announced July 2021.

Flexible nonstationary spatio-temporal modeling of high-frequency monitoring data

Authors: Christopher J. Geoga, Mihai Anitescu, Michael L. Stein

Abstract: Many physical datasets are generated by collections of instruments that make measurements at regular time intervals. For such regular monitoring data, we extend the framework of half-spectral covariance functions to the case of nonstationarity in space and time and demonstrate that this method provides a natural and tractable way to incorporate complex behaviors into a covariance model. Further, w… ▽ More Many physical datasets are generated by collections of instruments that make measurements at regular time intervals. For such regular monitoring data, we extend the framework of half-spectral covariance functions to the case of nonstationarity in space and time and demonstrate that this method provides a natural and tractable way to incorporate complex behaviors into a covariance model. Further, we use this method with fully time-domain computations to obtain bona fide maximum likelihood estimators---as opposed to using Whittle-type likelihood approximations, for example---that can still be computed efficiently. We apply this method to very high-frequency Doppler LIDAR vertical wind velocity measurements, demonstrating that the model can expressively capture the extreme nonstationarity of dynamics above and below the atmospheric boundary layer and, more importantly, the interaction of the process dynamics across it. △ Less

Submitted 22 July, 2020; originally announced July 2020.

Scalable Gaussian Process Computations Using Hierarchical Matrices

Authors: Christopher J. Geoga, Mihai Anitescu, Michael L. Stein

Abstract: We present a kernel-independent method that applies hierarchical matrices to the problem of maximum likelihood estimation for Gaussian processes. The proposed approximation provides natural and scalable stochastic estimators for its gradient and Hessian, as well as the expected Fisher information matrix, that are computable in quasilinear $O(n \log^2 n)$ complexity for a large range of models. To… ▽ More We present a kernel-independent method that applies hierarchical matrices to the problem of maximum likelihood estimation for Gaussian processes. The proposed approximation provides natural and scalable stochastic estimators for its gradient and Hessian, as well as the expected Fisher information matrix, that are computable in quasilinear $O(n \log^2 n)$ complexity for a large range of models. To accomplish this, we (i) choose a specific hierarchical approximation for covariance matrices that enables the computation of their exact derivatives and (ii) use a stabilized form of the Hutchinson stochastic trace estimator. Since both the observed and expected information matrices can be computed in quasilinear complexity, covariance matrices for MLEs can also be estimated efficiently. After discussing the associated mathematics, we demonstrate the scalability of the method, discuss details of its implementation, and validate that the resulting MLEs and confidence intervals based on the inverse Fisher information matrix faithfully approach those obtained by the exact likelihood. △ Less

Submitted 22 March, 2019; v1 submitted 9 August, 2018; originally announced August 2018.

Linear-Cost Covariance Functions for Gaussian Random Fields

Authors: Jie Chen, Michael L. Stein

Abstract: Gaussian random fields (GRF) are a fundamental stochastic model for spatiotemporal data analysis. An essential ingredient of GRF is the covariance function that characterizes the joint Gaussian distribution of the field. Commonly used covariance functions give rise to fully dense and unstructured covariance matrices, for which required calculations are notoriously expensive to carry out for large… ▽ More Gaussian random fields (GRF) are a fundamental stochastic model for spatiotemporal data analysis. An essential ingredient of GRF is the covariance function that characterizes the joint Gaussian distribution of the field. Commonly used covariance functions give rise to fully dense and unstructured covariance matrices, for which required calculations are notoriously expensive to carry out for large data. In this work, we propose a construction of covariance functions that result in matrices with a hierarchical structure. Empowered by matrix algorithms that scale linearly with the matrix dimension, the hierarchical structure is proved to be efficient for a variety of random field computations, including sampling, kriging, and likelihood evaluation. Specifically, with $n$ scattered sites, sampling and likelihood evaluation has an $O(n)$ cost and kriging has an $O(\log n)$ cost after preprocessing, particularly favorable for the kriging of an extremely large number of sites (e.g., predicting on more sites than observed). We demonstrate comprehensive numerical experiments to show the use of the constructed covariance functions and their appealing computation time. Numerical examples on a laptop include simulated data of size up to one million, as well as a climate data product with over two million observations. △ Less

Submitted 7 November, 2020; v1 submitted 15 November, 2017; originally announced November 2017.

Comments: Code is available at https://github.com/jiechenjiechen/RLCM

Locally stationary spatio-temporal interpolation of Argo profiling float data

Authors: Mikael Kuusela, Michael L. Stein

Abstract: Argo floats measure seawater temperature and salinity in the upper 2,000 m of the global ocean. Statistical analysis of the resulting spatio-temporal dataset is challenging due to its nonstationary structure and large size. We propose mapping these data using locally stationary Gaussian process regression where covariance parameter estimation and spatio-temporal prediction are carried out in a mov… ▽ More Argo floats measure seawater temperature and salinity in the upper 2,000 m of the global ocean. Statistical analysis of the resulting spatio-temporal dataset is challenging due to its nonstationary structure and large size. We propose mapping these data using locally stationary Gaussian process regression where covariance parameter estimation and spatio-temporal prediction are carried out in a moving-window fashion. This yields computationally tractable nonstationary anomaly fields without the need to explicitly model the nonstationary covariance structure. We also investigate Student-$t$ distributed fine-scale variation as a means to account for non-Gaussian heavy tails in ocean temperature data. Cross-validation studies comparing the proposed approach with the existing state-of-the-art demonstrate clear improvements in point predictions and show that accounting for the nonstationarity and non-Gaussianity is crucial for obtaining well-calibrated uncertainties. This approach also provides data-driven local estimates of the spatial and temporal dependence scales for the global ocean which are of scientific interest in their own right. △ Less

Submitted 28 December, 2018; v1 submitted 1 November, 2017; originally announced November 2017.

Comments: 28 pages, 8 figures, changes to the presentation throughout, some material moved to the supplement, author version of the published paper

Journal ref: Proceedings of the Royal Society A 474: 20180400 (2018)

Estimating trends in the global mean temperature record

Authors: Andrew Poppick, Elisabeth J. Moyer, Michael L. Stein

Abstract: Given uncertainties in physical theory and numerical climate simulations, the historical temperature record is often used as a source of empirical information about climate change. Many historical trend analyses appear to deemphasize physical and statistical assumptions: examples include regression models that treat time rather than radiative forcing as the relevant covariate and time series metho… ▽ More Given uncertainties in physical theory and numerical climate simulations, the historical temperature record is often used as a source of empirical information about climate change. Many historical trend analyses appear to deemphasize physical and statistical assumptions: examples include regression models that treat time rather than radiative forcing as the relevant covariate and time series methods that account for internal variability nonparametrically. However, given a limited record and the presence of internal variability, estimating radiatively forced historical temperature trends necessarily requires assumptions. Ostensibly empirical methods can involve an inherent conflict in assumptions: they require data records that are short enough for naive trend models to apply but long enough for internal variability to be accounted for. In the context of global mean temperatures, methods that deemphasize assumptions can therefore produce misleading inferences, because the twentieth century trend is complex and the scale of correlation is long relative to the data length. We illustrate how a simple but physically motivated trend model can provide better-fitting and more broadly applicable trend estimates and can address a wider array of questions. The model allows one to distinguish, within a single framework, between uncertainties in the shorter-term versus longer-term response to radiative forcing, with implications not only on historical trends but also on uncertainties in future projections. We also investigate the consequence on inferred uncertainties of the choice of a statistical description of internal variability. While nonparametric methods may seem to avoid making explicit assumptions, we demonstrate how even misspecified parametric methods, if attuned to important characteristics of internal variability, can result in more accurate statements about trend uncertainty. △ Less

Submitted 14 May, 2017; v1 submitted 13 July, 2016; originally announced July 2016.

Comments: 38 pages, 14 figures

A stochastic space-time model for intermittent precipitation occurrences

Authors: Ying Sun, Michael L. Stein

Abstract: Modeling a precipitation field is challenging due to its intermittent and highly scale-dependent nature. Motivated by the features of high-frequency precipitation data from a network of rain gauges, we propose a threshold space-time $t$ random field (tRF) model for 15-minute precipitation occurrences. This model is constructed through a space-time Gaussian random field (GRF) with random scaling va… ▽ More Modeling a precipitation field is challenging due to its intermittent and highly scale-dependent nature. Motivated by the features of high-frequency precipitation data from a network of rain gauges, we propose a threshold space-time $t$ random field (tRF) model for 15-minute precipitation occurrences. This model is constructed through a space-time Gaussian random field (GRF) with random scaling varying along time or space and time. It can be viewed as a generalization of the purely spatial tRF, and has a hierarchical representation that allows for Bayesian interpretation. Developing appropriate tools for evaluating precipitation models is a crucial part of the model-building process, and we focus on evaluating whether models can produce the observed conditional dry and rain probabilities given that some set of neighboring sites all have rain or all have no rain. These conditional probabilities show that the proposed space-time model has noticeable improvements in some characteristics of joint rainfall occurrences for the data we have considered. △ Less

Submitted 9 February, 2016; originally announced February 2016.

Comments: Published at http://dx.doi.org/10.1214/15-AOAS875 in the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS875

Journal ref: Annals of Applied Statistics 2015, Vol. 9, No. 4, 2110-2132

Changes in Spatio-temporal Precipitation Patterns in Changing Climate Conditions

Authors: Won Chang, Michael L. Stein, Jiali Wang, V. Rao Kotamarthi, Elisabeth J. Moyer

Abstract: Climate models robustly imply that some significant change in precipitation patterns will occur. Models consistently project that the intensity of individual precipitation events increases by approximately 6-7%/K, following the increase in atmospheric water content, but that total precipitation increases by a lesser amount (1-2 %/K in the global average in transient runs). Some other aspect of pre… ▽ More Climate models robustly imply that some significant change in precipitation patterns will occur. Models consistently project that the intensity of individual precipitation events increases by approximately 6-7%/K, following the increase in atmospheric water content, but that total precipitation increases by a lesser amount (1-2 %/K in the global average in transient runs). Some other aspect of precipitation events must then change to compensate for this difference. We develop here a new methodology for identifying individual rainstorms and studying their physical characteristics - including starting location, intensity, spatial extent, duration, and trajectory - that allows identifying that compensating mechanism. We apply this technique to precipitation over the contiguous U.S. from both radar-based data products and high-resolution model runs simulating 80 years of business-as-usual warming. In model studies, we find that the dominant compensating mechanism is a reduction of storm size. In summer, rainstorms become more intense but smaller, in winter, rainstorm shrinkage still dominates, but storms also become less numerous and shorter duration. These results imply that flood impacts from climate change will be less severe than would be expected from changes in precipitation intensity alone. We show also that projected changes are smaller than model-observation biases, implying that the best means of incorporating them into impact assessments is via "data-driven simulations" that apply model-projected changes to observational data. We therefore develop a simulation algorithm that statistically describes model changes in precipitation characteristics and adjusts data accordingly, and show that, especially for summertime precipitation, it outperforms simulation approaches that do not include spatial information. △ Less

Submitted 24 May, 2016; v1 submitted 6 January, 2016; originally announced January 2016.

Comments: This work has been submitted for publication. Copyright in this work may be transferred without further notice, and this version may no longer be accessible

Estimating changes in temperature extremes from millennial scale climate simulations using generalized extreme value (GEV) distributions

Authors: Whitney K. Huang, Michael L. Stein, David J. McInerney, Shanshan Sun, Elisabeth J. Moyer

Abstract: Changes in extreme weather may produce some of the largest societal impacts of anthropogenic climate change. However, it is intrinsically difficult to estimate changes in extreme events from the short observational record. In this work we use millennial runs from the CCSM3 in equilibrated pre-industrial and possible future conditions to examine both how extremes change in this model and how well t… ▽ More Changes in extreme weather may produce some of the largest societal impacts of anthropogenic climate change. However, it is intrinsically difficult to estimate changes in extreme events from the short observational record. In this work we use millennial runs from the CCSM3 in equilibrated pre-industrial and possible future conditions to examine both how extremes change in this model and how well these changes can be estimated as a function of run length. We estimate changes to distributions of future temperature extremes (annual minima and annual maxima) in the contiguous United States by fitting generalized extreme value (GEV) distributions. Using 1000-year pre-industrial and future time series, we show that the magnitude of warm extremes largely shifts in accordance with mean shifts in summertime temperatures. In contrast, cold extremes warm more than mean shifts in wintertime temperatures, but changes in GEV location parameters are largely explainable by mean shifts combined with reduced wintertime temperature variability. In addition, changes in the spread and shape of the GEV distributions of cold extremes at inland locations can lead to discernible changes in tail behavior. We then examine uncertainties that result from using shorter model runs. In principle, the GEV distribution provides theoretical justification to predict infrequent events using time series shorter than the recurrence frequency of those events. To investigate how well this approach works in practice, we estimate 20-, 50-, and 100-year extreme events using segments of varying lengths. We find that even using GEV distributions, time series that are of comparable or shorter length than the return period of interest can lead to very poor estimates. These results suggest caution when attempting to use short observational time series or model runs to infer infrequent extremes. △ Less

Submitted 14 June, 2016; v1 submitted 29 December, 2015; originally announced December 2015.

Comments: 33 pages, 22 figures, 1 table

Temperatures in transient climates: improved methods for simulations with evolving temporal covariances

Authors: Andrew Poppick, David J. McInerney, Elisabeth J. Moyer, Michael L. Stein

Abstract: Future climate change impacts depend on temperatures not only through changes in their means but also through changes in their variability. General circulation models (GCMs) predict changes in both means and variability; however, GCM output should not be used directly as simulations for impacts assessments because GCMs do not fully reproduce present-day temperature distributions. This paper addres… ▽ More Future climate change impacts depend on temperatures not only through changes in their means but also through changes in their variability. General circulation models (GCMs) predict changes in both means and variability; however, GCM output should not be used directly as simulations for impacts assessments because GCMs do not fully reproduce present-day temperature distributions. This paper addresses an ensuing need for simulations of future temperatures that combine both the observational record and GCM projections of changes in means and temporal covariances. Our perspective is that such simulations should be based on transforming observations to account for GCM projected changes, in contrast to methods that transform GCM output to account for discrepancies with observations. Our methodology is designed for simulating transient (non-stationary) climates, which are evolving in response to changes in CO$_2$ concentrations (as is the Earth at present). This work builds on previously described methods for simulating equilibrium (stationary) climates. Since the proposed simulation relies on GCM projected changes in covariance, we describe a statistical model for the evolution of temporal covariances in a GCM under future forcing scenarios, and apply this model to an ensemble of runs from one GCM, CCSM3. We find that, at least in CCSM3, changes in the local covariance structure can be explained as a function of the regional mean change in temperature and the rate of change of warming. This feature means that the statistical model can be used to emulate the evolving covariance structure of GCM temperatures under scenarios for which the GCM has not been run. When combined with an emulator for mean temperature, our methodology can simulate evolving temperatures under such scenarios, in a way that accounts for projections of changes while still retaining fidelity with the observational record. △ Less

Submitted 2 November, 2015; v1 submitted 2 July, 2015; originally announced July 2015.

Comments: 35 pages, 15 figures

Half-Spectral Space-Time Covariance Models

Authors: Michael T. Horrell, Michael L. Stein

Abstract: We develop two new classes of space-time Gaussian process models by specifying covariance functions using what we call a half-spectral representation. The half-spectral representation of a covariance function, $K$, is a special case of standard spectral representations. In addition to the introduction of two new model classes, we also develop desirable theoretical properties of certain half-spectr… ▽ More We develop two new classes of space-time Gaussian process models by specifying covariance functions using what we call a half-spectral representation. The half-spectral representation of a covariance function, $K$, is a special case of standard spectral representations. In addition to the introduction of two new model classes, we also develop desirable theoretical properties of certain half-spectral forms. In particular, for a half-spectral model, $K$, we determine spatial and temporal mean-square differentiability properties of a Gaussian process governed by $K$, and we determine whether or not the spectral density of $K$ meets a regularity condition motivated by a screening effect analysis. We fit models we develop in this paper to a wind power dataset, and we show our models fit these data better than other separable and non-separable space-time models. △ Less

Submitted 5 May, 2015; originally announced May 2015.

Comments: 24 pages, 9 figures

Bayesian and Maximum Likelihood Estimation for Gaussian Processes on an Incomplete Lattice

Authors: Jonathan R. Stroud, Michael L. Stein, Shaun Lysen

Abstract: This paper proposes a new approach for Bayesian and maximum likelihood parameter estimation for stationary Gaussian processes observed on a large lattice with missing values. We propose an MCMC approach for Bayesian inference, and a Monte Carlo EM algorithm for maximum likelihood inference. Our approach uses data augmentation and circulant embedding of the covariance matrix, and provides exact inf… ▽ More This paper proposes a new approach for Bayesian and maximum likelihood parameter estimation for stationary Gaussian processes observed on a large lattice with missing values. We propose an MCMC approach for Bayesian inference, and a Monte Carlo EM algorithm for maximum likelihood inference. Our approach uses data augmentation and circulant embedding of the covariance matrix, and provides exact inference for the parameters and the missing data. Using simulated data and an application to satellite sea surface temperatures in the Pacific Ocean, we show that our method provides accurate inference on lattices of sizes up to 512 x 512, and outperforms two popular methods: composite likelihood and spectral approximations. △ Less

Submitted 18 February, 2014; originally announced February 2014.

Comments: 29 pages, 5 figures, 3 tables

MSC Class: 62H11

Stochastic approximation of score functions for Gaussian processes

Authors: Michael L. Stein, Jie Chen, Mihai Anitescu

Abstract: We discuss the statistical properties of a recently introduced unbiased stochastic approximation to the score equations for maximum likelihood calculation for Gaussian processes. Under certain conditions, including bounded condition number of the covariance matrix, the approach achieves $O(n)$ storage and nearly $O(n)$ computational effort per optimization step, where $n$ is the number of data sit… ▽ More We discuss the statistical properties of a recently introduced unbiased stochastic approximation to the score equations for maximum likelihood calculation for Gaussian processes. Under certain conditions, including bounded condition number of the covariance matrix, the approach achieves $O(n)$ storage and nearly $O(n)$ computational effort per optimization step, where $n$ is the number of data sites. Here, we prove that if the condition number of the covariance matrix is bounded, then the approximate score equations are nearly optimal in a well-defined sense. Therefore, not only is the approximation efficient to compute, but it also has comparable statistical properties to the exact maximum likelihood estimates. We discuss a modification of the stochastic approximation in which design elements of the stochastic terms mimic patterns from a $2^n$ factorial design. We prove these designs are always at least as good as the unstructured design, and we demonstrate through simulation that they can produce a substantial improvement over random designs. Our findings are validated by numerical experiments on simulated data sets of up to 1 million observations. We apply the approach to fit a space-time model to over 80,000 observations of total column ozone contained in the latitude band $40^{\circ}$-$50^{\circ}$N during April 2012. △ Less

Submitted 10 December, 2013; originally announced December 2013.

Comments: Published in at http://dx.doi.org/10.1214/13-AOAS627 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS627

Journal ref: Annals of Applied Statistics 2013, Vol. 7, No. 2, 1162-1191

Interpolation of nonstationary high frequency spatial-temporal temperature data

Authors: Joseph Guinness, Michael L. Stein

Abstract: The Atmospheric Radiation Measurement program is a U.S. Department of Energy project that collects meteorological observations at several locations around the world in order to study how weather processes affect global climate change. As one of its initiatives, it operates a set of fixed but irregularly-spaced monitoring facilities in the Southern Great Plains region of the U.S. We describe method… ▽ More The Atmospheric Radiation Measurement program is a U.S. Department of Energy project that collects meteorological observations at several locations around the world in order to study how weather processes affect global climate change. As one of its initiatives, it operates a set of fixed but irregularly-spaced monitoring facilities in the Southern Great Plains region of the U.S. We describe methods for interpolating temperature records from these fixed facilities to locations at which no observations were made, which can be useful when values are required on a spatial grid. We interpolate by conditionally simulating from a fitted nonstationary Gaussian process model that accounts for the time-varying statistical characteristics of the temperatures, as well as the dependence on solar radiation. The model is fit by maximizing an approximate likelihood, and the conditional simulations result in well-calibrated confidence intervals for the predicted temperatures. We also describe methods for handling spatial-temporal jumps in the data to interpolate a slow-moving cold front. △ Less

Submitted 29 November, 2013; originally announced November 2013.

Comments: Published in at http://dx.doi.org/10.1214/13-AOAS633 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS633

Journal ref: Annals of Applied Statistics 2013, Vol. 7, No. 3, 1684-1708

Global space-time models for climate ensembles

Authors: Stefano Castruccio, Michael L. Stein

Abstract: Global climate models aim to reproduce physical processes on a global scale and predict quantities such as temperature given some forcing inputs. We consider climate ensembles made of collections of such runs with different initial conditions and forcing scenarios. The purpose of this work is to show how the simulated temperatures in the ensemble can be reproduced (emulated) with a global space/ti… ▽ More Global climate models aim to reproduce physical processes on a global scale and predict quantities such as temperature given some forcing inputs. We consider climate ensembles made of collections of such runs with different initial conditions and forcing scenarios. The purpose of this work is to show how the simulated temperatures in the ensemble can be reproduced (emulated) with a global space/time statistical model that addresses the issue of capturing nonstationarities in latitude more effectively than current alternatives in the literature. The model we propose leads to a computationally efficient estimation procedure and, by exploiting the gridded geometry of the data, we can fit massive data sets with millions of simulated data within a few hours. Given a training set of runs, the model efficiently emulates temperature for very different scenarios and therefore is an appealing tool for impact assessment. △ Less

Submitted 28 November, 2013; originally announced November 2013.

Comments: Published in at http://dx.doi.org/10.1214/13-AOAS656 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS656

Journal ref: Annals of Applied Statistics 2013, Vol. 7, No. 3, 1593-1611

Editorial

Authors: Michael L. Stein

Abstract: Many of you reading these words will have been attracted by the discussion paper [McShane and Wyner (2011)], in which case, this may be the first, but hopefully not the last, time you will have read anything in a statistics journal. I would like to take this opportunity to discuss the review process in our journal and to make some comments about the role of statistics and uncertainty assessment in… ▽ More Many of you reading these words will have been attracted by the discussion paper [McShane and Wyner (2011)], in which case, this may be the first, but hopefully not the last, time you will have read anything in a statistics journal. I would like to take this opportunity to discuss the review process in our journal and to make some comments about the role of statistics and uncertainty assessment in paleoclimatology and the broader debate about climate change. △ Less

Submitted 14 April, 2011; originally announced April 2011.

Comments: Published in at http://dx.doi.org/10.1214/10-AOAS449 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS449

Journal ref: Annals of Applied Statistics 2011, Vol. 5, No. 1, 1-4

Spatial interpolation of high-frequency monitoring data

Authors: Michael L. Stein

Abstract: Climate modelers generally require meteorological information on regular grids, but monitoring stations are, in practice, sited irregularly. Thus, there is a need to produce public data records that interpolate available data to a high density grid, which can then be used to generate meteorological maps at a broad range of spatial and temporal scales. In addition to point predictions, quantifica… ▽ More Climate modelers generally require meteorological information on regular grids, but monitoring stations are, in practice, sited irregularly. Thus, there is a need to produce public data records that interpolate available data to a high density grid, which can then be used to generate meteorological maps at a broad range of spatial and temporal scales. In addition to point predictions, quantifications of uncertainty are also needed. One way to accomplish this is to provide multiple simulations of the relevant meteorological quantities conditional on the observed data taking into account the various uncertainties in predicting a space-time process at locations with no monitoring data. Using a high-quality dataset of minute-by-minute measurements of atmospheric pressure in north-central Oklahoma, this work describes a statistical approach to carrying out these conditional simulations. Based on observations at 11 stations, conditional simulations were produced at two other sites with monitoring stations. The resulting point predictions are very accurate and the multiple simulations produce well-calibrated prediction uncertainties for temporal changes in atmospheric pressure but are substantially overconservative for the uncertainties in the predictions of (undifferenced) pressure. △ Less

Submitted 5 June, 2009; originally announced June 2009.

Comments: Published in at http://dx.doi.org/10.1214/08-AOAS208 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS208

Journal ref: Annals of Applied Statistics 2009, Vol. 3, No. 1, 272-291

Introduction to papers on astrostatistics

Authors: Thomas J. Loredo, John Rice, Michael L. Stein

Abstract: We are pleased to present a Special Section on Statistics and Astronomy in this issue of the The Annals of Applied Statistics. Astronomy is an observational rather than experimental science; as a result, astronomical data sets both small and large present particularly challenging problems to analysts who must make the best of whatever the sky offers their instruments. The resulting statistical p… ▽ More We are pleased to present a Special Section on Statistics and Astronomy in this issue of the The Annals of Applied Statistics. Astronomy is an observational rather than experimental science; as a result, astronomical data sets both small and large present particularly challenging problems to analysts who must make the best of whatever the sky offers their instruments. The resulting statistical problems have enormous diversity. In one problem, one may have to carefully quantify uncertainty in a hard-won, sparse data set; in another, the sheer volume of data may forbid a formally optimal analysis, requiring judicious balancing of model sophistication, approximations, and clever algorithms. Often the data bear a complex relationship to the underlying phenomenon producing them, much in the manner of inverse problems. △ Less

Submitted 14 May, 2009; originally announced May 2009.

Comments: Published in at http://dx.doi.org/10.1214/09-AOAS234 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS234

Journal ref: Annals of Applied Statistics 2009, Vol. 3, No. 1, 1-5

Nonstationary covariance models for global data

Authors: Mikyoung Jun, Michael L. Stein

Abstract: With the widespread availability of satellite-based instruments, many geophysical processes are measured on a global scale and they often show strong nonstationarity in the covariance structure. In this paper we present a flexible class of parametric covariance models that can capture the nonstationarity in global data, especially strong dependency of covariance structure on latitudes. We apply… ▽ More With the widespread availability of satellite-based instruments, many geophysical processes are measured on a global scale and they often show strong nonstationarity in the covariance structure. In this paper we present a flexible class of parametric covariance models that can capture the nonstationarity in global data, especially strong dependency of covariance structure on latitudes. We apply the Discrete Fourier Transform to data on regular grids, which enables us to calculate the exact likelihood for large data sets. Our covariance model is applied to global total column ozone level data on a given day. We discuss how our covariance model compares with some existing models. △ Less

Submitted 26 January, 2009; originally announced January 2009.

Comments: Published in at http://dx.doi.org/10.1214/08-AOAS183 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS183

Journal ref: Annals of Applied Statistics 2008, Vol. 2, No. 4, 1271-1289

Special section on statistics in the atmospheric sciences

Authors: Montserrat Fuentes, Peter Guttorp, Michael L. Stein

Abstract: With the possible exception of gambling, meteorology, particularly precipitation forecasting, may be the area with which the general public is most familiar with probabilistic assessments of uncertainty. Despite the heavy use of stochastic models and statistical methods in weather forecasting and other areas of the atmospheric sciences, papers in these areas have traditionally been somewhat unco… ▽ More With the possible exception of gambling, meteorology, particularly precipitation forecasting, may be the area with which the general public is most familiar with probabilistic assessments of uncertainty. Despite the heavy use of stochastic models and statistical methods in weather forecasting and other areas of the atmospheric sciences, papers in these areas have traditionally been somewhat uncommon in statistics journals. We see signs of this changing in recent years and we have sought to highlight some present research directions at the interface of statistics and the atmospheric sciences in this special section. △ Less

Submitted 22 January, 2009; originally announced January 2009.

Comments: Published in at http://dx.doi.org/10.1214/08-AOAS209 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS209

Journal ref: Annals of Applied Statistics 2008, Vol. 2, No. 4, 1143-1147

Spatial variation of total column ozone on a global scale

Authors: Michael L. Stein

Abstract: The spatial dependence of total column ozone varies strongly with latitude, so that homogeneous models (invariant to all rotations) are clearly unsuitable. However, an assumption of axial symmetry, which means that the process model is invariant to rotations about the Earth's axis, is much more plausible and considerably simplifies the modeling. Using TOMS (Total Ozone Mapping Spectrometer) meas… ▽ More The spatial dependence of total column ozone varies strongly with latitude, so that homogeneous models (invariant to all rotations) are clearly unsuitable. However, an assumption of axial symmetry, which means that the process model is invariant to rotations about the Earth's axis, is much more plausible and considerably simplifies the modeling. Using TOMS (Total Ozone Mapping Spectrometer) measurements of total column ozone over a six-day period, this work investigates the modeling of axially symmetric processes on the sphere using expansions in spherical harmonics. It turns out that one can capture many of the large scale features of the spatial covariance structure using a relatively small number of terms in such an expansion, but the resulting fitted model provides a horrible fit to the data when evaluated via its likelihood because of its inability to describe accurately the process's local behavior. Thus, there remains the challenge of developing computationally tractable models that capture both the large and small scale structure of these data. △ Less

Submitted 4 September, 2007; originally announced September 2007.

Comments: Published at http://dx.doi.org/10.1214/07-AOAS106 in the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOAS-AOAS106

Journal ref: Annals of Applied Statistics 2007, Vol. 1, No. 1, 191-210