Showing 1–21 of 21 results for author: Lee, K H

Inference for Log-Gaussian Cox Point Processes using Bayesian Deep Learning: Application to Human Oral Microbiome Image Data

Authors: Shuwan Wang, Christopher K. Wikle, Athanasios C. Micheas, Jessica L. Mark Welch, Jacqueline R. Starr, Kyu Ha Lee

Abstract: It is common in nature to see aggregation of objects in space. Exploring the mechanism associated with the locations of such clustered observations can be essential to understanding the phenomenon, such as the source of spatial heterogeneity, or comparison to other event generating processes in the same domain. Log-Gaussian Cox processes (LGCPs) represent an important class of models for quantifyi… ▽ More It is common in nature to see aggregation of objects in space. Exploring the mechanism associated with the locations of such clustered observations can be essential to understanding the phenomenon, such as the source of spatial heterogeneity, or comparison to other event generating processes in the same domain. Log-Gaussian Cox processes (LGCPs) represent an important class of models for quantifying aggregation in a spatial point pattern. However, implementing likelihood-based Bayesian inference for such models presents many computational challenges, particularly in high dimensions. In this paper, we propose a novel likelihood-free inference approach for LGCPs using the recently developed BayesFlow approach, where invertible neural networks are employed to approximate the posterior distribution of the parameters of interest. BayesFlow is a neural simulation-based method based on "amortized" posterior estimation. That is, after an initial training procedure, fast feed-forward operations allow rapid posterior inference for any data within the same model family. Comprehensive numerical studies validate the reliability of the framework and show that BayesFlow achieves substantial computational gain in repeated application, especially for two-dimensional LGCPs. We demonstrate the utility and robustness of the method by applying it to two distinct oral microbial biofilm images. △ Less

Submitted 18 March, 2025; v1 submitted 17 February, 2025; originally announced February 2025.

A Bayesian Multivariate Spatial Point Pattern Model: Application to Oral Microbiome FISH Image Data

Authors: Kyu Ha Lee, Brent A. Coull, Suman Majumder, Patrick J. La Riviere, Jessica L. Mark Welch, Jacqueline R. Starr

Abstract: Advances in cellular imaging technologies, especially those based on fluorescence in situ hybridization (FISH) now allow detailed visualization of the spatial organization of human or bacterial cells. Quantifying this spatial organization is crucial for understanding the function of multicellular tissues or biofilms, with implications for human health and disease. To address the need for better me… ▽ More Advances in cellular imaging technologies, especially those based on fluorescence in situ hybridization (FISH) now allow detailed visualization of the spatial organization of human or bacterial cells. Quantifying this spatial organization is crucial for understanding the function of multicellular tissues or biofilms, with implications for human health and disease. To address the need for better methods to achieve such quantification, we propose a flexible multivariate point process model that characterizes and estimates complex spatial interactions among multiple cell types. The proposed Bayesian framework is appealing due to its unified estimation process and the ability to directly quantify uncertainty in key estimates of interest, such as those of inter-type correlation and the proportion of variance due to inter-type relationships. To ensure stable and interpretable estimation, we consider shrinkage priors for coefficients associated with latent processes. Model selection and comparison are conducted by using a deviance information criterion designed for models with latent variables, effectively balancing the risk of overfitting with that of oversimplifying key quantities. Furthermore, we develop a hierarchical modeling approach to integrate multiple image-specific estimates from a given subject, allowing inference at both the global and subject-specific levels. We apply the proposed method to microbial biofilm image data from the human tongue dorsum and find that specific taxon pairs, such as Streptococcus mitis-Streptococcus salivarius and Streptococcus mitis-Veillonella, exhibit strong positive spatial correlations, while others, such as Actinomyces-Rothia, show slight negative correlations. For most of the taxa, a substantial portion of spatial variance can be attributed to inter-taxon relationships. △ Less

Submitted 14 February, 2025; originally announced February 2025.

Bayesian Variable Selection for High-Dimensional Mediation Analysis: Application to Metabolomics Data in Epidemiological Studies

Authors: Youngho Bae, Chanmin Kim, Fenglei Wang, Qi Sun, Kyu Ha Lee

Abstract: In epidemiological research, causal models incorporating potential mediators along a pathway are crucial for understanding how exposures influence health outcomes. This work is motivated by integrated epidemiological and blood biomarker studies, investigating the relationship between long-term adherence to a Mediterranean diet and cardiometabolic health, with plasma metabolomes as potential mediat… ▽ More In epidemiological research, causal models incorporating potential mediators along a pathway are crucial for understanding how exposures influence health outcomes. This work is motivated by integrated epidemiological and blood biomarker studies, investigating the relationship between long-term adherence to a Mediterranean diet and cardiometabolic health, with plasma metabolomes as potential mediators. Analyzing causal mediation in such high-dimensional omics data presents substantial challenges, including complex dependencies among mediators and the need for advanced regularization or Bayesian techniques to ensure stable and interpretable estimation and selection of indirect effects. To this end, we propose a novel Bayesian framework for identifying active pathways and estimating indirect effects in the presence of high-dimensional multivariate mediators. Our approach adopts a multivariate stochastic search variable selection method, tailored for such complex mediation scenarios. Central to our method is the introduction of a set of priors for the selection: a Markov random field prior and sequential subsetting Bernoulli priors. The first prior's Markov property leverages the inherent correlations among mediators, thereby increasing power to detect mediated effects. The sequential subsetting aspect of the second prior encourages the simultaneous selection of relevant mediators and their corresponding indirect effects from the two model parts, providing a more coherent and efficient variable selection framework, specific to mediation analysis. Comprehensive simulation studies demonstrate that the proposed method provides superior power in detecting active mediating pathways. We further illustrate the practical utility of the method through its application to metabolome data from two cohort studies, highlighting its effectiveness in real data setting. △ Less

Submitted 26 November, 2024; originally announced November 2024.

Group lasso priors for Bayesian accelerated failure time models with left-truncated and interval-censored data

Authors: Harrison T. Reeder, Sebastien Haneuse, Kyu Ha Lee

Abstract: An important task in health research is to characterize time-to-event outcomes such as disease onset or mortality in terms of a potentially high-dimensional set of risk factors. For example, prospective cohort studies of Alzheimer's disease typically enroll older adults for observation over several decades to assess the long-term impact of genetic and other factors on cognitive decline and mortali… ▽ More An important task in health research is to characterize time-to-event outcomes such as disease onset or mortality in terms of a potentially high-dimensional set of risk factors. For example, prospective cohort studies of Alzheimer's disease typically enroll older adults for observation over several decades to assess the long-term impact of genetic and other factors on cognitive decline and mortality. The accelerated failure time model is particularly well-suited to such studies, structuring covariate effects as `horizontal' changes to the survival quantiles that conceptually reflect shifts in the outcome distribution due to lifelong exposures. However, this modeling task is complicated by the enrollment of adults at differing ages, and intermittent followup visits leading to interval censored outcome information. Moreover, genetic and clinical risk factors are not only high-dimensional, but characterized by underlying grouping structure, such as by function or gene location. Such grouped high-dimensional covariates require shrinkage methods that directly acknowledge this structure to facilitate variable selection and estimation. In this paper, we address these considerations directly by proposing a Bayesian accelerated failure time model with a group-structured lasso penalty, designed for left-truncated and interval-censored time-to-event data. We develop a custom Markov chain Monte Carlo sampler for efficient estimation, and investigate the impact of various methods of penalty tuning and thresholding for variable selection. We present a simulation study examining the performance of this method relative to models with an ordinary lasso penalty, and apply the proposed method to identify groups of predictive genetic and clinical risk factors for Alzheimer's disease in the Religious Orders Study and Memory and Aging Project (ROSMAP) prospective cohort studies of AD and dementia. △ Less

Submitted 11 January, 2024; v1 submitted 9 January, 2024; originally announced January 2024.

Understanding an Acquisition Function Family for Bayesian Optimization

Authors: Jiajie Kong, Tony Pourmohamad, Herbert K. H. Lee

Abstract: Bayesian optimization (BO) developed as an approach for the efficient optimization of expensive black-box functions without gradient information. A typical BO paper introduces a new approach and compares it to some alternatives on simulated and possibly real examples to show its efficacy. Yet on a different example, this new algorithm might not be as effective as the alternatives. This paper looks… ▽ More Bayesian optimization (BO) developed as an approach for the efficient optimization of expensive black-box functions without gradient information. A typical BO paper introduces a new approach and compares it to some alternatives on simulated and possibly real examples to show its efficacy. Yet on a different example, this new algorithm might not be as effective as the alternatives. This paper looks at a broader family of approaches to explain the strengths and weaknesses of algorithms in the family, with guidance on what choices might work best on different classes of problems. △ Less

Submitted 16 October, 2023; originally announced October 2023.

Characterizing quantile-varying covariate effects under the accelerated failure time model

Authors: Harrison T. Reeder, Kyu Ha Lee, Sebastien Haneuse

Abstract: An important task in survival analysis is choosing a structure for the relationship between covariates of interest and the time-to-event outcome. For example, the accelerated failure time (AFT) model structures each covariate effect as a constant multiplicative shift in the outcome distribution across all survival quantiles. Though parsimonious, this structure cannot detect or capture effects that… ▽ More An important task in survival analysis is choosing a structure for the relationship between covariates of interest and the time-to-event outcome. For example, the accelerated failure time (AFT) model structures each covariate effect as a constant multiplicative shift in the outcome distribution across all survival quantiles. Though parsimonious, this structure cannot detect or capture effects that differ across quantiles of the distribution, a limitation that is analogous to only permitting proportional hazards in the Cox model. To address this, we propose a general framework for quantile-varying multiplicative effects under the AFT model. Specifically, we embed flexible regression structures within the AFT model, and derive a novel formula for interpretable effects on the quantile scale. A regression standardization scheme based on the g-formula is proposed to enable estimation of both covariate-conditional and marginal effects for an exposure of interest. We implement a user-friendly Bayesian approach for estimation and quantification of uncertainty, while accounting for left truncation and complex censoring. We emphasize the intuitive interpretation of this model through numerical and graphical tools, and illustrate its performance by application to a study of Alzheimer's disease and dementia. △ Less

Submitted 8 January, 2023; originally announced January 2023.

Comments: This is the pre-peer reviewed, "submitted" version of the manuscript published in final form in Biostatistics by Oxford University Press at the below citation/doi. This upload will be updated with the final peer-reviewed "accepted" version of the manuscript following a 24 month embargo period

Journal ref: Biostatistics (Oxford, England), kxac052 (2023)

Multivariate cluster point process to quantify and explore multi-entity configurations: Application to biofilm image data

Authors: Suman Majumder, Brent A. Coull, Jessica L. Mark Welch, Patrick J. La Riviere, Floyd E. Dewhirst, Jacqueline R. Starr, Kyu Ha Lee

Abstract: Clusters of similar or dissimilar objects are encountered in many fields. Frequently used approaches treat the central object of each cluster as latent. Yet, often objects of one or more types cluster around objects of another type. Such arrangements are common in biomedical images of cells, in which nearby cell types likely interact. Quantifying spatial relationships may elucidate biological mech… ▽ More Clusters of similar or dissimilar objects are encountered in many fields. Frequently used approaches treat the central object of each cluster as latent. Yet, often objects of one or more types cluster around objects of another type. Such arrangements are common in biomedical images of cells, in which nearby cell types likely interact. Quantifying spatial relationships may elucidate biological mechanisms. Parent-offspring statistical frameworks can be usefully applied even when central objects (parents) differ from peripheral ones (offspring). We propose the novel multivariate cluster point process (MCPP) to quantify multi-object (e.g., multi-cellular) arrangements. Unlike commonly used approaches, the MCPP exploits locations of the central parent object in clusters. It accounts for possibly multilayered, multivariate clustering. The model formulation requires specification of which object types function as cluster centers and which reside peripherally. If such information is unknown, the relative roles of object types may be explored by comparing fit of different models via the deviance information criterion (DIC). In simulated data, we compared DIC of a series of models; the MCPP correctly identified simulated relationships. It also produced more accurate and precise parameter estimates than the classical univariate Neyman-Scott process model. We also used the MCPP to quantify proposed configurations and explore new ones in human dental plaque biofilm image data. MCPP models quantified simultaneous clustering of Streptococcus and Porphyromonas around Corynebacterium and of Pasteurellaceae around Streptococcus and successfully captured hypothesized structures for all taxa. Further exploration suggested the presence of clustering between Fusobacterium and Leptotrichia, a previously unreported relationship. △ Less

Submitted 8 October, 2024; v1 submitted 8 February, 2022; originally announced February 2022.

MSC Class: 62

Measuring performance for end-of-life care

Authors: Sebastien Haneuse, Deborah Schrag, Francesca Dominici, Sharon-Lise Normand, Kyu Ha Lee

Abstract: Although not without controversy, readmission is entrenched as a hospital quality metric, with statistical analyses generally based on fitting a logistic-Normal generalized linear mixed model. Such analyses, however, ignore death as a competing risk, although doing so for clinical conditions with high mortality can have profound effects; a hospitals seemingly good performance for readmission may b… ▽ More Although not without controversy, readmission is entrenched as a hospital quality metric, with statistical analyses generally based on fitting a logistic-Normal generalized linear mixed model. Such analyses, however, ignore death as a competing risk, although doing so for clinical conditions with high mortality can have profound effects; a hospitals seemingly good performance for readmission may be an artifact of it having poor performance for mortality. In this paper we propose novel multivariate hospital-level performance measures for readmission and mortality, that derive from framing the analysis as one of cluster-correlated semi-competing risks data. We also consider a number of profiling-related goals, including the identification of extreme performers and a bivariate classification of whether the hospital has higher-/lower-than-expected readmission and mortality rates, via a Bayesian decision-theoretic approach that characterizes hospitals on the basis of minimizing the posterior expected loss for an appropriate loss function. In some settings, particularly if the number of hospitals is large, the computational burden may be prohibitive. To resolve this, we propose a series of analysis strategies that will be useful in practice. Throughout the methods are illustrated with data from CMS on N=17,685 patients diagnosed with pancreatic cancer between 2000-2012 at one of J=264 hospitals in California. △ Less

Submitted 18 May, 2021; originally announced May 2021.

Bayesian Survival Analysis Using Gamma Processes with Adaptive Time Partition

Authors: Yi Li, Sumi Seo, Kyu Ha Lee

Abstract: In Bayesian semi-parametric analyses of time-to-event data, non-parametric process priors are adopted for the baseline hazard function or the cumulative baseline hazard function for a given finite partition of the time axis. However, it would be controversial to suggest a general guideline to construct an optimal time partition. While a great deal of research has been done to relax the assumption… ▽ More In Bayesian semi-parametric analyses of time-to-event data, non-parametric process priors are adopted for the baseline hazard function or the cumulative baseline hazard function for a given finite partition of the time axis. However, it would be controversial to suggest a general guideline to construct an optimal time partition. While a great deal of research has been done to relax the assumption of the fixed split times for other non-parametric processes, to our knowledge, no methods have been developed for a gamma process prior, which is one of the most widely used in Bayesian survival analysis. In this paper, we propose a new Bayesian framework for proportional hazards models where the cumulative baseline hazard function is modeled a priori by a gamma process. A key feature of the proposed framework is that the number and position of interval cutpoints are treated as random and estimated based on their posterior distributions. △ Less

Submitted 5 August, 2020; originally announced August 2020.

Graph-Aware Transformer: Is Attention All Graphs Need?

Authors: Sanghyun Yoo, Young-Seok Kim, Kang Hyun Lee, Kuhwan Jeong, Junhwi Choi, Hoshik Lee, Young Sang Choi

Abstract: Graphs are the natural data structure to represent relational and structural information in many domains. To cover the broad range of graph-data applications including graph classification as well as graph generation, it is desirable to have a general and flexible model consisting of an encoder and a decoder that can handle graph data. Although the representative encoder-decoder model, Transformer… ▽ More Graphs are the natural data structure to represent relational and structural information in many domains. To cover the broad range of graph-data applications including graph classification as well as graph generation, it is desirable to have a general and flexible model consisting of an encoder and a decoder that can handle graph data. Although the representative encoder-decoder model, Transformer, shows superior performance in various tasks especially of natural language processing, it is not immediately available for graphs due to their non-sequential characteristics. To tackle this incompatibility, we propose GRaph-Aware Transformer (GRAT), the first Transformer-based model which can encode and decode whole graphs in end-to-end fashion. GRAT is featured with a self-attention mechanism adaptive to the edge information and an auto-regressive decoding mechanism based on the two-path approach consisting of sub-graph encoding path and node-and-edge generation path for each decoding step. We empirically evaluated GRAT on multiple setups including encoder-based tasks such as molecule property predictions on QM9 datasets and encoder-decoder-based tasks such as molecule graph generation in the organic molecule synthesis domain. GRAT has shown very promising results including state-of-the-art performance on 4 regression tasks in QM9 benchmark. △ Less

Submitted 9 June, 2020; originally announced June 2020.

A comprehensive study on the prediction reliability of graph neural networks for virtual screening

Authors: Soojung Yang, Kyung Hoon Lee, Seongok Ryu

Abstract: Prediction models based on deep neural networks are increasingly gaining attention for fast and accurate virtual screening systems. For decision makings in virtual screening, researchers find it useful to interpret an output of classification system as probability, since such interpretation allows them to filter out more desirable compounds. However, probabilistic interpretation cannot be correct… ▽ More Prediction models based on deep neural networks are increasingly gaining attention for fast and accurate virtual screening systems. For decision makings in virtual screening, researchers find it useful to interpret an output of classification system as probability, since such interpretation allows them to filter out more desirable compounds. However, probabilistic interpretation cannot be correct for models that hold over-parameterization problems or inappropriate regularizations, leading to unreliable prediction and decision making. In this regard, we concern the reliability of neural prediction models on molecular properties, especially when models are trained with sparse data points and imbalanced distributions. This work aims to propose guidelines for training reliable models, we thus provide methodological details and ablation studies on the following train principles. We investigate the effects of model architectures, regularization methods, and loss functions on the prediction performance and reliability of classification results. Moreover, we evaluate prediction reliability of models on virtual screening scenario. Our result highlights that correct choice of regularization and inference methods is evidently important to achieve high success rate, especially in data imbalanced situation. All experiments were performed under a single unified model implementation to alleviate external randomness in model training and to enable precise comparison of results. △ Less

Submitted 17 March, 2020; originally announced March 2020.

SemiCompRisks: An R Package for Independent and Cluster-Correlated Analyses of Semi-Competing Risks Data

Authors: Danilo Alvares, Sebastien Haneuse, Catherine Lee, Kyu Ha Lee

Abstract: Semi-competing risks refer to the setting where primary scientific interest lies in estimation and inference with respect to a non-terminal event, the occurrence of which is subject to a terminal event. In this paper, we present the R package SemiCompRisks that provides functions to perform the analysis of independent/clustered semi-competing risks data under the illness-death multi-state model. T… ▽ More Semi-competing risks refer to the setting where primary scientific interest lies in estimation and inference with respect to a non-terminal event, the occurrence of which is subject to a terminal event. In this paper, we present the R package SemiCompRisks that provides functions to perform the analysis of independent/clustered semi-competing risks data under the illness-death multi-state model. The package allows the user to choose the specification for model components from a range of options giving users substantial flexibility, including: accelerated failure time or proportional hazards regression models; parametric or non-parametric specifications for baseline survival functions; parametric or non-parametric specifications for random effects distributions when the data are cluster-correlated; and, a Markov or semi-Markov specification for terminal event following non-terminal event. While estimation is mainly performed within the Bayesian paradigm, the package also provides the maximum likelihood estimation for select parametric models. The package also includes functions for univariate survival analysis as complementary analysis tools. △ Less

Submitted 5 April, 2018; v1 submitted 10 January, 2018; originally announced January 2018.

Comments: 35 pages, 3 figures

Model-Based Clustering of Time-Evolving Networks through Temporal Exponential-Family Random Graph Models

Authors: Kevin H. Lee, Lingzhou Xue, David R. Hunter

Abstract: Dynamic networks are a general language for describing time-evolving complex systems, and discrete time network models provide an emerging statistical technique for various applications. It is a fundamental research question to detect the community structure in time-evolving networks. However, due to significant computational challenges and difficulties in modeling communities of time-evolving net… ▽ More Dynamic networks are a general language for describing time-evolving complex systems, and discrete time network models provide an emerging statistical technique for various applications. It is a fundamental research question to detect the community structure in time-evolving networks. However, due to significant computational challenges and difficulties in modeling communities of time-evolving networks, there is little progress in the current literature to effectively find communities in time-evolving networks. In this work, we propose a novel model-based clustering framework for time-evolving networks based on discrete time exponential-family random graph models. To choose the number of communities, we use conditional likelihood to construct an effective model selection criterion. Furthermore, we propose an efficient variational expectation-maximization (EM) algorithm to find approximate maximum likelihood estimates of network parameters and mixing proportions. By using variational methods and minorization-maximization (MM) techniques, our method has appealing scalability for large-scale time-evolving networks. The power of our method is demonstrated in simulation studies and empirical applications to international trade networks and the collaboration networks of a large American research university. △ Less

Submitted 20 December, 2017; originally announced December 2017.

Comments: 30 pages, 4 figures

Bayesian Variable Selection for Multivariate Zero-Inflated Models: Application to Microbiome Count Data

Authors: Kyu Ha Lee, Brent A. Coull, Anna-Barbara Moscicki, Bruce J. Paster, Jacqueline R. Starr

Abstract: Microorganisms play critical roles in human health and disease. It is well known that microbes live in diverse communities in which they interact synergistically or antagonistically. Thus for estimating microbial associations with clinical covariates, multivariate statistical models are preferred. Multivariate models allow one to estimate and exploit complex interdependencies among multiple taxa,… ▽ More Microorganisms play critical roles in human health and disease. It is well known that microbes live in diverse communities in which they interact synergistically or antagonistically. Thus for estimating microbial associations with clinical covariates, multivariate statistical models are preferred. Multivariate models allow one to estimate and exploit complex interdependencies among multiple taxa, yielding more powerful tests of exposure or treatment effects than application of taxon-specific univariate analyses. In addition, the analysis of microbial count data requires special attention because data commonly exhibit zero inflation. To meet these needs, we developed a Bayesian variable selection model for multivariate count data with excess zeros that incorporates information on the covariance structure of the outcomes (counts for multiple taxa), while estimating associations with the mean levels of these outcomes. Although there has been a great deal of effort in zero-inflated models for longitudinal data, little attention has been given to high-dimensional multivariate zero-inflated data modeled via a general correlation structure. Through simulation, we compared performance of the proposed method to that of existing univariate approaches, for both the binary and count parts of the model. When outcomes were correlated the proposed variable selection method maintained type I error while boosting the ability to identify true associations in the binary component of the model. For the count part of the model, in some scenarios the the univariate method had higher power than the multivariate approach. This higher power was at a cost of a highly inflated false discovery rate not observed with the proposed multivariate method. We applied the approach to oral microbiome data from the Pediatric HIV/AIDS Cohort Oral Health Study and identified five species (of 44) associated with HIV infection. △ Less

Submitted 20 May, 2018; v1 submitted 31 October, 2017; originally announced November 2017.

Hierarchical models for semi-competing risks data with application to quality of end-of-life care for pancreatic cancer

Authors: Kyu Ha Lee, Francesca Dominici, Deborah Schrag, Sebastien Haneuse

Abstract: Readmission following discharge from an initial hospitalization is a key marker of quality of health care in the United States. For the most part, readmission has been used to study quality of care for patients with acute health conditions, such as pneumonia and heart failure, with analyses typically based on a logistic-Normal generalized linear mixed model. Applying this model to the study readmi… ▽ More Readmission following discharge from an initial hospitalization is a key marker of quality of health care in the United States. For the most part, readmission has been used to study quality of care for patients with acute health conditions, such as pneumonia and heart failure, with analyses typically based on a logistic-Normal generalized linear mixed model. Applying this model to the study readmission among patients with increasingly prevalent advanced health conditions such as pancreatic cancer is problematic, however, because it ignores death as a competing risk. A more appropriate analysis is to imbed such studies within the semi-competing risks framework. To our knowledge, however, no comprehensive statistical methods have been developed for cluster-correlated semi-competing risks data. In this paper we propose a novel hierarchical modeling framework for the analysis of cluster-correlated semi-competing risks data. The framework permits parametric or non-parametric specifications for a range of model components, including baseline hazard functions and distributions for key random effects, giving analysts substantial flexibility as they consider their own analyses. Estimation and inference is performed within the Bayesian paradigm since it facilitates the straightforward characterization of (posterior) uncertainty for all model parameters including hospital-specific random effects. The proposed framework is used to study the risk of readmission among 5,298 Medicare beneficiaries diagnosed with pancreatic cancer at 112 hospitals in the six New England states between 2000-2009, specifically to investigate the role of patient-level risk factors and to characterize variation in risk across hospitals that is not explained by differences in patient case-mix. △ Less

Submitted 5 August, 2015; v1 submitted 2 February, 2015; originally announced February 2015.

Journal ref: Journal of the American Statistical Association 2016, Volume 111, Issue 515, pages 1075-1095

Modeling an Augmented Lagrangian for Blackbox Constrained Optimization

Authors: Robert B. Gramacy, Genetha A. Gray, Sebastien Le Digabel, Herbert K. H. Lee, Pritam Ranjan, Garth Wells, Stefan M. Wild

Abstract: Constrained blackbox optimization is a difficult problem, with most approaches coming from the mathematical programming literature. The statistical literature is sparse, especially in addressing problems with nontrivial constraints. This situation is unfortunate because statistical methods have many attractive properties: global scope, handling noisy objectives, sensitivity analysis, and so forth.… ▽ More Constrained blackbox optimization is a difficult problem, with most approaches coming from the mathematical programming literature. The statistical literature is sparse, especially in addressing problems with nontrivial constraints. This situation is unfortunate because statistical methods have many attractive properties: global scope, handling noisy objectives, sensitivity analysis, and so forth. To narrow that gap, we propose a combination of response surface modeling, expected improvement, and the augmented Lagrangian numerical optimization framework. This hybrid approach allows the statistical model to think globally and the augmented Lagrangian to act locally. We focus on problems where the constraints are the primary bottleneck, requiring expensive simulation to evaluate and substantial modeling effort to map out. In that context, our hybridization presents a simple yet effective solution that allows existing objective-oriented statistical approaches, like those based on Gaussian process surrogates and expected improvement heuristics, to be applied to the constrained setting with minor modification. This work is motivated by a challenging, real-data benchmark problem from hydrology where, even with a simple linear objective function, learning a nontrivial valid region complicates the search for a global minimum. △ Less

Submitted 3 March, 2015; v1 submitted 19 March, 2014; originally announced March 2014.

Comments: 22 Pages, 2 additional supplementary, 5 figures

Cases for the nugget in modeling computer experiments

Authors: Robert B. Gramacy, Herbert K. H. Lee

Abstract: Most surrogate models for computer experiments are interpolators, and the most common interpolator is a Gaussian process (GP) that deliberately omits a small-scale (measurement) error term called the nugget. The explanation is that computer experiments are, by definition, "deterministic", and so there is no measurement error. We think this is too narrow a focus for a computer experiment and a stat… ▽ More Most surrogate models for computer experiments are interpolators, and the most common interpolator is a Gaussian process (GP) that deliberately omits a small-scale (measurement) error term called the nugget. The explanation is that computer experiments are, by definition, "deterministic", and so there is no measurement error. We think this is too narrow a focus for a computer experiment and a statistically inefficient way to model them. We show that estimating a (non-zero) nugget can lead to surrogate models with better statistical properties, such as predictive accuracy and coverage, in a variety of common situations. △ Less

Submitted 21 November, 2010; v1 submitted 26 July, 2010; originally announced July 2010.

Comments: 17 pages, 4 figures, 3 tables; revised

Optimization Under Unknown Constraints

Authors: Robert B. Gramacy, Herbert K. H. Lee

Abstract: Optimization of complex functions, such as the output of computer simulators, is a difficult task that has received much attention in the literature. A less studied problem is that of optimization under unknown constraints, i.e., when the simulator must be invoked both to determine the typical real-valued response and to determine if a constraint has been violated, either for physical or policy re… ▽ More Optimization of complex functions, such as the output of computer simulators, is a difficult task that has received much attention in the literature. A less studied problem is that of optimization under unknown constraints, i.e., when the simulator must be invoked both to determine the typical real-valued response and to determine if a constraint has been violated, either for physical or policy reasons. We develop a statistical approach based on Gaussian processes and Bayesian learning to both approximate the unknown function and estimate the probability of meeting the constraints. A new integrated improvement criterion is proposed to recognize that responses from inputs that violate the constraint may still be informative about the function, and thus could potentially be useful in the optimization. The new criterion is illustrated on synthetic data, and on a motivating optimization problem from health care policy. △ Less

Submitted 5 July, 2010; v1 submitted 22 April, 2010; originally announced April 2010.

Comments: 19 pages, 8 figures, Valencia discussion paper

Adaptive design and analysis of supercomputer experiments

Authors: Robert B. Gramacy, Herbert K. H. Lee

Abstract: Computer experiments are often performed to allow modeling of a response surface of a physical experiment that can be too costly or difficult to run except using a simulator. Running the experiment over a dense grid can be prohibitively expensive, yet running over a sparse design chosen in advance can result in obtaining insufficient information in parts of the space, particularly when the surfa… ▽ More Computer experiments are often performed to allow modeling of a response surface of a physical experiment that can be too costly or difficult to run except using a simulator. Running the experiment over a dense grid can be prohibitively expensive, yet running over a sparse design chosen in advance can result in obtaining insufficient information in parts of the space, particularly when the surface calls for a nonstationary model. We propose an approach that automatically explores the space while simultaneously fitting the response surface, using predictive uncertainty to guide subsequent experimental runs. The newly developed Bayesian treed Gaussian process is used as the surrogate model, and a fully Bayesian approach allows explicit measures of uncertainty. We develop an adaptive sequential design framework to cope with an asynchronous, random, agent--based supercomputing environment, by using a hybrid approach that melds optimal strategies from the statistics literature with flexible strategies from the active learning literature. The merits of this approach are borne out in several examples, including the motivating computational fluid dynamics simulation of a rocket booster. △ Less

Submitted 25 May, 2009; v1 submitted 28 May, 2008; originally announced May 2008.

Comments: 42 pages, 8 Figures, 2 tables, to appear in Technometrics

Gaussian Processes and Limiting Linear Models

Authors: Robert B. Gramacy, Herbert K. H. Lee

Abstract: Gaussian processes retain the linear model either as a special case, or in the limit. We show how this relationship can be exploited when the data are at least partially linear. However from the perspective of the Bayesian posterior, the Gaussian processes which encode the linear model either have probability of nearly zero or are otherwise unattainable without the explicit construction of a pri… ▽ More Gaussian processes retain the linear model either as a special case, or in the limit. We show how this relationship can be exploited when the data are at least partially linear. However from the perspective of the Bayesian posterior, the Gaussian processes which encode the linear model either have probability of nearly zero or are otherwise unattainable without the explicit construction of a prior with the limiting linear model in mind. We develop such a prior, and show that its practical benefits extend well beyond the computational and conceptual simplicity of the linear model. For example, linearity can be extracted on a per-dimension basis, or can be combined with treed partition models to yield a highly efficient nonstationary model. Our approach is demonstrated on synthetic and real datasets of varying linearity and dimensionality. △ Less

Submitted 13 July, 2008; v1 submitted 29 April, 2008; originally announced April 2008.

Comments: 31 pages, 10 figures, 4 tables, accepted by CSDA, earlier version in JSM06 proceedings

Bayesian treed Gaussian process models with an application to computer modeling

Authors: Robert B. Gramacy, Herbert K. H. Lee

Abstract: Motivated by a computer experiment for the design of a rocket booster, this paper explores nonstationary modeling methodologies that couple stationary Gaussian processes with treed partitioning. Partitioning is a simple but effective method for dealing with nonstationarity. The methodological developments and statistical computing details which make this approach efficient are described in detai… ▽ More Motivated by a computer experiment for the design of a rocket booster, this paper explores nonstationary modeling methodologies that couple stationary Gaussian processes with treed partitioning. Partitioning is a simple but effective method for dealing with nonstationarity. The methodological developments and statistical computing details which make this approach efficient are described in detail. In addition to providing an analysis of the rocket booster simulator, our approach is demonstrated to be effective in other arenas. △ Less

Submitted 17 March, 2009; v1 submitted 24 October, 2007; originally announced October 2007.

Comments: 32 pages, 9 figures, to appear in the Journal of the American Statistical Association