Showing 1–48 of 48 results for author: Kosorok, M R

Linear Regression Using Hilbert-Space-Valued Covariates with Unknown Reproducing Kernel

Authors: Xinyi Li, Margaret Hoch, Michael R. Kosorok

Abstract: We present a new method of linear regression based on principal components using Hilbert-space-valued covariates with unknown reproducing kernels. We develop a computationally efficient approach to estimation and derive asymptotic theory for the regression parameter estimates under mild assumptions. We demonstrate the approach in simulation studies as well as in data analysis using two-dimensional… ▽ More We present a new method of linear regression based on principal components using Hilbert-space-valued covariates with unknown reproducing kernels. We develop a computationally efficient approach to estimation and derive asymptotic theory for the regression parameter estimates under mild assumptions. We demonstrate the approach in simulation studies as well as in data analysis using two-dimensional brain images as predictors. △ Less

Submitted 23 April, 2025; originally announced April 2025.

Statistical Design and Rationale of the Biomarkers for Evaluating Spine Treatments (BEST) Trial

Authors: John Sperger, Kelley M. Kidwell, Matthew C. Mauck, Beibo Zhao, Kevin J. Anstrom, Anna Batorsky, Timothy S. Carey, Daniel J. Clauw, Nikki L. B. Freeman, Carol M. Greco, Anastasia Ivanova, Sara Jones Berkeley, Samuel A. McLean, Matthew A. Psioda, Bryce Rowland, Gwendolyn A. Sowa, Ajay D. Wasan, Joshua P Zitovsky, Michael R. Kosorok

Abstract: Chronic low back pain (cLBP) is a prevalent condition with profound impacts on functioning and quality of life. While multiple evidence-based treatments exist, they all have modest average treatment effects$\unicode{x2013}$potentially due to individual variation in treatment response and the diverse etiologies of cLBP. This multi-site sequential, multiple-assignment randomized trial (SMART) invest… ▽ More Chronic low back pain (cLBP) is a prevalent condition with profound impacts on functioning and quality of life. While multiple evidence-based treatments exist, they all have modest average treatment effects$\unicode{x2013}$potentially due to individual variation in treatment response and the diverse etiologies of cLBP. This multi-site sequential, multiple-assignment randomized trial (SMART) investigated four treatment modalities with two stages of randomization and aimed to enroll 630 protocol completers. The primary objective was to develop a precision medicine approach by estimating optimal treatment or treatment combinations based on patient characteristics and initial treatment response. The analysis strategy focuses on estimating interpretable dynamic treatment regimes and identifying subgroups most responsive to specific interventions. Broad eligibility criteria were implemented to enhance generalizability and recruitment, most notably that participants could be eligible to enroll even if they could not be assigned to one (but no more) of the study interventions. Enrolling participants with restrictions on the treatment they could be assigned necessitated modifications to standard minimization methods for balancing covariates. The BEST trial represents one of the largest SMARTs focused on clinical decision-making to date and the largest in cLBP. By collecting an extensive array of biomarker and phenotypic measures, this trial may identify potential treatment mechanisms and establish a more evidence-based approach to individualizing cLBP treatment in clinical practice. △ Less

Submitted 24 March, 2025; originally announced March 2025.

Comments: 18 pages, 3 figures, 1 algorithm, 1 table

An optimal dynamic treatment regime estimator for indefinite-horizon survival outcomes

Authors: Jane She, Matthew Egberg, Michael R. Kosorok

Abstract: We propose a new method in indefinite-horizon settings for estimating optimal dynamic treatment regimes for time-to-event outcomes. This method allows patients to have different numbers of treatment stages and is constructed using generalized survival random forests to maximize mean survival time. We use summarized history and data pooling, preventing data from growing in dimension as a patient's… ▽ More We propose a new method in indefinite-horizon settings for estimating optimal dynamic treatment regimes for time-to-event outcomes. This method allows patients to have different numbers of treatment stages and is constructed using generalized survival random forests to maximize mean survival time. We use summarized history and data pooling, preventing data from growing in dimension as a patient's decision points increase. The algorithm operates through model re-fitting, resulting in a single model optimized for all patients and all stages. We derive theoretical properties of the estimator such as consistency of the estimator and value function and characterize the number of refitting iterations needed. We also conduct a simulation study of patients with a flexible number of treatment stages to examine finite-sample performance of the estimator. Finally, we illustrate use of the algorithm using administrative insurance claims data for pediatric Crohn's disease patients. △ Less

Submitted 29 January, 2025; originally announced January 2025.

Comments: 24 pages, 7 figures

Using Statistical Precision Medicine to Identify Optimal Treatments in a Heart Failure Setting

Authors: Arti Virkud, Jessie K. Edwards, Michele Jonsson Funk, Patricia Chang, Abhijit V. Kshirsagar, Emily W. Gower, Michael R. Kosorok

Abstract: Identifying optimal medical treatments to improve survival has long been a critical goal of pharmacoepidemiology. Traditionally, we use an average treatment effect measure to compare outcomes between treatment plans. However, new methods leveraging advantages of machine learning combined with the foundational tenets of causal inference are offering an alternative to the average treatment effect. H… ▽ More Identifying optimal medical treatments to improve survival has long been a critical goal of pharmacoepidemiology. Traditionally, we use an average treatment effect measure to compare outcomes between treatment plans. However, new methods leveraging advantages of machine learning combined with the foundational tenets of causal inference are offering an alternative to the average treatment effect. Here, we use three unique, precision medicine algorithms (random forests, residual weighted learning, efficient augmentation relaxed learning) to identify optimal treatment rules where patients receive the optimal treatment as indicated by their clinical history. First, we present a simple hypothetical example and a real-world application among heart failure patients using Medicare claims data. We next demonstrate how the optimal treatment rule improves the absolute risk in a hypothetical, three-modifier setting. Finally, we identify an optimal treatment rule that optimizes the time to outcome in a real-world heart failure setting. In both examples, we compare the average time to death under the optimized, tailored treatment rule with the average time to death under a universal treatment rule to show the benefit of precision medicine methods. The improvement under the optimal treatment rule in the real-world setting is greatest (additional ~9 days under the tailored rule) for survival time free of heart failure readmission. △ Less

Submitted 13 January, 2025; originally announced January 2025.

Optimal individualized treatment regimes for survival data with competing risks

Authors: Christina W. Zhou, Nikki L. B. Freeman, Katharine L. McGinigle, Michael R. Kosorok

Abstract: Precision medicine leverages patient heterogeneity to estimate individualized treatment regimens, formalized, data-driven approaches designed to match patients with optimal treatments. In the presence of competing events, where multiple causes of failure can occur and one cause precludes others, it is crucial to assess the risk of the specific outcome of interest, such as one type of failure over… ▽ More Precision medicine leverages patient heterogeneity to estimate individualized treatment regimens, formalized, data-driven approaches designed to match patients with optimal treatments. In the presence of competing events, where multiple causes of failure can occur and one cause precludes others, it is crucial to assess the risk of the specific outcome of interest, such as one type of failure over another. This helps clinicians tailor interventions based on the factors driving that particular cause, leading to more precise treatment strategies. Currently, no precision medicine methods simultaneously account for both survival and competing risk endpoints. To address this gap, we develop a nonparametric individualized treatment regime estimator. Our two-phase method accounts for both overall survival from all events as well as the cumulative incidence of a main event of interest. Additionally, we introduce a multi-utility value function that incorporates both outcomes. We develop random survival and random cumulative incidence forests to construct individual survival and cumulative incidence curves. Simulation studies demonstrated that our proposed method performs well, which we applied to a cohort of peripheral artery disease patients at high risk for limb loss and mortality. △ Less

Submitted 12 November, 2024; originally announced November 2024.

Comments: 22 pages, 4 figures

Uncertainty quantification for intervals

Authors: Carlos García Meixide, Michael R. Kosorok, Marcos Matabuena

Abstract: Data following an interval structure are increasingly prevalent in many scientific applications. In medicine, clinical events are often monitored between two clinical visits, making the exact time of the event unknown and generating outcomes with a range format. As interest in automating healthcare decisions grows, uncertainty quantification via predictive regions becomes essential for developing… ▽ More Data following an interval structure are increasingly prevalent in many scientific applications. In medicine, clinical events are often monitored between two clinical visits, making the exact time of the event unknown and generating outcomes with a range format. As interest in automating healthcare decisions grows, uncertainty quantification via predictive regions becomes essential for developing reliable and trustworthy predictive algorithms. However, the statistical literature currently lacks a general methodology for interval targets, especially when these outcomes are incomplete due to censoring. We propose an uncertainty quantification algorithm for interval responses and establish its theoretical properties using empirical process arguments based on a newly developed class of functions specifically designed for these interval data structures. Although this paper primarily focuses on deriving predictive regions for interval-censored data, the approach can also be applied to other statistical modeling tasks, such as goodness-of-fit assessments. Finally, the applicability of the method is demonstrated through simulations, showing up to a 60\% improvement in conditional coverage. Our new algorithm is also applied to various biomedical contexts, including two clinical examples: i) sleep duration and its association with cardiovascular diseases, and ii) survival time in relation to physical activity levels. △ Less

Submitted 30 March, 2025; v1 submitted 29 August, 2024; originally announced August 2024.

Effects Among the Affected

Authors: Lina M. Montoya, Elvin H. Geng, Michael Valancius, Michael R. Kosorok, Maya L. Petersen

Abstract: Many interventions are both beneficial to initiate and harmful to stop. Traditionally, to determine whether to deploy that intervention in a time-limited way depends on if, on average, the increase in the benefits of starting it outweigh the increase in the harms of stopping it. We propose a novel causal estimand that provides a more nuanced understanding of the effects of such treatments, particu… ▽ More Many interventions are both beneficial to initiate and harmful to stop. Traditionally, to determine whether to deploy that intervention in a time-limited way depends on if, on average, the increase in the benefits of starting it outweigh the increase in the harms of stopping it. We propose a novel causal estimand that provides a more nuanced understanding of the effects of such treatments, particularly, how response to an earlier treatment (e.g., treatment initiation) modifies the effect of a later treatment (e.g., treatment discontinuation), thus learning if there are effects among the (un)affected. Specifically, we consider a marginal structural working model summarizing how the average effect of a later treatment varies as a function of the (estimated) conditional average effect of an earlier treatment. We allow for estimation of this conditional average treatment effect using machine learning, such that the causal estimand is a data-adaptive parameter. We show how a sequentially randomized design can be used to identify this causal estimand, and we describe a targeted maximum likelihood estimator for the resulting statistical estimand, with influence curve-based inference. Throughout, we use the Adaptive Strategies for Preventing and Treating Lapses of Retention in HIV Care trial (NCT02338739) as an illustrative example, showing that discontinuation of conditional cash transfers for HIV care adherence was most harmful among those who most had an increase in benefits from them initially. △ Less

Submitted 26 August, 2024; originally announced August 2024.

Off-Policy Reinforcement Learning with High Dimensional Reward

Authors: Dong Neuck Lee, Michael R. Kosorok

Abstract: Conventional off-policy reinforcement learning (RL) focuses on maximizing the expected return of scalar rewards. Distributional RL (DRL), in contrast, studies the distribution of returns with the distributional Bellman operator in a Euclidean space, leading to highly flexible choices for utility. This paper establishes robust theoretical foundations for DRL. We prove the contraction property of th… ▽ More Conventional off-policy reinforcement learning (RL) focuses on maximizing the expected return of scalar rewards. Distributional RL (DRL), in contrast, studies the distribution of returns with the distributional Bellman operator in a Euclidean space, leading to highly flexible choices for utility. This paper establishes robust theoretical foundations for DRL. We prove the contraction property of the Bellman operator even when the reward space is an infinite-dimensional separable Banach space. Furthermore, we demonstrate that the behavior of high- or infinite-dimensional returns can be effectively approximated using a lower-dimensional Euclidean space. Leveraging these theoretical insights, we propose a novel DRL algorithm that tackles problems which have been previously intractable using conventional reinforcement learning approaches. △ Less

Submitted 14 August, 2024; originally announced August 2024.

Comments: 24 pages, 12 figures

MSC Class: 68T05; 46B09 (Primary) 46B06 (Secondary)

Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes

Authors: Sophia Yazzourh, Nicolas Savy, Philippe Saint-Pierre, Michael R. Kosorok

Abstract: The goal of precision medicine is to provide individualized treatment at each stage of chronic diseases, a concept formalized by Dynamic Treatment Regimes (DTR). These regimes adapt treatment strategies based on decision rules learned from clinical data to enhance therapeutic effectiveness. Reinforcement Learning (RL) algorithms allow to determine these decision rules conditioned by individual pat… ▽ More The goal of precision medicine is to provide individualized treatment at each stage of chronic diseases, a concept formalized by Dynamic Treatment Regimes (DTR). These regimes adapt treatment strategies based on decision rules learned from clinical data to enhance therapeutic effectiveness. Reinforcement Learning (RL) algorithms allow to determine these decision rules conditioned by individual patient data and their medical history. The integration of medical expertise into these models makes possible to increase confidence in treatment recommendations and facilitate the adoption of this approach by healthcare professionals and patients. In this work, we examine the mathematical foundations of RL, contextualize its application in the field of DTR, and present an overview of methods to improve its effectiveness by integrating medical expertise. △ Less

Submitted 29 June, 2024; originally announced July 2024.

Constructing a T-test for Value Function Comparison of Individualized Treatment Regimes in the Presence of Multiple Imputation for Missing Data

Authors: Minxin Lu, Annie Green Howard, Penny Gordon-Larsen, Katie A. Meyer, Hsiao-Chuan Tien, Shufa Du, Huijun Wang, Bing Zhang, Michael R. Kosorok

Abstract: Optimal individualized treatment decision-making has improved health outcomes in recent years. The value function is commonly used to evaluate the goodness of an individualized treatment decision rule. Despite recent advances, comparing value functions between different treatment decision rules or constructing confidence intervals around value functions remains difficult. We propose a t-test based… ▽ More Optimal individualized treatment decision-making has improved health outcomes in recent years. The value function is commonly used to evaluate the goodness of an individualized treatment decision rule. Despite recent advances, comparing value functions between different treatment decision rules or constructing confidence intervals around value functions remains difficult. We propose a t-test based method applied to a test set that generates valid p-values to compare value functions between a given pair of treatment decision rules when some of the data are missing. We demonstrate the ease in use of this method and evaluate its performance via simulation studies and apply it to the China Health and Nutrition Survey data. △ Less

Submitted 19 April, 2025; v1 submitted 23 December, 2023; originally announced December 2023.

Stochastic Step-wise Feature Selection for Exponential Random Graph Models (ERGMs)

Authors: Helal El-Zaatari, Fei Yu, Michael R Kosorok

Abstract: Statistical analysis of social networks provides valuable insights into complex network interactions across various scientific disciplines. However, accurate modeling of networks remains challenging due to the heavy computational burden and the need to account for observed network dependencies. Exponential Random Graph Models (ERGMs) have emerged as a promising technique used in social network mod… ▽ More Statistical analysis of social networks provides valuable insights into complex network interactions across various scientific disciplines. However, accurate modeling of networks remains challenging due to the heavy computational burden and the need to account for observed network dependencies. Exponential Random Graph Models (ERGMs) have emerged as a promising technique used in social network modeling to capture network dependencies by incorporating endogenous variables. Nevertheless, using ERGMs poses multiple challenges, including the occurrence of ERGM degeneracy, which generates unrealistic and meaningless network structures. To address these challenges and enhance the modeling of collaboration networks, we propose and test a novel approach that focuses on endogenous variable selection within ERGMs. Our method aims to overcome the computational burden and improve the accommodation of observed network dependencies, thereby facilitating more accurate and meaningful interpretations of network phenomena in various scientific fields. We conduct empirical testing and rigorous analysis to contribute to the advancement of statistical techniques and offer practical insights for network analysis. △ Less

Submitted 24 July, 2023; originally announced July 2023.

Comments: 23 pages, 6 tables and 18 figures

A Flexible Framework for Incorporating Patient Preferences Into Q-Learning

Authors: Joshua P. Zitovsky, Leslie Wilson, Michael R. Kosorok

Abstract: In real-world healthcare problems, there are often multiple competing outcomes of interest, such as treatment efficacy and side effect severity. However, statistical methods for estimating dynamic treatment regimes (DTRs) usually assume a single outcome of interest, and the few methods that deal with composite outcomes suffer from important limitations. This includes restrictions to a single time… ▽ More In real-world healthcare problems, there are often multiple competing outcomes of interest, such as treatment efficacy and side effect severity. However, statistical methods for estimating dynamic treatment regimes (DTRs) usually assume a single outcome of interest, and the few methods that deal with composite outcomes suffer from important limitations. This includes restrictions to a single time point and two outcomes, the inability to incorporate self-reported patient preferences and limited theoretical guarantees. To this end, we propose a new method to address these limitations, which we dub Latent Utility Q-Learning (LUQ-Learning). LUQ-Learning uses a latent model approach to naturally extend Q-learning to the composite outcome setting and adopt the ideal trade-off between outcomes to each patient. Unlike previous approaches, our framework allows for an arbitrary number of time points and outcomes, incorporates stated preferences and achieves strong asymptotic performance with realistic assumptions on the data. We conduct simulation experiments based on an ongoing trial for low back pain as well as a well-known completed trial for schizophrenia. In all experiments, our method achieves highly competitive empirical performance compared to several alternative baselines. △ Less

Submitted 22 July, 2023; originally announced July 2023.

Comments: Under Review

A Causal Inference Framework for Leveraging External Controls in Hybrid Trials

Authors: Michael Valancius, Herb Pang, Jiawen Zhu, Stephen R Cole, Michele Jonsson Funk, Michael R Kosorok

Abstract: We consider the challenges associated with causal inference in settings where data from a randomized trial is augmented with control data from an external source to improve efficiency in estimating the average treatment effect (ATE). Through the development of a formal causal inference framework, we outline sufficient causal assumptions about the exchangeability between the internal and external c… ▽ More We consider the challenges associated with causal inference in settings where data from a randomized trial is augmented with control data from an external source to improve efficiency in estimating the average treatment effect (ATE). Through the development of a formal causal inference framework, we outline sufficient causal assumptions about the exchangeability between the internal and external controls to identify the ATE and establish the connection to a novel graphical criteria. We propose estimators, review efficiency bounds, develop an approach for efficient doubly-robust estimation even when unknown nuisance models are estimated with flexible machine learning methods, and demonstrate finite-sample performance through a simulation study. To illustrate the ideas and methods, we apply the framework to a trial investigating the effect of risdisplam on motor function in patients with spinal muscular atrophy for which there exists an external set of control patients from a previous trial. △ Less

Submitted 15 May, 2023; originally announced May 2023.

Functional Individualized Treatment Regimes with Imaging Features

Authors: Xinyi Li, Michael R. Kosorok

Abstract: Precision medicine seeks to discover an optimal personalized treatment plan and thereby provide informed and principled decision support, based on the characteristics of individual patients. With recent advancements in medical imaging, it is crucial to incorporate patient-specific imaging features in the study of individualized treatment regimes. We propose a novel, data-driven method to construct… ▽ More Precision medicine seeks to discover an optimal personalized treatment plan and thereby provide informed and principled decision support, based on the characteristics of individual patients. With recent advancements in medical imaging, it is crucial to incorporate patient-specific imaging features in the study of individualized treatment regimes. We propose a novel, data-driven method to construct interpretable image features which can be incorporated, along with other features, to guide optimal treatment regimes. The proposed method treats imaging information as a realization of a stochastic process, and employs smoothing techniques in estimation. We show that the proposed estimators are consistent under mild conditions. The proposed method is applied to a dataset provided by the Alzheimer's Disease Neuroimaging Initiative. △ Less

Submitted 25 April, 2023; originally announced April 2023.

Revisiting Bellman Errors for Offline Model Selection

Authors: Joshua P. Zitovsky, Daniel de Marchi, Rishabh Agarwal, Michael R. Kosorok

Abstract: Offline model selection (OMS), that is, choosing the best policy from a set of many policies given only logged data, is crucial for applying offline RL in real-world settings. One idea that has been extensively explored is to select policies based on the mean squared Bellman error (MSBE) of the associated Q-functions. However, previous work has struggled to obtain adequate OMS performance with Bel… ▽ More Offline model selection (OMS), that is, choosing the best policy from a set of many policies given only logged data, is crucial for applying offline RL in real-world settings. One idea that has been extensively explored is to select policies based on the mean squared Bellman error (MSBE) of the associated Q-functions. However, previous work has struggled to obtain adequate OMS performance with Bellman errors, leading many researchers to abandon the idea. To this end, we elucidate why previous work has seen pessimistic results with Bellman errors and identify conditions under which OMS algorithms based on Bellman errors will perform well. Moreover, we develop a new estimator of the MSBE that is more accurate than prior methods. Our estimator obtains impressive OMS performance on diverse discrete control tasks, including Atari games. △ Less

Submitted 6 June, 2023; v1 submitted 31 January, 2023; originally announced February 2023.

Comments: Published in ICML 2023

ACM Class: I.2.8; I.6.4

Journal ref: In ICML (pp. 43369-43406). PMLR (2023)

Dynamic treatment regime characterization via value function surrogate with an application to partial compliance

Authors: Nikki L. B. Freeman, Sydney E. Browder, Katharine L. McGinigle, Michael R. Kosorok

Abstract: Precision medicine is a promising framework for generating evidence to improve health and health care. Yet, a gap persists between the ever-growing number of statistical precision medicine strategies for evidence generation and implementation in real world clinical settings, and the strategies for closing this gap will likely be context dependent. In this paper, we consider the specific context of… ▽ More Precision medicine is a promising framework for generating evidence to improve health and health care. Yet, a gap persists between the ever-growing number of statistical precision medicine strategies for evidence generation and implementation in real world clinical settings, and the strategies for closing this gap will likely be context dependent. In this paper, we consider the specific context of partial compliance to wound management among patients with peripheral artery disease. Through the use of a Gaussian process surrogate for the value function, we expand beyond the common precision medicine task of learning an optimal dynamic treatment regime to characterization of classes of dynamic treatment regimes and how those findings can be translated into clinical contexts. △ Less

Submitted 1 December, 2022; originally announced December 2022.

Comments: 18 pages, 7 figures

Efficient and Robust Approaches for Analysis of SMARTs: Illustration using the ADAPT-R Trial

Authors: Lina M. Montoya, Michael R. Kosorok, Elvin H. Geng, Joshua Schwab, Thomas A. Odeny, Maya L. Petersen

Abstract: Personalized intervention strategies, in particular those that modify treatment based on a participant's own response, are a core component of precision medicine approaches. Sequential Multiple Assignment Randomized Trials (SMARTs) are growing in popularity and are specifically designed to facilitate the evaluation of sequential adaptive strategies, in particular those embedded within the SMART. A… ▽ More Personalized intervention strategies, in particular those that modify treatment based on a participant's own response, are a core component of precision medicine approaches. Sequential Multiple Assignment Randomized Trials (SMARTs) are growing in popularity and are specifically designed to facilitate the evaluation of sequential adaptive strategies, in particular those embedded within the SMART. Advances in efficient estimation approaches that are able to incorporate machine learning while retaining valid inference can allow for more precise estimates of the effectiveness of these embedded regimes. However, to the best of our knowledge, such approaches have not yet been applied as the primary analysis in SMART trials. In this paper, we present a robust and efficient approach using Targeted Maximum Likelihood Estimation (TMLE) for estimating and contrasting expected outcomes under the dynamic regimes embedded in a SMART, together with generating simultaneous confidence intervals for the resulting estimates. We contrast this method with two alternatives (G-computation and Inverse Probability Weighting estimators). The precision gains and robust inference achievable through the use of TMLE to evaluate the effects of embedded regimes are illustrated using both outcome-blind simulations and a real data analysis from the Adaptive Strategies for Preventing and Treating Lapses of Retention in HIV Care (ADAPT-R) trial (NCT02338739), a SMART with a primary aim of identifying strategies to improve retention in HIV care among people living with HIV in sub-Saharan Africa. △ Less

Submitted 7 October, 2022; originally announced October 2022.

Offline Reinforcement Learning with Instrumental Variables in Confounded Markov Decision Processes

Authors: Zuyue Fu, Zhengling Qi, Zhaoran Wang, Zhuoran Yang, Yanxun Xu, Michael R. Kosorok

Abstract: We study the offline reinforcement learning (RL) in the face of unmeasured confounders. Due to the lack of online interaction with the environment, offline RL is facing the following two significant challenges: (i) the agent may be confounded by the unobserved state variables; (ii) the offline data collected a prior does not provide sufficient coverage for the environment. To tackle the above chal… ▽ More We study the offline reinforcement learning (RL) in the face of unmeasured confounders. Due to the lack of online interaction with the environment, offline RL is facing the following two significant challenges: (i) the agent may be confounded by the unobserved state variables; (ii) the offline data collected a prior does not provide sufficient coverage for the environment. To tackle the above challenges, we study the policy learning in the confounded MDPs with the aid of instrumental variables. Specifically, we first establish value function (VF)-based and marginalized importance sampling (MIS)-based identification results for the expected total reward in the confounded MDPs. Then by leveraging pessimism and our identification results, we propose various policy learning methods with the finite-sample suboptimality guarantee of finding the optimal in-class policy under minimal data coverage and modeling assumptions. Lastly, our extensive theoretical investigations and one numerical study motivated by the kidney transplantation demonstrate the promising performance of the proposed methods. △ Less

Submitted 18 September, 2022; originally announced September 2022.

Risk-Adjusted Incidence Modeling on Hierarchical Survival Data with Recurrent Events

Authors: Xiaotong Jiang, William Stoudemire, Marianne S. Muhlebach, Michael R. Kosorok

Abstract: There is a constant need for many healthcare programs to timely address problems with infection prevention and control (IP&C). For example, pathogens can be transmitted among patients with cystic fibrosis (CF) in both the inpatient and outpatient settings within the healthcare system even with the existing recommended IP&C practices, and these pathogens are often associated with negative clinical… ▽ More There is a constant need for many healthcare programs to timely address problems with infection prevention and control (IP&C). For example, pathogens can be transmitted among patients with cystic fibrosis (CF) in both the inpatient and outpatient settings within the healthcare system even with the existing recommended IP&C practices, and these pathogens are often associated with negative clinical outcomes. Because of limited and delayed data sharing, CF programs need a reliable method to track infection rates. There are three complex structures in CF registry data: recurrent infections, missing data, and multilevel correlation due to repeated measures within a patient and patient-to-patient transmissions. A step-by-step analysis pipeline was proposed to develop and validate a risk-adjusted model to help healthcare programs monitor the number of recurrent events while taking into account missing data and the hierarchies of repeated measures in right-censored data. We extended the mixed-effect Andersen-Gill model (the frailty model), adjusted for important risk factors, and provided confidence intervals for the predicted number of events where the variability of the prediction was estimated from three identified sources. The coverage of the estimated confidence intervals was used to evaluate model performance. Simulation results indicated that the coverage of our method was close to the desired confidence level. To demonstrate its clinical practicality, our pipeline was applied to monitor the infection incidence rate of two key CF pathogens using a U.S. registry. Results showed that years closer to the time of interest were better at predicting future incidence rates in the CF example. △ Less

Submitted 26 July, 2022; originally announced July 2022.

Neural interval-censored survival regression with feature selection

Authors: Carlos García Meixide, Marcos Matabuena, Louis Abraham, Michael R. Kosorok

Abstract: Survival analysis is a fundamental area of focus in biomedical research, particularly in the context of personalized medicine. This prominence is due to the increasing prevalence of large and high-dimensional datasets, such as omics and medical image data. However, the literature on non-linear regression algorithms and variable selection techniques for interval-censoring is either limited or non-e… ▽ More Survival analysis is a fundamental area of focus in biomedical research, particularly in the context of personalized medicine. This prominence is due to the increasing prevalence of large and high-dimensional datasets, such as omics and medical image data. However, the literature on non-linear regression algorithms and variable selection techniques for interval-censoring is either limited or non-existent, particularly in the context of neural networks. Our objective is to introduce a novel predictive framework tailored for interval-censored regression tasks, rooted in Accelerated Failure Time (AFT) models. Our strategy comprises two key components: i) a variable selection phase leveraging recent advances on sparse neural network architectures, ii) a regression model targeting prediction of the interval-censored response. To assess the performance of our novel algorithm, we conducted a comprehensive evaluation through both numerical experiments and real-world applications that encompass scenarios related to diabetes and physical activity. Our results outperform traditional AFT algorithms, particularly in scenarios featuring non-linear relationships. △ Less

Submitted 22 August, 2024; v1 submitted 14 June, 2022; originally announced June 2022.

Journal ref: Statistical Analysis and Data Mining: The ASA Data Science Journal 17.4 (2024):

Discussion of Multiscale Fisher's Independence Test for Multivariate Dependence

Authors: Duyeol Lee, Helal El-Zaatari, Michael R. Kosorok, Xinyi Li, Kai Zhang

Abstract: The multiscale Fisher's independence test (MULTIFIT hereafter) proposed by Gorsky & Ma (2022) is a novel method to test independence between two random vectors. By its design, this test is particularly useful in detecting local dependence. Moreover, by adopting a resampling-free approach, it can easily accommodate massive sample sizes. Another benefit of the proposed method is its ability to inter… ▽ More The multiscale Fisher's independence test (MULTIFIT hereafter) proposed by Gorsky & Ma (2022) is a novel method to test independence between two random vectors. By its design, this test is particularly useful in detecting local dependence. Moreover, by adopting a resampling-free approach, it can easily accommodate massive sample sizes. Another benefit of the proposed method is its ability to interpret the nature of dependency. We congratulate the authors, Shai Gorksy and Li Ma, for their very interesting and elegant work. In this comment, we would like to discuss a general framework unifying the MULTIFIT and other tests and compare it with the binary expansion randomized ensemble test (BERET hereafter) proposed by Lee et al. (In press). We also would like to contribute our thoughts on potential extensions of the method. △ Less

Submitted 26 April, 2022; originally announced April 2022.

Stabilized Direct Learning for Efficient Estimation of Individualized Treatment Rules

Authors: Kushal S. Shah, Haoda Fu, Michael R. Kosorok

Abstract: In recent years, the field of precision medicine has seen many advancements. Significant focus has been placed on creating algorithms to estimate individualized treatment rules (ITR), which map from patient covariates to the space of available treatments with the goal of maximizing patient outcome. Direct Learning (D-Learning) is a recent one-step method which estimates the ITR by directly modelin… ▽ More In recent years, the field of precision medicine has seen many advancements. Significant focus has been placed on creating algorithms to estimate individualized treatment rules (ITR), which map from patient covariates to the space of available treatments with the goal of maximizing patient outcome. Direct Learning (D-Learning) is a recent one-step method which estimates the ITR by directly modeling the treatment-covariate interaction. However, when the variance of the outcome is heterogeneous with respect to treatment and covariates, D-Learning does not leverage this structure. Stabilized Direct Learning (SD-Learning), proposed in this paper, utilizes potential heteroscedasticity in the error term through a residual reweighting which models the residual variance via flexible machine learning algorithms such as XGBoost and random forests. We also develop an internal cross-validation scheme which determines the best residual model amongst competing models. SD-Learning improves the efficiency of D-Learning estimates in binary and multi-arm treatment scenarios. The method is simple to implement and an easy way to improve existing algorithms within the D-Learning family, including original D-Learning, Angle-based D-Learning (AD-Learning), and Robust D-Learning (RD-Learning). We provide theoretical properties and justification of the optimality of SD-Learning. Head-to-head performance comparisons with D-Learning methods are provided through simulations, which demonstrate improvement in terms of average prediction error (APE), misclassification rate, and empirical value, along with data analysis of an AIDS randomized clinical trial. △ Less

Submitted 7 December, 2021; originally announced December 2021.

Multi-stage optimal dynamic treatment regimes for survival outcomes with dependent censoring

Authors: Hunyong Cho, Shannon T. Holloway, David J. Couper, Michael R. Kosorok

Abstract: We propose a reinforcement learning method for estimating an optimal dynamic treatment regime for survival outcomes with dependent censoring. The estimator allows the failure time to be conditionally independent of censoring and dependent on the treatment decision times, supports a flexible number of treatment arms and treatment stages, and can maximize either the mean survival time or the surviva… ▽ More We propose a reinforcement learning method for estimating an optimal dynamic treatment regime for survival outcomes with dependent censoring. The estimator allows the failure time to be conditionally independent of censoring and dependent on the treatment decision times, supports a flexible number of treatment arms and treatment stages, and can maximize either the mean survival time or the survival probability at a certain time point. The estimator is constructed using generalized random survival forests and can have polynomial rates of convergence. Simulations and data analysis results suggest that the new estimator brings higher expected outcomes than existing methods in various settings. An R package dtrSurv is available on CRAN. △ Less

Submitted 12 May, 2022; v1 submitted 6 December, 2020; originally announced December 2020.

Kernel Assisted Learning for Personalized Dose Finding

Authors: Liangyu Zhu, Wenbin Lu, Michael R. Kosorok, Rui Song

Abstract: An individualized dose rule recommends a dose level within a continuous safe dose range based on patient level information such as physical conditions, genetic factors and medication histories. Traditionally, personalized dose finding process requires repeating clinical visits of the patient and frequent adjustments of the dosage. Thus the patient is constantly exposed to the risk of underdosing a… ▽ More An individualized dose rule recommends a dose level within a continuous safe dose range based on patient level information such as physical conditions, genetic factors and medication histories. Traditionally, personalized dose finding process requires repeating clinical visits of the patient and frequent adjustments of the dosage. Thus the patient is constantly exposed to the risk of underdosing and overdosing during the process. Statistical methods for finding an optimal individualized dose rule can lower the costs and risks for patients. In this article, we propose a kernel assisted learning method for estimating the optimal individualized dose rule. The proposed methodology can also be applied to all other continuous decision-making problems. Advantages of the proposed method include robustness to model misspecification and capability of providing statistical inference for the estimated parameters. In the simulation studies, we show that this method is capable of identifying the optimal individualized dose rule and produces favorable expected outcomes in the population. Finally, we illustrate our approach using data from a warfarin dosing study for thrombosis patients. △ Less

Submitted 19 July, 2020; originally announced July 2020.

Comments: Accepted for KDD 2020

Missing Data Imputation for Classification Problems

Authors: Arkopal Choudhury, Michael R. Kosorok

Abstract: Imputation of missing data is a common application in various classification problems where the feature training matrix has missingness. A widely used solution to this imputation problem is based on the lazy learning technique, $k$-nearest neighbor (kNN) approach. However, most of the previous work on missing data does not take into account the presence of the class label in the classification pro… ▽ More Imputation of missing data is a common application in various classification problems where the feature training matrix has missingness. A widely used solution to this imputation problem is based on the lazy learning technique, $k$-nearest neighbor (kNN) approach. However, most of the previous work on missing data does not take into account the presence of the class label in the classification problem. Also, existing kNN imputation methods use variants of Minkowski distance as a measure of distance, which does not work well with heterogeneous data. In this paper, we propose a novel iterative kNN imputation technique based on class weighted grey distance between the missing datum and all the training data. Grey distance works well in heterogeneous data with missing instances. The distance is weighted by Mutual Information (MI) which is a measure of feature relevance between the features and the class label. This ensures that the imputation of the training data is directed towards improving classification performance. This class weighted grey kNN imputation algorithm demonstrates improved performance when compared to other kNN imputation algorithms, as well as standard imputation algorithms such as MICE and missForest, in imputation and classification problems. These problems are based on simulated scenarios and UCI datasets with various rates of missingness. △ Less

Submitted 25 February, 2020; originally announced February 2020.

Comments: 27 pages, 5 figures

Technical Background for "A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis"

Authors: Xiaotong Jiang, Amanda E. Nelson, Rebecca J. Cleveland, Daniel P. Beavers, Todd A. Schwartz, Liubov Arbeeva, Carolina Alvarez, Leigh F. Callahan, Stephen Messier, Richard Loeser, Michael R. Kosorok

Abstract: We provide additional statistical background for the methodology developed in the clinical analysis of knee osteoarthritis in "A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis" (Jiang et al. 2020). Jiang et al. 2020 proposed a pipeline to learn optimal treatment rules with precision… ▽ More We provide additional statistical background for the methodology developed in the clinical analysis of knee osteoarthritis in "A Precision Medicine Approach to Develop and Internally Validate Optimal Exercise and Weight Loss Treatments for Overweight and Obese Adults with Knee Osteoarthritis" (Jiang et al. 2020). Jiang et al. 2020 proposed a pipeline to learn optimal treatment rules with precision medicine models and compared them with zero-order models with a Z-test. The model performance was based on value functions, a scalar that predicts the future reward of each decision rule. The jackknife (i.e., leave-one-out cross validation) method was applied to estimate the value function and its variance of several outcomes in IDEA. IDEA is a randomized clinical trial studying three interventions (exercise (E), dietary weight loss (D), and D+E) on overweight and obese participants with knee osteoarthritis. In this report, we expand the discussion and justification with additional statistical background. We elaborate more on the background of precision medicine, the derivation of the jackknife estimator of value function and its estimated variance, the consistency property of jackknife estimator, as well as additional simulation results that reflect more of the performance of jackknife estimators. We recommend reading Jiang et al. 2020 for clinical application and interpretation of the optimal ITR of knee osteoarthritis as well as the overall understanding of the pipeline and recommend using this article to understand the underlying statistical derivation and methodology. △ Less

Submitted 20 February, 2020; v1 submitted 27 January, 2020; originally announced January 2020.

Estimating heterogeneous treatment effects with right-censored data via causal survival forests

Authors: Yifan Cui, Michael R. Kosorok, Erik Sverdrup, Stefan Wager, Ruoqing Zhu

Abstract: Forest-based methods have recently gained in popularity for non-parametric treatment effect estimation. Building on this line of work, we introduce causal survival forests, which can be used to estimate heterogeneous treatment effects in a survival and observational setting where outcomes may be right-censored. Our approach relies on orthogonal estimating equations to robustly adjust for both cens… ▽ More Forest-based methods have recently gained in popularity for non-parametric treatment effect estimation. Building on this line of work, we introduce causal survival forests, which can be used to estimate heterogeneous treatment effects in a survival and observational setting where outcomes may be right-censored. Our approach relies on orthogonal estimating equations to robustly adjust for both censoring and selection effects under unconfoundedness. In our experiments, we find our approach to perform well relative to a number of baselines. △ Less

Submitted 28 February, 2023; v1 submitted 27 January, 2020; originally announced January 2020.

Comments: To appear in the Journal of the Royal Statistical Society, Series B

MSC Class: 62N01

Interval censored recursive forests

Authors: Hunyong Cho, Nicholas P. Jewell, Michael R. Kosorok

Abstract: We propose the interval censored recursive forests (ICRF) which is an iterative tree ensemble method for interval censored survival data. This nonparametric regression estimator makes the best use of censored information by iteratively updating the survival estimate, and can be viewed as a self-consistent estimator with convergence monitored using out-of-bag samples. Splitting rules optimized for… ▽ More We propose the interval censored recursive forests (ICRF) which is an iterative tree ensemble method for interval censored survival data. This nonparametric regression estimator makes the best use of censored information by iteratively updating the survival estimate, and can be viewed as a self-consistent estimator with convergence monitored using out-of-bag samples. Splitting rules optimized for interval censored data are developed and kernel-smoothing is applied. The ICRF displays the highest prediction accuracy among competing nonparametric methods in most of the simulations and in an applied example to avalanche data. An R package icrf is available for implementation. △ Less

Submitted 20 May, 2021; v1 submitted 20 December, 2019; originally announced December 2019.

High dimensional precision medicine from patient-derived xenografts

Authors: Naim U. Rashid, Daniel J. Luckett, Jingxiang Chen, Michael T. Lawson, Longshaokan Wang, Yunshu Zhang, Eric B. Laber, Yufeng Liu, Jen Jen Yeh, Donglin Zeng, Michael R. Kosorok

Abstract: The complexity of human cancer often results in significant heterogeneity in response to treatment. Precision medicine offers potential to improve patient outcomes by leveraging this heterogeneity. Individualized treatment rules (ITRs) formalize precision medicine as maps from the patient covariate space into the space of allowable treatments. The optimal ITR is that which maximizes the mean of a… ▽ More The complexity of human cancer often results in significant heterogeneity in response to treatment. Precision medicine offers potential to improve patient outcomes by leveraging this heterogeneity. Individualized treatment rules (ITRs) formalize precision medicine as maps from the patient covariate space into the space of allowable treatments. The optimal ITR is that which maximizes the mean of a clinical outcome in a population of interest. Patient-derived xenograft (PDX) studies permit the evaluation of multiple treatments within a single tumor and thus are ideally suited for estimating optimal ITRs. PDX data are characterized by correlated outcomes, a high-dimensional feature space, and a large number of treatments. Existing methods for estimating optimal ITRs do not take advantage of the unique structure of PDX data or handle the associated challenges well. In this paper, we explore machine learning methods for estimating optimal ITRs from PDX data. We analyze data from a large PDX study to identify biomarkers that are informative for developing personalized treatment recommendations in multiple cancers. We estimate optimal ITRs using regression-based approaches such as Q-learning and direct search methods such as outcome weighted learning. Finally, we implement a superlearner approach to combine a set of estimated ITRs and show that the resulting ITR performs better than any of the input ITRs, mitigating uncertainty regarding user choice of any particular ITR estimation methodology. Our results indicate that PDX data are a valuable resource for developing individualized treatment strategies in oncology. △ Less

Submitted 13 December, 2019; originally announced December 2019.

The Binary Expansion Randomized Ensemble Test (BERET)

Authors: Duyeol Lee, Kai Zhang, Michael R. Kosorok

Abstract: Recently, the binary expansion testing framework was introduced to test the independence of two continuous random variables by utilizing symmetry statistics that are complete sufficient statistics for dependence. We develop a new test based on an ensemble approach that uses the sum of squared symmetry statistics and distance correlation. Simulation studies suggest that this method improves the pow… ▽ More Recently, the binary expansion testing framework was introduced to test the independence of two continuous random variables by utilizing symmetry statistics that are complete sufficient statistics for dependence. We develop a new test based on an ensemble approach that uses the sum of squared symmetry statistics and distance correlation. Simulation studies suggest that this method improves the power while preserving the clear interpretation of the binary expansion testing. We extend this method to tests of independence of random vectors in arbitrary dimension. Through random projections, the proposed binary expansion randomized ensemble test transforms the multivariate independence testing problem into a univariate problem. Simulation studies and data example analyses show that the proposed method provides relatively robust performance compared with existing methods. △ Less

Submitted 7 January, 2021; v1 submitted 8 December, 2019; originally announced December 2019.

Balanced Policy Evaluation and Learning for Right Censored Data

Authors: Owen E. Leete, Nathan Kallus, Michael G. Hudgens, Sonia Napravnik, Michael R. Kosorok

Abstract: Individualized treatment rules can lead to better health outcomes when patients have heterogeneous responses to treatment. Very few individualized treatment rule estimation methods are compatible with a multi-treatment observational study with right censored survival outcomes. In this paper we extend policy evaluation methods to the right censored data setting. Existing approaches either make rest… ▽ More Individualized treatment rules can lead to better health outcomes when patients have heterogeneous responses to treatment. Very few individualized treatment rule estimation methods are compatible with a multi-treatment observational study with right censored survival outcomes. In this paper we extend policy evaluation methods to the right censored data setting. Existing approaches either make restrictive assumptions about the structure of the data, or use inverse weighting methods that increase the variance of the estimator resulting in decreased performance. We propose a method which uses balanced policy evaluation combined with an imputation approach to remove right censoring. We show that the proposed imputation approach is compatible with a large number of existing survival models and can be used to extend any individualized treatment rule estimation method to the right censored data setting. We establish the rate at which the imputed values converge to the conditional expected survival times, as well as consistency guarantees and regret bounds for the combined balanced policy with imputation approach. In simulation studies, we demonstrate the improved performance of our approach compared to existing methods. We also apply our method to data from the University of North Carolina Center for AIDS Research HIV Clinical Cohort. △ Less

Submitted 13 November, 2019; originally announced November 2019.

Sample Size Calculations for SMARTs

Authors: Eric J. Rose, Eric B. Laber, Marie Davidian, Anastasios A. Tsiatis, Ying-Qi Zhao, Michael R. Kosorok

Abstract: Sequential Multiple Assignment Randomized Trials (SMARTs) are considered the gold standard for estimation and evaluation of treatment regimes. SMARTs are typically sized to ensure sufficient power for a simple comparison, e.g., the comparison of two fixed treatment sequences. Estimation of an optimal treatment regime is conducted as part of a secondary and hypothesis-generating analysis with forma… ▽ More Sequential Multiple Assignment Randomized Trials (SMARTs) are considered the gold standard for estimation and evaluation of treatment regimes. SMARTs are typically sized to ensure sufficient power for a simple comparison, e.g., the comparison of two fixed treatment sequences. Estimation of an optimal treatment regime is conducted as part of a secondary and hypothesis-generating analysis with formal evaluation of the estimated optimal regime deferred to a follow-up trial. However, running a follow-up trial to evaluate an estimated optimal treatment regime is costly and time-consuming; furthermore, the estimated optimal regime that is to be evaluated in such a follow-up trial may be far from optimal if the original trial was underpowered for estimation of an optimal regime. We derive sample size procedures for a SMART that ensure: (i) sufficient power for comparing the optimal treatment regime with standard of care; and (ii) the estimated optimal regime is within a given tolerance of the true optimal regime with high-probability. We establish asymptotic validity of the proposed procedures and demonstrate their finite sample performance in a series of simulation experiments. △ Less

Submitted 16 June, 2019; originally announced June 2019.

Genome analysis and pleiotropy assessment using causal networks with loss of function mutation and metabolomics

Authors: Azam Yazdani, Akram Yazdani, Sarah H. Elsea, Daniel J. Schaid, Michael R. Kosorok, Gita Dangol, Ahmad Samiei

Abstract: Background: Many genome-wide association studies have detected genomic regions associated with traits, yet understanding the functional causes of association often remains elusive. Utilizing systems approaches and focusing on intermediate molecular phenotypes might facilitate biologic understanding. Results: The availability of exome sequencing of two populations of African-Americans and European-… ▽ More Background: Many genome-wide association studies have detected genomic regions associated with traits, yet understanding the functional causes of association often remains elusive. Utilizing systems approaches and focusing on intermediate molecular phenotypes might facilitate biologic understanding. Results: The availability of exome sequencing of two populations of African-Americans and European-Americans from the Atherosclerosis Risk in Communities study allowed us to investigate the effects of annotated loss-of-function (LoF) mutations on 122 serum metabolites. To assess the findings, we built metabolomic causal networks for each population separately and utilized structural equation modeling. We then validated our findings with a set of independent samples. By use of methods based on concepts of Mendelian randomization of genetic variants, we showed that some of the affected metabolites are risk predictors in the causal pathway of disease. For example, LoF mutations in the gene KIAA1755 were identified to elevate the levels of eicosapentaenoate (p-value=5E-14), an essential fatty acid clinically identified to increase essential hypertension. We showed that this gene is in the pathway to triglycerides, where both triglycerides and essential hypertension are risk factors of metabolomic disorder and heart attack. We also identified that the gene CLDN17, harboring loss-of-function mutations, had pleiotropic actions on metabolites from amino acid and lipid pathways. Conclusion: Using systems biology approaches for the analysis of metabolomics and genetic data, we integrated several biological processes, which lead to findings that may functionally connect genetic variants with complex diseases. △ Less

Submitted 29 April, 2019; originally announced April 2019.

Estimating Individualized Treatment Regimes from Crossover Designs

Authors: Crystal T. Nguyen, Daniel J. Luckett, Anna R. Kahkoska, Grace E. Shearrer, Donna Spruijt-Metz, Jaimie N. Davis, Michael R. Kosorok

Abstract: The field of precision medicine aims to tailor treatment based on patient-specific factors in a reproducible way. To this end, estimating an optimal individualized treatment regime (ITR) that recommends treatment decisions based on patient characteristics to maximize the mean of a pre-specified outcome is of particular interest. Several methods have been proposed for estimating an optimal ITR from… ▽ More The field of precision medicine aims to tailor treatment based on patient-specific factors in a reproducible way. To this end, estimating an optimal individualized treatment regime (ITR) that recommends treatment decisions based on patient characteristics to maximize the mean of a pre-specified outcome is of particular interest. Several methods have been proposed for estimating an optimal ITR from clinical trial data in the parallel group setting where each subject is randomized to a single intervention. However, little work has been done in the area of estimating the optimal ITR from crossover study designs. Such designs naturally lend themselves to precision medicine, because they allow for observing the response to multiple treatments for each patient. In this paper, we introduce a method for estimating the optimal ITR using data from a 2x2 crossover study with or without carryover effects. The proposed method is similar to policy search methods such as outcome weighted learning; however, we take advantage of the crossover design by using the difference in responses under each treatment as the observed reward. We establish Fisher and global consistency, present numerical experiments, and analyze data from a feeding trial to demonstrate the improved performance of the proposed method compared to standard methods for a parallel study design. △ Less

Submitted 4 February, 2019; originally announced February 2019.

Receiver Operating Characteristic Curves and Confidence Bands for Support Vector Machines

Authors: Daniel J. Luckett, Eric B. Laber, Samer S. El-Kamary, Cheng Fan, Ravi Jhaveri, Charles M. Perou, Fatma M. Shebl, Michael R. Kosorok

Abstract: Many problems that appear in biomedical decision making, such as diagnosing disease and predicting response to treatment, can be expressed as binary classification problems. The costs of false positives and false negatives vary across application domains and receiver operating characteristic (ROC) curves provide a visual representation of this trade-off. Nonparametric estimators for the ROC curve,… ▽ More Many problems that appear in biomedical decision making, such as diagnosing disease and predicting response to treatment, can be expressed as binary classification problems. The costs of false positives and false negatives vary across application domains and receiver operating characteristic (ROC) curves provide a visual representation of this trade-off. Nonparametric estimators for the ROC curve, such as a weighted support vector machine (SVM), are desirable because they are robust to model misspecification. While weighted SVMs have great potential for estimating ROC curves, their theoretical properties were heretofore underdeveloped. We propose a method for constructing confidence bands for the SVM ROC curve and provide the theoretical justification for the SVM ROC curve by showing that the risk function of the estimated decision rule is uniformly consistent across the weight parameter. We demonstrate the proposed confidence band method and the superior sensitivity and specificity of the weighted SVM compared to commonly used methods in diagnostic medicine using simulation studies. We present two illustrative examples: diagnosis of hepatitis C and a predictive model for treatment response in breast cancer. △ Less

Submitted 17 July, 2018; originally announced July 2018.

A proportional hazards model for interval-censored data subject to instantaneous failures

Authors: Prabhashi W. Withana Gamage, Monica Chaudari, Christopher S. McMahan, Michael R. Kosorok

Abstract: The proportional hazards (PH) model is arguably one of the most popular models used to analyze time to event data arising from clinical trials and longitudinal studies, among many others. In many such studies, the event time of interest is not directly observed but is known relative to periodic examination times; i.e., practitioners observe either current status or interval-censored data. The anal… ▽ More The proportional hazards (PH) model is arguably one of the most popular models used to analyze time to event data arising from clinical trials and longitudinal studies, among many others. In many such studies, the event time of interest is not directly observed but is known relative to periodic examination times; i.e., practitioners observe either current status or interval-censored data. The analysis of data of this structure is often fraught with many difficulties. Further exacerbating this issue, in some such studies the observed data also consists of instantaneous failures; i.e., the event times for several study units coincide exactly with the time at which the study begins. In light of these difficulties, this work focuses on developing a mixture model, under the PH assumptions, which can be used to analyze interval-censored data subject to instantaneous failures. To allow for modeling flexibility, two methods of estimating the unknown cumulative baseline hazard function are proposed; a fully parametric and a monotone spline representation are considered. Through a novel data augmentation procedure involving latent Poisson random variables, an expectation-maximization (EM) algorithm was developed to complete model fitting. The resulting EM algorithm is easy to implement and is computationally efficient. Moreover, through extensive simulation studies the proposed approach is shown to provide both reliable estimation and inference. △ Less

Submitted 3 April, 2018; v1 submitted 30 March, 2018; originally announced April 2018.

Augmented Outcome-weighted Learning for Optimal Treatment Regimes

Authors: Xin Zhou, Michael R. Kosorok

Abstract: Precision medicine is of considerable interest in clinical, academic and regulatory parties. The key to precision medicine is the optimal treatment regime. Recently, Zhou et al. (2017) developed residual weighted learning (RWL) to construct the optimal regime that directly optimize the clinical outcome. However, this method involves computationally intensive non-convex optimization, which cannot g… ▽ More Precision medicine is of considerable interest in clinical, academic and regulatory parties. The key to precision medicine is the optimal treatment regime. Recently, Zhou et al. (2017) developed residual weighted learning (RWL) to construct the optimal regime that directly optimize the clinical outcome. However, this method involves computationally intensive non-convex optimization, which cannot guarantee a global solution. Furthermore, this method does not possess fully semiparametrical efficiency. In this article, we propose augmented outcome-weighted learning (AOL). The method is built on a doubly robust augmented inverse probability weighted estimator, and hence constructs semiparametrically efficient regimes. Our proposed AOL is closely related to RWL. The weights are obtained from counterfactual residuals, where negative residuals are reflected to positive and accordingly their treatment assignments are switched to opposites. Convex loss functions are thus applied to guarantee a global solution and to reduce computations. We show that AOL is universally consistent, i.e., the estimated regime of AOL converges the Bayes regime when the sample size approaches infinity, without knowing any specifics of the distribution of the data. We also propose variable selection methods for linear and nonlinear regimes, respectively, to further improve performance. The performance of the proposed AOL methods is illustrated in simulation studies and in an analysis of the Nefazodone-CBASP clinical trial data. △ Less

Submitted 28 November, 2017; originally announced November 2017.

Estimation and Optimization of Composite Outcomes

Authors: Daniel J. Luckett, Eric B. Laber, Michael R. Kosorok

Abstract: There is tremendous interest in precision medicine as a means to improve patient outcomes by tailoring treatment to individual characteristics. An individualized treatment rule formalizes precision medicine as a map from patient information to a recommended treatment. A treatment rule is defined to be optimal if it maximizes the mean of a scalar outcome in a population of interest, e.g., symptom r… ▽ More There is tremendous interest in precision medicine as a means to improve patient outcomes by tailoring treatment to individual characteristics. An individualized treatment rule formalizes precision medicine as a map from patient information to a recommended treatment. A treatment rule is defined to be optimal if it maximizes the mean of a scalar outcome in a population of interest, e.g., symptom reduction. However, clinical and intervention scientists often must balance multiple and possibly competing outcomes, e.g., symptom reduction and the risk of an adverse event. One approach to precision medicine in this setting is to elicit a composite outcome which balances all competing outcomes; unfortunately, eliciting a composite outcome directly from patients is difficult without a high-quality instrument, and an expert-derived composite outcome may not account for heterogeneity in patient preferences. We propose a new paradigm for the study of precision medicine using observational data that relies solely on the assumption that clinicians are approximately (i.e., imperfectly) making decisions to maximize individual patient utility. Estimated composite outcomes are subsequently used to construct an estimator of an individualized treatment rule which maximizes the mean of patient-specific composite outcomes. The estimated composite outcomes and estimated optimal individualized treatment rule provide new insights into patient preference heterogeneity, clinician behavior, and the value of precision medicine in a given domain. We derive inference procedures for the proposed estimators under mild conditions and demonstrate their finite sample performance through a suite of simulation experiments and an illustrative application to data from a study of bipolar depression. △ Less

Submitted 26 May, 2020; v1 submitted 28 November, 2017; originally announced November 2017.

Causal nearest neighbor rules for optimal treatment regimes

Authors: Xin Zhou, Michael R. Kosorok

Abstract: The estimation of optimal treatment regimes is of considerable interest to precision medicine. In this work, we propose a causal $k$-nearest neighbor method to estimate the optimal treatment regime. The method roots in the framework of causal inference, and estimates the causal treatment effects within the nearest neighborhood. Although the method is simple, it possesses nice theoretical propertie… ▽ More The estimation of optimal treatment regimes is of considerable interest to precision medicine. In this work, we propose a causal $k$-nearest neighbor method to estimate the optimal treatment regime. The method roots in the framework of causal inference, and estimates the causal treatment effects within the nearest neighborhood. Although the method is simple, it possesses nice theoretical properties. We show that the causal $k$-nearest neighbor regime is universally consistent. That is, the causal $k$-nearest neighbor regime will eventually learn the optimal treatment regime as the sample size increases. We also establish its convergence rate. However, the causal $k$-nearest neighbor regime may suffer from the curse of dimensionality, i.e. performance deteriorates as dimensionality increases. To alleviate this problem, we develop an adaptive causal $k$-nearest neighbor method to perform metric selection and variable selection simultaneously. The performance of the proposed methods is illustrated in simulation studies and in an analysis of a chronic depression clinical trial. △ Less

Submitted 22 November, 2017; originally announced November 2017.

Double Sparsity Kernel Learning with Automatic Variable Selection and Data Extraction

Authors: Jingxiang Chen, Chong Zhang, Michael R. Kosorok, Yufeng Liu

Abstract: Learning with Reproducing Kernel Hilbert Spaces (RKHS) has been widely used in many scientific disciplines. Because a RKHS can be very flexible, it is common to impose a regularization term in the optimization to prevent overfitting. Standard RKHS learning employs the squared norm penalty of the learning function. Despite its success, many challenges remain. In particular, one cannot directly use… ▽ More Learning with Reproducing Kernel Hilbert Spaces (RKHS) has been widely used in many scientific disciplines. Because a RKHS can be very flexible, it is common to impose a regularization term in the optimization to prevent overfitting. Standard RKHS learning employs the squared norm penalty of the learning function. Despite its success, many challenges remain. In particular, one cannot directly use the squared norm penalty for variable selection or data extraction. Therefore, when there exists noise predictors, or the underlying function has a sparse representation in the dual space, the performance of standard RKHS learning can be suboptimal. In the literature,work has been proposed on how to perform variable selection in RKHS learning, and a data sparsity constraint was considered for data extraction. However, how to learn in a RKHS with both variable selection and data extraction simultaneously remains unclear. In this paper, we propose a unified RKHS learning method, namely, DOuble Sparsity Kernel (DOSK) learning, to overcome this challenge. An efficient algorithm is provided to solve the corresponding optimization problem. We prove that under certain conditions, our new method can asymptotically achieve variable selection consistency. Simulated and real data results demonstrate that DOSK is highly competitive among existing approaches for RKHS learning. △ Less

Submitted 5 June, 2017; originally announced June 2017.

Estimating Individualized Treatment Rules for Ordinal Treatments

Authors: Jingxiang Chen, Haoda Fu, Xuanyao He, Michael R. Kosorok, Yufeng Liu

Abstract: Precision medicine is an emerging scientific topic for disease treatment and prevention that takes into account individual patient characteristics. It is an important direction for clinical research, and many statistical methods have been recently proposed. One of the primary goals of precision medicine is to obtain an optimal individual treatment rule (ITR), which can help make decisions on treat… ▽ More Precision medicine is an emerging scientific topic for disease treatment and prevention that takes into account individual patient characteristics. It is an important direction for clinical research, and many statistical methods have been recently proposed. One of the primary goals of precision medicine is to obtain an optimal individual treatment rule (ITR), which can help make decisions on treatment selection according to each patient's specific characteristics. Recently, outcome weighted learning (OWL) has been proposed to estimate such an optimal ITR in a binary treatment setting by maximizing the expected clinical outcome. However, for ordinal treatment settings, such as individualized dose finding, it is unclear how to use OWL. In this paper, we propose a new technique for estimating ITR with ordinal treatments. In particular, we propose a data duplication technique with a piecewise convex loss function. We establish Fisher consistency for the resulting estimated ITR under certain conditions, and obtain the convergence and risk bound properties. Simulated examples and two applications to datasets from an irritable bowel problem and a type 2 diabetes mellitus observational study demonstrate the highly competitive performance of the proposed method compared to existing alternatives. △ Less

Submitted 15 February, 2017; originally announced February 2017.

Estimating Dynamic Treatment Regimes in Mobile Health Using V-learning

Authors: Daniel J. Luckett, Eric B. Laber, Anna R. Kahkoska, David M. Maahs, Elizabeth Mayer-Davis, Michael R. Kosorok

Abstract: The vision for precision medicine is to use individual patient characteristics to inform a personalized treatment plan that leads to the best healthcare possible for each patient. Mobile technologies have an important role to play in this vision as they offer a means to monitor a patient's health status in real-time and subsequently to deliver interventions if, when, and in the dose that they are… ▽ More The vision for precision medicine is to use individual patient characteristics to inform a personalized treatment plan that leads to the best healthcare possible for each patient. Mobile technologies have an important role to play in this vision as they offer a means to monitor a patient's health status in real-time and subsequently to deliver interventions if, when, and in the dose that they are needed. Dynamic treatment regimes formalize individualized treatment plans as sequences of decision rules, one per stage of clinical intervention, that map current patient information to a recommended treatment. However, existing methods for estimating optimal dynamic treatment regimes are designed for a small number of fixed decision points occurring on a coarse time-scale. We propose a new reinforcement learning method for estimating an optimal treatment regime that is applicable to data collected using mobile technologies in an outpatient setting. The proposed method accommodates an indefinite time horizon and minute-by-minute decision making that are common in mobile health applications. We show the proposed estimators are consistent and asymptotically normal under mild conditions. The proposed methods are applied to estimate an optimal dynamic treatment regime for controlling blood glucose levels in patients with type 1 diabetes. △ Less

Submitted 14 October, 2017; v1 submitted 10 November, 2016; originally announced November 2016.

Robust Hybrid Learning for Estimating Personalized Dynamic Treatment Regimens

Authors: Ying Liu, Yuanjia Wang, Michael R. Kosorok, Yingqi Zhao, Donglin Zeng

Abstract: Dynamic treatment regimens (DTRs) are sequential decision rules tailored at each stage by potentially time-varying patient features and intermediate outcomes observed in previous stages. The complexity, patient heterogeneity and chronicity of many diseases and disorders call for learning optimal DTRs which best dynamically tailor treatment to each individual's response over time. Proliferation of… ▽ More Dynamic treatment regimens (DTRs) are sequential decision rules tailored at each stage by potentially time-varying patient features and intermediate outcomes observed in previous stages. The complexity, patient heterogeneity and chronicity of many diseases and disorders call for learning optimal DTRs which best dynamically tailor treatment to each individual's response over time. Proliferation of personalized data (e.g., genetic and imaging data) provides opportunities for deep tailoring as well as new challenges for statistical methodology. In this work, we propose a robust hybrid approach referred as Augmented Multistage Outcome-Weighted Learning (AMOL) to integrate outcome-weighted learning and Q-learning to identify optimal DTRs from the Sequential Multiple Assignment Randomization Trials (SMARTs). We generalize outcome weighted learning (O-learning; Zhao et al.~2012) to allow for negative outcomes; we propose methods to reduce variability of weights in O-learning to achieve numeric stability and higher efficiency; finally, for multiple-stage SMART studies, we introduce doubly robust augmentation to machine learning based O-learning to improve efficiency by drawing information from regression model-based Q-learning at each stage. The proposed AMOL remains valid even if the Q-learning model is misspecified. We establish the theoretical properties of AMOL, including the consistency of the estimated rules and the rates of convergence to the optimal value function. The comparative advantage of AMOL over existing methods is demonstrated in extensive simulation studies and applications to two SMART data sets: a two-stage trial for attention deficit and hyperactive disorder (ADHD) and the STAR*D trial for major depressive disorder (MDD). △ Less

Submitted 7 November, 2016; originally announced November 2016.

Residual Weighted Learning for Estimating Individualized Treatment Rules

Authors: Xin Zhou, Nicole Mayer-Hamblett, Umer Khan, Michael R. Kosorok

Abstract: Personalized medicine has received increasing attention among statisticians, computer scientists, and clinical practitioners. A major component of personalized medicine is the estimation of individualized treatment rules (ITRs). Recently, Zhao et al. (2012) proposed outcome weighted learning (OWL) to construct ITRs that directly optimize the clinical outcome. Although OWL opens the door to introdu… ▽ More Personalized medicine has received increasing attention among statisticians, computer scientists, and clinical practitioners. A major component of personalized medicine is the estimation of individualized treatment rules (ITRs). Recently, Zhao et al. (2012) proposed outcome weighted learning (OWL) to construct ITRs that directly optimize the clinical outcome. Although OWL opens the door to introducing machine learning techniques to optimal treatment regimes, it still has some problems in performance. In this article, we propose a general framework, called Residual Weighted Learning (RWL), to improve finite sample performance. Unlike OWL which weights misclassification errors by clinical outcomes, RWL weights these errors by residuals of the outcome from a regression fit on clinical covariates excluding treatment assignment. We utilize the smoothed ramp loss function in RWL, and provide a difference of convex (d.c.) algorithm to solve the corresponding non-convex optimization problem. By estimating residuals with linear models or generalized linear models, RWL can effectively deal with different types of outcomes, such as continuous, binary and count outcomes. We also propose variable selection methods for linear and nonlinear rules, respectively, to further improve the performance. We show that the resulting estimator of the treatment rule is consistent. We further obtain a rate of convergence for the difference between the expected outcome using the estimated ITR and that of the optimal treatment rule. The performance of the proposed RWL methods is illustrated in simulation studies and in an analysis of cystic fibrosis clinical trial data. △ Less

Submitted 13 August, 2015; originally announced August 2015.

Comments: 48 pages, 3 figures

Biclustering Via Sparse Clustering

Authors: Qian Liu, Guanhua Chen, Michael R. Kosorok, Eric Bair

Abstract: In many situations it is desirable to identify clusters that differ with respect to only a subset of features. Such clusters may represent homogeneous subgroups of patients with a disease, such as cancer or chronic pain. We define a bicluster to be a submatrix U of a larger data matrix X such that the features and observations in U differ from those not contained in U. For example, the observation… ▽ More In many situations it is desirable to identify clusters that differ with respect to only a subset of features. Such clusters may represent homogeneous subgroups of patients with a disease, such as cancer or chronic pain. We define a bicluster to be a submatrix U of a larger data matrix X such that the features and observations in U differ from those not contained in U. For example, the observations in U could have different means or variances with respect to the features in U. We propose a general framework for biclustering based on the sparse clustering method of Witten and Tibshirani (2010). We develop a method for identifying features that belong to biclusters. This framework can be used to identify biclusters that differ with respect to the means of the features, the variance of the features, or more general differences. We apply these methods to several simulated and real-world data sets and compare the results of our method with several previously published methods. The results of our method compare favorably with existing methods with respect to both predictive accuracy and computing time. △ Less

Submitted 10 July, 2014; originally announced July 2014.

Comments: 40 pages, 8 figures, 10 tables

Support Vector Regression for Right Censored Data

Authors: Yair Goldberg, Michael R. Kosorok

Abstract: We develop a unified approach for classification and regression support vector machines for data subject to right censoring. We provide finite sample bounds on the generalization error of the algorithm, prove risk consistency for a wide class of probability measures, and study the associated learning rates. We apply the general methodology to estimation of the (truncated) mean, median, quantiles,… ▽ More We develop a unified approach for classification and regression support vector machines for data subject to right censoring. We provide finite sample bounds on the generalization error of the algorithm, prove risk consistency for a wide class of probability measures, and study the associated learning rates. We apply the general methodology to estimation of the (truncated) mean, median, quantiles, and for classification problems. We present a simulation study that demonstrates the performance of the proposed approach. △ Less

Submitted 12 January, 2013; v1 submitted 23 February, 2012; originally announced February 2012.

Comments: In this version, we strengthened the theoretical results and corrected a few mistakes

Penalized Q-Learning for Dynamic Treatment Regimes

Authors: Rui Song, Weiwei Wang, Donglin Zeng, Michael R. Kosorok

Abstract: A dynamic treatment regime effectively incorporates both accrued information and long-term effects of treatment from specially designed clinical trials. As these become more and more popular in conjunction with longitudinal data from clinical studies, the development of statistical inference for optimal dynamic treatment regimes is a high priority. This is very challenging due to the difficulties… ▽ More A dynamic treatment regime effectively incorporates both accrued information and long-term effects of treatment from specially designed clinical trials. As these become more and more popular in conjunction with longitudinal data from clinical studies, the development of statistical inference for optimal dynamic treatment regimes is a high priority. This is very challenging due to the difficulties arising form non-regularities in the treatment effect parameters. In this paper, we propose a new reinforcement learning framework called penalized Q-learning (PQ-learning), under which the non-regularities can be resolved and valid statistical inference established. We also propose a new statistical procedure---individual selection---and corresponding methods for incorporating individual selection within PQ-learning. Extensive numerical studies are presented which compare the proposed methods with existing methods, under a variety of non-regular scenarios, and demonstrate that the proposed approach is both inferentially and computationally superior. The proposed method is demonstrated with the data from a depression clinical trial study. △ Less

Submitted 26 August, 2011; originally announced August 2011.

Discussion of: Brownian distance covariance

Authors: Michael R. Kosorok

Abstract: We discuss briefly the very interesting concept of Brownian distance covariance developed by Székely and Rizzo [Ann. Appl. Statist. (2009), to appear] and describe two possible extensions. The first extension is for high dimensional data that can be coerced into a Hilbert space, including certain high throughput screening and functional data settings. The second extension involves very simple modi… ▽ More We discuss briefly the very interesting concept of Brownian distance covariance developed by Székely and Rizzo [Ann. Appl. Statist. (2009), to appear] and describe two possible extensions. The first extension is for high dimensional data that can be coerced into a Hilbert space, including certain high throughput screening and functional data settings. The second extension involves very simple modifications that may yield increased power in some settings. We commend Székely and Rizzo for their very interesting work and recognize that this general idea has potential to have a large impact on the way in which statisticians evaluate dependency in data. [arXiv:1010.0297] △ Less

Submitted 9 December, 2013; v1 submitted 5 October, 2010; originally announced October 2010.

Comments: Published in at http://dx.doi.org/10.1214/09-AOAS312B the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org). With Corrections

Report number: IMS-AOAS-AOAS312B

Journal ref: Annals of Applied Statistics 2009, Vol. 3, No. 4, 1270-1278