-
clustra: A multi-platform k-means clustering algorithm for analysis of longitudinal trajectories in large electronic health records data
Authors:
Nimish Adhikari,
Hanna Gerlovin,
George Ostrouchov,
Rachel Ehrbar,
Alyssa B. Dufour,
Brian R. Ferolito,
Serkalem Demissie,
Lauren Costa,
Yuk-Lam Ho,
Laura Tarko,
Edmon Begoli,
Kelly Cho,
David R. Gagnon
Abstract:
Background and Objective: Variables collected over time, or longitudinally, such as biologic measurements in electronic health records data, are not simple to summarize with a single time-point, and thus can be more holistically conceptualized as trajectories over time. Cluster analysis with longitudinal data further allows for clinical representation of groups of subjects with similar trajectorie…
▽ More
Background and Objective: Variables collected over time, or longitudinally, such as biologic measurements in electronic health records data, are not simple to summarize with a single time-point, and thus can be more holistically conceptualized as trajectories over time. Cluster analysis with longitudinal data further allows for clinical representation of groups of subjects with similar trajectories and identification of unique characteristics, or phenotypes, that can be investigated as risk factors or disease outcomes. Some of the challenges in estimating these clustered trajectories lie in the handling of observations at inconsistent time intervals and the usability of algorithms across programming languages.
Methods: We propose longitudinal trajectory clustering using a k-means algorithm with thin-plate regression splines, implemented across multiple platforms, the R package clustra and corresponding \SAS macros. The \SAS macros accommodate flexible clustering approaches, and also include visualization of the clusters, and silhouette plots for diagnostic evaluation of the appropriate cluster number. The R package, designed in parallel, has similar functionality, with additional multi-core processing and Rand-index-based diagnostics.
Results: The package and macros achieve comparable results when applied to an example of simulated blood pressure measurements based on real data from Veterans Affairs Healthcare recipients who were initiated on anti-hypertensive medication.
Conclusion: The R package clustra and the SAS macros integrate a K-means clustering algorithm for longitudinal trajectories that operates with large electronic health record data. The implementations provide comparable results in both platforms, satisfying the needs of investigators familiar with, or constrained by access to, one or the other platform.
△ Less
Submitted 1 July, 2025;
originally announced July 2025.
-
A Spatially Correlated Competing Risks Time-to-Event Model for Supercomputer GPU Failure Data
Authors:
Jie Min,
Yili Hong,
William Q. Meeker,
George Ostrouchov
Abstract:
Graphics processing units (GPUs) are widely used in many high-performance computing (HPC) applications such as imaging/video processing and training deep-learning models in artificial intelligence. GPUs installed in HPC systems are often heavily used, and GPU failures occur during HPC system operations. Thus, the reliability of GPUs is of interest for the overall reliability of HPC systems. The Cr…
▽ More
Graphics processing units (GPUs) are widely used in many high-performance computing (HPC) applications such as imaging/video processing and training deep-learning models in artificial intelligence. GPUs installed in HPC systems are often heavily used, and GPU failures occur during HPC system operations. Thus, the reliability of GPUs is of interest for the overall reliability of HPC systems. The Cray XK7 Titan supercomputer was one of the top ten supercomputers in the world. The failure event times of more than 30,000 GPUs in Titan were recorded and previous data analysis suggested that the failure time of a GPU may be affected by the GPU's connectivity location inside the supercomputer among other factors. In this paper, we conduct in-depth statistical modeling of GPU failure times to study the effect of location on GPU failures under competing risks with covariates and spatially correlated random effects. In particular, two major failure types of GPUs in Titan are considered. The connectivity locations of cabinets are modeled as spatially correlated random effects, and the positions of GPUs inside each cabinet are treated as covariates. A Bayesian framework is used for statistical inference. We also compare different methods of estimation such as the maximum likelihood, which is implemented via an expectation-maximization algorithm. Our results provide interesting insights into GPU failures in HPC systems.
△ Less
Submitted 28 March, 2023;
originally announced March 2023.
-
Parallel Statistical Computing with R: An Illustration on Two Architectures
Authors:
George Ostrouchov,
Wei-Chen Chen,
Drew Schmidt
Abstract:
To harness the full benefit of new computing platforms, it is necessary to develop software with parallel computing capabilities. This is no less true for statisticians than for astrophysicists. The R programming language, which is perhaps the most popular software environment for statisticians today, has many packages available for parallel computing. Their diversity in approach can be difficult…
▽ More
To harness the full benefit of new computing platforms, it is necessary to develop software with parallel computing capabilities. This is no less true for statisticians than for astrophysicists. The R programming language, which is perhaps the most popular software environment for statisticians today, has many packages available for parallel computing. Their diversity in approach can be difficult to navigate. Some have attempted to alleviate this problem by designing common interfaces. However, these approaches offer limited flexibility to the user; additionally, they often serve as poor abstractions to the reality of modern hardware, leading to poor performance. We give a short introduction to two basic parallel computing approaches that closely align with hardware reality, allow the user to understand its performance, and provide sufficient capability to fully utilize multicore and multinode environments.
We illustrate both approaches by working through a simple example fitting a random forest model. Beginning with a serial algorithm, we derive two parallel versions. Our objective is to illustrate the use of multiple cores on a single processor and the use of multiple processors in a cluster computer. We discuss the differences between the two versions and how the underlying hardware is used in each case.
△ Less
Submitted 6 September, 2017; v1 submitted 4 September, 2017;
originally announced September 2017.
