-
Foundation models for electronic health records: representation dynamics and transferability
Authors:
Michael C. Burkhart,
Bashar Ramadan,
Zewei Liao,
Kaveri Chhikara,
Juan C. Rojas,
William F. Parker,
Brett K. Beaulieu-Jones
Abstract:
Foundation models (FMs) trained on electronic health records (EHRs) have shown strong performance on a range of clinical prediction tasks. However, adapting these models to local health systems remains challenging due to limited data availability and resource constraints. In this study, we investigated what these models learn and evaluated the transferability of an FM trained on MIMIC-IV to an ins…
▽ More
Foundation models (FMs) trained on electronic health records (EHRs) have shown strong performance on a range of clinical prediction tasks. However, adapting these models to local health systems remains challenging due to limited data availability and resource constraints. In this study, we investigated what these models learn and evaluated the transferability of an FM trained on MIMIC-IV to an institutional EHR dataset at the University of Chicago Medical Center. We assessed their ability to identify outlier patients and examined representation-space patient trajectories in relation to future clinical outcomes. We also evaluated the performance of supervised fine-tuned classifiers on both source and target datasets. Our findings offer insights into the adaptability of FMs across different healthcare systems, highlight considerations for their effective implementation, and provide an empirical analysis of the underlying factors that contribute to their predictive performance.
△ Less
Submitted 14 April, 2025;
originally announced April 2025.
-
ProtoECGNet: Case-Based Interpretable Deep Learning for Multi-Label ECG Classification with Contrastive Learning
Authors:
Sahil Sethi,
David Chen,
Thomas Statchen,
Michael C. Burkhart,
Nipun Bhandari,
Bashar Ramadan,
Brett Beaulieu-Jones
Abstract:
Deep learning-based electrocardiogram (ECG) classification has shown impressive performance but clinical adoption has been slowed by the lack of transparent and faithful explanations. Post hoc methods such as saliency maps may fail to reflect a model's true decision process. Prototype-based reasoning offers a more transparent alternative by grounding decisions in similarity to learned representati…
▽ More
Deep learning-based electrocardiogram (ECG) classification has shown impressive performance but clinical adoption has been slowed by the lack of transparent and faithful explanations. Post hoc methods such as saliency maps may fail to reflect a model's true decision process. Prototype-based reasoning offers a more transparent alternative by grounding decisions in similarity to learned representations of real ECG segments, enabling faithful, case-based explanations. We introduce ProtoECGNet, a prototype-based deep learning model for interpretable, multi-label ECG classification. ProtoECGNet employs a structured, multi-branch architecture that reflects clinical interpretation workflows: it integrates a 1D CNN with global prototypes for rhythm classification, a 2D CNN with time-localized prototypes for morphology-based reasoning, and a 2D CNN with global prototypes for diffuse abnormalities. Each branch is trained with a prototype loss designed for multi-label learning, combining clustering, separation, diversity, and a novel contrastive loss that encourages appropriate separation between prototypes of unrelated classes while allowing clustering for frequently co-occurring diagnoses. We evaluate ProtoECGNet on all 71 diagnostic labels from the PTB-XL dataset, demonstrating competitive performance relative to state-of-the-art black-box models while providing structured, case-based explanations. To assess prototype quality, we conduct a structured clinician review of the final model's projected prototypes, finding that they are rated as representative and clear. ProtoECGNet shows that prototype learning can be effectively scaled to complex, multi-label time-series classification, offering a practical path toward transparent and trustworthy deep learning models for clinical decision support.
△ Less
Submitted 17 May, 2025; v1 submitted 11 April, 2025;
originally announced April 2025.
-
Safe AI for health and beyond -- Monitoring to transform a health service
Authors:
Mahed Abroshan,
Michael Burkhart,
Oscar Giles,
Sam Greenbury,
Zoe Kourtzi,
Jack Roberts,
Mihaela van der Schaar,
Jannetta S Steyn,
Alan Wilson,
May Yong
Abstract:
Machine learning techniques are effective for building predictive models because they identify patterns in large datasets. Development of a model for complex real-life problems often stop at the point of publication, proof of concept or when made accessible through some mode of deployment. However, a model in the medical domain risks becoming obsolete as patient demographics, systems and clinical…
▽ More
Machine learning techniques are effective for building predictive models because they identify patterns in large datasets. Development of a model for complex real-life problems often stop at the point of publication, proof of concept or when made accessible through some mode of deployment. However, a model in the medical domain risks becoming obsolete as patient demographics, systems and clinical practices change. The maintenance and monitoring of predictive model performance post-publication is crucial to enable their safe and effective long-term use. We will assess the infrastructure required to monitor the outputs of a machine learning algorithm, and present two scenarios with examples of monitoring and updates of models, firstly on a breast cancer prognosis model trained on public longitudinal data, and secondly on a neurodegenerative stratification algorithm that is currently being developed and tested in clinic.
△ Less
Submitted 6 June, 2023; v1 submitted 2 March, 2023;
originally announced March 2023.
-
Deep Low-Density Separation for Semi-Supervised Classification
Authors:
Michael C. Burkhart,
Kyle Shan
Abstract:
Given a small set of labeled data and a large set of unlabeled data, semi-supervised learning (SSL) attempts to leverage the location of the unlabeled datapoints in order to create a better classifier than could be obtained from supervised methods applied to the labeled training set alone. Effective SSL imposes structural assumptions on the data, e.g. that neighbors are more likely to share a clas…
▽ More
Given a small set of labeled data and a large set of unlabeled data, semi-supervised learning (SSL) attempts to leverage the location of the unlabeled datapoints in order to create a better classifier than could be obtained from supervised methods applied to the labeled training set alone. Effective SSL imposes structural assumptions on the data, e.g. that neighbors are more likely to share a classification or that the decision boundary lies in an area of low density. For complex and high-dimensional data, neural networks can learn feature embeddings to which traditional SSL methods can then be applied in what we call hybrid methods.
Previously-developed hybrid methods iterate between refining a latent representation and performing graph-based SSL on this representation. In this paper, we introduce a novel hybrid method that instead applies low-density separation to the embedded features. We describe it in detail and discuss why low-density separation may be better suited for SSL on neural network-based embeddings than graph-based algorithms. We validate our method using in-house customer survey data and compare it to other state-of-the-art learning methods. Our approach effectively classifies thousands of unlabeled users from a relatively small number of hand-classified examples.
△ Less
Submitted 22 May, 2022;
originally announced May 2022.
-
Neuroevolutionary Feature Representations for Causal Inference
Authors:
Michael C. Burkhart,
Gabriel Ruiz
Abstract:
Within the field of causal inference, we consider the problem of estimating heterogeneous treatment effects from data. We propose and validate a novel approach for learning feature representations to aid the estimation of the conditional average treatment effect or CATE. Our method focuses on an intermediate layer in a neural network trained to predict the outcome from the features. In contrast to…
▽ More
Within the field of causal inference, we consider the problem of estimating heterogeneous treatment effects from data. We propose and validate a novel approach for learning feature representations to aid the estimation of the conditional average treatment effect or CATE. Our method focuses on an intermediate layer in a neural network trained to predict the outcome from the features. In contrast to previous approaches that encourage the distribution of representations to be treatment-invariant, we leverage a genetic algorithm that optimizes over representations useful for predicting the outcome to select those less useful for predicting the treatment. This allows us to retain information within the features useful for predicting outcome even if that information may be related to treatment assignment. We validate our method on synthetic examples and illustrate its use on a real life dataset.
△ Less
Submitted 21 May, 2022;
originally announced May 2022.
-
Discriminative Bayesian filtering lends momentum to the stochastic Newton method for minimizing log-convex functions
Authors:
Michael C. Burkhart
Abstract:
To minimize the average of a set of log-convex functions, the stochastic Newton method iteratively updates its estimate using subsampled versions of the full objective's gradient and Hessian. We contextualize this optimization problem as sequential Bayesian inference on a latent state-space model with a discriminatively-specified observation process. Applying Bayesian filtering then yields a novel…
▽ More
To minimize the average of a set of log-convex functions, the stochastic Newton method iteratively updates its estimate using subsampled versions of the full objective's gradient and Hessian. We contextualize this optimization problem as sequential Bayesian inference on a latent state-space model with a discriminatively-specified observation process. Applying Bayesian filtering then yields a novel optimization algorithm that considers the entire history of gradients and Hessians when forming an update. We establish matrix-based conditions under which the effect of older observations diminishes over time, in a manner analogous to Polyak's heavy ball momentum. We illustrate various aspects of our approach with an example and review other relevant innovations for the stochastic Newton method.
△ Less
Submitted 21 August, 2023; v1 submitted 26 April, 2021;
originally announced April 2021.
-
A Discriminative Approach to Bayesian Filtering with Applications to Human Neural Decoding
Authors:
Michael C. Burkhart
Abstract:
Given a stationary state-space model that relates a sequence of hidden states and corresponding measurements or observations, Bayesian filtering provides a principled statistical framework for inferring the posterior distribution of the current state given all measurements up to the present time. For example, the Apollo lunar module implemented a Kalman filter to infer its location from a sequence…
▽ More
Given a stationary state-space model that relates a sequence of hidden states and corresponding measurements or observations, Bayesian filtering provides a principled statistical framework for inferring the posterior distribution of the current state given all measurements up to the present time. For example, the Apollo lunar module implemented a Kalman filter to infer its location from a sequence of earth-based radar measurements and land safely on the moon.
To perform Bayesian filtering, we require a measurement model that describes the conditional distribution of each observation given state. The Kalman filter takes this measurement model to be linear, Gaussian. Here we show how a nonlinear, Gaussian approximation to the distribution of state given observation can be used in conjunction with Bayes' rule to build a nonlinear, non-Gaussian measurement model. The resulting approach, called the Discriminative Kalman Filter (DKF), retains fast closed-form updates for the posterior. We argue there are many cases where the distribution of state given measurement is better-approximated as Gaussian, especially when the dimensionality of measurements far exceeds that of states and the Bernstein-von Mises theorem applies. Online neural decoding for brain-computer interfaces provides a motivating example, where filtering incorporates increasingly detailed measurements of neural activity to provide users control over external devices. Within the BrainGate2 clinical trial, the DKF successfully enabled three volunteers with quadriplegia to control an on-screen cursor in real-time using mental imagery alone. Participant "T9" used the DKF to type out messages on a tablet PC.
△ Less
Submitted 16 July, 2018;
originally announced July 2018.
-
Reduce to the Max: A Simple Approach for Massive-Scale Privacy-Preserving Collaborative Network Measurements (Extended Version)
Authors:
Fabio Ricciato,
Martin Burkhart
Abstract:
Privacy-preserving techniques for distributed computation have been proposed recently as a promising framework in collaborative inter-domain network monitoring. Several different approaches exist to solve such class of problems, e.g., Homomorphic Encryption (HE) and Secure Multiparty Computation (SMC) based on Shamir's Secret Sharing algorithm (SSS). Such techniques are complete from a computation…
▽ More
Privacy-preserving techniques for distributed computation have been proposed recently as a promising framework in collaborative inter-domain network monitoring. Several different approaches exist to solve such class of problems, e.g., Homomorphic Encryption (HE) and Secure Multiparty Computation (SMC) based on Shamir's Secret Sharing algorithm (SSS). Such techniques are complete from a computation-theoretic perspective: given a set of private inputs, it is possible to perform arbitrary computation tasks without revealing any of the intermediate results. In fact, HE and SSS can operate also on secret inputs and/or provide secret outputs. However, they are computationally expensive and do not scale well in the number of players and/or in the rate of computation tasks. In this paper we advocate the use of "elementary" (as opposite to "complete") Secure Multiparty Computation (E-SMC) procedures for traffic monitoring. E-SMC supports only simple computations with private input and public output, i.e., it can not handle secret input nor secret (intermediate) output. Such a simplification brings a dramatic reduction in complexity and enables massive-scale implementation with acceptable delay and overhead. Notwithstanding its simplicity, we claim that an E-SMC scheme is sufficient to perform a great variety of computation tasks of practical relevance to collaborative network monitoring, including, e.g., anonymous publishing and set operations. This is achieved by combining a E-SMC scheme with data structures like Bloom Filters and bitmap strings.
△ Less
Submitted 28 January, 2011;
originally announced January 2011.
-
The Risk-Utility Tradeoff for IP Address Truncation
Authors:
Martin Burkhart,
Daniela Brauckhoff,
Martin May,
Elisa Boschi
Abstract:
Network operators are reluctant to share traffic data due to security and privacy concerns. Consequently, there is a lack of publicly available traces for validating and generalizing the latest results in network and security research. Anonymization is a possible solution in this context; however, it is unclear how the sanitization of data preserves characteristics important for traffic analysis…
▽ More
Network operators are reluctant to share traffic data due to security and privacy concerns. Consequently, there is a lack of publicly available traces for validating and generalizing the latest results in network and security research. Anonymization is a possible solution in this context; however, it is unclear how the sanitization of data preserves characteristics important for traffic analysis. In addition, the privacy-preserving property of state-of-the-art IP address anonymization techniques has come into question by recent attacks that successfully identified a large number of hosts in anonymized traces.
In this paper, we examine the tradeoff between data utility for anomaly detection and the risk of host identification for IP address truncation. Specifically, we analyze three weeks of unsampled and non-anonymized network traces from a medium-sized backbone network to assess data utility. The risk of de-anonymizing individual IP addresses is formally evaluated, using a metric based on conditional entropy.
Our results indicate that truncation effectively prevents host identification but degrades the utility of data for anomaly detection. However, the degree of degradation depends on the metric used and whether network-internal or external addresses are considered. Entropy metrics are more resistant to truncation than unique counts and the detection quality of anomalies degrades much faster in internal addresses than in external addresses. In particular, the usefulness of internal address counts is lost even for truncation of only 4 bits whereas utility of external address entropy is virtually unchanged even for truncation of 20 bits.
△ Less
Submitted 25 March, 2009;
originally announced March 2009.
-
SEPIA: Security through Private Information Aggregation
Authors:
Martin Burkhart,
Mario Strasser,
Dilip Many,
Xenofontas Dimitropoulos
Abstract:
Secure multiparty computation (MPC) allows joint privacy-preserving computations on data of multiple parties. Although MPC has been studied substantially, building solutions that are practical in terms of computation and communication cost is still a major challenge. In this paper, we investigate the practical usefulness of MPC for multi-domain network security and monitoring. We first optimize…
▽ More
Secure multiparty computation (MPC) allows joint privacy-preserving computations on data of multiple parties. Although MPC has been studied substantially, building solutions that are practical in terms of computation and communication cost is still a major challenge. In this paper, we investigate the practical usefulness of MPC for multi-domain network security and monitoring. We first optimize MPC comparison operations for processing high volume data in near real-time. We then design privacy-preserving protocols for event correlation and aggregation of network traffic statistics, such as addition of volume metrics, computation of feature entropy, and distinct item count. Optimizing performance of parallel invocations, we implement our protocols along with a complete set of basic operations in a library called SEPIA. We evaluate the running time and bandwidth requirements of our protocols in realistic settings on a local cluster as well as on PlanetLab and show that they work in near real-time for up to 140 input providers and 9 computation nodes. Compared to implementations using existing general-purpose MPC frameworks, our protocols are significantly faster, requiring, for example, 3 minutes for a task that takes 2 days with general-purpose frameworks. This improvement paves the way for new applications of MPC in the area of networking. Finally, we run SEPIA's protocols on real traffic traces of 17 networks and show how they provide new possibilities for distributed troubleshooting and early anomaly detection.
△ Less
Submitted 16 February, 2010; v1 submitted 25 March, 2009;
originally announced March 2009.
-
On the Utility of Anonymized Flow Traces for Anomaly Detection
Authors:
Martin Burkhart,
Daniela Brauckhoff,
Martin May
Abstract:
The sharing of network traces is an important prerequisite for the development and evaluation of efficient anomaly detection mechanisms. Unfortunately, privacy concerns and data protection laws prevent network operators from sharing these data. Anonymization is a promising solution in this context; however, it is unclear if the sanitization of data preserves the traffic characteristics or introd…
▽ More
The sharing of network traces is an important prerequisite for the development and evaluation of efficient anomaly detection mechanisms. Unfortunately, privacy concerns and data protection laws prevent network operators from sharing these data. Anonymization is a promising solution in this context; however, it is unclear if the sanitization of data preserves the traffic characteristics or introduces artifacts that may falsify traffic analysis results. In this paper, we examine the utility of anonymized flow traces for anomaly detection. We quantitatively evaluate the impact of IP address anonymization, namely variations of permutation and truncation, on the detectability of large-scale anomalies. Specifically, we analyze three weeks of un-sampled and non-anonymized network traces from a medium-sized backbone network. We find that all anonymization techniques, except prefix-preserving permutation, degrade the utility of data for anomaly detection. We show that the degree of degradation depends to a large extent on the nature and mix of anomalies present in a trace. Moreover, we present a case study that illustrates how traffic characteristics of individual hosts are distorted by anonymization.
△ Less
Submitted 9 October, 2008;
originally announced October 2008.
-
Flow-level Characteristics of Spam and Ham
Authors:
Dominik Schatzmann,
Martin Burkhart,
Thrasyvoulos Spyropoulos
Abstract:
Despite a large amount of effort devoted in the past years trying to limit unsolicited mail, spam is still a major global concern. Content-analysis techniques and blacklists, the most popular methods used to identify and block spam, are beginning to lose their edge in the battle. We argue here that one not only needs to look into the network-related characteristics of spam traffic, as has been r…
▽ More
Despite a large amount of effort devoted in the past years trying to limit unsolicited mail, spam is still a major global concern. Content-analysis techniques and blacklists, the most popular methods used to identify and block spam, are beginning to lose their edge in the battle. We argue here that one not only needs to look into the network-related characteristics of spam traffic, as has been recently suggested, but also to look deeper into the network core, in order to counter the increasing sophistication of spam-ing methods. Yet, at the same time, local knowledge available at a given server can often be irreplaceable in identifying specific spammers. To this end, in this paper we show how the local intelligence of mail servers can be gathered and correlated pas- sively at the ISP-level providing valuable network-wide information. Specifically, we use first a large network flow trace from a medium size, national ISP, to demonstrate that the pre-filtering decisions of individual mail servers can be tracked and combined at the flow level. Then, we argue that such aggregated knowledge not only can allow ISPs to develop and evaluate powerful new methods for fighting spam, but also to monitor remotely what their own servers are doing.
△ Less
Submitted 29 August, 2008;
originally announced August 2008.
