Multi-Modal Dataset Creation for Federated Learning with DICOM Structured Reports
Authors:
Malte Tölle,
Lukas Burger,
Halvar Kelm,
Florian André,
Peter Bannas,
Gerhard Diller,
Norbert Frey,
Philipp Garthe,
Stefan Groß,
Anja Hennemuth,
Lars Kaderali,
Nina Krüger,
Andreas Leha,
Simon Martin,
Alexander Meyer,
Eike Nagel,
Stefan Orwat,
Clemens Scherer,
Moritz Seiffert,
Jan Moritz Seliger,
Stefan Simm,
Tim Friede,
Tim Seidler,
Sandy Engelhardt
Abstract:
Purpose: Federated training is often hindered by heterogeneous datasets due to divergent data storage options, inconsistent naming schemes, varied annotation procedures, and disparities in label quality. This is particularly evident in the emerging multi-modal learning paradigms, where dataset harmonization including a uniform data representation and filtering options are of paramount importance.…
▽ More
Purpose: Federated training is often hindered by heterogeneous datasets due to divergent data storage options, inconsistent naming schemes, varied annotation procedures, and disparities in label quality. This is particularly evident in the emerging multi-modal learning paradigms, where dataset harmonization including a uniform data representation and filtering options are of paramount importance.
Methods: DICOM structured reports enable the standardized linkage of arbitrary information beyond the imaging domain and can be used within Python deep learning pipelines with highdicom. Building on this, we developed an open platform for data integration and interactive filtering capabilities that simplifies the process of assembling multi-modal datasets.
Results: In this study, we extend our prior work by showing its applicability to more and divergent data types, as well as streamlining datasets for federated training within an established consortium of eight university hospitals in Germany. We prove its concurrent filtering ability by creating harmonized multi-modal datasets across all locations for predicting the outcome after minimally invasive heart valve replacement. The data includes DICOM data (i.e. computed tomography images, electrocardiography scans) as well as annotations (i.e. calcification segmentations, pointsets and pacemaker dependency), and metadata (i.e. prosthesis and diagnoses).
Conclusion: Structured reports bridge the traditional gap between imaging systems and information systems. Utilizing the inherent DICOM reference system arbitrary data types can be queried concurrently to create meaningful cohorts for clinical studies. The graphical interface as well as example structured report templates will be made publicly available.
△ Less
Submitted 6 August, 2024; v1 submitted 12 July, 2024;
originally announced July 2024.
Real World Federated Learning with a Knowledge Distilled Transformer for Cardiac CT Imaging
Authors:
Malte Tölle,
Philipp Garthe,
Clemens Scherer,
Jan Moritz Seliger,
Andreas Leha,
Nina Krüger,
Stefan Simm,
Simon Martin,
Sebastian Eble,
Halvar Kelm,
Moritz Bednorz,
Florian André,
Peter Bannas,
Gerhard Diller,
Norbert Frey,
Stefan Groß,
Anja Hennemuth,
Lars Kaderali,
Alexander Meyer,
Eike Nagel,
Stefan Orwat,
Moritz Seiffert,
Tim Friede,
Tim Seidler,
Sandy Engelhardt
Abstract:
Federated learning is a renowned technique for utilizing decentralized data while preserving privacy. However, real-world applications often face challenges like partially labeled datasets, where only a few locations have certain expert annotations, leaving large portions of unlabeled data unused. Leveraging these could enhance transformer architectures ability in regimes with small and diversely…
▽ More
Federated learning is a renowned technique for utilizing decentralized data while preserving privacy. However, real-world applications often face challenges like partially labeled datasets, where only a few locations have certain expert annotations, leaving large portions of unlabeled data unused. Leveraging these could enhance transformer architectures ability in regimes with small and diversely annotated sets. We conduct the largest federated cardiac CT analysis to date (n=8,104) in a real-world setting across eight hospitals. Our two-step semi-supervised strategy distills knowledge from task-specific CNNs into a transformer. First, CNNs predict on unlabeled data per label type and then the transformer learns from these predictions with label-specific heads. This improves predictive accuracy and enables simultaneous learning of all partial labels across the federation, and outperforms UNet-based models in generalizability on downstream tasks. Code and model weights are made openly available for leveraging future cardiac CT analysis.
△ Less
Submitted 2 January, 2025; v1 submitted 10 July, 2024;
originally announced July 2024.
