-
MedLoRD: A Medical Low-Resource Diffusion Model for High-Resolution 3D CT Image Synthesis
Authors:
Marvin Seyfarth,
Salman Ul Hassan Dar,
Isabelle Ayx,
Matthias Alexander Fink,
Stefan O. Schoenberg,
Hans-Ulrich Kauczor,
Sandy Engelhardt
Abstract:
Advancements in AI for medical imaging offer significant potential. However, their applications are constrained by the limited availability of data and the reluctance of medical centers to share it due to patient privacy concerns. Generative models present a promising solution by creating synthetic data as a substitute for real patient data. However, medical images are typically high-dimensional,…
▽ More
Advancements in AI for medical imaging offer significant potential. However, their applications are constrained by the limited availability of data and the reluctance of medical centers to share it due to patient privacy concerns. Generative models present a promising solution by creating synthetic data as a substitute for real patient data. However, medical images are typically high-dimensional, and current state-of-the-art methods are often impractical for computational resource-constrained healthcare environments. These models rely on data sub-sampling, raising doubts about their feasibility and real-world applicability. Furthermore, many of these models are evaluated on quantitative metrics that alone can be misleading in assessing the image quality and clinical meaningfulness of the generated images. To address this, we introduce MedLoRD, a generative diffusion model designed for computational resource-constrained environments. MedLoRD is capable of generating high-dimensional medical volumes with resolutions up to 512$\times$512$\times$256, utilizing GPUs with only 24GB VRAM, which are commonly found in standard desktop workstations. MedLoRD is evaluated across multiple modalities, including Coronary Computed Tomography Angiography and Lung Computed Tomography datasets. Extensive evaluations through radiological evaluation, relative regional volume analysis, adherence to conditional masks, and downstream tasks show that MedLoRD generates high-fidelity images closely adhering to segmentation mask conditions, surpassing the capabilities of current state-of-the-art generative models for medical image synthesis in computational resource-constrained environments.
△ Less
Submitted 17 March, 2025;
originally announced March 2025.
-
Latent Pollution Model: The Hidden Carbon Footprint in 3D Image Synthesis
Authors:
Marvin Seyfarth,
Salman Ul Hassan Dar,
Sandy Engelhardt
Abstract:
Contemporary developments in generative AI are rapidly transforming the field of medical AI. These developments have been predominantly driven by the availability of large datasets and high computing power, which have facilitated a significant increase in model capacity. Despite their considerable potential, these models demand substantially high power, leading to high carbon dioxide (CO2) emissio…
▽ More
Contemporary developments in generative AI are rapidly transforming the field of medical AI. These developments have been predominantly driven by the availability of large datasets and high computing power, which have facilitated a significant increase in model capacity. Despite their considerable potential, these models demand substantially high power, leading to high carbon dioxide (CO2) emissions. Given the harm such models are causing to the environment, there has been little focus on the carbon footprints of such models. This study analyzes carbon emissions from 2D and 3D latent diffusion models (LDMs) during training and data generation phases, revealing a surprising finding: the synthesis of large images contributes most significantly to these emissions. We assess different scenarios including model sizes, image dimensions, distributed training, and data generation steps. Our findings reveal substantial carbon emissions from these models, with training 2D and 3D models comparable to driving a car for 10 km and 90 km, respectively. The process of data generation is even more significant, with CO2 emissions equivalent to driving 160 km for 2D models and driving for up to 3345 km for 3D synthesis. Additionally, we found that the location of the experiment can increase carbon emissions by up to 94 times, and even the time of year can influence emissions by up to 50%. These figures are alarming, considering they represent only a single training and data generation phase for each model. Our results emphasize the urgent need for developing environmentally sustainable strategies in generative AI.
△ Less
Submitted 20 July, 2024;
originally announced July 2024.
-
Unconditional Latent Diffusion Models Memorize Patient Imaging Data: Implications for Openly Sharing Synthetic Data
Authors:
Salman Ul Hassan Dar,
Marvin Seyfarth,
Isabelle Ayx,
Theano Papavassiliu,
Stefan O. Schoenberg,
Robert Malte Siepmann,
Fabian Christopher Laqua,
Jannik Kahmann,
Norbert Frey,
Bettina Baeßler,
Sebastian Foersch,
Daniel Truhn,
Jakob Nikolas Kather,
Sandy Engelhardt
Abstract:
AI models present a wide range of applications in the field of medicine. However, achieving optimal performance requires access to extensive healthcare data, which is often not readily available. Furthermore, the imperative to preserve patient privacy restricts patient data sharing with third parties and even within institutes. Recently, generative AI models have been gaining traction for facilita…
▽ More
AI models present a wide range of applications in the field of medicine. However, achieving optimal performance requires access to extensive healthcare data, which is often not readily available. Furthermore, the imperative to preserve patient privacy restricts patient data sharing with third parties and even within institutes. Recently, generative AI models have been gaining traction for facilitating open-data sharing by proposing synthetic data as surrogates of real patient data. Despite the promise, some of these models are susceptible to patient data memorization, where models generate patient data copies instead of novel synthetic samples. Considering the importance of the problem, surprisingly it has received relatively little attention in the medical imaging community. To this end, we assess memorization in unconditional latent diffusion models. We train latent diffusion models on CT, MR, and X-ray datasets for synthetic data generation. We then detect the amount of training data memorized utilizing our novel self-supervised copy detection approach and further investigate various factors that can influence memorization. Our findings show a surprisingly high degree of patient data memorization across all datasets. Comparison with non-diffusion generative models, such as autoencoders and generative adversarial networks, indicates that while latent diffusion models are more susceptible to memorization, overall they outperform non-diffusion models in synthesis quality. Further analyses reveal that using augmentation strategies, small architecture, and increasing dataset can reduce memorization while over-training the models can enhance it. Collectively, our results emphasize the importance of carefully training generative models on private medical imaging datasets, and examining the synthetic data to ensure patient privacy before sharing it for medical research and applications.
△ Less
Submitted 7 January, 2025; v1 submitted 1 February, 2024;
originally announced February 2024.
-
Stable Numerical Integration of an Epitaxial Growth Model with Slope Selection
Authors:
Gregory M. Seyfarth,
Benjamin P. Vollmayr-Lee
Abstract:
We consider a continuum phase field model for crystal growth via molecular beam epitaxy, with the goal of determining stable numerical time integration methods for the dynamics. We parametrize a class of semi-implicit methods that are linear in the updated field, which allows for efficient implementation with fast Fourier transforms. We perform unconditional von Neumann stability analysis to ident…
▽ More
We consider a continuum phase field model for crystal growth via molecular beam epitaxy, with the goal of determining stable numerical time integration methods for the dynamics. We parametrize a class of semi-implicit methods that are linear in the updated field, which allows for efficient implementation with fast Fourier transforms. We perform unconditional von Neumann stability analysis to identify the region of stability in parameter space, and then test these predictions numerically for gradient stability. We find strong agreement between the approaches.
△ Less
Submitted 4 July, 2013; v1 submitted 21 March, 2013;
originally announced March 2013.
