MediSyn: A Generalist Text-Guided Latent Diffusion Model For Diverse Medical Image Synthesis
Authors:
Joseph Cho,
Mrudang Mathur,
Cyril Zakka,
Dhamanpreet Kaur,
Matthew Leipzig,
Alex Dalal,
Aravind Krishnan,
Eubee Koo,
Karen Wai,
Cindy S. Zhao,
Rohan Shad,
Robyn Fong,
Ross Wightman,
Akshay Chaudhari,
William Hiesinger
Abstract:
Deep learning algorithms require extensive data to achieve robust performance. However, data availability is often restricted in the medical domain due to patient privacy concerns. Synthetic data presents a possible solution to these challenges. Recently, image generative models have found increasing use for medical applications but are often designed for singular medical specialties and imaging m…
▽ More
Deep learning algorithms require extensive data to achieve robust performance. However, data availability is often restricted in the medical domain due to patient privacy concerns. Synthetic data presents a possible solution to these challenges. Recently, image generative models have found increasing use for medical applications but are often designed for singular medical specialties and imaging modalities, thus limiting their broader utility. To address this, we introduce MediSyn: a text-guided, latent diffusion model capable of generating synthetic images from 6 medical specialties and 10 image types. The synthetic images are validated by expert clinicians for alignment with their corresponding text prompts. Furthermore, a direct comparison of the synthetic images against the real images confirms that our model synthesizes novel images and, crucially, may preserve patient privacy. Finally, classifiers trained on a mixture of synthetic and real data achieve similar performance to those trained on twice the amount of real data. Our findings highlight the immense potential for generalist image generative models to accelerate algorithmic research and development in medicine.
△ Less
Submitted 10 February, 2025; v1 submitted 16 May, 2024;
originally announced May 2024.
A Generalizable Deep Learning System for Cardiac MRI
Authors:
Rohan Shad,
Cyril Zakka,
Dhamanpreet Kaur,
Robyn Fong,
Ross Warren Filice,
John Mongan,
Kimberly Kalianos,
Nishith Khandwala,
David Eng,
Matthew Leipzig,
Walter Witschey,
Alejandro de Feria,
Victor Ferrari,
Euan Ashley,
Michael A. Acker,
Curtis Langlotz,
William Hiesinger
Abstract:
Cardiac MRI allows for a comprehensive assessment of myocardial structure, function, and tissue characteristics. Here we describe a foundational vision system for cardiac MRI, capable of representing the breadth of human cardiovascular disease and health. Our deep learning model is trained via self-supervised contrastive learning, by which visual concepts in cine-sequence cardiac MRI scans are lea…
▽ More
Cardiac MRI allows for a comprehensive assessment of myocardial structure, function, and tissue characteristics. Here we describe a foundational vision system for cardiac MRI, capable of representing the breadth of human cardiovascular disease and health. Our deep learning model is trained via self-supervised contrastive learning, by which visual concepts in cine-sequence cardiac MRI scans are learned from the raw text of the accompanying radiology reports. We train and evaluate our model on data from four large academic clinical institutions in the United States. We additionally showcase the performance of our models on the UK BioBank, and two additional publicly available external datasets. We explore emergent zero-shot capabilities of our system, and demonstrate remarkable performance across a range of tasks; including the problem of left ventricular ejection fraction regression, and the diagnosis of 35 different conditions such as cardiac amyloidosis and hypertrophic cardiomyopathy. We show that our deep learning system is capable of not only understanding the staggering complexity of human cardiovascular disease, but can be directed towards clinical problems of interest yielding impressive, clinical grade diagnostic accuracy with a fraction of the training data typically required for such tasks.
△ Less
Submitted 1 December, 2023;
originally announced December 2023.
