-
Multi-environment lifelong deep reinforcement learning for medical imaging
Authors:
Guangyao Zheng,
Shuhao Lai,
Vladimir Braverman,
Michael A. Jacobs,
Vishwa S. Parekh
Abstract:
Deep reinforcement learning(DRL) is increasingly being explored in medical imaging. However, the environments for medical imaging tasks are constantly evolving in terms of imaging orientations, imaging sequences, and pathologies. To that end, we developed a Lifelong DRL framework, SERIL to continually learn new tasks in changing imaging environments without catastrophic forgetting. SERIL was devel…
▽ More
Deep reinforcement learning(DRL) is increasingly being explored in medical imaging. However, the environments for medical imaging tasks are constantly evolving in terms of imaging orientations, imaging sequences, and pathologies. To that end, we developed a Lifelong DRL framework, SERIL to continually learn new tasks in changing imaging environments without catastrophic forgetting. SERIL was developed using selective experience replay based lifelong learning technique for the localization of five anatomical landmarks in brain MRI on a sequence of twenty-four different imaging environments. The performance of SERIL, when compared to two baseline setups: MERT(multi-environment-best-case) and SERT(single-environment-worst-case) demonstrated excellent performance with an average distance of $9.90\pm7.35$ pixels from the desired landmark across all 120 tasks, compared to $10.29\pm9.07$ for MERT and $36.37\pm22.41$ for SERT($p<0.05$), demonstrating the excellent potential for continuously learning multiple tasks across dynamically changing imaging environments.
△ Less
Submitted 31 May, 2023;
originally announced June 2023.
-
Asynchronous Decentralized Federated Lifelong Learning for Landmark Localization in Medical Imaging
Authors:
Guangyao Zheng,
Michael A. Jacobs,
Vladimir Braverman,
Vishwa S. Parekh
Abstract:
Federated learning is a recent development in the machine learning area that allows a system of devices to train on one or more tasks without sharing their data to a single location or device. However, this framework still requires a centralized global model to consolidate individual models into one, and the devices train synchronously, which both can be potential bottlenecks for using federated l…
▽ More
Federated learning is a recent development in the machine learning area that allows a system of devices to train on one or more tasks without sharing their data to a single location or device. However, this framework still requires a centralized global model to consolidate individual models into one, and the devices train synchronously, which both can be potential bottlenecks for using federated learning. In this paper, we propose a novel method of asynchronous decentralized federated lifelong learning (ADFLL) method that inherits the merits of federated learning and can train on multiple tasks simultaneously without the need for a central node or synchronous training. Thus, overcoming the potential drawbacks of conventional federated learning. We demonstrate excellent performance on the brain tumor segmentation (BRATS) dataset for localizing the left ventricle on multiple image sequences and image orientation. Our framework allows agents to achieve the best performance with a mean distance error of 7.81, better than the conventional all-knowing agent's mean distance error of 11.78, and significantly (p=0.01) better than a conventional lifelong learning agent with a distance error of 15.17 after eight rounds of training. In addition, all ADFLL agents have comparable or better performance than a conventional LL agent. In conclusion, we developed an ADFLL framework with excellent performance and speed-up compared to conventional RL agents.
△ Less
Submitted 10 January, 2024; v1 submitted 12 March, 2023;
originally announced March 2023.
-
Cross-Domain Federated Learning in Medical Imaging
Authors:
Vishwa S Parekh,
Shuhao Lai,
Vladimir Braverman,
Jeff Leal,
Steven Rowe,
Jay J Pillai,
Michael A Jacobs
Abstract:
Federated learning is increasingly being explored in the field of medical imaging to train deep learning models on large scale datasets distributed across different data centers while preserving privacy by avoiding the need to transfer sensitive patient information. In this manuscript, we explore federated learning in a multi-domain, multi-task setting wherein different participating nodes may con…
▽ More
Federated learning is increasingly being explored in the field of medical imaging to train deep learning models on large scale datasets distributed across different data centers while preserving privacy by avoiding the need to transfer sensitive patient information. In this manuscript, we explore federated learning in a multi-domain, multi-task setting wherein different participating nodes may contain datasets sourced from different domains and are trained to solve different tasks. We evaluated cross-domain federated learning for the tasks of object detection and segmentation across two different experimental settings: multi-modal and multi-organ. The result from our experiments on cross-domain federated learning framework were very encouraging with an overlap similarity of 0.79 for organ localization and 0.65 for lesion segmentation. Our results demonstrate the potential of federated learning in developing multi-domain, multi-task deep learning models without sharing data from different domains.
△ Less
Submitted 18 December, 2021;
originally announced December 2021.
-
Multiparametric Deep Learning Tissue Signatures for Muscular Dystrophy: Preliminary Results
Authors:
Alex E. Bocchieri,
Vishwa S. Parekh,
Kathryn R. Wagner. Shivani Ahlawat,
Vladimir Braverman,
Doris G. Leung,
Michael A. Jacobs
Abstract:
A current clinical challenge is identifying limb girdle muscular dystrophy 2I(LGMD2I)tissue changes in the thighs, in particular, separating fat, fat-infiltrated muscle, and muscle tissue. Deep learning algorithms have the ability to learn different features by using the inherent tissue contrasts from multiparametric magnetic resonance imaging (mpMRI). To that end, we developed a novel multiparame…
▽ More
A current clinical challenge is identifying limb girdle muscular dystrophy 2I(LGMD2I)tissue changes in the thighs, in particular, separating fat, fat-infiltrated muscle, and muscle tissue. Deep learning algorithms have the ability to learn different features by using the inherent tissue contrasts from multiparametric magnetic resonance imaging (mpMRI). To that end, we developed a novel multiparametric deep learning network (MPDL) tissue signature model based on mpMRI and applied it to LGMD2I. We demonstrate a new tissue signature model of muscular dystrophy with the MPDL algorithm segments different tissue types with excellent results.
△ Less
Submitted 31 July, 2019;
originally announced August 2019.
-
Multiparametric Deep Learning and Radiomics for Tumor Grading and Treatment Response Assessment of Brain Cancer: Preliminary Results
Authors:
Vishwa S. Parekh,
John Laterra,
Chetan Bettegowda,
Alex E. Bocchieri,
Jay J. Pillai,
Michael A. Jacobs
Abstract:
Radiomics is an exciting new area of texture research for extracting quantitative and morphological characteristics of pathological tissue. However, to date, only single images have been used for texture analysis. We have extended radiomic texture methods to use multiparametric (mp) data to get more complete information from all the images. These mpRadiomic methods could potentially provide a plat…
▽ More
Radiomics is an exciting new area of texture research for extracting quantitative and morphological characteristics of pathological tissue. However, to date, only single images have been used for texture analysis. We have extended radiomic texture methods to use multiparametric (mp) data to get more complete information from all the images. These mpRadiomic methods could potentially provide a platform for stratification of tumor grade as well as assessment of treatment response in brain tumors. In brain, multiparametric MRI (mpMRI) are based on contrast enhanced T1-weighted imaging (T1WI), T2WI, Fluid Attenuated Inversion Recovery (FLAIR), Diffusion Weighted Imaging (DWI) and Perfusion Weighted Imaging (PWI). Therefore, we applied our multiparametric radiomic framework (mpRadiomic) on 24 patients with brain tumors (8 grade II and 16 grade IV). The mpRadiomic framework classified grade IV tumors from grade II tumors with a sensitivity and specificity of 93% and 100%, respectively, with an AUC of 0.95. For treatment response, the mpRadiomic framework classified pseudo-progression from true-progression with an AUC of 0.93. In conclusion, the mpRadiomic analysis was able to effectively capture the multiparametric brain MRI texture and could be used as potential biomarkers for distinguishing grade IV from grade II tumors as well as determining true-progression from pseudo-progression.
△ Less
Submitted 10 June, 2019;
originally announced June 2019.
-
DreamNLP: Novel NLP System for Clinical Report Metadata Extraction using Count Sketch Data Streaming Algorithm: Preliminary Results
Authors:
Sanghyun Choi,
Nikita Ivkin,
Vladimir Braverman,
Michael A. Jacobs
Abstract:
Extracting information from electronic health records (EHR) is a challenging task since it requires prior knowledge of the reports and some natural language processing algorithm (NLP). With the growing number of EHR implementations, such knowledge is increasingly challenging to obtain in an efficient manner. We address this challenge by proposing a novel methodology to analyze large sets of EHRs u…
▽ More
Extracting information from electronic health records (EHR) is a challenging task since it requires prior knowledge of the reports and some natural language processing algorithm (NLP). With the growing number of EHR implementations, such knowledge is increasingly challenging to obtain in an efficient manner. We address this challenge by proposing a novel methodology to analyze large sets of EHRs using a modified Count Sketch data streaming algorithm termed DreamNLP. By using DreamNLP, we generate a dictionary of frequently occurring terms or heavy hitters in the EHRs using low computational memory compared to conventional counting approach other NLP programs use. We demonstrate the extraction of the most important breast diagnosis features from the EHRs in a set of patients that underwent breast imaging. Based on the analysis, extraction of these terms would be useful for defining important features for downstream tasks such as machine learning for precision medicine.
△ Less
Submitted 25 August, 2018;
originally announced September 2018.
