-
Robust and Efficient Medical Imaging with Self-Supervision
Authors:
Shekoofeh Azizi,
Laura Culp,
Jan Freyberg,
Basil Mustafa,
Sebastien Baur,
Simon Kornblith,
Ting Chen,
Patricia MacWilliams,
S. Sara Mahdavi,
Ellery Wulczyn,
Boris Babenko,
Megan Wilson,
Aaron Loh,
Po-Hsuan Cameron Chen,
Yuan Liu,
Pinal Bavishi,
Scott Mayer McKinney,
Jim Winkens,
Abhijit Guha Roy,
Zach Beaver,
Fiona Ryan,
Justin Krogue,
Mozziyar Etemadi,
Umesh Telang,
Yun Liu
, et al. (9 additional authors not shown)
Abstract:
Recent progress in Medical Artificial Intelligence (AI) has delivered systems that can reach clinical expert level performance. However, such systems tend to demonstrate sub-optimal "out-of-distribution" performance when evaluated in clinical settings different from the training environment. A common mitigation strategy is to develop separate systems for each clinical setting using site-specific d…
▽ More
Recent progress in Medical Artificial Intelligence (AI) has delivered systems that can reach clinical expert level performance. However, such systems tend to demonstrate sub-optimal "out-of-distribution" performance when evaluated in clinical settings different from the training environment. A common mitigation strategy is to develop separate systems for each clinical setting using site-specific data [1]. However, this quickly becomes impractical as medical data is time-consuming to acquire and expensive to annotate [2]. Thus, the problem of "data-efficient generalization" presents an ongoing difficulty for Medical AI development. Although progress in representation learning shows promise, their benefits have not been rigorously studied, specifically for out-of-distribution settings. To meet these challenges, we present REMEDIS, a unified representation learning strategy to improve robustness and data-efficiency of medical imaging AI. REMEDIS uses a generic combination of large-scale supervised transfer learning with self-supervised learning and requires little task-specific customization. We study a diverse range of medical imaging tasks and simulate three realistic application scenarios using retrospective data. REMEDIS exhibits significantly improved in-distribution performance with up to 11.5% relative improvement in diagnostic accuracy over a strong supervised baseline. More importantly, our strategy leads to strong data-efficient generalization of medical imaging AI, matching strong supervised baselines using between 1% to 33% of retraining data across tasks. These results suggest that REMEDIS can significantly accelerate the life-cycle of medical imaging AI development thereby presenting an important step forward for medical imaging AI to deliver broad impact.
△ Less
Submitted 3 July, 2022; v1 submitted 19 May, 2022;
originally announced May 2022.
-
Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions
Authors:
Abhijit Guha Roy,
Jie Ren,
Shekoofeh Azizi,
Aaron Loh,
Vivek Natarajan,
Basil Mustafa,
Nick Pawlowski,
Jan Freyberg,
Yuan Liu,
Zach Beaver,
Nam Vo,
Peggy Bui,
Samantha Winter,
Patricia MacWilliams,
Greg S. Corrado,
Umesh Telang,
Yun Liu,
Taylan Cemgil,
Alan Karthikesalingam,
Balaji Lakshminarayanan,
Jim Winkens
Abstract:
We develop and rigorously evaluate a deep learning based system that can accurately classify skin conditions while detecting rare conditions for which there is not enough data available for training a confident classifier. We frame this task as an out-of-distribution (OOD) detection problem. Our novel approach, hierarchical outlier detection (HOD) assigns multiple abstention classes for each train…
▽ More
We develop and rigorously evaluate a deep learning based system that can accurately classify skin conditions while detecting rare conditions for which there is not enough data available for training a confident classifier. We frame this task as an out-of-distribution (OOD) detection problem. Our novel approach, hierarchical outlier detection (HOD) assigns multiple abstention classes for each training outlier class and jointly performs a coarse classification of inliers vs. outliers, along with fine-grained classification of the individual classes. We demonstrate the effectiveness of the HOD loss in conjunction with modern representation learning approaches (BiT, SimCLR, MICLe) and explore different ensembling strategies for further improving the results. We perform an extensive subgroup analysis over conditions of varying risk levels and different skin types to investigate how the OOD detection performance changes over each subgroup and demonstrate the gains of our framework in comparison to baselines. Finally, we introduce a cost metric to approximate downstream clinical impact. We use this cost metric to compare the proposed method against a baseline system, thereby making a stronger case for the overall system effectiveness in a real-world deployment scenario.
△ Less
Submitted 8 April, 2021;
originally announced April 2021.
-
Big Self-Supervised Models Advance Medical Image Classification
Authors:
Shekoofeh Azizi,
Basil Mustafa,
Fiona Ryan,
Zachary Beaver,
Jan Freyberg,
Jonathan Deaton,
Aaron Loh,
Alan Karthikesalingam,
Simon Kornblith,
Ting Chen,
Vivek Natarajan,
Mohammad Norouzi
Abstract:
Self-supervised pretraining followed by supervised fine-tuning has seen success in image recognition, especially when labeled examples are scarce, but has received limited attention in medical image analysis. This paper studies the effectiveness of self-supervised learning as a pretraining strategy for medical image classification. We conduct experiments on two distinct tasks: dermatology skin con…
▽ More
Self-supervised pretraining followed by supervised fine-tuning has seen success in image recognition, especially when labeled examples are scarce, but has received limited attention in medical image analysis. This paper studies the effectiveness of self-supervised learning as a pretraining strategy for medical image classification. We conduct experiments on two distinct tasks: dermatology skin condition classification from digital camera images and multi-label chest X-ray classification, and demonstrate that self-supervised learning on ImageNet, followed by additional self-supervised learning on unlabeled domain-specific medical images significantly improves the accuracy of medical image classifiers. We introduce a novel Multi-Instance Contrastive Learning (MICLe) method that uses multiple images of the underlying pathology per patient case, when available, to construct more informative positive pairs for self-supervised learning. Combining our contributions, we achieve an improvement of 6.7% in top-1 accuracy and an improvement of 1.1% in mean AUC on dermatology and chest X-ray classification respectively, outperforming strong supervised baselines pretrained on ImageNet. In addition, we show that big self-supervised models are robust to distribution shift and can learn efficiently with a small number of labeled medical images.
△ Less
Submitted 1 April, 2021; v1 submitted 13 January, 2021;
originally announced January 2021.
