Downstream Analysis of Foundational Medical Vision Models for Disease Progression
Authors:
Basar Demir,
Soumitri Chattopadhyay,
Thomas Hastings Greer,
Boqi Chen,
Marc Niethammer
Abstract:
Medical vision foundational models are used for a wide variety of tasks, including medical image segmentation and registration. This work evaluates the ability of these models to predict disease progression using a simple linear probe. We hypothesize that intermediate layer features of segmentation models capture structural information, while those of registration models encode knowledge of change…
▽ More
Medical vision foundational models are used for a wide variety of tasks, including medical image segmentation and registration. This work evaluates the ability of these models to predict disease progression using a simple linear probe. We hypothesize that intermediate layer features of segmentation models capture structural information, while those of registration models encode knowledge of change over time. Beyond demonstrating that these features are useful for disease progression prediction, we also show that registration model features do not require spatially aligned input images. However, for segmentation models, spatial alignment is essential for optimal performance. Our findings highlight the importance of spatial alignment and the utility of foundation model features for image registration.
△ Less
Submitted 21 March, 2025;
originally announced March 2025.
multiGradICON: A Foundation Model for Multimodal Medical Image Registration
Authors:
Basar Demir,
Lin Tian,
Thomas Hastings Greer,
Roland Kwitt,
Francois-Xavier Vialard,
Raul San Jose Estepar,
Sylvain Bouix,
Richard Jarrett Rushmore,
Ebrahim Ebrahim,
Marc Niethammer
Abstract:
Modern medical image registration approaches predict deformations using deep networks. These approaches achieve state-of-the-art (SOTA) registration accuracy and are generally fast. However, deep learning (DL) approaches are, in contrast to conventional non-deep-learning-based approaches, anatomy-specific. Recently, a universal deep registration approach, uniGradICON, has been proposed. However, u…
▽ More
Modern medical image registration approaches predict deformations using deep networks. These approaches achieve state-of-the-art (SOTA) registration accuracy and are generally fast. However, deep learning (DL) approaches are, in contrast to conventional non-deep-learning-based approaches, anatomy-specific. Recently, a universal deep registration approach, uniGradICON, has been proposed. However, uniGradICON focuses on monomodal image registration. In this work, we therefore develop multiGradICON as a first step towards universal *multimodal* medical image registration. Specifically, we show that 1) we can train a DL registration model that is suitable for monomodal *and* multimodal registration; 2) loss function randomization can increase multimodal registration accuracy; and 3) training a model with multimodal data helps multimodal generalization. Our code and the multiGradICON model are available at https://github.com/uncbiag/uniGradICON.
△ Less
Submitted 7 February, 2025; v1 submitted 31 July, 2024;
originally announced August 2024.
