multiGradICON: A Foundation Model for Multimodal Medical Image Registration
Authors:
Basar Demir,
Lin Tian,
Thomas Hastings Greer,
Roland Kwitt,
Francois-Xavier Vialard,
Raul San Jose Estepar,
Sylvain Bouix,
Richard Jarrett Rushmore,
Ebrahim Ebrahim,
Marc Niethammer
Abstract:
Modern medical image registration approaches predict deformations using deep networks. These approaches achieve state-of-the-art (SOTA) registration accuracy and are generally fast. However, deep learning (DL) approaches are, in contrast to conventional non-deep-learning-based approaches, anatomy-specific. Recently, a universal deep registration approach, uniGradICON, has been proposed. However, u…
▽ More
Modern medical image registration approaches predict deformations using deep networks. These approaches achieve state-of-the-art (SOTA) registration accuracy and are generally fast. However, deep learning (DL) approaches are, in contrast to conventional non-deep-learning-based approaches, anatomy-specific. Recently, a universal deep registration approach, uniGradICON, has been proposed. However, uniGradICON focuses on monomodal image registration. In this work, we therefore develop multiGradICON as a first step towards universal *multimodal* medical image registration. Specifically, we show that 1) we can train a DL registration model that is suitable for monomodal *and* multimodal registration; 2) loss function randomization can increase multimodal registration accuracy; and 3) training a model with multimodal data helps multimodal generalization. Our code and the multiGradICON model are available at https://github.com/uncbiag/uniGradICON.
△ Less
Submitted 7 February, 2025; v1 submitted 31 July, 2024;
originally announced August 2024.
$\texttt{GradICON}$: Approximate Diffeomorphisms via Gradient Inverse Consistency
Authors:
Lin Tian,
Hastings Greer,
François-Xavier Vialard,
Roland Kwitt,
Raúl San José Estépar,
Richard Jarrett Rushmore,
Nikolaos Makris,
Sylvain Bouix,
Marc Niethammer
Abstract:
We present an approach to learning regular spatial transformations between image pairs in the context of medical image registration. Contrary to optimization-based registration techniques and many modern learning-based methods, we do not directly penalize transformation irregularities but instead promote transformation regularity via an inverse consistency penalty. We use a neural network to predi…
▽ More
We present an approach to learning regular spatial transformations between image pairs in the context of medical image registration. Contrary to optimization-based registration techniques and many modern learning-based methods, we do not directly penalize transformation irregularities but instead promote transformation regularity via an inverse consistency penalty. We use a neural network to predict a map between a source and a target image as well as the map when swapping the source and target images. Different from existing approaches, we compose these two resulting maps and regularize deviations of the $\bf{Jacobian}$ of this composition from the identity matrix. This regularizer -- $\texttt{GradICON}$ -- results in much better convergence when training registration models compared to promoting inverse consistency of the composition of maps directly while retaining the desirable implicit regularization effects of the latter. We achieve state-of-the-art registration performance on a variety of real-world medical image datasets using a single set of hyperparameters and a single non-dataset-specific training protocol.
△ Less
Submitted 9 October, 2023; v1 submitted 13 June, 2022;
originally announced June 2022.
