-
Motion-Robust T2* Quantification from Gradient Echo MRI with Physics-Informed Deep Learning
Authors:
Hannah Eichhorn,
Veronika Spieker,
Kerstin Hammernik,
Elisa Saks,
Lina Felsner,
Kilian Weiss,
Christine Preibisch,
Julia A. Schnabel
Abstract:
Purpose: T2* quantification from gradient echo magnetic resonance imaging is particularly affected by subject motion due to the high sensitivity to magnetic field inhomogeneities, which are influenced by motion and might cause signal loss. Thus, motion correction is crucial to obtain high-quality T2* maps.
Methods: We extend our previously introduced learning-based physics-informed motion correc…
▽ More
Purpose: T2* quantification from gradient echo magnetic resonance imaging is particularly affected by subject motion due to the high sensitivity to magnetic field inhomogeneities, which are influenced by motion and might cause signal loss. Thus, motion correction is crucial to obtain high-quality T2* maps.
Methods: We extend our previously introduced learning-based physics-informed motion correction method, PHIMO, by utilizing acquisition knowledge to enhance the reconstruction performance for challenging motion patterns and increase PHIMO's robustness to varying strengths of magnetic field inhomogeneities across the brain. We perform comprehensive evaluations regarding motion detection accuracy and image quality for data with simulated and real motion.
Results: Our extended version of PHIMO outperforms the learning-based baseline methods both qualitatively and quantitatively with respect to line detection and image quality. Moreover, PHIMO performs on-par with a conventional state-of-the-art motion correction method for T2* quantification from gradient echo MRI, which relies on redundant data acquisition.
Conclusion: PHIMO's competitive motion correction performance, combined with a reduction in acquisition time by over 40% compared to the state-of-the-art method, make it a promising solution for motion-robust T2* quantification in research settings and clinical routine.
△ Less
Submitted 24 February, 2025;
originally announced February 2025.
-
Graph Neural Networks: A suitable Alternative to MLPs in Latent 3D Medical Image Classification?
Authors:
Johannes Kiechle,
Daniel M. Lang,
Stefan M. Fischer,
Lina Felsner,
Jan C. Peeken,
Julia A. Schnabel
Abstract:
Recent studies have underscored the capabilities of natural imaging foundation models to serve as powerful feature extractors, even in a zero-shot setting for medical imaging data. Most commonly, a shallow multi-layer perceptron (MLP) is appended to the feature extractor to facilitate end-to-end learning and downstream prediction tasks such as classification, thus representing the de facto standar…
▽ More
Recent studies have underscored the capabilities of natural imaging foundation models to serve as powerful feature extractors, even in a zero-shot setting for medical imaging data. Most commonly, a shallow multi-layer perceptron (MLP) is appended to the feature extractor to facilitate end-to-end learning and downstream prediction tasks such as classification, thus representing the de facto standard. However, as graph neural networks (GNNs) have become a practicable choice for various tasks in medical research in the recent past, we direct attention to the question of how effective GNNs are compared to MLP prediction heads for the task of 3D medical image classification, proposing them as a potential alternative. In our experiments, we devise a subject-level graph for each volumetric dataset instance. Therein latent representations of all slices in the volume, encoded through a DINOv2 pretrained vision transformer (ViT), constitute the nodes and their respective node features. We use public datasets to compare the classification heads numerically and evaluate various graph construction and graph convolution methods in our experiments. Our findings show enhancements of the GNN in classification performance and substantial improvements in runtime compared to an MLP prediction head. Additional robustness evaluations further validate the promising performance of the GNN, promoting them as a suitable alternative to traditional MLP classification heads. Our code is publicly available at: https://github.com/compai-lab/2024-miccai-grail-kiechle
△ Less
Submitted 24 July, 2024;
originally announced July 2024.
-
Progressive Growing of Patch Size: Resource-Efficient Curriculum Learning for Dense Prediction Tasks
Authors:
Stefan M. Fischer,
Lina Felsner,
Richard Osuala,
Johannes Kiechle,
Daniel M. Lang,
Jan C. Peeken,
Julia A. Schnabel
Abstract:
In this work, we introduce Progressive Growing of Patch Size, a resource-efficient implicit curriculum learning approach for dense prediction tasks. Our curriculum approach is defined by growing the patch size during model training, which gradually increases the task's difficulty. We integrated our curriculum into the nnU-Net framework and evaluated the methodology on all 10 tasks of the Medical S…
▽ More
In this work, we introduce Progressive Growing of Patch Size, a resource-efficient implicit curriculum learning approach for dense prediction tasks. Our curriculum approach is defined by growing the patch size during model training, which gradually increases the task's difficulty. We integrated our curriculum into the nnU-Net framework and evaluated the methodology on all 10 tasks of the Medical Segmentation Decathlon. With our approach, we are able to substantially reduce runtime, computational costs, and CO2 emissions of network training compared to classical constant patch size training. In our experiments, the curriculum approach resulted in improved convergence. We are able to outperform standard nnU-Net training, which is trained with constant patch size, in terms of Dice Score on 7 out of 10 MSD tasks while only spending roughly 50% of the original training runtime. To the best of our knowledge, our Progressive Growing of Patch Size is the first successful employment of a sample-length curriculum in the form of patch size in the field of computer vision. Our code is publicly available at https://github.com/compai-lab/2024-miccai-fischer.
△ Less
Submitted 11 July, 2024; v1 submitted 10 July, 2024;
originally announced July 2024.
-
Data-Driven Tissue- and Subject-Specific Elastic Regularization for Medical Image Registration
Authors:
Anna Reithmeir,
Lina Felsner,
Rickmer Braren,
Julia A. Schnabel,
Veronika A. Zimmer
Abstract:
Physics-inspired regularization is desired for intra-patient image registration since it can effectively capture the biomechanical characteristics of anatomical structures. However, a major challenge lies in the reliance on physical parameters: Parameter estimations vary widely across the literature, and the physical properties themselves are inherently subject-specific. In this work, we introduce…
▽ More
Physics-inspired regularization is desired for intra-patient image registration since it can effectively capture the biomechanical characteristics of anatomical structures. However, a major challenge lies in the reliance on physical parameters: Parameter estimations vary widely across the literature, and the physical properties themselves are inherently subject-specific. In this work, we introduce a novel data-driven method that leverages hypernetworks to learn the tissue-dependent elasticity parameters of an elastic regularizer. Notably, our approach facilitates the estimation of patient-specific parameters without the need to retrain the network. We evaluate our method on three publicly available 2D and 3D lung CT and cardiac MR datasets. We find that with our proposed subject-specific tissue-dependent regularization, a higher registration quality is achieved across all datasets compared to using a global regularizer. The code is available at https://github.com/compai-lab/2024-miccai-reithmeir.
△ Less
Submitted 5 July, 2024;
originally announced July 2024.
-
Language Models Meet Anomaly Detection for Better Interpretability and Generalizability
Authors:
Jun Li,
Su Hwan Kim,
Philip Müller,
Lina Felsner,
Daniel Rueckert,
Benedikt Wiestler,
Julia A. Schnabel,
Cosmin I. Bercea
Abstract:
This research explores the integration of language models and unsupervised anomaly detection in medical imaging, addressing two key questions: (1) Can language models enhance the interpretability of anomaly detection maps? and (2) Can anomaly maps improve the generalizability of language models in open-set anomaly detection tasks? To investigate these questions, we introduce a new dataset for mult…
▽ More
This research explores the integration of language models and unsupervised anomaly detection in medical imaging, addressing two key questions: (1) Can language models enhance the interpretability of anomaly detection maps? and (2) Can anomaly maps improve the generalizability of language models in open-set anomaly detection tasks? To investigate these questions, we introduce a new dataset for multi-image visual question-answering on brain magnetic resonance images encompassing multiple conditions. We propose KQ-Former (Knowledge Querying Transformer), which is designed to optimally align visual and textual information in limited-sample contexts. Our model achieves a 60.81% accuracy on closed questions, covering disease classification and severity across 15 different classes. For open questions, KQ-Former demonstrates a 70% improvement over the baseline with a BLEU-4 score of 0.41, and achieves the highest entailment ratios (up to 71.9%) and lowest contradiction ratios (down to 10.0%) among various natural language inference models. Furthermore, integrating anomaly maps results in an 18% accuracy increase in detecting open-set anomalies, thereby enhancing the language model's generalizability to previously unseen medical conditions. The code and dataset are available at https://github.com/compai-lab/miccai-2024-junli?tab=readme-ov-file
△ Less
Submitted 23 July, 2024; v1 submitted 11 April, 2024;
originally announced April 2024.
-
Learned Cone-Beam CT Reconstruction Using Neural Ordinary Differential Equations
Authors:
Mareike Thies,
Fabian Wagner,
Mingxuan Gu,
Lukas Folle,
Lina Felsner,
Andreas Maier
Abstract:
Learned iterative reconstruction algorithms for inverse problems offer the flexibility to combine analytical knowledge about the problem with modules learned from data. This way, they achieve high reconstruction performance while ensuring consistency with the measured data. In computed tomography, extending such approaches from 2D fan-beam to 3D cone-beam data is challenging due to the prohibitive…
▽ More
Learned iterative reconstruction algorithms for inverse problems offer the flexibility to combine analytical knowledge about the problem with modules learned from data. This way, they achieve high reconstruction performance while ensuring consistency with the measured data. In computed tomography, extending such approaches from 2D fan-beam to 3D cone-beam data is challenging due to the prohibitively high GPU memory that would be needed to train such models. This paper proposes to use neural ordinary differential equations to solve the reconstruction problem in a residual formulation via numerical integration. For training, there is no need to backpropagate through several unrolled network blocks nor through the internals of the solver. Instead, the gradients are obtained very memory-efficiently in the neural ODE setting allowing for training on a single consumer graphics card. The method is able to reduce the root mean squared error by over 30% compared to the best performing classical iterative reconstruction algorithm and produces high quality cone-beam reconstructions even in a sparse view scenario.
△ Less
Submitted 19 January, 2022;
originally announced January 2022.
-
X-ray Scatter Estimation Using Deep Splines
Authors:
Philipp Roser,
Annette Birkhold,
Alexander Preuhs,
Christopher Syben,
Lina Felsner,
Elisabeth Hoppe,
Norbert Strobel,
Markus Korwarschik,
Rebecca Fahrig,
Andreas Maier
Abstract:
Algorithmic X-ray scatter compensation is a desirable technique in flat-panel X-ray imaging and cone-beam computed tomography. State-of-the-art U-net based image translation approaches yielded promising results. As there are no physics constraints applied to the output of the U-Net, it cannot be ruled out that it yields spurious results. Unfortunately, those may be misleading in the context of med…
▽ More
Algorithmic X-ray scatter compensation is a desirable technique in flat-panel X-ray imaging and cone-beam computed tomography. State-of-the-art U-net based image translation approaches yielded promising results. As there are no physics constraints applied to the output of the U-Net, it cannot be ruled out that it yields spurious results. Unfortunately, those may be misleading in the context of medical imaging. To overcome this problem, we propose to embed B-splines as a known operator into neural networks. This inherently limits their predictions to well-behaved and smooth functions. In a study using synthetic head and thorax data as well as real thorax phantom data, we found that our approach performed on par with U-net when comparing both algorithms based on quantitative performance metrics. However, our approach not only reduces runtime and parameter complexity, but we also found it much more robust to unseen noise levels. While the U-net responded with visible artifacts, our approach preserved the X-ray signal's frequency characteristics.
△ Less
Submitted 22 January, 2021;
originally announced January 2021.
-
2-D Respiration Navigation Framework for 3-D Continuous Cardiac Magnetic Resonance Imaging
Authors:
Elisabeth Hoppe,
Jens Wetzl,
Philipp Roser,
Lina Felsner,
Alexander Preuhs,
Andreas Maier
Abstract:
Continuous protocols for cardiac magnetic resonance imaging enable sampling of the cardiac anatomy simultaneously resolved into cardiac phases. To avoid respiration artifacts, associated motion during the scan has to be compensated for during reconstruction. In this paper, we propose a sampling adaption to acquire 2-D respiration information during a continuous scan. Further, we develop a pipeline…
▽ More
Continuous protocols for cardiac magnetic resonance imaging enable sampling of the cardiac anatomy simultaneously resolved into cardiac phases. To avoid respiration artifacts, associated motion during the scan has to be compensated for during reconstruction. In this paper, we propose a sampling adaption to acquire 2-D respiration information during a continuous scan. Further, we develop a pipeline to extract the different respiration states from the acquired signals, which are used to reconstruct data from one respiration phase. Our results show the benefit of the proposed workflow on the image quality compared to no respiration compensation, as well as a previous 1-D respiration navigation approach.
△ Less
Submitted 26 December, 2020;
originally announced December 2020.
-
Reconstruction of Voxels with Position- and Angle-Dependent Weightings
Authors:
Lina Felsner,
Tobias Würfl,
Christopher Syben,
Philipp Roser,
Alexander Preuhs,
Andreas Maier,
Christian Riess
Abstract:
The reconstruction problem of voxels with individual weightings can be modeled a position- and angle- dependent function in the forward-projection. This changes the system matrix and prohibits to use standard filtered backprojection. In this work we first formulate this reconstruction problem in terms of a system matrix and weighting part. We compute the pseudoinverse and show that the solution is…
▽ More
The reconstruction problem of voxels with individual weightings can be modeled a position- and angle- dependent function in the forward-projection. This changes the system matrix and prohibits to use standard filtered backprojection. In this work we first formulate this reconstruction problem in terms of a system matrix and weighting part. We compute the pseudoinverse and show that the solution is rank-deficient and hence very ill posed. This is a fundamental limitation for reconstruction. We then derive an iterative solution and experimentally show its uperiority to any closed-form solution.
△ Less
Submitted 27 October, 2020;
originally announced October 2020.
-
A 3-D Projection Model for X-ray Dark-field Imaging
Authors:
Shiyang Hu,
Lina Felsner,
Andreas Maier,
Veronika Ludwig,
Gisela Anton,
Christian Riess
Abstract:
Talbot-Lau X-ray phase-contrast imaging is a novel imaging modality, which provides not only an X-ray absorption image, but also additionally a differential phase image and a dark-field image. The dark-field image is related to small angle scattering and has an interesting property when canning oriented structures: the recorded signal depends on the relative orientation of the structure in the ima…
▽ More
Talbot-Lau X-ray phase-contrast imaging is a novel imaging modality, which provides not only an X-ray absorption image, but also additionally a differential phase image and a dark-field image. The dark-field image is related to small angle scattering and has an interesting property when canning oriented structures: the recorded signal depends on the relative orientation of the structure in the imaging system. Exactly this property allows to draw conclusions about the orientation and to reconstruct the structure. However, the reconstruction is a complex, non-trivial challenge. A lot of research was conducted towards this goal in the last years and several reconstruction algorithms were proposed. A key step of the reconstruction algorithm is the inversion of a forward projection model. Up until now, only 2-D projection models are available, with effectively limit the scanning trajectory to a 2-D plane. To obtain true 3-D information, this limitation requires to combine several 2-D scans, which leads to quite complex, impractical acquisitions schemes. Furthermore, it is not possible with these models to use 3-D trajectories that might allow simpler protocols, like for example a helical trajectory. To address these limitations, we propose in this work a very general 3-D projection model. Our projection model defines the dark-field signal dependent on an arbitrarily chosen ray and sensitivity direction. We derive the projection model under the assumption that the observed scatter distribution has a Gaussian shape. We theoretically show the consistency of our model with more constrained existing 2-D models. Furthermore, we experimentally show the compatibility of our model with dark-field measurements of two matchsticks. We believe that this 3-D projection model is an important step towards more flexible trajectories and imaging protocols that are much better applicable in practice.
△ Less
Submitted 4 March, 2019; v1 submitted 11 November, 2018;
originally announced November 2018.
-
Phase-Sensitive Region-of-Interest Computed Tomography
Authors:
Lina Felsner,
Martin Berger,
Sebastian Kaeppler,
Johannes Bopp,
Veronika Ludwig,
Thomas Weber,
Georg Pelzer,
Thilo Michel,
Andreas Maier,
Gisela Anton,
Christian Riess
Abstract:
X-Ray Phase-Contrast Imaging (PCI) yields absorption, differential phase, and dark-field images. Computed Tomography (CT) of grating-based PCI can in principle provide high-resolution soft-tissue contrast. Recently, grating-based PCI took several hurdles towards clinical implementation by addressing, for example, acquisition speed, high X-ray energies, and system vibrations. However, a critical im…
▽ More
X-Ray Phase-Contrast Imaging (PCI) yields absorption, differential phase, and dark-field images. Computed Tomography (CT) of grating-based PCI can in principle provide high-resolution soft-tissue contrast. Recently, grating-based PCI took several hurdles towards clinical implementation by addressing, for example, acquisition speed, high X-ray energies, and system vibrations. However, a critical impediment in all grating-based systems lies in limits that constrain the grating diameter to few centimeters. In this work, we propose a system and a reconstruction algorithm to circumvent this constraint in a clinically compatible way. We propose to perform a phase-sensitive Region-of-Interest (ROI) CT within a full-field absorption CT. The biggest advantage of this approach is that it allows to correct for phase truncation artifacts, and to obtain quantitative phase values. Our method is robust, and shows high-quality results on simulated data and on a biological mouse sample. This work is a proof of concept showing the potential to use PCI in CT on large specimen, such as humans, in clinical applications.
△ Less
Submitted 24 May, 2018;
originally announced May 2018.
