-
Uncertainty-Aware Test-Time Adaptation for Inverse Consistent Diffeomorphic Lung Image Registration
Authors:
Muhammad F. A. Chaudhary,
Stephanie M. Aguilera,
Arie Nakhmani,
Joseph M. Reinhardt,
Surya P. Bhatt,
Sandeep Bodduluri
Abstract:
Diffeomorphic deformable image registration ensures smooth invertible transformations across inspiratory and expiratory chest CT scans. Yet, in practice, deep learning-based diffeomorphic methods struggle to capture large deformations between inspiratory and expiratory volumes, and therefore lack inverse consistency. Existing methods also fail to account for model uncertainty, which can be useful…
▽ More
Diffeomorphic deformable image registration ensures smooth invertible transformations across inspiratory and expiratory chest CT scans. Yet, in practice, deep learning-based diffeomorphic methods struggle to capture large deformations between inspiratory and expiratory volumes, and therefore lack inverse consistency. Existing methods also fail to account for model uncertainty, which can be useful for improving performance. We propose an uncertainty-aware test-time adaptation framework for inverse consistent diffeomorphic lung registration. Our method uses Monte Carlo (MC) dropout to estimate spatial uncertainty that is used to improve model performance. We train and evaluate our method for inspiratory-to-expiratory CT registration on a large cohort of 675 subjects from the COPDGene study, achieving a higher Dice similarity coefficient (DSC) between the lung boundaries (0.966) compared to both VoxelMorph (0.953) and TransMorph (0.953). Our method demonstrates consistent improvements in the inverse registration direction as well with an overall DSC of 0.966, higher than VoxelMorph (0.958) and TransMorph (0.956). Paired t-tests indicate statistically significant improvements.
△ Less
Submitted 12 November, 2024;
originally announced November 2024.
-
Earth System Data Cubes: Avenues for advancing Earth system research
Authors:
David Montero,
Guido Kraemer,
Anca Anghelea,
César Aybar,
Gunnar Brandt,
Gustau Camps-Valls,
Felix Cremer,
Ida Flik,
Fabian Gans,
Sarah Habershon,
Chaonan Ji,
Teja Kattenborn,
Laura Martínez-Ferrer,
Francesco Martinuzzi,
Martin Reinhardt,
Maximilian Söchting,
Khalil Teber,
Miguel D. Mahecha
Abstract:
Recent advancements in Earth system science have been marked by the exponential increase in the availability of diverse, multivariate datasets characterised by moderate to high spatio-temporal resolutions. Earth System Data Cubes (ESDCs) have emerged as one suitable solution for transforming this flood of data into a simple yet robust data structure. ESDCs achieve this by organising data into an a…
▽ More
Recent advancements in Earth system science have been marked by the exponential increase in the availability of diverse, multivariate datasets characterised by moderate to high spatio-temporal resolutions. Earth System Data Cubes (ESDCs) have emerged as one suitable solution for transforming this flood of data into a simple yet robust data structure. ESDCs achieve this by organising data into an analysis-ready format aligned with a spatio-temporal grid, facilitating user-friendly analysis and diminishing the need for extensive technical data processing knowledge. Despite these significant benefits, the completion of the entire ESDC life cycle remains a challenging task. Obstacles are not only of a technical nature but also relate to domain-specific problems in Earth system research. There exist barriers to realising the full potential of data collections in light of novel cloud-based technologies, particularly in curating data tailored for specific application domains. These include transforming data to conform to a spatio-temporal grid with minimum distortions and managing complexities such as spatio-temporal autocorrelation issues. Addressing these challenges is pivotal for the effective application of Artificial Intelligence (AI) approaches. Furthermore, adhering to open science principles for data dissemination, reproducibility, visualisation, and reuse is crucial for fostering sustainable research. Overcoming these challenges offers a substantial opportunity to advance data-driven Earth system research, unlocking the full potential of an integrated, multidimensional view of Earth system processes. This is particularly true when such research is coupled with innovative research paradigms and technological progress.
△ Less
Submitted 5 August, 2024;
originally announced August 2024.
-
TaleMate: Exploring the use of Voice Agents for Parent-Child Joint Reading Experiences
Authors:
Daniel Vargas-Diaz,
Jisun Kim,
Sulakna Karunaratna,
Maegan Reinhardt,
Caroline Hornburg,
Koeun Choi,
Sang Won Lee
Abstract:
Joint reading is a key activity for early learners, with caregiver-child interactions such as questioning and feedback playing an essential role in children's cognitive and linguistic development. However, for some parents, actively engaging children in storytelling can be challenging. To address this, we introduce TaleMate a platform designed to enhance shared reading by leveraging conversational…
▽ More
Joint reading is a key activity for early learners, with caregiver-child interactions such as questioning and feedback playing an essential role in children's cognitive and linguistic development. However, for some parents, actively engaging children in storytelling can be challenging. To address this, we introduce TaleMate a platform designed to enhance shared reading by leveraging conversational agents that have been shown to support children's engagement and learning. TaleMate enables a dynamic, participatory reading experience where parents and children can choose which characters they wish to embody. Moreover, the system navigates the challenges posed by digital reading tools, such as decreased parent-child interaction, and builds upon the benefits of traditional and digital reading techniques. TaleMate offers an innovative approach to fostering early reading habits, bridging the gap between traditional joint reading practices and the digital reading landscape.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
MEDPSeg: Hierarchical polymorphic multitask learning for the segmentation of ground-glass opacities, consolidation, and pulmonary structures on computed tomography
Authors:
Diedre S. Carmo,
Jean A. Ribeiro,
Alejandro P. Comellas,
Joseph M. Reinhardt,
Sarah E. Gerard,
Letícia Rittner,
Roberto A. Lotufo
Abstract:
The COVID-19 pandemic response highlighted the potential of deep learning methods in facilitating the diagnosis, prognosis and understanding of lung diseases through automated segmentation of pulmonary structures and lesions in chest computed tomography (CT). Automated separation of lung lesion into ground-glass opacity (GGO) and consolidation is hindered due to the labor-intensive and subjective…
▽ More
The COVID-19 pandemic response highlighted the potential of deep learning methods in facilitating the diagnosis, prognosis and understanding of lung diseases through automated segmentation of pulmonary structures and lesions in chest computed tomography (CT). Automated separation of lung lesion into ground-glass opacity (GGO) and consolidation is hindered due to the labor-intensive and subjective nature of this task, resulting in scarce availability of ground truth for supervised learning. To tackle this problem, we propose MEDPSeg. MEDPSeg learns from heterogeneous chest CT targets through hierarchical polymorphic multitask learning (HPML). HPML explores the hierarchical nature of GGO and consolidation, lung lesions, and the lungs, with further benefits achieved through multitasking airway and pulmonary artery segmentation. Over 6000 volumetric CT scans from different partially labeled sources were used for training and testing. Experiments show PML enabling new state-of-the-art performance for GGO and consolidation segmentation tasks. In addition, MEDPSeg simultaneously performs segmentation of the lung parenchyma, airways, pulmonary artery, and lung lesions, all in a single forward prediction, with performance comparable to state-of-the-art methods specialized in each of those targets. Finally, we provide an open-source implementation with a graphical user interface at https://github.com/MICLab-Unicamp/medpseg.
△ Less
Submitted 25 March, 2024; v1 submitted 4 December, 2023;
originally announced December 2023.
-
Automatic segmentation of lung findings in CT and application to Long COVID
Authors:
Diedre S. Carmo,
Rosarie A. Tudas,
Alejandro P. Comellas,
Leticia Rittner,
Roberto A. Lotufo,
Joseph M. Reinhardt,
Sarah E. Gerard
Abstract:
Automated segmentation of lung abnormalities in computed tomography is an important step for diagnosing and characterizing lung disease. In this work, we improve upon a previous method and propose S-MEDSeg, a deep learning based approach for accurate segmentation of lung lesions in chest CT images. S-MEDSeg combines a pre-trained EfficientNet backbone, bidirectional feature pyramid network, and mo…
▽ More
Automated segmentation of lung abnormalities in computed tomography is an important step for diagnosing and characterizing lung disease. In this work, we improve upon a previous method and propose S-MEDSeg, a deep learning based approach for accurate segmentation of lung lesions in chest CT images. S-MEDSeg combines a pre-trained EfficientNet backbone, bidirectional feature pyramid network, and modern network advancements to achieve improved segmentation performance. A comprehensive ablation study was performed to evaluate the contribution of the proposed network modifications. The results demonstrate modifications introduced in S-MEDSeg significantly improves segmentation performance compared to the baseline approach. The proposed method is applied to an independent dataset of long COVID inpatients to study the effect of post-acute infection vaccination on extent of lung findings. Open-source code, graphical user interface and pip package are available at https://github.com/MICLab-Unicamp/medseg.
△ Less
Submitted 13 October, 2023;
originally announced October 2023.
-
Path Weight Sampling: Exact Monte Carlo Computation of the Mutual Information between Stochastic Trajectories
Authors:
Manuel Reinhardt,
Gašper Tkačik,
Pieter Rein ten Wolde
Abstract:
Most natural and engineered information-processing systems transmit information via signals that vary in time. Computing the information transmission rate or the information encoded in the temporal characteristics of these signals, requires the mutual information between the input and output signals as a function of time, i.e. between the input and output trajectories. Yet, this is notoriously dif…
▽ More
Most natural and engineered information-processing systems transmit information via signals that vary in time. Computing the information transmission rate or the information encoded in the temporal characteristics of these signals, requires the mutual information between the input and output signals as a function of time, i.e. between the input and output trajectories. Yet, this is notoriously difficult because of the high-dimensional nature of the trajectory space, and all existing techniques require approximations. We present an exact Monte Carlo technique called Path Weight Sampling (PWS) that, for the first time, makes it possible to compute the mutual information between input and output trajectories for any stochastic system that is described by a master equation. The principal idea is to use the master equation to evaluate the exact conditional probability of an individual output trajectory for a given input trajectory, and average this via Monte Carlo sampling in trajectory space to obtain the mutual information. We present three variants of PWS, which all generate the trajectories using the standard stochastic simulation algorithm. While Direct PWS is a brute-force method, Rosenbluth-Rosenbluth PWS exploits the analogy between signal trajectory sampling and polymer sampling, and Thermodynamic Integration PWS is based on a reversible work calculation in trajectory space. PWS also makes it possible to compute the mutual information between input and output trajectories for systems with hidden internal states as well as systems with feedback from output to input. Applying PWS to the bacterial chemotaxis system, consisting of 182 coupled chemical reactions, demonstrates not only that the scheme is highly efficient, but also that the number of receptor clusters is much smaller than hitherto believed, while their size is much larger.
△ Less
Submitted 4 October, 2023; v1 submitted 7 March, 2022;
originally announced March 2022.
-
Single volume lung biomechanics from chest computed tomography using a mode preserving generative adversarial network
Authors:
Muhammad F. A. Chaudhary,
Sarah E. Gerard,
Di Wang,
Gary E. Christensen,
Christopher B. Cooper,
Joyce D. Schroeder,
Eric A. Hoffman,
Joseph M. Reinhardt
Abstract:
Local tissue expansion of the lungs is typically derived by registering computed tomography (CT) scans acquired at multiple lung volumes. However, acquiring multiple scans incurs increased radiation dose, time, and cost, and may not be possible in many cases, thus restricting the applicability of registration-based biomechanics. We propose a generative adversarial learning approach for estimating…
▽ More
Local tissue expansion of the lungs is typically derived by registering computed tomography (CT) scans acquired at multiple lung volumes. However, acquiring multiple scans incurs increased radiation dose, time, and cost, and may not be possible in many cases, thus restricting the applicability of registration-based biomechanics. We propose a generative adversarial learning approach for estimating local tissue expansion directly from a single CT scan. The proposed framework was trained and evaluated on 2500 subjects from the SPIROMICS cohort. Once trained, the framework can be used as a registration-free method for predicting local tissue expansion. We evaluated model performance across varying degrees of disease severity and compared its performance with two image-to-image translation frameworks - UNet and Pix2Pix. Our model achieved an overall PSNR of 18.95 decibels, SSIM of 0.840, and Spearman's correlation of 0.61 at a high spatial resolution of 1 mm3.
△ Less
Submitted 15 October, 2021;
originally announced October 2021.
-
CT Image Segmentation for Inflamed and Fibrotic Lungs Using a Multi-Resolution Convolutional Neural Network
Authors:
Sarah E. Gerard,
Jacob Herrmann,
Yi Xin,
Kevin T. Martin,
Emanuele Rezoagli,
Davide Ippolito,
Giacomo Bellani,
Maurizio Cereda,
Junfeng Guo,
Eric A. Hoffman,
David W. Kaczka,
Joseph M. Reinhardt
Abstract:
The purpose of this study was to develop a fully-automated segmentation algorithm, robust to various density enhancing lung abnormalities, to facilitate rapid quantitative analysis of computed tomography images. A polymorphic training approach is proposed, in which both specifically labeled left and right lungs of humans with COPD, and nonspecifically labeled lungs of animals with acute lung injur…
▽ More
The purpose of this study was to develop a fully-automated segmentation algorithm, robust to various density enhancing lung abnormalities, to facilitate rapid quantitative analysis of computed tomography images. A polymorphic training approach is proposed, in which both specifically labeled left and right lungs of humans with COPD, and nonspecifically labeled lungs of animals with acute lung injury, were incorporated into training a single neural network. The resulting network is intended for predicting left and right lung regions in humans with or without diffuse opacification and consolidation. Performance of the proposed lung segmentation algorithm was extensively evaluated on CT scans of subjects with COPD, confirmed COVID-19, lung cancer, and IPF, despite no labeled training data of the latter three diseases. Lobar segmentations were obtained using the left and right lung segmentation as input to the LobeNet algorithm. Regional lobar analysis was performed using hierarchical clustering to identify radiographic subtypes of COVID-19. The proposed lung segmentation algorithm was quantitatively evaluated using semi-automated and manually-corrected segmentations in 87 COVID-19 CT images, achieving an average symmetric surface distance of $0.495 \pm 0.309$ mm and Dice coefficient of $0.985 \pm 0.011$. Hierarchical clustering identified four radiographical phenotypes of COVID-19 based on lobar fractions of consolidated and poorly aerated tissue. Lower left and lower right lobes were consistently more afflicted with poor aeration and consolidation. However, the most severe cases demonstrated involvement of all lobes. The polymorphic training approach was able to accurately segment COVID-19 cases with diffuse consolidation without requiring COVID-19 cases for training.
△ Less
Submitted 14 January, 2021; v1 submitted 16 October, 2020;
originally announced October 2020.
