Showing 1–6 of 6 results for author: Stayman, W

Abn-BLIP: Abnormality-aligned Bootstrapping Language-Image Pre-training for Pulmonary Embolism Diagnosis and Report Generation from CTPA

Authors: Zhusi Zhong, Yuli Wang, Lulu Bi, Zhuoqi Ma, Sun Ho Ahn, Christopher J. Mullin, Colin F. Greineder, Michael K. Atalay, Scott Collins, Grayson L. Baird, Cheng Ting Lin, Webster Stayman, Todd M. Kolb, Ihab Kamel, Harrison X. Bai, Zhicheng Jiao

Abstract: Medical imaging plays a pivotal role in modern healthcare, with computed tomography pulmonary angiography (CTPA) being a critical tool for diagnosing pulmonary embolism and other thoracic conditions. However, the complexity of interpreting CTPA scans and generating accurate radiology reports remains a significant challenge. This paper introduces Abn-BLIP (Abnormality-aligned Bootstrapping Language… ▽ More Medical imaging plays a pivotal role in modern healthcare, with computed tomography pulmonary angiography (CTPA) being a critical tool for diagnosing pulmonary embolism and other thoracic conditions. However, the complexity of interpreting CTPA scans and generating accurate radiology reports remains a significant challenge. This paper introduces Abn-BLIP (Abnormality-aligned Bootstrapping Language-Image Pretraining), an advanced diagnosis model designed to align abnormal findings to generate the accuracy and comprehensiveness of radiology reports. By leveraging learnable queries and cross-modal attention mechanisms, our model demonstrates superior performance in detecting abnormalities, reducing missed findings, and generating structured reports compared to existing methods. Our experiments show that Abn-BLIP outperforms state-of-the-art medical vision-language models and 3D report generation methods in both accuracy and clinical relevance. These results highlight the potential of integrating multimodal learning strategies for improving radiology reporting. The source code is available at https://github.com/zzs95/abn-blip. △ Less

Submitted 3 March, 2025; originally announced March 2025.

Conformal Risk Control for Semantic Uncertainty Quantification in Computed Tomography

Authors: Jacopo Teneggi, J Webster Stayman, Jeremias Sulam

Abstract: Uncertainty quantification is necessary for developers, physicians, and regulatory agencies to build trust in machine learning predictors and improve patient care. Beyond measuring uncertainty, it is crucial to express it in clinically meaningful terms that provide actionable insights. This work introduces a conformal risk control (CRC) procedure for organ-dependent uncertainty estimation, ensurin… ▽ More Uncertainty quantification is necessary for developers, physicians, and regulatory agencies to build trust in machine learning predictors and improve patient care. Beyond measuring uncertainty, it is crucial to express it in clinically meaningful terms that provide actionable insights. This work introduces a conformal risk control (CRC) procedure for organ-dependent uncertainty estimation, ensuring high-probability coverage of the ground-truth image. We first present a high-dimensional CRC procedure that leverages recent ideas of length minimization. We make this procedure semantically adaptive to each patient's anatomy and positioning of organs. Our method, sem-CRC, provides tighter uncertainty intervals with valid coverage on real-world computed tomography (CT) data while communicating uncertainty with clinically relevant features. △ Less

Submitted 28 February, 2025; originally announced March 2025.

Deep Learning CT Image Restoration using System Blur and Noise Models

Authors: Yijie Yuan, Grace J. Gang, J. Webster Stayman

Abstract: The restoration of images affected by blur and noise has been widely studied and has broad potential for applications including in medical imaging modalities like computed tomography (CT). Although the blur and noise in CT images can be attributed to a variety of system factors, these image properties can often be modeled and predicted accurately and used in classical restoration approaches for de… ▽ More The restoration of images affected by blur and noise has been widely studied and has broad potential for applications including in medical imaging modalities like computed tomography (CT). Although the blur and noise in CT images can be attributed to a variety of system factors, these image properties can often be modeled and predicted accurately and used in classical restoration approaches for deconvolution and denoising. In classical approaches, simultaneous deconvolution and denoising can be challenging and often represent competing goals. Recently, deep learning approaches have demonstrated the potential to enhance image quality beyond classic limits; however, most deep learning models attempt a blind restoration problem and base their restoration on image inputs alone without direct knowledge of the image noise and blur properties. In this work, we present a method that leverages both degraded image inputs and a characterization of the system blur and noise to combine modeling and deep learning approaches. Different methods to integrate these auxiliary inputs are presented. Namely, an input-variant and a weight-variant approach wherein the auxiliary inputs are incorporated as a parameter vector before and after the convolutional block, respectively, allowing easy integration into any CNN architecture. The proposed model shows superior performance compared to baseline models lacking auxiliary inputs. Evaluations are based on the average Peak Signal-to-Noise Ratio (PSNR), selected examples of good and poor performance for varying approaches, and an input space analysis to assess the effect of different noise and blur on performance. Results demonstrate the efficacy of providing a deep learning model with auxiliary inputs, representing system blur and noise characteristics, to enhance the performance of the model in image restoration tasks. △ Less

Submitted 20 July, 2024; originally announced July 2024.

CT Material Decomposition using Spectral Diffusion Posterior Sampling

Authors: Xiao Jiang, Grace J. Gang, J. Webster Stayman

Abstract: In this work, we introduce a new deep learning approach based on diffusion posterior sampling (DPS) to perform material decomposition from spectral CT measurements. This approach combines sophisticated prior knowledge from unsupervised training with a rigorous physical model of the measurements. A faster and more stable variant is proposed that uses a jumpstarted process to reduce the number of ti… ▽ More In this work, we introduce a new deep learning approach based on diffusion posterior sampling (DPS) to perform material decomposition from spectral CT measurements. This approach combines sophisticated prior knowledge from unsupervised training with a rigorous physical model of the measurements. A faster and more stable variant is proposed that uses a jumpstarted process to reduce the number of time steps required in the reverse process and a gradient approximation to reduce the computational cost. Performance is investigated for two spectral CT systems: dual-kVp and dual-layer detector CT. On both systems, DPS achieves high Structure Similarity Index Metric Measure(SSIM) with only 10% of iterations as used in the model-based material decomposition(MBMD). Jumpstarted DPS (JSDPS) further reduces computational time by over 85% and achieves the highest accuracy, the lowest uncertainty, and the lowest computational costs compared to classic DPS and MBMD. The results demonstrate the potential of JSDPS for providing relatively fast and accurate material decomposition based on spectral CT data. △ Less

Submitted 5 February, 2024; originally announced February 2024.

Comments: 5 pages, 4 figures

CT Reconstruction using Diffusion Posterior Sampling conditioned on a Nonlinear Measurement Model

Authors: Shudong Li, Xiao Jiang, Matthew Tivnan, Grace J. Gang, Yuan Shen, J. Webster Stayman

Abstract: Diffusion models have been demonstrated as powerful deep learning tools for image generation in CT reconstruction and restoration. Recently, diffusion posterior sampling, where a score-based diffusion prior is combined with a likelihood model, has been used to produce high quality CT images given low-quality measurements. This technique is attractive since it permits a one-time, unsupervised train… ▽ More Diffusion models have been demonstrated as powerful deep learning tools for image generation in CT reconstruction and restoration. Recently, diffusion posterior sampling, where a score-based diffusion prior is combined with a likelihood model, has been used to produce high quality CT images given low-quality measurements. This technique is attractive since it permits a one-time, unsupervised training of a CT prior; which can then be incorporated with an arbitrary data model. However, current methods rely on a linear model of x-ray CT physics to reconstruct or restore images. While it is common to linearize the transmission tomography reconstruction problem, this is an approximation to the true and inherently nonlinear forward model. We propose a new method that solves the inverse problem of nonlinear CT image reconstruction via diffusion posterior sampling. We implement a traditional unconditional diffusion model by training a prior score function estimator, and apply Bayes rule to combine this prior with a measurement likelihood score function derived from the nonlinear physical model to arrive at a posterior score function that can be used to sample the reverse-time diffusion process. This plug-and-play method allows incorporation of a diffusion-based prior with generalized nonlinear CT image reconstruction into multiple CT system designs with different forward models, without the need for any additional training. We develop the algorithm that performs this reconstruction, including an ordered-subsets variant for accelerated processing and demonstrate the technique in both fully sampled low dose data and sparse-view geometries using a single unsupervised training of the prior. △ Less

Submitted 11 June, 2024; v1 submitted 3 December, 2023; originally announced December 2023.

Comments: 24 pages, 12 figures, 1 table, submitted to SPIE Journal of Medical Imaging. Updated with more realistic phantom data, Poisson likelihood, and additional evaluations including hallucination evaluation, performance under multiple noise levels, inference time evaluation, and etc. Changes in authorship is based on unanimous agreement to acknowledge the adding authors' contributions in this work

ACM Class: J.3; I.4.4; I.4.5

Journal ref: Journal of Medical Imaging 11(4), 043504 (2024)

How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Control

Authors: Jacopo Teneggi, Matthew Tivnan, J. Webster Stayman, Jeremias Sulam

Abstract: Score-based generative modeling, informally referred to as diffusion models, continue to grow in popularity across several important domains and tasks. While they provide high-quality and diverse samples from empirical distributions, important questions remain on the reliability and trustworthiness of these sampling procedures for their responsible use in critical scenarios. Conformal prediction i… ▽ More Score-based generative modeling, informally referred to as diffusion models, continue to grow in popularity across several important domains and tasks. While they provide high-quality and diverse samples from empirical distributions, important questions remain on the reliability and trustworthiness of these sampling procedures for their responsible use in critical scenarios. Conformal prediction is a modern tool to construct finite-sample, distribution-free uncertainty guarantees for any black-box predictor. In this work, we focus on image-to-image regression tasks and we present a generalization of the Risk-Controlling Prediction Sets (RCPS) procedure, that we term $K$-RCPS, which allows to $(i)$ provide entrywise calibrated intervals for future samples of any diffusion model, and $(ii)$ control a certain notion of risk with respect to a ground truth image with minimal mean interval length. Differently from existing conformal risk control procedures, ours relies on a novel convex optimization approach that allows for multidimensional risk control while provably minimizing the mean interval length. We illustrate our approach on two real-world image denoising problems: on natural images of faces as well as on computed tomography (CT) scans of the abdomen, demonstrating state of the art performance. △ Less

Submitted 27 December, 2023; v1 submitted 7 February, 2023; originally announced February 2023.

Journal ref: International Conference on Machine Learning (2023)