-
Structural Attention: Rethinking Transformer for Unpaired Medical Image Synthesis
Authors:
Vu Minh Hieu Phan,
Yutong Xie,
Bowen Zhang,
Yuankai Qi,
Zhibin Liao,
Antonios Perperidis,
Son Lam Phung,
Johan W. Verjans,
Minh-Son To
Abstract:
Unpaired medical image synthesis aims to provide complementary information for an accurate clinical diagnostics, and address challenges in obtaining aligned multi-modal medical scans. Transformer-based models excel in imaging translation tasks thanks to their ability to capture long-range dependencies. Although effective in supervised training settings, their performance falters in unpaired image…
▽ More
Unpaired medical image synthesis aims to provide complementary information for an accurate clinical diagnostics, and address challenges in obtaining aligned multi-modal medical scans. Transformer-based models excel in imaging translation tasks thanks to their ability to capture long-range dependencies. Although effective in supervised training settings, their performance falters in unpaired image synthesis, particularly in synthesizing structural details. This paper empirically demonstrates that, lacking strong inductive biases, Transformer can converge to non-optimal solutions in the absence of paired data. To address this, we introduce UNet Structured Transformer (UNest), a novel architecture incorporating structural inductive biases for unpaired medical image synthesis. We leverage the foundational Segment-Anything Model to precisely extract the foreground structure and perform structural attention within the main anatomy. This guides the model to learn key anatomical regions, thus improving structural synthesis under the lack of supervision in unpaired training. Evaluated on two public datasets, spanning three modalities, i.e., MR, CT, and PET, UNest improves recent methods by up to 19.30% across six medical image synthesis tasks. Our code is released at https://github.com/HieuPhan33/MICCAI2024-UNest.
△ Less
Submitted 28 August, 2024; v1 submitted 27 June, 2024;
originally announced June 2024.
-
Patch-Based Sparse Representation For Bacterial Detection
Authors:
Ahmed Karam Eldaly,
Yoann Altmann,
Ahsan Akram,
Antonios Perperidis,
Kevin Dhaliwal,
Stephen McLaughlin
Abstract:
In this paper, we propose an unsupervised approach for bacterial detection in optical endomicroscopy images. This approach splits each image into a set of overlapping patches and assumes that observed intensities are linear combinations of the actual intensity values associated with background image structures, corrupted by additive Gaussian noise and potentially by a sparse outlier term modelling…
▽ More
In this paper, we propose an unsupervised approach for bacterial detection in optical endomicroscopy images. This approach splits each image into a set of overlapping patches and assumes that observed intensities are linear combinations of the actual intensity values associated with background image structures, corrupted by additive Gaussian noise and potentially by a sparse outlier term modelling anomalies (which are considered to be candidate bacteria). The actual intensity term representing background structures is modelled as a linear combination of a few atoms drawn from a dictionary which is learned from bacteria-free data and then fixed while analyzing new images. The bacteria detection task is formulated as a minimization problem and an alternating direction method of multipliers (ADMM) is then used to estimate the unknown parameters. Simulations conducted using two ex vivo lung datasets show good detection and correlation performance between bacteria counts identified by a trained clinician and those of the proposed method.
△ Less
Submitted 24 January, 2019; v1 submitted 29 October, 2018;
originally announced October 2018.
-
Image computing for fibre-bundle endomicroscopy: A review
Authors:
Antonios Perperidis,
Kevin Dhaliwal,
Stephen McLaughlin,
Tom Vercauteren
Abstract:
Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for cli…
▽ More
Endomicroscopy is an emerging imaging modality, that facilitates the acquisition of in vivo, in situ optical biopsies, assisting diagnostic and potentially therapeutic interventions. While there is a diverse and constantly expanding range of commercial and experimental optical biopsy platforms available, fibre-bundle endomicroscopy is currently the most widely used platform and is approved for clinical use in a range of clinical indications. Miniaturised, flexible fibre-bundles, guided through the working channel of endoscopes, needles and catheters, enable high-resolution imaging across a variety of organ systems. Yet, the nature of image acquisition though a fibre-bundle gives rise to several inherent characteristics and limitations necessitating novel and effective image pre- and post-processing algorithms, ranging from image formation, enhancement and mosaicing to pathology detection and quantification. This paper introduces the underlying technology and most prevalent clinical applications of fibre-bundle endomicroscopy, and provides a comprehensive, up-to-date, review of relevant image reconstruction, analysis and understanding/inference methodologies. Furthermore, current limitations as well as future challenges and opportunities in fibre-bundle endomicroscopy computing are identified and discussed.
△ Less
Submitted 3 September, 2018;
originally announced September 2018.
-
Deconvolution and Restoration of Optical Endomicroscopy Images
Authors:
Ahmed Karam Eldaly,
Yoann Altmann,
Antonios Perperidis,
Nikola Krstajic,
Tushar Choudhary,
Kevin Dhaliwal,
Stephen McLaughlin
Abstract:
Optical endomicroscopy (OEM) is an emerging technology platform with preclinical and clinical imaging applications. Pulmonary OEM via fibre bundles has the potential to provide in vivo, in situ molecular signatures of disease such as infection and inflammation. However, enhancing the quality of data acquired by this technique for better visualization and subsequent analysis remains a challenging p…
▽ More
Optical endomicroscopy (OEM) is an emerging technology platform with preclinical and clinical imaging applications. Pulmonary OEM via fibre bundles has the potential to provide in vivo, in situ molecular signatures of disease such as infection and inflammation. However, enhancing the quality of data acquired by this technique for better visualization and subsequent analysis remains a challenging problem. Cross coupling between fiber cores and sparse sampling by imaging fiber bundles are the main reasons for image degradation, and poor detection performance (i.e., inflammation, bacteria, etc.). In this work, we address the problem of deconvolution and restoration of OEM data. We propose a hierarchical Bayesian model to solve this problem and compare three estimation algorithms to exploit the resulting joint posterior distribution. The first method is based on Markov chain Monte Carlo (MCMC) methods, however, it exhibits a relatively long computational time. The second and third algorithms deal with this issue and are based on a variational Bayes (VB) approach and an alternating direction method of multipliers (ADMM) algorithm respectively. Results on both synthetic and real datasets illustrate the effectiveness of the proposed methods for restoration of OEM images.
△ Less
Submitted 28 August, 2018; v1 submitted 27 January, 2017;
originally announced January 2017.
