-
Blind Restoration of High-Resolution Ultrasound Video
Authors:
Chu Chen,
Kangning Cui,
Pasquale Cascarano,
Wei Tang,
Elena Loli Piccolomini,
Raymond H. Chan
Abstract:
Ultrasound imaging is widely applied in clinical practice, yet ultrasound videos often suffer from low signal-to-noise ratios (SNR) and limited resolutions, posing challenges for diagnosis and analysis. Variations in equipment and acquisition settings can further exacerbate differences in data distribution and noise levels, reducing the generalizability of pre-trained models. This work presents a…
▽ More
Ultrasound imaging is widely applied in clinical practice, yet ultrasound videos often suffer from low signal-to-noise ratios (SNR) and limited resolutions, posing challenges for diagnosis and analysis. Variations in equipment and acquisition settings can further exacerbate differences in data distribution and noise levels, reducing the generalizability of pre-trained models. This work presents a self-supervised ultrasound video super-resolution algorithm called Deep Ultrasound Prior (DUP). DUP employs a video-adaptive optimization process of a neural network that enhances the resolution of given ultrasound videos without requiring paired training data while simultaneously removing noise. Quantitative and visual evaluations demonstrate that DUP outperforms existing super-resolution algorithms, leading to substantial improvements for downstream applications.
△ Less
Submitted 20 May, 2025;
originally announced May 2025.
-
Deep Guess acceleration for explainable image reconstruction in sparse-view CT
Authors:
Elena Loli Piccolomini,
Davide Evangelista,
Elena Morotti
Abstract:
Sparse-view Computed Tomography (CT) is an emerging protocol designed to reduce X-ray dose radiation in medical imaging. Traditional Filtered Back Projection algorithm reconstructions suffer from severe artifacts due to sparse data. In contrast, Model-Based Iterative Reconstruction (MBIR) algorithms, though better at mitigating noise through regularization, are too computationally costly for clini…
▽ More
Sparse-view Computed Tomography (CT) is an emerging protocol designed to reduce X-ray dose radiation in medical imaging. Traditional Filtered Back Projection algorithm reconstructions suffer from severe artifacts due to sparse data. In contrast, Model-Based Iterative Reconstruction (MBIR) algorithms, though better at mitigating noise through regularization, are too computationally costly for clinical use. This paper introduces a novel technique, denoted as the Deep Guess acceleration scheme, using a trained neural network both to quicken the regularized MBIR and to enhance the reconstruction accuracy. We integrate state-of-the-art deep learning tools to initialize a clever starting guess for a proximal algorithm solving a non-convex model and thus computing an interpretable solution image in a few iterations. Experimental results on real CT images demonstrate the Deep Guess effectiveness in (very) sparse tomographic protocols, where it overcomes its mere variational counterpart and many data-driven approaches at the state of the art. We also consider a ground truth-free implementation and test the robustness of the proposed framework to noise.
△ Less
Submitted 2 December, 2024;
originally announced December 2024.
-
Space-Variant Total Variation boosted by learning techniques in few-view tomographic imaging
Authors:
Elena Morotti,
Davide Evangelista,
Andrea Sebastiani,
Elena Loli Piccolomini
Abstract:
This paper focuses on the development of a space-variant regularization model for solving an under-determined linear inverse problem. The case study is a medical image reconstruction from few-view tomographic noisy data. The primary objective of the proposed optimization model is to achieve a good balance between denoising and the preservation of fine details and edges, overcoming the performance…
▽ More
This paper focuses on the development of a space-variant regularization model for solving an under-determined linear inverse problem. The case study is a medical image reconstruction from few-view tomographic noisy data. The primary objective of the proposed optimization model is to achieve a good balance between denoising and the preservation of fine details and edges, overcoming the performance of the popular and largely used Total Variation (TV) regularization through the application of appropriate pixel-dependent weights. The proposed strategy leverages the role of gradient approximations for the computation of the space-variant TV weights. For this reason, a convolutional neural network is designed, to approximate both the ground truth image and its gradient using an elastic loss function in its training. Additionally, the paper provides a theoretical analysis of the proposed model, showing the uniqueness of its solution, and illustrates a Chambolle-Pock algorithm tailored to address the specific problem at hand. This comprehensive framework integrates innovative regularization techniques with advanced neural network capabilities, demonstrating promising results in achieving high-quality reconstructions from low-sampled tomographic data.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
Deep image prior inpainting of ancient frescoes in the Mediterranean Alpine arc
Authors:
Fabio Merizzi,
Perrine Saillard,
Oceane Acquier,
Elena Morotti,
Elena Loli Piccolomini,
Luca Calatroni,
Rosa Maria Dessì
Abstract:
The unprecedented success of image reconstruction approaches based on deep neural networks has revolutionised both the processing and the analysis paradigms in several applied disciplines. In the field of digital humanities, the task of digital reconstruction of ancient frescoes is particularly challenging due to the scarce amount of available training data caused by ageing, wear, tear and retouch…
▽ More
The unprecedented success of image reconstruction approaches based on deep neural networks has revolutionised both the processing and the analysis paradigms in several applied disciplines. In the field of digital humanities, the task of digital reconstruction of ancient frescoes is particularly challenging due to the scarce amount of available training data caused by ageing, wear, tear and retouching over time. To overcome these difficulties, we consider the Deep Image Prior (DIP) inpainting approach which computes appropriate reconstructions by relying on the progressive updating of an untrained convolutional neural network so as to match the reliable piece of information in the image at hand while promoting regularisation elsewhere. In comparison with state-of-the-art approaches (based on variational/PDEs and patch-based methods), DIP-based inpainting reduces artefacts and better adapts to contextual/non-local information, thus providing a valuable and effective tool for art historians. As a case study, we apply such approach to reconstruct missing image contents in a dataset of highly damaged digital images of medieval paintings located into several chapels in the Mediterranean Alpine Arc and provide a detailed description on how visible and invisible (e.g., infrared) information can be integrated for identifying and reconstructing damaged image regions.
△ Less
Submitted 11 December, 2023; v1 submitted 25 June, 2023;
originally announced June 2023.
-
Ambiguity in solving imaging inverse problems with deep learning based operators
Authors:
Davide Evangelista,
Elena Morotti,
Elena Loli Piccolomini,
James Nagy
Abstract:
In recent years, large convolutional neural networks have been widely used as tools for image deblurring, because of their ability in restoring images very precisely. It is well known that image deblurring is mathematically modeled as an ill-posed inverse problem and its solution is difficult to approximate when noise affects the data. Really, one limitation of neural networks for deblurring is th…
▽ More
In recent years, large convolutional neural networks have been widely used as tools for image deblurring, because of their ability in restoring images very precisely. It is well known that image deblurring is mathematically modeled as an ill-posed inverse problem and its solution is difficult to approximate when noise affects the data. Really, one limitation of neural networks for deblurring is their sensitivity to noise and other perturbations, which can lead to instability and produce poor reconstructions. In addition, networks do not necessarily take into account the numerical formulation of the underlying imaging problem, when trained end-to-end. In this paper, we propose some strategies to improve stability without losing to much accuracy to deblur images with deep-learning based methods. First, we suggest a very small neural architecture, which reduces the execution time for training, satisfying a green AI need, and does not extremely amplify noise in the computed image. Second, we introduce a unified framework where a pre-processing step balances the lack of stability of the following, neural network-based, step. Two different pre-processors are presented: the former implements a strong parameter-free denoiser, and the latter is a variational model-based regularized formulation of the latent imaging problem. This framework is also formally characterized by mathematical analysis. Numerical experiments are performed to verify the accuracy and stability of the proposed approaches for image deblurring when unknown or not-quantified noise is present; the results confirm that they improve the network stability with respect to noise. In particular, the model-based framework represents the most reliable trade-off between visual precision and robustness.
△ Less
Submitted 31 May, 2023;
originally announced May 2023.
-
To be or not to be stable, that is the question: understanding neural networks for inverse problems
Authors:
Davide Evangelista,
James Nagy,
Elena Morotti,
Elena Loli Piccolomini
Abstract:
The solution of linear inverse problems arising, for example, in signal and image processing is a challenging problem since the ill-conditioning amplifies, in the solution, the noise present in the data. Recently introduced algorithms based on deep learning overwhelm the more traditional model-based approaches in performance, but they typically suffer from instability with respect to data perturba…
▽ More
The solution of linear inverse problems arising, for example, in signal and image processing is a challenging problem since the ill-conditioning amplifies, in the solution, the noise present in the data. Recently introduced algorithms based on deep learning overwhelm the more traditional model-based approaches in performance, but they typically suffer from instability with respect to data perturbation. In this paper, we theoretically analyze the trade-off between stability and accuracy of neural networks, when used to solve linear imaging inverse problems for not under-determined cases. Moreover, we propose different supervised and unsupervised solutions to increase the network stability and maintain a good accuracy, by means of regularization properties inherited from a model-based iterative scheme during the network training and pre-processing stabilizing operator in the neural networks. Extensive numerical experiments on image deblurring confirm the theoretical results and the effectiveness of the proposed deep learning-based approaches to handle noise on the data.
△ Less
Submitted 7 February, 2024; v1 submitted 24 November, 2022;
originally announced November 2022.
-
DeepCEL0 for 2D Single Molecule Localization in Fluorescence Microscopy
Authors:
Pasquale Cascarano,
Maria Colomba Comes,
Andrea Sebastiani,
Arianna Mencattini,
Elena Loli Piccolomini,
Eugenio Martinelli
Abstract:
In fluorescence microscopy, Single Molecule Localization Microscopy (SMLM) techniques aim at localizing with high precision high density fluorescent molecules by stochastically activating and imaging small subsets of blinking emitters. Super Resolution (SR) plays an important role in this field since it allows to go beyond the intrinsic light diffraction limit. In this work, we propose a deep lear…
▽ More
In fluorescence microscopy, Single Molecule Localization Microscopy (SMLM) techniques aim at localizing with high precision high density fluorescent molecules by stochastically activating and imaging small subsets of blinking emitters. Super Resolution (SR) plays an important role in this field since it allows to go beyond the intrinsic light diffraction limit. In this work, we propose a deep learning-based algorithm for precise molecule localization of high density frames acquired by SMLM techniques whose $\ell_{2}$-based loss function is regularized by positivity and $\ell_{0}$-based constraints. The $\ell_{0}$ is relaxed through its Continuous Exact $\ell_{0}$ (CEL0) counterpart. The arising approach, named DeepCEL0, is parameter-free, more flexible, faster and provides more precise molecule localization maps if compared to the other state-of-the-art methods. We validate our approach on both simulated and real fluorescence microscopy data.
△ Less
Submitted 5 July, 2021;
originally announced July 2021.
-
A survey on Variational Autoencoders from a GreenAI perspective
Authors:
A. Asperti,
D. Evangelista,
E. Loli Piccolomini
Abstract:
Variational AutoEncoders (VAEs) are powerful generative models that merge elements from statistics and information theory with the flexibility offered by deep neural networks to efficiently solve the generation problem for high dimensional data. The key insight of VAEs is to learn the latent distribution of data in such a way that new meaningful samples can be generated from it. This approach led…
▽ More
Variational AutoEncoders (VAEs) are powerful generative models that merge elements from statistics and information theory with the flexibility offered by deep neural networks to efficiently solve the generation problem for high dimensional data. The key insight of VAEs is to learn the latent distribution of data in such a way that new meaningful samples can be generated from it. This approach led to tremendous research and variations in the architectural design of VAEs, nourishing the recent field of research known as unsupervised representation learning. In this article, we provide a comparative evaluation of some of the most successful, recent variations of VAEs. We particularly focus the analysis on the energetic efficiency of the different models, in the spirit of the so called Green AI, aiming both to reduce the carbon footprint and the financial cost of generative techniques. For each architecture we provide its mathematical formulation, the ideas underlying its design, a detailed model description, a running implementation and quantitative results.
△ Less
Submitted 1 March, 2021;
originally announced March 2021.
-
Plug-and-Play gradient-based denoisers applied to CT image enhancement
Authors:
Pasquale Cascarano,
Elena Loli Piccolomini,
Elena Morotti,
Andrea Sebastiani
Abstract:
Blur and noise corrupting Computed Tomography (CT) images can hide or distort small but important details, negatively affecting the diagnosis. In this paper, we present a novel gradient-based Plug-and-Play algorithm, constructed on the Half-Quadratic Splitting scheme, and we apply it to restore CT images. In particular, we consider different schemes encompassing external and internal denoisers as…
▽ More
Blur and noise corrupting Computed Tomography (CT) images can hide or distort small but important details, negatively affecting the diagnosis. In this paper, we present a novel gradient-based Plug-and-Play algorithm, constructed on the Half-Quadratic Splitting scheme, and we apply it to restore CT images. In particular, we consider different schemes encompassing external and internal denoisers as priors, defined on the image gradient domain. The internal prior is based on the Total Variation functional. The external denoiser is implemented by a deep Convolutional Neural Network (CNN) trained on the gradient domain (and not on the image one, as in state-of-the-art works). We also prove a general fixed-point convergence theorem under weak assumptions on both internal and external denoisers. The experiments confirm the effectiveness of the proposed framework in restoring blurred noisy CT images, both in simulated and real medical settings. The achieved enhancements in the restored images are really remarkable, if compared to the results of many state-of-the-art methods.
△ Less
Submitted 19 March, 2021; v1 submitted 15 February, 2021;
originally announced February 2021.
-
Recursive Deep Prior Video: a Super Resolution algorithm for Time-Lapse Microscopy of organ-on-chip experiments
Authors:
Pasquale Cascarano,
Maria Colomba Comes,
Arianna Mencattini,
Maria Carla Parrini,
Elena Loli Piccolomini,
Eugenio Martinelli
Abstract:
Biological experiments based on organ-on-chips (OOCs) exploit light Time-Lapse Microscopy (TLM) for a direct observation of cell movement that is an observable signature of underlying biological processes. A high spatial resolution is essential to capture cell dynamics and interactions from recorded experiments by TLM. Unfortunately, due to physical and cost limitations, acquiring high resolution…
▽ More
Biological experiments based on organ-on-chips (OOCs) exploit light Time-Lapse Microscopy (TLM) for a direct observation of cell movement that is an observable signature of underlying biological processes. A high spatial resolution is essential to capture cell dynamics and interactions from recorded experiments by TLM. Unfortunately, due to physical and cost limitations, acquiring high resolution videos is not always possible. To overcome the problem, we present here a new deep learning-based algorithm that extends the well known Deep Image Prior (DIP) to TLM Video Super Resolution (SR) without requiring any training. The proposed Recursive Deep Prior Video (RDPV) method introduces some novelties. The weights of the DIP network architecture are initialized for each of the frames according to a new recursive updating rule combined with an efficient early stopping criterion. Moreover, the DIP loss function is penalized by two different Total Variation (TV) based terms. The method has been validated on synthetic, i.e., artificially generated, as well as real videos from OOC experiments related to tumor-immune interaction. Achieved results are compared with several state-of-the-art trained deep learning SR algorithms showing outstanding performances.
△ Less
Submitted 19 November, 2020;
originally announced November 2020.
-
A fast nonconvex Compressed Sensing algorithm for highly low-sampled MR images reconstruction
Authors:
Damiana Lazzaro,
Elena Loli Piccolomini,
Fabiana Zama
Abstract:
In this paper we present a fast and efficient method for the reconstruction of Magnetic Resonance Images (MRI) from severely under-sampled data. From the Compressed Sensing theory we have mathematically modeled the problem as a constrained minimization problem with a family of non-convex regularizing objective functions depending on a parameter and a least squares data fit constraint. We propose a…
▽ More
In this paper we present a fast and efficient method for the reconstruction of Magnetic Resonance Images (MRI) from severely under-sampled data. From the Compressed Sensing theory we have mathematically modeled the problem as a constrained minimization problem with a family of non-convex regularizing objective functions depending on a parameter and a least squares data fit constraint. We propose a fast and efficient algorithm, named Fast NonConvex Reweighting (FNCR) algorithm, based on an iterative scheme where the non-convex problem is approximated by its convex linearization and the penalization parameter is automatically updated. The convex problem is solved by a Forward-Backward procedure, where the Backward step is performed by a Split Bregman strategy. Moreover, we propose a new efficient iterative solver for the arising linear systems. We prove the convergence of the proposed FNCR method. The results on synthetic phantoms and real images show that the algorithm is very well performing and computationally efficient, even when compared to the best performing methods proposed in the literature.
△ Less
Submitted 29 November, 2017;
originally announced November 2017.
