-
DeepInverse: A Python package for solving imaging inverse problems with deep learning
Authors:
Julián Tachella,
Matthieu Terris,
Samuel Hurault,
Andrew Wang,
Dongdong Chen,
Minh-Hai Nguyen,
Maxime Song,
Thomas Davies,
Leo Davy,
Jonathan Dong,
Paul Escande,
Johannes Hertrich,
Zhiyuan Hu,
Tobías I. Liaudat,
Nils Laurent,
Brett Levac,
Mathurin Massias,
Thomas Moreau,
Thibaut Modrzyk,
Brayan Monroy,
Sebastian Neumayer,
Jérémy Scanvic,
Florian Sarron,
Victor Sechaud,
Georg Schramm
, et al. (2 additional authors not shown)
Abstract:
DeepInverse is an open-source PyTorch-based library for solving imaging inverse problems. The library covers all crucial steps in image reconstruction from the efficient implementation of forward operators (e.g., optics, MRI, tomography), to the definition and resolution of variational problems and the design and training of advanced neural network architectures. In this paper, we describe the mai…
▽ More
DeepInverse is an open-source PyTorch-based library for solving imaging inverse problems. The library covers all crucial steps in image reconstruction from the efficient implementation of forward operators (e.g., optics, MRI, tomography), to the definition and resolution of variational problems and the design and training of advanced neural network architectures. In this paper, we describe the main functionality of the library and discuss the main design choices.
△ Less
Submitted 17 June, 2025; v1 submitted 26 May, 2025;
originally announced May 2025.
-
DEALing with Image Reconstruction: Deep Attentive Least Squares
Authors:
Mehrsa Pourya,
Erich Kobler,
Michael Unser,
Sebastian Neumayer
Abstract:
State-of-the-art image reconstruction often relies on complex, highly parameterized deep architectures. We propose an alternative: a data-driven reconstruction method inspired by the classic Tikhonov regularization. Our approach iteratively refines intermediate reconstructions by solving a sequence of quadratic problems. These updates have two key components: (i) learned filters to extract salient…
▽ More
State-of-the-art image reconstruction often relies on complex, highly parameterized deep architectures. We propose an alternative: a data-driven reconstruction method inspired by the classic Tikhonov regularization. Our approach iteratively refines intermediate reconstructions by solving a sequence of quadratic problems. These updates have two key components: (i) learned filters to extract salient image features, and (ii) an attention mechanism that locally adjusts the penalty of filter responses. Our method achieves performance on par with leading plug-and-play and learned regularizer approaches while offering interpretability, robustness, and convergent behavior. In effect, we bridge traditional regularization and deep learning with a principled reconstruction approach.
△ Less
Submitted 6 February, 2025;
originally announced February 2025.
-
Learning of Patch-Based Smooth-Plus-Sparse Models for Image Reconstruction
Authors:
Stanislas Ducotterd,
Sebastian Neumayer,
Michael Unser
Abstract:
We aim at the solution of inverse problems in imaging, by combining a penalized sparse representation of image patches with an unconstrained smooth one. This allows for a straightforward interpretation of the reconstruction. We formulate the optimization as a bilevel problem. The inner problem deploys classical algorithms while the outer problem optimizes the dictionary and the regularizer paramet…
▽ More
We aim at the solution of inverse problems in imaging, by combining a penalized sparse representation of image patches with an unconstrained smooth one. This allows for a straightforward interpretation of the reconstruction. We formulate the optimization as a bilevel problem. The inner problem deploys classical algorithms while the outer problem optimizes the dictionary and the regularizer parameters through supervised learning. The process is carried out via implicit differentiation and gradient-based optimization. We evaluate our method for denoising, super-resolution, and compressed-sensing magnetic-resonance imaging. We compare it to other classical models as well as deep-learning-based methods and show that it always outperforms the former and also the latter in some instances.
△ Less
Submitted 17 March, 2025; v1 submitted 17 December, 2024;
originally announced December 2024.
-
Iteratively Refined Image Reconstruction with Learned Attentive Regularizers
Authors:
Mehrsa Pourya,
Sebastian Neumayer,
Michael Unser
Abstract:
We propose a regularization scheme for image reconstruction that leverages the power of deep learning while hinging on classic sparsity-promoting models. Many deep-learning-based models are hard to interpret and cumbersome to analyze theoretically. In contrast, our scheme is interpretable because it corresponds to the minimization of a series of convex problems. For each problem in the series, a m…
▽ More
We propose a regularization scheme for image reconstruction that leverages the power of deep learning while hinging on classic sparsity-promoting models. Many deep-learning-based models are hard to interpret and cumbersome to analyze theoretically. In contrast, our scheme is interpretable because it corresponds to the minimization of a series of convex problems. For each problem in the series, a mask is generated based on the previous solution to refine the regularization strength spatially. In this way, the model becomes progressively attentive to the image structure. For the underlying update operator, we prove the existence of a fixed point. As a special case, we investigate a mask generator for which the fixed-point iterations converge to a critical point of an explicit energy functional. In our experiments, we match the performance of state-of-the-art learned variational models for the solution of inverse problems. Additionally, we offer a promising balance between interpretability, theoretical guarantees, reliability, and performance.
△ Less
Submitted 9 July, 2024;
originally announced July 2024.
-
Learning Weakly Convex Regularizers for Convergent Image-Reconstruction Algorithms
Authors:
Alexis Goujon,
Sebastian Neumayer,
Michael Unser
Abstract:
We propose to learn non-convex regularizers with a prescribed upper bound on their weak-convexity modulus. Such regularizers give rise to variational denoisers that minimize a convex energy. They rely on few parameters (less than 15,000) and offer a signal-processing interpretation as they mimic handcrafted sparsity-promoting regularizers. Through numerical experiments, we show that such denoisers…
▽ More
We propose to learn non-convex regularizers with a prescribed upper bound on their weak-convexity modulus. Such regularizers give rise to variational denoisers that minimize a convex energy. They rely on few parameters (less than 15,000) and offer a signal-processing interpretation as they mimic handcrafted sparsity-promoting regularizers. Through numerical experiments, we show that such denoisers outperform convex-regularization methods as well as the popular BM3D denoiser. Additionally, the learned regularizer can be deployed to solve inverse problems with iterative schemes that provably converge. For both CT and MRI reconstruction, the regularizer generalizes well and offers an excellent tradeoff between performance, number of parameters, guarantees, and interpretability when compared to other data-driven approaches.
△ Less
Submitted 20 December, 2023; v1 submitted 21 August, 2023;
originally announced August 2023.
-
A Neural-Network-Based Convex Regularizer for Inverse Problems
Authors:
Alexis Goujon,
Sebastian Neumayer,
Pakshal Bohra,
Stanislas Ducotterd,
Michael Unser
Abstract:
The emergence of deep-learning-based methods to solve image-reconstruction problems has enabled a significant increase in reconstruction quality. Unfortunately, these new methods often lack reliability and explainability, and there is a growing interest to address these shortcomings while retaining the boost in performance. In this work, we tackle this issue by revisiting regularizers that are the…
▽ More
The emergence of deep-learning-based methods to solve image-reconstruction problems has enabled a significant increase in reconstruction quality. Unfortunately, these new methods often lack reliability and explainability, and there is a growing interest to address these shortcomings while retaining the boost in performance. In this work, we tackle this issue by revisiting regularizers that are the sum of convex-ridge functions. The gradient of such regularizers is parameterized by a neural network that has a single hidden layer with increasing and learnable activation functions. This neural network is trained within a few minutes as a multistep Gaussian denoiser. The numerical experiments for denoising, CT, and MRI reconstruction show improvements over methods that offer similar reliability guarantees.
△ Less
Submitted 25 August, 2023; v1 submitted 22 November, 2022;
originally announced November 2022.
-
Stability of Image-Reconstruction Algorithms
Authors:
Pol del Aguila Pla,
Sebastian Neumayer,
Michael Unser
Abstract:
Robustness and stability of image-reconstruction algorithms have recently come under scrutiny. Their importance to medical imaging cannot be overstated. We review the known results for the topical variational regularization strategies ($\ell_2$ and $\ell_1$ regularization) and present novel stability results for $\ell_p$-regularized linear inverse problems for $p\in(1,\infty)$. Our results guarant…
▽ More
Robustness and stability of image-reconstruction algorithms have recently come under scrutiny. Their importance to medical imaging cannot be overstated. We review the known results for the topical variational regularization strategies ($\ell_2$ and $\ell_1$ regularization) and present novel stability results for $\ell_p$-regularized linear inverse problems for $p\in(1,\infty)$. Our results guarantee Lipschitz continuity for small $p$ and Hölder continuity for larger $p$. They generalize well to the $L_p(Ω)$ function spaces.
△ Less
Submitted 20 January, 2023; v1 submitted 14 June, 2022;
originally announced June 2022.
-
Convolutional Proximal Neural Networks and Plug-and-Play Algorithms
Authors:
Johannes Hertrich,
Sebastian Neumayer,
Gabriele Steidl
Abstract:
In this paper, we introduce convolutional proximal neural networks (cPNNs), which are by construction averaged operators. For filters of full length, we propose a stochastic gradient descent algorithm on a submanifold of the Stiefel manifold to train cPNNs. In case of filters with limited length, we design algorithms for minimizing functionals that approximate the orthogonality constraints imposed…
▽ More
In this paper, we introduce convolutional proximal neural networks (cPNNs), which are by construction averaged operators. For filters of full length, we propose a stochastic gradient descent algorithm on a submanifold of the Stiefel manifold to train cPNNs. In case of filters with limited length, we design algorithms for minimizing functionals that approximate the orthogonality constraints imposed on the operators by penalizing the least squares distance to the identity operator. Then, we investigate how scaled cPNNs with a prescribed Lipschitz constant can be used for denoising signals and images, where the achieved quality depends on the Lipschitz constant. Finally, we apply cPNN based denoisers within a Plug-and-Play (PnP) framework and provide convergence results for the corresponding PnP forward-backward splitting algorithm based on an oracle construction.
△ Less
Submitted 4 November, 2020;
originally announced November 2020.
