-
Fast and stable schemes for non-linear osmosis filtering
Authors:
L. Calatroni,
S. Morigi,
S. Parisotto,
G. A. Recupero
Abstract:
We consider a non-linear variant of the transport-diffusion osmosis model for solving a variety of imaging problems such as shadow/soft-light removal and compact data representation. The non-linear behaviour is encoded in terms of a general scalar function g with suitable properties, which allows to balance the diffusion intensity on the different regions of the image while preventing smoothing ar…
▽ More
We consider a non-linear variant of the transport-diffusion osmosis model for solving a variety of imaging problems such as shadow/soft-light removal and compact data representation. The non-linear behaviour is encoded in terms of a general scalar function g with suitable properties, which allows to balance the diffusion intensity on the different regions of the image while preventing smoothing artefacts. For the proposed model, conservation properties (intensity and non-negativity) are proved and a variational interpretation is showed for specific choices of g. Upon suitable spatial discretisation, both an explicit and a semi-implicit iterative scheme are considered, for which convergence restrictions and unconditional stability are proved, respectively. To validate the proposed modelling and the computational speed of the numerical schemes considered, we report several results and comparisons for the problem of shadow/light-spot removal and compact data representation, showing that artefact-free and computationally efficient results are obtained in comparison to standard linear and anisotropic models, and state-of-the art approaches.
△ Less
Submitted 29 March, 2022;
originally announced March 2022.
-
Mathematical osmosis imaging for multi-modal and multi-spectral applications in Cultural Heritage conservation
Authors:
Simone Parisotto,
Luca Calatroni,
Claudia Daffara
Abstract:
In this work we present a dual-mode mid-infrared workflow [6], for detecting sub-superficial mural damages in frescoes artworks. Due to the large nature of frescoes, multiple thermal images are recorded. Thus, the experimental setup may introduce measurements errors, seen as inter-frame changes in the image contrast, after mosaicking. An approach to lowering errors is to post-process the mosaic [1…
▽ More
In this work we present a dual-mode mid-infrared workflow [6], for detecting sub-superficial mural damages in frescoes artworks. Due to the large nature of frescoes, multiple thermal images are recorded. Thus, the experimental setup may introduce measurements errors, seen as inter-frame changes in the image contrast, after mosaicking. An approach to lowering errors is to post-process the mosaic [10] via osmosis partial differential equation (PDE) [12, 13], which preserves details, mass and balance the lights: efficient numerical study for osmosis on large images is proposed [2, 11], based on operator splitting [8]. Our range of Cultural Heritage applications include the detection of sub-superficial voids in Monocromo (L. Da Vinci, Castello Sforzesco, Milan) [5], the light-balance for multi-spectral imaging and the data integration on the Archimedes Palimpsest [10].
△ Less
Submitted 29 June, 2021;
originally announced June 2021.
-
Total Variation Regularisation with Spatially Variable Lipschitz Constraints
Authors:
Martin Burger,
Yury Korolev,
Simone Parisotto,
Carola-Bibiane Schönlieb
Abstract:
We introduce a first order Total Variation type regulariser that decomposes a function into a part with a given Lipschitz constant (which is also allowed to vary spatially) and a jump part. The kernel of this regulariser contains all functions whose Lipschitz constant does not exceed a given value, hence by locally adjusting this value one can determine how much variation is the reconstruction all…
▽ More
We introduce a first order Total Variation type regulariser that decomposes a function into a part with a given Lipschitz constant (which is also allowed to vary spatially) and a jump part. The kernel of this regulariser contains all functions whose Lipschitz constant does not exceed a given value, hence by locally adjusting this value one can determine how much variation is the reconstruction allowed to have. We prove regularising properties of this functional, study its connections to other Total Variation type regularisers and propose a primal dual optimisation scheme. Our numerical experiments demonstrate that the proposed first order regulariser can achieve reconstruction quality similar to that of second order regularisers such as Total Generalised Variation, while requiring significantly less computational time.
△ Less
Submitted 5 December, 2019;
originally announced December 2019.
-
Variational Osmosis for Non-linear Image Fusion
Authors:
Simone Parisotto,
Luca Calatroni,
Aurélie Bugeau,
Nicolas Papadakis,
Carola-Bibiane Schönlieb
Abstract:
We propose a new variational model for non-linear image fusion. Our approach is based on the use of an osmosis energy term related to the one studied in Vogel et al. (2013) and Weickert et al. (2013) The minimization of the proposed non-convex energy realizes visually plausible image data fusion, invariant to multiplicative brightness changes. On the practical side, it requires minimal supervision…
▽ More
We propose a new variational model for non-linear image fusion. Our approach is based on the use of an osmosis energy term related to the one studied in Vogel et al. (2013) and Weickert et al. (2013) The minimization of the proposed non-convex energy realizes visually plausible image data fusion, invariant to multiplicative brightness changes. On the practical side, it requires minimal supervision and parameter tuning and can encode prior information on the structure of the images to be fused. For the numerical solution of the proposed model, we develop a primal-dual algorithm and we apply the resulting minimization scheme to solve multi-modal face fusion, color transfer and cultural heritage conservation problems. Visual and quantitative comparisons to state-of-the-art approaches prove the out-performance and the flexibility of our method.
△ Less
Submitted 24 March, 2020; v1 submitted 4 October, 2019;
originally announced October 2019.
-
Total Directional Variation for Video Denoising
Authors:
Simone Parisotto,
Carola-Bibiane Schönlieb
Abstract:
In this paper, we propose a variational approach for video denoising, based on a total directional variation (TDV) regulariser proposed in Parisotto et al. (2018), for image denoising and interpolation. In the TDV regulariser, the underlying image structure is encoded by means of weighted derivatives so as to enhance the anisotropic structures in images, e.g. stripes or curves with a dominant loca…
▽ More
In this paper, we propose a variational approach for video denoising, based on a total directional variation (TDV) regulariser proposed in Parisotto et al. (2018), for image denoising and interpolation. In the TDV regulariser, the underlying image structure is encoded by means of weighted derivatives so as to enhance the anisotropic structures in images, e.g. stripes or curves with a dominant local directionality. For the extension of TDV to video denoising, the space-time structure is captured by the volumetric structure tensor guiding the smoothing process. We discuss this and present our whole video denoising work-flow. Our numerical results are compared with some state-of-the-art video denoising methods.
△ Less
Submitted 29 March, 2019; v1 submitted 12 December, 2018;
originally announced December 2018.
-
Higher-Order Total Directional Variation: Analysis
Authors:
Simone Parisotto,
Simon Masnou,
Carola-Bibiane Schönlieb
Abstract:
We analyse a new notion of total anisotropic higher-order variation which, differently from the Total Generalized Variation by Bredies et al., quantifies for possibly non-symmetric tensor fields their variations at arbitrary order weighted by possibly inhomogeneous, smooth elliptic anisotropies. We prove some properties of this total variation and of the associated spaces of tensors with finite va…
▽ More
We analyse a new notion of total anisotropic higher-order variation which, differently from the Total Generalized Variation by Bredies et al., quantifies for possibly non-symmetric tensor fields their variations at arbitrary order weighted by possibly inhomogeneous, smooth elliptic anisotropies. We prove some properties of this total variation and of the associated spaces of tensors with finite variations. We show the existence of solutions to a related regularity-fidelity optimisation problem. We also prove a decomposition formula which appears to be helpful for the design of numerical schemes, as shown in a companion paper, where several applications to image processing are studied.
△ Less
Submitted 8 January, 2020; v1 submitted 12 December, 2018;
originally announced December 2018.
-
Higher-Order Total Directional Variation: Imaging Applications
Authors:
Simone Parisotto,
Jan Lellmann,
Simon Masnou,
Carola-Bibiane Schönlieb
Abstract:
We introduce a class of higher-order anisotropic total variation regularisers, which are defined for possibly inhomogeneous, smooth elliptic anisotropies, that extends the Total Generalized Variation (TGV) regulariser and its variants. We propose a primal-dual hybrid gradient approach to approximate numerically the associated gradient flow. This choice of regularisers allows to preserve and enhanc…
▽ More
We introduce a class of higher-order anisotropic total variation regularisers, which are defined for possibly inhomogeneous, smooth elliptic anisotropies, that extends the Total Generalized Variation (TGV) regulariser and its variants. We propose a primal-dual hybrid gradient approach to approximate numerically the associated gradient flow. This choice of regularisers allows to preserve and enhance intrinsic anisotropic features in images. This is illustrated on various examples from different imaging applications: image denoising, wavelet-based image zooming, and reconstruction of surfaces from scattered height measurements.
△ Less
Submitted 6 July, 2020; v1 submitted 12 December, 2018;
originally announced December 2018.
-
Anisotropic osmosis filtering for shadow removal in images
Authors:
Simone Parisotto,
Luca Calatroni,
Marco Caliari,
Carola-Bibiane Schönlieb,
Joachim Weickert
Abstract:
We present an anisotropic extension of the isotropic osmosis model that has been introduced by Weickert et al.~(Weickert, 2013) for visual computing applications, and we adapt it specifically to shadow removal applications. We show that in the integrable setting, linear anisotropic osmosis minimises an energy that involves a suitable quadratic form which models local directional structures. In our…
▽ More
We present an anisotropic extension of the isotropic osmosis model that has been introduced by Weickert et al.~(Weickert, 2013) for visual computing applications, and we adapt it specifically to shadow removal applications. We show that in the integrable setting, linear anisotropic osmosis minimises an energy that involves a suitable quadratic form which models local directional structures. In our shadow removal applications we estimate the local structure via a modified tensor voting approach (Moreno, 2012) and use this information within an anisotropic diffusion inpainting that resembles edge-enhancing anisotropic diffusion inpainting (Weickert, 2006, Galić, 2008). Our numerical scheme combines the nonnegativity preserving stencil of Fehrenbach and Mirebeau (Fehrenbach, 2014) with an exact time stepping based on highly accurate polynomial approximations of the matrix exponential. The resulting anisotropic model is tested on several synthetic and natural images corrupted by constant shadows. We show that it outperforms isotropic osmosis, since it does not suffer from blurring artefacts at the shadow boundaries.
△ Less
Submitted 17 September, 2018;
originally announced September 2018.
-
Unveiling the invisible - mathematical methods for restoring and interpreting illuminated manuscripts
Authors:
Luca Calatroni,
Marie d'Autume,
Rob Hocking,
Stella Panayotova,
Simone Parisotto,
Paola Ricciardi,
Carola-Bibiane Schönlieb
Abstract:
The last fifty years have seen an impressive development of mathematical methods for the analysis and processing of digital images, mostly in the context of photography, biomedical imaging and various forms of engineering. The arts have been mostly overlooked in this process, apart from a few exceptional works in the last ten years. With the rapid emergence of digitisation in the arts, however, th…
▽ More
The last fifty years have seen an impressive development of mathematical methods for the analysis and processing of digital images, mostly in the context of photography, biomedical imaging and various forms of engineering. The arts have been mostly overlooked in this process, apart from a few exceptional works in the last ten years. With the rapid emergence of digitisation in the arts, however, the arts domain is becoming increasingly receptive to digital image processing methods and the importance of paying attention to this therefore increases. In this paper we discuss a range of mathematical methods for digital image restoration and digital visualisation for illuminated manuscripts. The latter provide an interesting opportunity for digital manipulation because they traditionally remain physically untouched. At the same time they also serve as an example for the possibilities mathematics and digital restoration offer as a generic and objective toolkit for the arts.
△ Less
Submitted 19 March, 2018;
originally announced March 2018.
-
Digital Cultural Heritage imaging via osmosis filtering
Authors:
Simone Parisotto,
Luca Calatroni,
Claudia Daffara
Abstract:
In Cultural Heritage (CH) imaging, data acquired within different spectral regions are often used to inspect surface and sub-surface features. Due to the experimental setup, these images may suffer from intensity inhomogeneities, which may prevent conservators from distinguishing the physical properties of the object under restoration. Furthermore, in multi-modal imaging, the transfer of informati…
▽ More
In Cultural Heritage (CH) imaging, data acquired within different spectral regions are often used to inspect surface and sub-surface features. Due to the experimental setup, these images may suffer from intensity inhomogeneities, which may prevent conservators from distinguishing the physical properties of the object under restoration. Furthermore, in multi-modal imaging, the transfer of information between one modality to another is often used to integrate image contents. In this paper, we apply the image osmosis model proposed in (Weickert et al. 2013) to solve similar problems arising when using diagnostic CH imaging techniques based on reflectance, emission and fluorescence mode in the optical and thermal range. For an efficient computation, we use stable operator splitting techniques. We test our methods on real artwork datasets: the thermal measurements of the mural painting "Monocromo" by Leonardo Da Vinci, the UV-VIS-IR imaging of an ancient Russian icon and the Archimedes Palimpsest dataset.
△ Less
Submitted 16 April, 2018; v1 submitted 15 February, 2018;
originally announced February 2018.
-
Efficient Osmosis Filtering of Thermal-Quasi Reflectography Images for Cultural Heritage
Authors:
Simone Parisotto,
Luca Calatroni,
Claudia Daffara
Abstract:
In Cultural Heritage, non-invasive infrared imaging techniques are used to analyse portions of deep structures behind wall paintings. When mosaicked, these images usually suffer from light inhomogeneities due to the experimental setup, which may prevent restorers from distinguishing the physical properties of the object under restoration. A light-balanced image is therefore essential for inter-fra…
▽ More
In Cultural Heritage, non-invasive infrared imaging techniques are used to analyse portions of deep structures behind wall paintings. When mosaicked, these images usually suffer from light inhomogeneities due to the experimental setup, which may prevent restorers from distinguishing the physical properties of the object under restoration. A light-balanced image is therefore essential for inter-frame comparisons, while preserving intra-frames details. In this paper we apply the image osmosis model proposed in (Weickert, 2013) to solve the light balance problem in Thermal-Quasi Reflectography (TQR) imaging. Due to the large amount of image data, the computation of the numerical solution of the model may be prohibitively costly. To overcome this issue, we make use of efficient operator splitting techniques. We test the proposed numerical schemes on the TQR measurement dataset of the mural painting "Monocromo" by Leonardo Da Vinci at Castello Sforzesco (Milan, Italy). The light corrected result is registered to a visible orthophoto, which makes it re-usable for further restorations.
△ Less
Submitted 13 April, 2017;
originally announced April 2017.
-
Alternating Direction Implicit (ADI) schemes for a PDE-based image osmosis model
Authors:
Luca Calatroni,
Claudio Estatico,
Nicola Garibaldi,
Simone Parisotto
Abstract:
We consider \emph{Alternating Direction Implicit} (ADI) splitting schemes to compute efficiently the numerical solution of the PDE osmosis model considered by Weickert et al. for several imaging applications. The discretised scheme is shown to preserve analogous properties to the continuous model. The dimensional splitting strategy traduces numerically into the solution of simple tridiagonal syste…
▽ More
We consider \emph{Alternating Direction Implicit} (ADI) splitting schemes to compute efficiently the numerical solution of the PDE osmosis model considered by Weickert et al. for several imaging applications. The discretised scheme is shown to preserve analogous properties to the continuous model. The dimensional splitting strategy traduces numerically into the solution of simple tridiagonal systems for which standard matrix factorisation techniques can be used to improve upon the performance of classical implicit methods, even for large time steps. Applications to the shadow removal problem are presented.
△ Less
Submitted 3 August, 2017; v1 submitted 13 March, 2017;
originally announced March 2017.
