-
Single-shot Star-convex Polygon-based Instance Segmentation for Spatially-correlated Biomedical Objects
Authors:
Trina De,
Adrian Urbanski,
Artur Yakimovich
Abstract:
Biomedical images often contain objects known to be spatially correlated or nested due to their inherent properties, leading to semantic relations. Examples include cell nuclei being nested within eukaryotic cells and colonies growing exclusively within their culture dishes. While these semantic relations bear key importance, detection tasks are often formulated independently, requiring multi-shot…
▽ More
Biomedical images often contain objects known to be spatially correlated or nested due to their inherent properties, leading to semantic relations. Examples include cell nuclei being nested within eukaryotic cells and colonies growing exclusively within their culture dishes. While these semantic relations bear key importance, detection tasks are often formulated independently, requiring multi-shot analysis pipelines. Importantly, spatial correlation could constitute a fundamental prior facilitating learning of more meaningful representations for tasks like instance segmentation. This knowledge has, thus far, not been utilised by the biomedical computer vision community. We argue that the instance segmentation of two or more categories of objects can be achieved in parallel. We achieve this via two architectures HydraStarDist (HSD) and the novel (HSD-WBR) based on the widely-used StarDist (SD), to take advantage of the star-convexity of our target objects. HSD and HSD-WBR are constructed to be capable of incorporating their interactions as constraints into account. HSD implicitly incorporates spatial correlation priors based on object interaction through a joint encoder. HSD-WBR further enforces the prior in a regularisation layer with the penalty we proposed named Within Boundary Regularisation Penalty (WBR). Both architectures achieve nested instance segmentation in a single shot. We demonstrate their competitiveness based on $IoU_R$ and AP and superiority in a new, task-relevant criteria, Joint TP rate (JTPR) compared to their baseline SD and Cellpose. Our approach can be further modified to capture partial-inclusion/-exclusion in multi-object interactions in fluorescent or brightfield microscopy or digital imaging. Finally, our strategy suggests gains by making this learning single-shot and computationally efficient.
△ Less
Submitted 16 April, 2025;
originally announced April 2025.
-
Heterogeneity of tumor biophysical properties and their potential role as prognostic markers
Authors:
Anja Madleine Markl,
Daniel Nieder,
Diana Isabel Sandoval-Bojorquez,
Anna Taubenberger,
Jean-François Berret,
Artur Yakimovich,
Eduardo Sergio Oliveros- Mata,
Larysa Baraban,
Anna Dubrovska
Abstract:
Progress in our knowledge of tumor mechanisms and complexity led to the understanding of the physical parameters of cancer cells and their microenvironment, including the mechanical, thermal, and electrical properties, solid stress, and liquid pressure, as critical regulators of tumor progression and potential prognostic traits associated with clinical outcomes. The biological hallmarks of cancer…
▽ More
Progress in our knowledge of tumor mechanisms and complexity led to the understanding of the physical parameters of cancer cells and their microenvironment, including the mechanical, thermal, and electrical properties, solid stress, and liquid pressure, as critical regulators of tumor progression and potential prognostic traits associated with clinical outcomes. The biological hallmarks of cancer and physical abnormalities of tumors are mutually reinforced, promoting a vicious cycle of tumor progression. A comprehensive analysis of the biological and physical tumor parameters is critical for developing more robust prognostic and diagnostic markers and improving treatment efficiency. Like the biological tumor traits, physical tumor features are characterized by inter-and intratumoral heterogeneity. The dynamic changes of physical tumor traits during tumor progression and as a result of tumor treatment highlight the necessity of their spatial and temporal analysis in clinical settings. This review focuses on the biological basis of the tumor-specific physical traits, the state-of-the-art methods of their analyses, and the perspective of clinical translation. The importance of tumor physical parameters for disease progression and therapy resistance, as well as current treatment strategies to monitor and target tumor physical traits in clinics, is highlighted.
△ Less
Submitted 29 November, 2024;
originally announced November 2024.
-
Solving the inverse problem of microscopy deconvolution with a residual Beylkin-Coifman-Rokhlin neural network
Authors:
Rui Li,
Mikhail Kudryashev,
Artur Yakimovich
Abstract:
Optic deconvolution in light microscopy (LM) refers to recovering the object details from images, revealing the ground truth of samples. Traditional explicit methods in LM rely on the point spread function (PSF) during image acquisition. Yet, these approaches often fall short due to inaccurate PSF models and noise artifacts, hampering the overall restoration quality. In this paper, we approached t…
▽ More
Optic deconvolution in light microscopy (LM) refers to recovering the object details from images, revealing the ground truth of samples. Traditional explicit methods in LM rely on the point spread function (PSF) during image acquisition. Yet, these approaches often fall short due to inaccurate PSF models and noise artifacts, hampering the overall restoration quality. In this paper, we approached the optic deconvolution as an inverse problem. Motivated by the nonstandard-form compression scheme introduced by Beylkin, Coifman, and Rokhlin (BCR), we proposed an innovative physics-informed neural network Multi-Stage Residual-BCR Net (m-rBCR) to approximate the optic deconvolution. We validated the m-rBCR model on four microscopy datasets - two simulated microscopy datasets from ImageNet and BioSR, real dSTORM microscopy images, and real widefield microscopy images. In contrast to the explicit deconvolution methods (e.g. Richardson-Lucy) and other state-of-the-art NN models (U-Net, DDPM, CARE, DnCNN, ESRGAN, RCAN, Noise2Noise, MPRNet, and MIMO-U-Net), the m-rBCR model demonstrates superior performance to other candidates by PSNR and SSIM in two real microscopy datasets and the simulated BioSR dataset. In the simulated ImageNet dataset, m-rBCR ranks the second-best place (right after MIMO-U-Net). With the backbone from the optical physics, m-rBCR exploits the trainable parameters with better performances (from ~30 times fewer than the benchmark MIMO-U-Net to ~210 times than ESRGAN). This enables m-rBCR to achieve a shorter runtime (from ~3 times faster than MIMO-U-Net to ~300 times faster than DDPM). To summarize, by leveraging physics constraints our model reduced potentially redundant parameters significantly in expertise-oriented NN candidates and achieved high efficiency with superior performance.
△ Less
Submitted 15 July, 2024; v1 submitted 3 July, 2024;
originally announced July 2024.
-
Phenotype-preserving metric design for high-content image reconstruction by generative inpainting
Authors:
Vaibhav Sharma,
Artur Yakimovich
Abstract:
In the past decades, automated high-content microscopy demonstrated its ability to deliver large quantities of image-based data powering the versatility of phenotypic drug screening and systems biology applications. However, as the sizes of image-based datasets grew, it became infeasible for humans to control, avoid and overcome the presence of imaging and sample preparation artefacts in the image…
▽ More
In the past decades, automated high-content microscopy demonstrated its ability to deliver large quantities of image-based data powering the versatility of phenotypic drug screening and systems biology applications. However, as the sizes of image-based datasets grew, it became infeasible for humans to control, avoid and overcome the presence of imaging and sample preparation artefacts in the images. While novel techniques like machine learning and deep learning may address these shortcomings through generative image inpainting, when applied to sensitive research data this may come at the cost of undesired image manipulation. Undesired manipulation may be caused by phenomena such as neural hallucinations, to which some artificial neural networks are prone. To address this, here we evaluate the state-of-the-art inpainting methods for image restoration in a high-content fluorescence microscopy dataset of cultured cells with labelled nuclei. We show that architectures like DeepFill V2 and Edge Connect can faithfully restore microscopy images upon fine-tuning with relatively little data. Our results demonstrate that the area of the region to be restored is of higher importance than shape. Furthermore, to control for the quality of restoration, we propose a novel phenotype-preserving metric design strategy. In this strategy, the size and count of the restored biological phenotypes like cell nuclei are quantified to penalise undesirable manipulation. We argue that the design principles of our approach may also generalise to other applications.
△ Less
Submitted 22 August, 2023; v1 submitted 26 July, 2023;
originally announced July 2023.
-
Microscopy image reconstruction with physics-informed denoising diffusion probabilistic model
Authors:
Rui Li,
Gabriel della Maggiora,
Vardan Andriasyan,
Anthony Petkidis,
Artsemi Yushkevich,
Mikhail Kudryashev,
Artur Yakimovich
Abstract:
Light microscopy is a widespread and inexpensive imaging technique facilitating biomedical discovery and diagnostics. However, light diffraction barrier and imperfections in optics limit the level of detail of the acquired images. The details lost can be reconstructed among others by deep learning models. Yet, deep learning models are prone to introduce artefacts and hallucinations into the recons…
▽ More
Light microscopy is a widespread and inexpensive imaging technique facilitating biomedical discovery and diagnostics. However, light diffraction barrier and imperfections in optics limit the level of detail of the acquired images. The details lost can be reconstructed among others by deep learning models. Yet, deep learning models are prone to introduce artefacts and hallucinations into the reconstruction. Recent state-of-the-art image synthesis models like the denoising diffusion probabilistic models (DDPMs) are no exception to this. We propose to address this by incorporating the physical problem of microscopy image formation into the model's loss function. To overcome the lack of microscopy data, we train this model with synthetic data. We simulate the effects of the microscope optics through the theoretical point spread function and varying the noise levels to obtain synthetic data. Furthermore, we incorporate the physical model of a light microscope into the reverse process of a conditioned DDPM proposing a physics-informed DDPM (PI-DDPM). We show consistent improvement and artefact reductions when compared to model-based methods, deep-learning regression methods and regular conditioned DDPMs.
△ Less
Submitted 8 June, 2023; v1 submitted 5 June, 2023;
originally announced June 2023.
-
MatlabCompat.jl: helping Julia understand Your Matlab/Octave Code
Authors:
Vardan Andriasyan,
Yauhen Yakimovich,
Artur Yakimovich
Abstract:
Scientific legacy code in MATLAB/Octave not compatible with modernization of research workflows is vastly abundant throughout academic community. Performance of non-vectorized code written in MATLAB/Octave represents a major burden. A new programming language for technical computing Julia, promises to address these issues. Although Julia syntax is similar to MATLAB/Octave, porting code to Julia ma…
▽ More
Scientific legacy code in MATLAB/Octave not compatible with modernization of research workflows is vastly abundant throughout academic community. Performance of non-vectorized code written in MATLAB/Octave represents a major burden. A new programming language for technical computing Julia, promises to address these issues. Although Julia syntax is similar to MATLAB/Octave, porting code to Julia may be cumbersome for researchers. Here we present MatlabCompat.jl - a library aimed at simplifying the conversion of your MATLAB/Octave code to Julia. We show using a simplistic image analysis use case that MATLAB/Octave code can be easily ported to high performant Julia using MatlabCompat.jl.
△ Less
Submitted 6 January, 2017;
originally announced January 2017.
