-
3D fluorescence microscopy data synthesis for segmentation and benchmarking
Authors:
Dennis Eschweiler,
Malte Rethwisch,
Mareike Jarchow,
Simon Koppers,
Johannes Stegmaier
Abstract:
Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general la…
▽ More
Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotation masks of 3D cellular structures. In combination with mask simulation approaches, we demonstrate the generation of fully-annotated 3D microscopy data sets that we make publicly available for training or benchmarking. An additional positional conditioning of the cellular structures enables the reconstruction of position-dependent intensity characteristics and allows to generate image data of different quality levels. A patch-wise working principle and a subsequent full-size reassemble strategy is used to generate image data of arbitrary size and different organisms. We present this as a proof-of-concept for the automated generation of fully-annotated training data sets requiring only a minimum of manual interaction to alleviate the need of manual annotations.
△ Less
Submitted 21 July, 2021;
originally announced July 2021.
-
Spherical Harmonics for Shape-Constrained 3D Cell Segmentation
Authors:
Dennis Eschweiler,
Malte Rethwisch,
Simon Koppers,
Johannes Stegmaier
Abstract:
Recent microscopy imaging techniques allow to precisely analyze cell morphology in 3D image data. To process the vast amount of image data generated by current digitized imaging techniques, automated approaches are demanded more than ever. Segmentation approaches used for morphological analyses, however, are often prone to produce unnaturally shaped predictions, which in conclusion could lead to i…
▽ More
Recent microscopy imaging techniques allow to precisely analyze cell morphology in 3D image data. To process the vast amount of image data generated by current digitized imaging techniques, automated approaches are demanded more than ever. Segmentation approaches used for morphological analyses, however, are often prone to produce unnaturally shaped predictions, which in conclusion could lead to inaccurate experimental outcomes. In order to minimize further manual interaction, shape priors help to constrain the predictions to the set of natural variations. In this paper, we show how spherical harmonics can be used as an alternative way to inherently constrain the predictions of neural networks for the segmentation of cells in 3D microscopy image data. Benefits and limitations of the spherical harmonic representation are analyzed and final results are compared to other state-of-the-art approaches on two different data sets.
△ Less
Submitted 23 October, 2020;
originally announced October 2020.
-
Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge
Authors:
Spyridon Bakas,
Mauricio Reyes,
Andras Jakab,
Stefan Bauer,
Markus Rempfler,
Alessandro Crimi,
Russell Takeshi Shinohara,
Christoph Berger,
Sung Min Ha,
Martin Rozycki,
Marcel Prastawa,
Esther Alberts,
Jana Lipkova,
John Freymann,
Justin Kirby,
Michel Bilello,
Hassan Fathallah-Shaykh,
Roland Wiest,
Jan Kirschke,
Benedikt Wiestler,
Rivka Colen,
Aikaterini Kotrotsou,
Pamela Lamontagne,
Daniel Marcus,
Mikhail Milchenko
, et al. (402 additional authors not shown)
Abstract:
Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles dissem…
▽ More
Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles disseminated across multi-parametric magnetic resonance imaging (mpMRI) scans, reflecting varying biological properties. Their heterogeneous shape, extent, and location are some of the factors that make these tumors difficult to resect, and in some cases inoperable. The amount of resected tumor is a factor also considered in longitudinal scans, when evaluating the apparent tumor for potential diagnosis of progression. Furthermore, there is mounting evidence that accurate segmentation of the various tumor sub-regions can offer the basis for quantitative image analysis towards prediction of patient overall survival. This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i.e., 2012-2018. Specifically, we focus on i) evaluating segmentations of the various glioma sub-regions in pre-operative mpMRI scans, ii) assessing potential tumor progression by virtue of longitudinal growth of tumor sub-regions, beyond use of the RECIST/RANO criteria, and iii) predicting the overall survival from pre-operative mpMRI scans of patients that underwent gross total resection. Finally, we investigate the challenge of identifying the best ML algorithms for each of these tasks, considering that apart from being diverse on each instance of the challenge, the multi-institutional mpMRI BraTS dataset has also been a continuously evolving/growing dataset.
△ Less
Submitted 23 April, 2019; v1 submitted 5 November, 2018;
originally announced November 2018.
-
Spherical Harmonic Residual Network for Diffusion Signal Harmonization
Authors:
Simon Koppers,
Luke Bloy,
Jeffrey I. Berman,
Chantal M. W. Tax,
J. Christopher Edgar,
Dorit Merhof
Abstract:
Diffusion imaging is an important method in the field of neuroscience, as it is sensitive to changes within the tissue microstructure of the human brain. However, a major challenge when using MRI to derive quantitative measures is that the use of different scanners, as used in multi-site group studies, introduces measurement variability. This can lead to an increased variance in quantitative metri…
▽ More
Diffusion imaging is an important method in the field of neuroscience, as it is sensitive to changes within the tissue microstructure of the human brain. However, a major challenge when using MRI to derive quantitative measures is that the use of different scanners, as used in multi-site group studies, introduces measurement variability. This can lead to an increased variance in quantitative metrics, even if the same brain is scanned.
Contrary to the assumption that these characteristics are comparable and similar, small changes in these values are observed in many clinical studies, hence harmonization of the signals is essential.
In this paper, we present a method that does not require additional preprocessing, such as segmentation or registration, and harmonizes the signal based on a deep learning residual network. For this purpose, a training database is required, which consist of the same subjects, scanned on different scanners.
The results show that harmonized signals are significantly more similar to the ground truth signal compared to no harmonization, but also improve in comparison to another deep learning method. The same effect is also demonstrated in commonly used metrics derived from the diffusion MRI signal.
△ Less
Submitted 5 August, 2018;
originally announced August 2018.
-
DELIMIT PyTorch - An extension for Deep Learning in Diffusion Imaging
Authors:
Simon Koppers,
Dorit Merhof
Abstract:
DELIMIT is a framework extension for deep learning in diffusion imaging, which extends the basic framework PyTorch towards spherical signals. Based on several novel layers, deep learning can be applied to spherical diffusion imaging data in a very convenient way. First, two spherical harmonic interpolation layers are added to the extension, which allow to transform the signal from spherical surfac…
▽ More
DELIMIT is a framework extension for deep learning in diffusion imaging, which extends the basic framework PyTorch towards spherical signals. Based on several novel layers, deep learning can be applied to spherical diffusion imaging data in a very convenient way. First, two spherical harmonic interpolation layers are added to the extension, which allow to transform the signal from spherical surface space into the spherical harmonic space, and vice versa. In addition, a local spherical convolution layer is introduced that adds the possibility to include gradient neighborhood information within the network. Furthermore, these extensions can also be utilized for the preprocessing of diffusion signals.
△ Less
Submitted 4 August, 2018;
originally announced August 2018.
