Showing 1–49 of 49 results for author: Stegmaier, J

Towards Robust and Generalizable Gerchberg Saxton based Physics Inspired Neural Networks for Computer Generated Holography: A Sensitivity Analysis Framework

Authors: Ankit Amrutkar, Björn Kampa, Volkmar Schulz, Johannes Stegmaier, Markus Rothermel, Dorit Merhof

Abstract: Computer-generated holography (CGH) enables applications in holographic augmented reality (AR), 3D displays, systems neuroscience, and optical trapping. The fundamental challenge in CGH is solving the inverse problem of phase retrieval from intensity measurements. Physics-inspired neural networks (PINNs), especially Gerchberg-Saxton-based PINNs (GS-PINNs), have advanced phase retrieval capabilitie… ▽ More Computer-generated holography (CGH) enables applications in holographic augmented reality (AR), 3D displays, systems neuroscience, and optical trapping. The fundamental challenge in CGH is solving the inverse problem of phase retrieval from intensity measurements. Physics-inspired neural networks (PINNs), especially Gerchberg-Saxton-based PINNs (GS-PINNs), have advanced phase retrieval capabilities. However, their performance strongly depends on forward models (FMs) and their hyperparameters (FMHs), limiting generalization, complicating benchmarking, and hindering hardware optimization. We present a systematic sensitivity analysis framework based on Saltelli's extension of Sobol's method to quantify FMH impacts on GS-PINN performance. Our analysis demonstrates that SLM pixel-resolution is the primary factor affecting neural network sensitivity, followed by pixel-pitch, propagation distance, and wavelength. Free space propagation forward models demonstrate superior neural network performance compared to Fourier holography, providing enhanced parameterization and generalization. We introduce a composite evaluation metric combining performance consistency, generalization capability, and hyperparameter perturbation resilience, establishing a unified benchmarking standard across CGH configurations. Our research connects physics-inspired deep learning theory with practical CGH implementations through concrete guidelines for forward model selection, neural network architecture, and performance evaluation. Our contributions advance the development of robust, interpretable, and generalizable neural networks for diverse holographic applications, supporting evidence-based decisions in CGH research and implementation. △ Less

Submitted 30 April, 2025; originally announced May 2025.

Comparative Analysis of Unsupervised and Supervised Autoencoders for Nuclei Classification in Clear Cell Renal Cell Carcinoma Images

Authors: Fatemeh Javadian, Zahra Aminparast, Johannes Stegmaier, Abin Jose

Abstract: This study explores the application of supervised and unsupervised autoencoders (AEs) to automate nuclei classification in clear cell renal cell carcinoma (ccRCC) images, a diagnostic task traditionally reliant on subjective visual grading by pathologists. We evaluate various AE architectures, including standard AEs, contractive AEs (CAEs), and discriminative AEs (DAEs), as well as a classifier-ba… ▽ More This study explores the application of supervised and unsupervised autoencoders (AEs) to automate nuclei classification in clear cell renal cell carcinoma (ccRCC) images, a diagnostic task traditionally reliant on subjective visual grading by pathologists. We evaluate various AE architectures, including standard AEs, contractive AEs (CAEs), and discriminative AEs (DAEs), as well as a classifier-based discriminative AE (CDAE), optimized using the hyperparameter tuning tool Optuna. Bhattacharyya distance is selected from several metrics to assess class separability in the latent space, revealing challenges in distinguishing adjacent grades using unsupervised models. CDAE, integrating a supervised classifier branch, demonstrated superior performance in both latent space separation and classification accuracy. Given that CDAE-CNN achieved notable improvements in classification metrics, affirming the value of supervised learning for class-specific feature extraction, F1 score was incorporated into the tuning process to optimize classification performance. Results show significant improvements in identifying aggressive ccRCC grades by leveraging the classification capability of AE through latent clustering followed by fine-grained classification. Our model outperforms the current state of the art, CHR-Network, across all evaluated metrics. These findings suggest that integrating a classifier branch in AEs, combined with neural architecture search and contrastive learning, enhances grading automation in ccRCC pathology, particularly in detecting aggressive tumor grades, and may improve diagnostic accuracy. △ Less

Submitted 3 April, 2025; originally announced April 2025.

Comments: Accepted 4-page paper at IEEE ISBI 2025. 3 figures, 3 tables

ACM Class: I.2.10; I.4.9

Eye on the Target: Eye Tracking Meets Rodent Tracking

Authors: Emil Mededovic, Yuli Wu, Henning Konermann, Marcin Kopaczka, Mareike Schulz, Rene Tolba, Johannes Stegmaier

Abstract: Analyzing animal behavior from video recordings is crucial for scientific research, yet manual annotation remains labor-intensive and prone to subjectivity. Efficient segmentation methods are needed to automate this process while maintaining high accuracy. In this work, we propose a novel pipeline that utilizes eye-tracking data from Aria glasses to generate prompt points, which are then used to p… ▽ More Analyzing animal behavior from video recordings is crucial for scientific research, yet manual annotation remains labor-intensive and prone to subjectivity. Efficient segmentation methods are needed to automate this process while maintaining high accuracy. In this work, we propose a novel pipeline that utilizes eye-tracking data from Aria glasses to generate prompt points, which are then used to produce segmentation masks via a fast zero-shot segmentation model. Additionally, we apply post-processing to refine the prompts, leading to improved segmentation quality. Through our approach, we demonstrate that combining eye-tracking-based annotation with smart prompt refinement can enhance segmentation accuracy, achieving an improvement of 70.6% from 38.8 to 66.2 in the Jaccard Index for segmentation results in the rats dataset. △ Less

Submitted 13 March, 2025; originally announced March 2025.

CellStyle: Improved Zero-Shot Cell Segmentation via Style Transfer

Authors: Rüveyda Yilmaz, Zhu Chen, Yuli Wu, Johannes Stegmaier

Abstract: Cell microscopy data are abundant; however, corresponding segmentation annotations remain scarce. Moreover, variations in cell types, imaging devices, and staining techniques introduce significant domain gaps between datasets. As a result, even large, pretrained segmentation models trained on diverse datasets (source datasets) struggle to generalize to unseen datasets (target datasets). To overcom… ▽ More Cell microscopy data are abundant; however, corresponding segmentation annotations remain scarce. Moreover, variations in cell types, imaging devices, and staining techniques introduce significant domain gaps between datasets. As a result, even large, pretrained segmentation models trained on diverse datasets (source datasets) struggle to generalize to unseen datasets (target datasets). To overcome this generalization problem, we propose CellStyle, which improves the segmentation quality of such models without requiring labels for the target dataset, thereby enabling zero-shot adaptation. CellStyle transfers the attributes of an unannotated target dataset, such as texture, color, and noise, to the annotated source dataset. This transfer is performed while preserving the cell shapes of the source images, ensuring that the existing source annotations can still be used while maintaining the visual characteristics of the target dataset. The styled synthetic images with the existing annotations enable the finetuning of a generalist segmentation model for application to the unannotated target data. We demonstrate that CellStyle significantly improves zero-shot cell segmentation performance across diverse datasets by finetuning multiple segmentation models on the style-transferred data. The code will be made publicly available. △ Less

Submitted 11 March, 2025; originally announced March 2025.

A Pragmatic Note on Evaluating Generative Models with Fréchet Inception Distance for Retinal Image Synthesis

Authors: Yuli Wu, Fucheng Liu, Rüveyda Yilmaz, Henning Konermann, Peter Walter, Johannes Stegmaier

Abstract: Fréchet Inception Distance (FID), computed with an ImageNet pretrained Inception-v3 network, is widely used as a state-of-the-art evaluation metric for generative models. It assumes that feature vectors from Inception-v3 follow a multivariate Gaussian distribution and calculates the 2-Wasserstein distance based on their means and covariances. While FID effectively measures how closely synthetic da… ▽ More Fréchet Inception Distance (FID), computed with an ImageNet pretrained Inception-v3 network, is widely used as a state-of-the-art evaluation metric for generative models. It assumes that feature vectors from Inception-v3 follow a multivariate Gaussian distribution and calculates the 2-Wasserstein distance based on their means and covariances. While FID effectively measures how closely synthetic data match real data in many image synthesis tasks, the primary goal in biomedical generative models is often to enrich training datasets ideally with corresponding annotations. For this purpose, the gold standard for evaluating generative models is to incorporate synthetic data into downstream task training, such as classification and segmentation, to pragmatically assess its performance. In this paper, we examine cases from retinal imaging modalities, including color fundus photography and optical coherence tomography, where FID and its related metrics misalign with task-specific evaluation goals in classification and segmentation. We highlight the limitations of using various metrics, represented by FID and its variants, as evaluation criteria for these applications and address their potential caveats in broader biomedical imaging modalities and downstream tasks. △ Less

Submitted 26 February, 2025; v1 submitted 24 February, 2025; originally announced February 2025.

Unsupervised Learning for Feature Extraction and Temporal Alignment of 3D+t Point Clouds of Zebrafish Embryos

Authors: Zhu Chen, Ina Laube, Johannes Stegmaier

Abstract: Zebrafish are widely used in biomedical research and developmental stages of their embryos often need to be synchronized for further analysis. We present an unsupervised approach to extract descriptive features from 3D+t point clouds of zebrafish embryos and subsequently use those features to temporally align corresponding developmental stages. An autoencoder architecture is proposed to learn a de… ▽ More Zebrafish are widely used in biomedical research and developmental stages of their embryos often need to be synchronized for further analysis. We present an unsupervised approach to extract descriptive features from 3D+t point clouds of zebrafish embryos and subsequently use those features to temporally align corresponding developmental stages. An autoencoder architecture is proposed to learn a descriptive representation of the point clouds and we designed a deep regression network for their temporal alignment. We achieve a high alignment accuracy with an average mismatch of only 3.83 minutes over an experimental duration of 5.3 hours. As a fully-unsupervised approach, there is no manual labeling effort required and unlike manual analyses the method easily scales. Besides, the alignment without human annotation of the data also avoids any influence caused by subjective bias. △ Less

Submitted 10 February, 2025; originally announced February 2025.

Evaluating Cross-Subject and Cross-Device Consistency in Visual Fixation Prediction

Authors: Yuli Wu, Henning Konermann, Emil Mededovic, Peter Walter, Johannes Stegmaier

Abstract: Understanding cross-subject and cross-device consistency in visual fixation prediction is essential for advancing eye-tracking applications, including visual attention modeling and neuroprosthetics. This study evaluates fixation consistency using an embedded eye tracker integrated into regular-sized glasses, comparing its performance with high-end standalone eye-tracking systems. Nine participants… ▽ More Understanding cross-subject and cross-device consistency in visual fixation prediction is essential for advancing eye-tracking applications, including visual attention modeling and neuroprosthetics. This study evaluates fixation consistency using an embedded eye tracker integrated into regular-sized glasses, comparing its performance with high-end standalone eye-tracking systems. Nine participants viewed 300 images from the MIT1003 dataset in subjective experiments, allowing us to analyze cross-device and cross-subject variations in fixation patterns with various evaluation metrics. Our findings indicate that average visual fixations can be reliably transferred across devices for relatively simple stimuli. However, individual-to-average consistency remains weak, highlighting the challenges of predicting individual fixations across devices. These results provide an empirical foundation for leveraging predicted average visual fixation data to enhance neuroprosthetic applications. △ Less

Submitted 8 February, 2025; originally announced February 2025.

No Free Lunch in Annotation either: An objective evaluation of foundation models for streamlining annotation in animal tracking

Authors: Emil Mededovic, Valdy Laurentius, Yuli Wu, Marcin Kopaczka, Zhu Chen, Mareike Schulz, René Tolba, Johannes Stegmaier

Abstract: We analyze the capabilities of foundation models addressing the tedious task of generating annotations for animal tracking. Annotating a large amount of data is vital and can be a make-or-break factor for the robustness of a tracking model. Robustness is particularly crucial in animal tracking, as accurate tracking over long time horizons is essential for capturing the behavior of animals. However… ▽ More We analyze the capabilities of foundation models addressing the tedious task of generating annotations for animal tracking. Annotating a large amount of data is vital and can be a make-or-break factor for the robustness of a tracking model. Robustness is particularly crucial in animal tracking, as accurate tracking over long time horizons is essential for capturing the behavior of animals. However, generating additional annotations using foundation models can be counterproductive, as the quality of the annotations is just as important. Poorly annotated data can introduce noise and inaccuracies, ultimately compromising the performance and accuracy of the trained model. Over-reliance on automated annotations without ensuring precision can lead to diminished results, making careful oversight and quality control essential in the annotation process. Ultimately, we demonstrate that a thoughtful combination of automated annotations and manually annotated data is a valuable strategy, yielding an IDF1 score of 80.8 against blind usage of SAM2 video with an IDF1 score of 65.6. △ Less

Submitted 6 February, 2025; originally announced February 2025.

Comments: \c{opyright} 2025 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

LogicAD: Explainable Anomaly Detection via VLM-based Text Feature Extraction

Authors: Er Jin, Qihui Feng, Yongli Mou, Stefan Decker, Gerhard Lakemeyer, Oliver Simons, Johannes Stegmaier

Abstract: Logical image understanding involves interpreting and reasoning about the relationships and consistency within an image's visual content. This capability is essential in applications such as industrial inspection, where logical anomaly detection is critical for maintaining high-quality standards and minimizing costly recalls. Previous research in anomaly detection (AD) has relied on prior knowledg… ▽ More Logical image understanding involves interpreting and reasoning about the relationships and consistency within an image's visual content. This capability is essential in applications such as industrial inspection, where logical anomaly detection is critical for maintaining high-quality standards and minimizing costly recalls. Previous research in anomaly detection (AD) has relied on prior knowledge for designing algorithms, which often requires extensive manual annotations, significant computing power, and large amounts of data for training. Autoregressive, multimodal Vision Language Models (AVLMs) offer a promising alternative due to their exceptional performance in visual reasoning across various domains. Despite this, their application to logical AD remains unexplored. In this work, we investigate using AVLMs for logical AD and demonstrate that they are well-suited to the task. Combining AVLMs with format embedding and a logic reasoner, we achieve SOTA performance on public benchmarks, MVTec LOCO AD, with an AUROC of 86.0% and F1-max of 83.7%, along with explanations of anomalies. This significantly outperforms the existing SOTA method by a large margin. △ Less

Submitted 8 January, 2025; v1 submitted 3 January, 2025; originally announced January 2025.

Comments: Accepted for publication at aaai25, project page: https://jasonjin34.github.io/logicad.github.io/

Effortless Efficiency: Low-Cost Pruning of Diffusion Models

Authors: Yang Zhang, Er Jin, Yanfei Dong, Ashkan Khakzar, Philip Torr, Johannes Stegmaier, Kenji Kawaguchi

Abstract: Diffusion models have achieved impressive advancements in various vision tasks. However, these gains often rely on increasing model size, which escalates computational complexity and memory demands, complicating deployment, raising inference costs, and causing environmental impact. While some studies have explored pruning techniques to improve the memory efficiency of diffusion models, most existi… ▽ More Diffusion models have achieved impressive advancements in various vision tasks. However, these gains often rely on increasing model size, which escalates computational complexity and memory demands, complicating deployment, raising inference costs, and causing environmental impact. While some studies have explored pruning techniques to improve the memory efficiency of diffusion models, most existing methods require extensive retraining to retain the model performance. Retraining a modern large diffusion model is extremely costly and resource-intensive, which limits the practicality of these methods. In this work, we achieve low-cost diffusion pruning without retraining by proposing a model-agnostic structural pruning framework for diffusion models that learns a differentiable mask to sparsify the model. To ensure effective pruning that preserves the quality of the final denoised latent, we design a novel end-to-end pruning objective that spans the entire diffusion process. As end-to-end pruning is memory-intensive, we further propose time step gradient checkpointing, a technique that significantly reduces memory usage during optimization, enabling end-to-end pruning within a limited memory budget. Results on state-of-the-art U-Net diffusion models SDXL and diffusion transformers (FLUX) demonstrate that our method can effectively prune up to 20% parameters with minimal perceptible performance degradation, and notably, without the need for model retraining. We also showcase that our method can still prune on top of time step distilled diffusion models. △ Less

Submitted 3 December, 2024; originally announced December 2024.

Comments: Project page: https://yangzhang-v5.github.io/EcoDiff

Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation

Authors: Rüveyda Yilmaz, Kaan Keven, Yuli Wu, Johannes Stegmaier

Abstract: Automated cell segmentation in microscopy images is essential for biomedical research, yet conventional methods are labor-intensive and prone to error. While deep learning-based approaches have proven effective, they often require large annotated datasets, which are scarce due to the challenges of manual annotation. To overcome this, we propose a novel framework for synthesizing densely annotated… ▽ More Automated cell segmentation in microscopy images is essential for biomedical research, yet conventional methods are labor-intensive and prone to error. While deep learning-based approaches have proven effective, they often require large annotated datasets, which are scarce due to the challenges of manual annotation. To overcome this, we propose a novel framework for synthesizing densely annotated 2D and 3D cell microscopy images using cascaded diffusion models. Our method synthesizes 2D and 3D cell masks from sparse 2D annotations using multi-level diffusion models and NeuS, a 3D surface reconstruction approach. Following that, a pretrained 2D Stable Diffusion model is finetuned to generate realistic cell textures and the final outputs are combined to form cell populations. We show that training a segmentation model with a combination of our synthetic data and real data improves cell segmentation performance by up to 9\% across multiple datasets. Additionally, the FID scores indicate that the synthetic data closely resembles real data. The code for our proposed approach will be available at https://github.com/ruveydayilmaz0/cascaded_diffusion. △ Less

Submitted 19 November, 2024; v1 submitted 18 November, 2024; originally announced November 2024.

Visual Fixation-Based Retinal Prosthetic Simulation

Authors: Yuli Wu, Do Dinh Tan Nguyen, Henning Konermann, Rüveyda Yilmaz, Peter Walter, Johannes Stegmaier

Abstract: This study proposes a retinal prosthetic simulation framework driven by visual fixations, inspired by the saccade mechanism, and assesses performance improvements through end-to-end optimization in a classification task. Salient patches are predicted from input images using the self-attention map of a vision transformer to mimic visual fixations. These patches are then encoded by a trainable U-Net… ▽ More This study proposes a retinal prosthetic simulation framework driven by visual fixations, inspired by the saccade mechanism, and assesses performance improvements through end-to-end optimization in a classification task. Salient patches are predicted from input images using the self-attention map of a vision transformer to mimic visual fixations. These patches are then encoded by a trainable U-Net and simulated using the pulse2percept framework to predict visual percepts. By incorporating a learnable encoder, we aim to optimize the visual information transmitted to the retinal implant, addressing both the limited resolution of the electrode array and the distortion between the input stimuli and resulting phosphenes. The predicted percepts are evaluated using the self-supervised DINOv2 foundation model, with an optional learnable linear layer for classification accuracy. On a subset of the ImageNet validation set, the fixation-based framework achieves a classification accuracy of 87.72%, using computational parameters based on a real subject's physiological data, significantly outperforming the downsampling-based accuracy of 40.59% and approaching the healthy upper bound of 92.76%. Our approach shows promising potential for producing more semantically understandable percepts with the limited resolution available in retinal prosthetics. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Annotated Biomedical Video Generation using Denoising Diffusion Probabilistic Models and Flow Fields

Authors: Rüveyda Yilmaz, Dennis Eschweiler, Johannes Stegmaier

Abstract: The segmentation and tracking of living cells play a vital role within the biomedical domain, particularly in cancer research, drug development, and developmental biology. These are usually tedious and time-consuming tasks that are traditionally done by biomedical experts. Recently, to automatize these processes, deep learning based segmentation and tracking methods have been proposed. These metho… ▽ More The segmentation and tracking of living cells play a vital role within the biomedical domain, particularly in cancer research, drug development, and developmental biology. These are usually tedious and time-consuming tasks that are traditionally done by biomedical experts. Recently, to automatize these processes, deep learning based segmentation and tracking methods have been proposed. These methods require large-scale datasets and their full potential is constrained by the scarcity of annotated data in the biomedical imaging domain. To address this limitation, we propose Biomedical Video Diffusion Model (BVDM), capable of generating realistic-looking synthetic microscopy videos. Trained only on a single real video, BVDM can generate videos of arbitrary length with pixel-level annotations that can be used for training data-hungry models. It is composed of a denoising diffusion probabilistic model (DDPM) generating high-fidelity synthetic cell microscopy images and a flow prediction model (FPM) predicting the non-rigid transformation between consecutive video frames. During inference, initially, the DDPM imposes realistic cell textures on synthetic cell masks which are generated based on real data statistics. The flow prediction model predicts the flow field between consecutive masks and applies that to the DDPM output from the previous time frame to create the next one while keeping temporal consistency. BVDM outperforms state-of-the-art synthetic live cell microscopy video generation models. Furthermore, we demonstrate that a sufficiently large synthetic dataset enhances the performance of cell segmentation and tracking models compared to using a limited amount of available real data. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Optimizing Retinal Prosthetic Stimuli with Conditional Invertible Neural Networks

Authors: Yuli Wu, Julian Wittmann, Peter Walter, Johannes Stegmaier

Abstract: Implantable retinal prostheses offer a promising solution to restore partial vision by circumventing damaged photoreceptor cells in the retina and directly stimulating the remaining functional retinal cells. However, the information transmission between the camera and retinal cells is often limited by the low resolution of the electrode array and the lack of specificity for different ganglion cell… ▽ More Implantable retinal prostheses offer a promising solution to restore partial vision by circumventing damaged photoreceptor cells in the retina and directly stimulating the remaining functional retinal cells. However, the information transmission between the camera and retinal cells is often limited by the low resolution of the electrode array and the lack of specificity for different ganglion cell types, resulting in suboptimal stimulations. In this work, we propose to utilize normalizing flow-based conditional invertible neural networks to optimize retinal implant stimulation in an unsupervised manner. The invertibility of these networks allows us to use them as a surrogate for the computational model of the visual system, while also encoding input camera signals into optimized electrical stimuli on the electrode array. Compared to other methods, such as trivial downsampling, linear models, and feed-forward convolutional neural networks, the flow-based invertible neural network and its conditional extension yield better visual reconstruction qualities w.r.t. various metrics using a physiologically validated simulation tool. △ Less

Submitted 15 July, 2024; v1 submitted 7 March, 2024; originally announced March 2024.

Enhancing Lidar-based Object Detection in Adverse Weather using Offset Sequences in Time

Authors: Raphael van Kempen, Tim Rehbronn, Abin Jose, Johannes Stegmaier, Bastian Lampe, Timo Woopen, Lutz Eckstein

Abstract: Automated vehicles require an accurate perception of their surroundings for safe and efficient driving. Lidar-based object detection is a widely used method for environment perception, but its performance is significantly affected by adverse weather conditions such as rain and fog. In this work, we investigate various strategies for enhancing the robustness of lidar-based object detection by proce… ▽ More Automated vehicles require an accurate perception of their surroundings for safe and efficient driving. Lidar-based object detection is a widely used method for environment perception, but its performance is significantly affected by adverse weather conditions such as rain and fog. In this work, we investigate various strategies for enhancing the robustness of lidar-based object detection by processing sequential data samples generated by lidar sensors. Our approaches leverage temporal information to improve a lidar object detection model, without the need for additional filtering or pre-processing steps. We compare $10$ different neural network architectures that process point cloud sequences including a novel augmentation strategy introducing a temporal offset between frames of a sequence during training and evaluate the effectiveness of all strategies on lidar point clouds under adverse weather conditions through experiments. Our research provides a comprehensive study of effective methods for mitigating the effects of adverse weather on the reliability of lidar-based object detection using sequential data that are evaluated using public datasets such as nuScenes, Dense, and the Canadian Adverse Driving Conditions Dataset. Our findings demonstrate that our novel method, involving temporal offset augmentation through randomized frame skipping in sequences, enhances object detection accuracy compared to both the baseline model (Pillar-based Object Detection) and no augmentation. △ Less

Submitted 17 January, 2024; originally announced January 2024.

Comments: Published as part of the III. International Conference on Electrical, Computer and Energy Technologies (ICECET 2023), Cape Town, South Africa, November 16-17, 2023

Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentation

Authors: Yuli Wu, Weidong He, Dennis Eschweiler, Ningxin Dou, Zixin Fan, Shengli Mi, Peter Walter, Johannes Stegmaier

Abstract: Modern biomedical image analysis using deep learning often encounters the challenge of limited annotated data. To overcome this issue, deep generative models can be employed to synthesize realistic biomedical images. In this regard, we propose an image synthesis method that utilizes denoising diffusion probabilistic models (DDPMs) to automatically generate retinal optical coherence tomography (OCT… ▽ More Modern biomedical image analysis using deep learning often encounters the challenge of limited annotated data. To overcome this issue, deep generative models can be employed to synthesize realistic biomedical images. In this regard, we propose an image synthesis method that utilizes denoising diffusion probabilistic models (DDPMs) to automatically generate retinal optical coherence tomography (OCT) images. By providing rough layer sketches, the trained DDPMs can generate realistic circumpapillary OCT images. We further find that more accurate pseudo labels can be obtained through knowledge adaptation, which greatly benefits the segmentation task. Through this, we observe a consistent improvement in layer segmentation accuracy, which is validated using various neural networks. Furthermore, we have discovered that a layer segmentation model trained solely with synthesized images can achieve comparable results to a model trained exclusively with real images. These findings demonstrate the promising potential of DDPMs in reducing the need for manual annotations of retinal OCT images. △ Less

Submitted 6 March, 2024; v1 submitted 9 November, 2023; originally announced November 2023.

Comments: ISBI 2024

SortedAP: Rethinking evaluation metrics for instance segmentation

Authors: Long Chen, Yuli Wu, Johannes Stegmaier, Dorit Merhof

Abstract: Designing metrics for evaluating instance segmentation revolves around comprehensively considering object detection and segmentation accuracy. However, other important properties, such as sensitivity, continuity, and equality, are overlooked in the current study. In this paper, we reveal that most existing metrics have a limited resolution of segmentation quality. They are only conditionally sensi… ▽ More Designing metrics for evaluating instance segmentation revolves around comprehensively considering object detection and segmentation accuracy. However, other important properties, such as sensitivity, continuity, and equality, are overlooked in the current study. In this paper, we reveal that most existing metrics have a limited resolution of segmentation quality. They are only conditionally sensitive to the change of masks or false predictions. For certain metrics, the score can change drastically in a narrow range which could provide a misleading indication of the quality gap between results. Therefore, we propose a new metric called sortedAP, which strictly decreases with both object- and pixel-level imperfections and has an uninterrupted penalization scale over the entire domain. We provide the evaluation toolkit and experiment code at https://www.github.com/looooongChen/sortedAP. △ Less

Submitted 9 September, 2023; originally announced September 2023.

Transformers for CT Reconstruction From Monoplanar and Biplanar Radiographs

Authors: Firas Khader, Gustav Müller-Franzes, Tianyu Han, Sven Nebelung, Christiane Kuhl, Johannes Stegmaier, Daniel Truhn

Abstract: Computed Tomography (CT) scans provide detailed and accurate information of internal structures in the body. They are constructed by sending x-rays through the body from different directions and combining this information into a three-dimensional volume. Such volumes can then be used to diagnose a wide range of conditions and allow for volumetric measurements of organs. In this work, we tackle the… ▽ More Computed Tomography (CT) scans provide detailed and accurate information of internal structures in the body. They are constructed by sending x-rays through the body from different directions and combining this information into a three-dimensional volume. Such volumes can then be used to diagnose a wide range of conditions and allow for volumetric measurements of organs. In this work, we tackle the problem of reconstructing CT images from biplanar x-rays only. X-rays are widely available and even if the CT reconstructed from these radiographs is not a replacement of a complete CT in the diagnostic setting, it might serve to spare the patients from radiation where a CT is only acquired for rough measurements such as determining organ size. We propose a novel method based on the transformer architecture, by framing the underlying task as a language translation problem. Radiographs and CT images are first embedded into latent quantized codebook vectors using two different autoencoder networks. We then train a GPT model, to reconstruct the codebook vectors of the CT image, conditioned on the codebook vectors of the x-rays and show that this approach leads to realistic looking images. To encourage further research in this direction, we make our code publicly available on GitHub: XXX. △ Less

Submitted 11 May, 2023; originally announced May 2023.

Cascaded Cross-Attention Networks for Data-Efficient Whole-Slide Image Classification Using Transformers

Authors: Firas Khader, Jakob Nikolas Kather, Tianyu Han, Sven Nebelung, Christiane Kuhl, Johannes Stegmaier, Daniel Truhn

Abstract: Whole-Slide Imaging allows for the capturing and digitization of high-resolution images of histological specimen. An automated analysis of such images using deep learning models is therefore of high demand. The transformer architecture has been proposed as a possible candidate for effectively leveraging the high-resolution information. Here, the whole-slide image is partitioned into smaller image… ▽ More Whole-Slide Imaging allows for the capturing and digitization of high-resolution images of histological specimen. An automated analysis of such images using deep learning models is therefore of high demand. The transformer architecture has been proposed as a possible candidate for effectively leveraging the high-resolution information. Here, the whole-slide image is partitioned into smaller image patches and feature tokens are extracted from these image patches. However, while the conventional transformer allows for a simultaneous processing of a large set of input tokens, the computational demand scales quadratically with the number of input tokens and thus quadratically with the number of image patches. To address this problem we propose a novel cascaded cross-attention network (CCAN) based on the cross-attention mechanism that scales linearly with the number of extracted patches. Our experiments demonstrate that this architecture is at least on-par with and even outperforms other attention-based state-of-the-art methods on two public datasets: On the use-case of lung cancer (TCGA NSCLC) our model reaches a mean area under the receiver operating characteristic (AUC) of 0.970 $\pm$ 0.008 and on renal cancer (TCGA RCC) reaches a mean AUC of 0.985 $\pm$ 0.004. Furthermore, we show that our proposed model is efficient in low-data regimes, making it a promising approach for analyzing whole-slide images in resource-limited settings. To foster research in this direction, we make our code publicly available on GitHub: XXX. △ Less

Submitted 11 May, 2023; originally announced May 2023.

A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation

Authors: Yuli Wu, Ivan Karetic, Johannes Stegmaier, Peter Walter, Dorit Merhof

Abstract: We propose a neural network-based framework to optimize the perceptions simulated by the in silico retinal implant model pulse2percept. The overall pipeline consists of a trainable encoder, a pre-trained retinal implant model and a pre-trained evaluator. The encoder is a U-Net, which takes the original image and outputs the stimulus. The pre-trained retinal implant model is also a U-Net, which is… ▽ More We propose a neural network-based framework to optimize the perceptions simulated by the in silico retinal implant model pulse2percept. The overall pipeline consists of a trainable encoder, a pre-trained retinal implant model and a pre-trained evaluator. The encoder is a U-Net, which takes the original image and outputs the stimulus. The pre-trained retinal implant model is also a U-Net, which is trained to mimic the biomimetic perceptual model implemented in pulse2percept. The evaluator is a shallow VGG classifier, which is trained with original images. Based on 10,000 test images from the MNIST dataset, we show that the convolutional neural network-based encoder performs significantly better than the trivial downsampling approach, yielding a boost in the weighted F1-Score by 36.17% in the pre-trained classifier with 6x10 electrodes. With this fully neural network-based encoder, the quality of the downstream perceptions can be fine-tuned using gradient descent in an end-to-end fashion. △ Less

Submitted 7 February, 2023; originally announced February 2023.

Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets

Authors: Dennis Eschweiler, Rüveyda Yilmaz, Matisse Baumann, Ina Laube, Rijo Roy, Abin Jose, Daniel Brückner, Johannes Stegmaier

Abstract: Recent advances in computer vision have led to significant progress in the generation of realistic image data, with denoising diffusion probabilistic models proving to be a particularly effective method. In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets through an unsupervised and intuitive approach, using rough sketches of desi… ▽ More Recent advances in computer vision have led to significant progress in the generation of realistic image data, with denoising diffusion probabilistic models proving to be a particularly effective method. In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets through an unsupervised and intuitive approach, using rough sketches of desired structures as the starting point. The proposed pipeline helps to reduce the reliance on manual annotations when training deep learning-based segmentation approaches and enables the segmentation of diverse datasets without the need for human annotations. This approach holds great promise in streamlining the data generation process and enabling a more efficient and scalable training of segmentation models, as we show in the example of different practical experiments involving various organisms and cell types. △ Less

Submitted 8 August, 2023; v1 submitted 2 January, 2023; originally announced January 2023.

Comments: 9 pages, 2 figures

Medical Diagnosis with Large Scale Multimodal Transformers: Leveraging Diverse Data for More Accurate Diagnosis

Authors: Firas Khader, Gustav Mueller-Franzes, Tianci Wang, Tianyu Han, Soroosh Tayebi Arasteh, Christoph Haarburger, Johannes Stegmaier, Keno Bressem, Christiane Kuhl, Sven Nebelung, Jakob Nikolas Kather, Daniel Truhn

Abstract: Multimodal deep learning has been used to predict clinical endpoints and diagnoses from clinical routine data. However, these models suffer from scaling issues: they have to learn pairwise interactions between each piece of information in each data type, thereby escalating model complexity beyond manageable scales. This has so far precluded a widespread use of multimodal deep learning. Here, we pr… ▽ More Multimodal deep learning has been used to predict clinical endpoints and diagnoses from clinical routine data. However, these models suffer from scaling issues: they have to learn pairwise interactions between each piece of information in each data type, thereby escalating model complexity beyond manageable scales. This has so far precluded a widespread use of multimodal deep learning. Here, we present a new technical approach of "learnable synergies", in which the model only selects relevant interactions between data modalities and keeps an "internal memory" of relevant data. Our approach is easily scalable and naturally adapts to multimodal data inputs from clinical routine. We demonstrate this approach on three large multimodal datasets from radiology and ophthalmology and show that it outperforms state-of-the-art models in clinically relevant diagnosis tasks. Our new approach is transferable and will allow the application of multimodal deep learning to a broad set of clinically relevant problems. △ Less

Submitted 20 December, 2022; v1 submitted 18 December, 2022; originally announced December 2022.

Medical Diffusion: Denoising Diffusion Probabilistic Models for 3D Medical Image Generation

Authors: Firas Khader, Gustav Mueller-Franzes, Soroosh Tayebi Arasteh, Tianyu Han, Christoph Haarburger, Maximilian Schulze-Hagen, Philipp Schad, Sandy Engelhardt, Bettina Baessler, Sebastian Foersch, Johannes Stegmaier, Christiane Kuhl, Sven Nebelung, Jakob Nikolas Kather, Daniel Truhn

Abstract: Recent advances in computer vision have shown promising results in image generation. Diffusion probabilistic models in particular have generated realistic images from textual input, as demonstrated by DALL-E 2, Imagen and Stable Diffusion. However, their use in medicine, where image data typically comprises three-dimensional volumes, has not been systematically evaluated. Synthetic images may play… ▽ More Recent advances in computer vision have shown promising results in image generation. Diffusion probabilistic models in particular have generated realistic images from textual input, as demonstrated by DALL-E 2, Imagen and Stable Diffusion. However, their use in medicine, where image data typically comprises three-dimensional volumes, has not been systematically evaluated. Synthetic images may play a crucial role in privacy preserving artificial intelligence and can also be used to augment small datasets. Here we show that diffusion probabilistic models can synthesize high quality medical imaging data, which we show for Magnetic Resonance Images (MRI) and Computed Tomography (CT) images. We provide quantitative measurements of their performance through a reader study with two medical experts who rated the quality of the synthesized images in three categories: Realistic image appearance, anatomical correctness and consistency between slices. Furthermore, we demonstrate that synthetic images can be used in a self-supervised pre-training and improve the performance of breast segmentation models when data is scarce (dice score 0.91 vs. 0.95 without vs. with synthetic data). The code is publicly available on GitHub: https://github.com/FirasGit/medicaldiffusion. △ Less

Submitted 3 January, 2023; v1 submitted 7 November, 2022; originally announced November 2022.

Semi- and Self-Supervised Multi-View Fusion of 3D Microscopy Images using Generative Adversarial Networks

Authors: Canyu Yang, Dennis Eschweiler, Johannes Stegmaier

Abstract: Recent developments in fluorescence microscopy allow capturing high-resolution 3D images over time for living model organisms. To be able to image even large specimens, techniques like multi-view light-sheet imaging record different orientations at each time point that can then be fused into a single high-quality volume. Based on measured point spread functions (PSF), deconvolution and content fus… ▽ More Recent developments in fluorescence microscopy allow capturing high-resolution 3D images over time for living model organisms. To be able to image even large specimens, techniques like multi-view light-sheet imaging record different orientations at each time point that can then be fused into a single high-quality volume. Based on measured point spread functions (PSF), deconvolution and content fusion are able to largely revert the inevitable degradation occurring during the imaging process. Classical multi-view deconvolution and fusion methods mainly use iterative procedures and content-based averaging. Lately, Convolutional Neural Networks (CNNs) have been deployed to approach 3D single-view deconvolution microscopy, but the multi-view case waits to be studied. We investigated the efficacy of CNN-based multi-view deconvolution and fusion with two synthetic data sets that mimic developing embryos and involve either two or four complementary 3D views. Compared with classical state-of-the-art methods, the proposed semi- and self-supervised models achieve competitive and superior deconvolution and fusion quality in the two-view and quad-view cases, respectively. △ Less

Submitted 5 August, 2021; originally announced August 2021.

Comments: 11 pages, 3 figures, 1 table, accepted for publication at MLMIR 2021

3D fluorescence microscopy data synthesis for segmentation and benchmarking

Authors: Dennis Eschweiler, Malte Rethwisch, Mareike Jarchow, Simon Koppers, Johannes Stegmaier

Abstract: Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general la… ▽ More Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotation masks of 3D cellular structures. In combination with mask simulation approaches, we demonstrate the generation of fully-annotated 3D microscopy data sets that we make publicly available for training or benchmarking. An additional positional conditioning of the cellular structures enables the reconstruction of position-dependent intensity characteristics and allows to generate image data of different quality levels. A patch-wise working principle and a subsequent full-size reassemble strategy is used to generate image data of arbitrary size and different organisms. We present this as a proof-of-concept for the automated generation of fully-annotated training data sets requiring only a minimum of manual interaction to alleviate the need of manual annotations. △ Less

Submitted 21 July, 2021; originally announced July 2021.

Robust 3D Cell Segmentation: Extending the View of Cellpose

Authors: Dennis Eschweiler, Richard S. Smith, Johannes Stegmaier

Abstract: Increasing data set sizes of 3D microscopy imaging experiments demand for an automation of segmentation processes to be able to extract meaningful biomedical information. Due to the shortage of annotated 3D image data that can be used for machine learning-based approaches, 3D segmentation approaches are required to be robust and to generalize well to unseen data. The Cellpose approach proposed by… ▽ More Increasing data set sizes of 3D microscopy imaging experiments demand for an automation of segmentation processes to be able to extract meaningful biomedical information. Due to the shortage of annotated 3D image data that can be used for machine learning-based approaches, 3D segmentation approaches are required to be robust and to generalize well to unseen data. The Cellpose approach proposed by Stringer et al. proved to be such a generalist approach for cell instance segmentation tasks. In this paper, we extend the Cellpose approach to improve segmentation accuracy on 3D image data and we further show how the formulation of the gradient maps can be simplified while still being robust and reaching similar segmentation accuracy. The code is publicly available and was integrated into two established open-source applications that allow using the 3D extension of Cellpose without any programming knowledge. △ Less

Submitted 1 February, 2022; v1 submitted 3 May, 2021; originally announced May 2021.

Spherical Harmonics for Shape-Constrained 3D Cell Segmentation

Authors: Dennis Eschweiler, Malte Rethwisch, Simon Koppers, Johannes Stegmaier

Abstract: Recent microscopy imaging techniques allow to precisely analyze cell morphology in 3D image data. To process the vast amount of image data generated by current digitized imaging techniques, automated approaches are demanded more than ever. Segmentation approaches used for morphological analyses, however, are often prone to produce unnaturally shaped predictions, which in conclusion could lead to i… ▽ More Recent microscopy imaging techniques allow to precisely analyze cell morphology in 3D image data. To process the vast amount of image data generated by current digitized imaging techniques, automated approaches are demanded more than ever. Segmentation approaches used for morphological analyses, however, are often prone to produce unnaturally shaped predictions, which in conclusion could lead to inaccurate experimental outcomes. In order to minimize further manual interaction, shape priors help to constrain the predictions to the set of natural variations. In this paper, we show how spherical harmonics can be used as an alternative way to inherently constrain the predictions of neural networks for the segmentation of cells in 3D microscopy image data. Benefits and limitations of the spherical harmonic representation are analyzed and final results are compared to other state-of-the-art approaches on two different data sets. △ Less

Submitted 23 October, 2020; originally announced October 2020.

CellCycleGAN: Spatiotemporal Microscopy Image Synthesis of Cell Populations using Statistical Shape Models and Conditional GANs

Authors: Dennis Bähr, Dennis Eschweiler, Anuk Bhattacharyya, Daniel Moreno-Andrés, Wolfram Antonin, Johannes Stegmaier

Abstract: Automatic analysis of spatio-temporal microscopy images is inevitable for state-of-the-art research in the life sciences. Recent developments in deep learning provide powerful tools for automatic analyses of such image data, but heavily depend on the amount and quality of provided training data to perform well. To this end, we developed a new method for realistic generation of synthetic 2D+t micro… ▽ More Automatic analysis of spatio-temporal microscopy images is inevitable for state-of-the-art research in the life sciences. Recent developments in deep learning provide powerful tools for automatic analyses of such image data, but heavily depend on the amount and quality of provided training data to perform well. To this end, we developed a new method for realistic generation of synthetic 2D+t microscopy image data of fluorescently labeled cellular nuclei. The method combines spatiotemporal statistical shape models of different cell cycle stages with a conditional GAN to generate time series of cell populations and provides instance-level control of cell cycle stage and the fluorescence intensity of generated cells. We show the effect of the GAN conditioning and create a set of synthetic images that can be readily used for training and benchmarking of cell segmentation and tracking approaches. △ Less

Submitted 26 January, 2021; v1 submitted 22 October, 2020; originally announced October 2020.

Comments: 5 pages, 3 figures

Making Logic Learnable With Neural Networks

Authors: Tobias Brudermueller, Dennis L. Shung, Adrian J. Stanley, Johannes Stegmaier, Smita Krishnaswamy

Abstract: While neural networks are good at learning unspecified functions from training samples, they cannot be directly implemented in hardware and are often not interpretable or formally verifiable. On the other hand, logic circuits are implementable, verifiable, and interpretable but are not able to learn from training data in a generalizable way. We propose a novel logic learning pipeline that combines… ▽ More While neural networks are good at learning unspecified functions from training samples, they cannot be directly implemented in hardware and are often not interpretable or formally verifiable. On the other hand, logic circuits are implementable, verifiable, and interpretable but are not able to learn from training data in a generalizable way. We propose a novel logic learning pipeline that combines the advantages of neural networks and logic circuits. Our pipeline first trains a neural network on a classification task, and then translates this, first to random forests, and then to AND-Inverter logic. We show that our pipeline maintains greater accuracy than naive translations to logic, and minimizes the logic such that it is more interpretable and has decreased hardware cost. We show the utility of our pipeline on a network that is trained on biomedical data. This approach could be applied to patient care to provide risk stratification and guide clinical decision-making. △ Less

Submitted 7 June, 2020; v1 submitted 10 February, 2020; originally announced February 2020.

Semi-Automatic Generation of Tight Binary Masks and Non-Convex Isosurfaces for Quantitative Analysis of 3D Biological Samples

Authors: Sourabh Bhide, Ralf Mikut, Maria Leptin, Johannes Stegmaier

Abstract: Current in vivo microscopy allows us detailed spatiotemporal imaging (3D+t) of complete organisms and offers insights into their development on the cellular level. Even though the imaging speed and quality is steadily improving, fully-automated segmentation and analysis methods are often not accurate enough. This is particularly true while imaging large samples (100um - 1mm) and deep inside the sp… ▽ More Current in vivo microscopy allows us detailed spatiotemporal imaging (3D+t) of complete organisms and offers insights into their development on the cellular level. Even though the imaging speed and quality is steadily improving, fully-automated segmentation and analysis methods are often not accurate enough. This is particularly true while imaging large samples (100um - 1mm) and deep inside the specimen. Drosophila embryogenesis, widely used as a developmental paradigm, presents an example for such a challenge, especially where cell outlines need to imaged - a general challenge in other systems as well. To deal with the current bottleneck in analyzing quantitatively the 3D+t light-sheet microscopy images of Drosophila embryos, we developed a collection of semi-automatic open-source tools. The presented methods include a semi-automatic masking procedure, automatic projection of non-convex 3D isosurfaces to 2D representations as well as cell segmentation and tracking. △ Less

Submitted 30 January, 2020; originally announced January 2020.

Comments: 5 pages, 2 figures

Towards Automatic Embryo Staging in 3D+T Microscopy Images using Convolutional Neural Networks and PointNets

Authors: Manuel Traub, Johannes Stegmaier

Abstract: Automatic analyses and comparisons of different stages of embryonic development largely depend on a highly accurate spatiotemporal alignment of the investigated data sets. In this contribution, we assess multiple approaches for automatic staging of developing embryos that were imaged with time-resolved 3D light-sheet microscopy. The methods comprise image-based convolutional neural networks as wel… ▽ More Automatic analyses and comparisons of different stages of embryonic development largely depend on a highly accurate spatiotemporal alignment of the investigated data sets. In this contribution, we assess multiple approaches for automatic staging of developing embryos that were imaged with time-resolved 3D light-sheet microscopy. The methods comprise image-based convolutional neural networks as well as an approach based on the PointNet architecture that directly operates on 3D point clouds of detected cell nuclei centroids. The experiments with four wild-type zebrafish embryos render both approaches suitable for automatic staging with average deviations of 21 - 34 minutes. Moreover, a proof-of-concept evaluation based on simulated 3D+t point cloud data sets shows that average deviations of less than 7 minutes are possible. △ Less

Submitted 29 July, 2020; v1 submitted 1 October, 2019; originally announced October 2019.

Comments: 10 pages, 3 figures, 1 table, accepted paper at the Simulation and Synthesis in Medical Imaging (SASHIMI) Workshop held at MICCAI 2020

Algorithms used for the Cell Segmentation Benchmark Competition at ISBI 2019 by RWTH-GE

Authors: Dennis Eschweiler, Johannes Stegmaier

Abstract: The presented algorithms for segmentation and tracking follow a 3-step approach where we detect, track and finally segment nuclei. In the preprocessing phase, we detect centroids of the cell nuclei using a convolutional neural network (CNN) for the 2D images and a Laplacian-of-Gaussian Scale Space Maximum Projection approach for the 3D data sets. Tracking was performed in a backwards fashion on th… ▽ More The presented algorithms for segmentation and tracking follow a 3-step approach where we detect, track and finally segment nuclei. In the preprocessing phase, we detect centroids of the cell nuclei using a convolutional neural network (CNN) for the 2D images and a Laplacian-of-Gaussian Scale Space Maximum Projection approach for the 3D data sets. Tracking was performed in a backwards fashion on the predicted seed points, i.e., starting at the last frame and sequentially connecting corresponding objects until the first frame was reached. Correspondences were identified by propagating detections of a frame t to its preceding frame t-1 and by combining redundant detections using a hierarchical clustering approach. The tracked centroids were then used as input to variants of the seeded watershed algorithm to obtain the final segmentation. △ Less

Submitted 15 April, 2019; originally announced April 2019.

Comments: 4 pages, algorithms used for the Cell Segmentation Benchmark competition at IEEE International Symposium on Biomedical Imaging (ISBI) 2019 in Venice, Italy

CNN-based Preprocessing to Optimize Watershed-based Cell Segmentation in 3D Confocal Microscopy Images

Authors: Dennis Eschweiler, Thiago V. Spina, Rohan C. Choudhury, Elliot Meyerowitz, Alexandre Cunha, Johannes Stegmaier

Abstract: The quantitative analysis of cellular membranes helps understanding developmental processes at the cellular level. Particularly 3D microscopic image data offers valuable insights into cell dynamics, but error-free automatic segmentation remains challenging due to the huge amount of data generated and strong variations in image intensities. In this paper, we propose a new 3D segmentation approach w… ▽ More The quantitative analysis of cellular membranes helps understanding developmental processes at the cellular level. Particularly 3D microscopic image data offers valuable insights into cell dynamics, but error-free automatic segmentation remains challenging due to the huge amount of data generated and strong variations in image intensities. In this paper, we propose a new 3D segmentation approach which combines the discriminative power of convolutional neural networks (CNNs) for preprocessing and investigates the performance of three watershed-based postprocessing strategies (WS), which are well suited to segment object shapes, even when supplied with vague seed and boundary constraints. To leverage the full potential of the watershed algorithm, the multi-instance segmentation problem is initially interpreted as three-class semantic segmentation problem, which in turn is well-suited for the application of CNNs. Using manually annotated 3D confocal microscopy images of Arabidopsis thaliana, we show the superior performance of the proposed method compared to the state of the art. △ Less

Submitted 16 October, 2018; originally announced October 2018.

Comments: 5 pages, 4 figures, 1 table

Transfer Learning with Human Corneal Tissues: An Analysis of Optimal Cut-Off Layer

Authors: Nadezhda Prodanova, Johannes Stegmaier, Stephan Allgeier, Sebastian Bohn, Oliver Stachs, Bernd Köhler, Ralf Mikut, Andreas Bartschat

Abstract: Transfer learning is a powerful tool to adapt trained neural networks to new tasks. Depending on the similarity of the original task to the new task, the selection of the cut-off layer is critical. For medical applications like tissue classification, the last layers of an object classification network might not be optimal. We found that on real data of human corneal tissues the best feature repres… ▽ More Transfer learning is a powerful tool to adapt trained neural networks to new tasks. Depending on the similarity of the original task to the new task, the selection of the cut-off layer is critical. For medical applications like tissue classification, the last layers of an object classification network might not be optimal. We found that on real data of human corneal tissues the best feature representation can be found in the middle layers of the Inception-v3 and in the rear layers of the VGG-19 architecture. △ Less

Submitted 22 June, 2018; v1 submitted 19 June, 2018; originally announced June 2018.

Comments: Extendend Abstract with 3 pages and 2 figures. Submitted to MIDL Amsterdam, see openreviews.org

SEGMENT3D: A Web-based Application for Collaborative Segmentation of 3D images used in the Shoot Apical Meristem

Authors: Thiago V. Spina, Johannes Stegmaier, Alexandre X. Falcão, Elliot Meyerowitz, Alexandre Cunha

Abstract: The quantitative analysis of 3D confocal microscopy images of the shoot apical meristem helps understanding the growth process of some plants. Cell segmentation in these images is crucial for computational plant analysis and many automated methods have been proposed. However, variations in signal intensity across the image mitigate the effectiveness of those approaches with no easy way for user co… ▽ More The quantitative analysis of 3D confocal microscopy images of the shoot apical meristem helps understanding the growth process of some plants. Cell segmentation in these images is crucial for computational plant analysis and many automated methods have been proposed. However, variations in signal intensity across the image mitigate the effectiveness of those approaches with no easy way for user correction. We propose a web-based collaborative 3D image segmentation application, SEGMENT3D, to leverage automatic segmentation results. The image is divided into 3D tiles that can be either segmented interactively from scratch or corrected from a pre-existing segmentation. Individual segmentation results per tile are then automatically merged via consensus analysis and then stitched to complete the segmentation for the entire image stack. SEGMENT3D is a comprehensive application that can be applied to other 3D imaging modalities and general objects. It also provides an easy way to create supervised data to advance segmentation using machine learning models. △ Less

Submitted 26 October, 2017; originally announced October 2017.

Cell Segmentation in 3D Confocal Images using Supervoxel Merge-Forests with CNN-based Hypothesis Selection

Authors: Johannes Stegmaier, Thiago V. Spina, Alexandre X. Falcão, Andreas Bartschat, Ralf Mikut, Elliot Meyerowitz, Alexandre Cunha

Abstract: Automated segmentation approaches are crucial to quantitatively analyze large-scale 3D microscopy images. Particularly in deep tissue regions, automatic methods still fail to provide error-free segmentations. To improve the segmentation quality throughout imaged samples, we present a new supervoxel-based 3D segmentation approach that outperforms current methods and reduces the manual correction ef… ▽ More Automated segmentation approaches are crucial to quantitatively analyze large-scale 3D microscopy images. Particularly in deep tissue regions, automatic methods still fail to provide error-free segmentations. To improve the segmentation quality throughout imaged samples, we present a new supervoxel-based 3D segmentation approach that outperforms current methods and reduces the manual correction effort. The algorithm consists of gentle preprocessing and a conservative super-voxel generation method followed by supervoxel agglomeration based on local signal properties and a postprocessing step to fix under-segmentation errors using a Convolutional Neural Network. We validate the functionality of the algorithm on manually labeled 3D confocal images of the plant Arabidopis thaliana and compare the results to a state-of-the-art meristem segmentation algorithm. △ Less

Submitted 18 October, 2017; originally announced October 2017.

Comments: 5 pages, 3 figures, 1 table

The MATLAB Toolbox SciXMiner: User's Manual and Programmer's Guide

Authors: Ralf Mikut, Andreas Bartschat, Wolfgang Doneit, Jorge Ángel González Ordiano, Benjamin Schott, Johannes Stegmaier, Simon Waczowicz, Markus Reischl

Abstract: The Matlab toolbox SciXMiner is designed for the visualization and analysis of time series and features with a special focus to classification problems. It was developed at the Institute of Applied Computer Science of the Karlsruhe Institute of Technology (KIT), a member of the Helmholtz Association of German Research Centres in Germany. The aim was to provide an open platform for the development… ▽ More The Matlab toolbox SciXMiner is designed for the visualization and analysis of time series and features with a special focus to classification problems. It was developed at the Institute of Applied Computer Science of the Karlsruhe Institute of Technology (KIT), a member of the Helmholtz Association of German Research Centres in Germany. The aim was to provide an open platform for the development and improvement of data mining methods and its applications to various medical and technical problems. SciXMiner bases on Matlab (tested for the version 2017a). Many functions do not require additional standard toolboxes but some parts of Signal, Statistics and Wavelet toolboxes are used for special cases. The decision to a Matlab-based solution was made to use the wide mathematical functionality of this package provided by The Mathworks Inc. SciXMiner is controlled by a graphical user interface (GUI) with menu items and control elements like popup lists, checkboxes and edit elements. This makes it easier to work with SciXMiner for inexperienced users. Furthermore, an automatization and batch standardization of analyzes is possible using macros. The standard Matlab style using the command line is also available. SciXMiner is an open source software. The download page is http://sourceforge.net/projects/SciXMiner. It is licensed under the conditions of the GNU General Public License (GNU-GPL) of The Free Software Foundation. △ Less

Submitted 11 April, 2017; originally announced April 2017.

3D Cell Nuclei Segmentation with Balanced Graph Partitioning

Authors: Julian Arz, Peter Sanders, Johannes Stegmaier, Ralf Mikut

Abstract: Cell nuclei segmentation is one of the most important tasks in the analysis of biomedical images. With ever-growing sizes and amounts of three-dimensional images to be processed, there is a need for better and faster segmentation methods. Graph-based image segmentation has seen a rise in popularity in recent years, but is seen as very costly with regard to computational demand. We propose a new se… ▽ More Cell nuclei segmentation is one of the most important tasks in the analysis of biomedical images. With ever-growing sizes and amounts of three-dimensional images to be processed, there is a need for better and faster segmentation methods. Graph-based image segmentation has seen a rise in popularity in recent years, but is seen as very costly with regard to computational demand. We propose a new segmentation algorithm which overcomes these limitations. Our method uses recursive balanced graph partitioning to segment foreground components of a fast and efficient binarization. We construct a model for the cell nuclei to guide the partitioning process. Our algorithm is compared to other state-of-the-art segmentation algorithms in an experimental evaluation on two sets of realistically simulated inputs. Our method is faster, has similar or better quality and an acceptable memory overhead. △ Less

Submitted 17 February, 2017; originally announced February 2017.

New Methods to Improve Large-Scale Microscopy Image Analysis with Prior Knowledge and Uncertainty

Authors: Johannes Stegmaier

Abstract: Multidimensional imaging techniques provide powerful ways to examine various kinds of scientific questions. The routinely produced datasets in the terabyte-range, however, can hardly be analyzed manually and require an extensive use of automated image analysis. The present thesis introduces a new concept for the estimation and propagation of uncertainty involved in image analysis operators and new… ▽ More Multidimensional imaging techniques provide powerful ways to examine various kinds of scientific questions. The routinely produced datasets in the terabyte-range, however, can hardly be analyzed manually and require an extensive use of automated image analysis. The present thesis introduces a new concept for the estimation and propagation of uncertainty involved in image analysis operators and new segmentation algorithms that are suitable for terabyte-scale analyses of 3D+t microscopy images. △ Less

Submitted 30 August, 2016; originally announced August 2016.

Comments: 218 pages, 58 figures, PhD thesis, Department of Mechanical Engineering, Karlsruhe Institute of Technology, published online with KITopen (License: CC BY-SA 3.0, http://dx.doi.org/10.5445/IR/1000057821)

Fuzzy-based Propagation of Prior Knowledge to Improve Large-Scale Image Analysis Pipelines

Authors: Johannes Stegmaier, Ralf Mikut

Abstract: Many automatically analyzable scientific questions are well-posed and offer a variety of information about the expected outcome a priori. Although often being neglected, this prior knowledge can be systematically exploited to make automated analysis operations sensitive to a desired phenomenon or to evaluate extracted content with respect to this prior knowledge. For instance, the performance of p… ▽ More Many automatically analyzable scientific questions are well-posed and offer a variety of information about the expected outcome a priori. Although often being neglected, this prior knowledge can be systematically exploited to make automated analysis operations sensitive to a desired phenomenon or to evaluate extracted content with respect to this prior knowledge. For instance, the performance of processing operators can be greatly enhanced by a more focused detection strategy and the direct information about the ambiguity inherent in the extracted data. We present a new concept for the estimation and propagation of uncertainty involved in image analysis operators. This allows using simple processing operators that are suitable for analyzing large-scale 3D+t microscopy images without compromising the result quality. On the foundation of fuzzy set theory, we transform available prior knowledge into a mathematical representation and extensively use it enhance the result quality of various processing operators. All presented concepts are illustrated on a typical bioimage analysis pipeline comprised of seed point detection, segmentation, multiview fusion and tracking. Furthermore, the functionality of the proposed approach is validated on a comprehensive simulated 3D+t benchmark data set that mimics embryonic development and on large-scale light-sheet microscopy data of a zebrafish embryo. The general concept introduced in this contribution represents a new approach to efficiently exploit prior knowledge to improve the result quality of image analysis pipelines. Especially, the automated analysis of terabyte-scale microscopy data will benefit from sophisticated and efficient algorithms that enable a quantitative and fast readout. The generality of the concept, however, makes it also applicable to practically any other field with processing strategies that are arranged as linear pipelines. △ Less

Submitted 3 August, 2016; originally announced August 2016.

Comments: 39 pages, 12 figures

Generating Semi-Synthetic Validation Benchmarks for Embryomics

Authors: Johannes Stegmaier, Julian Arz, Benjamin Schott, Jens C. Otte, Andrei Kobitski, G. Ulrich Nienhaus, Uwe Strähle, Peter Sanders, Ralf Mikut

Abstract: Systematic validation is an essential part of algorithm development. The enormous dataset sizes and the complexity observed in many recent time-resolved 3D fluorescence microscopy imaging experiments, however, prohibit a comprehensive manual ground truth generation. Moreover, existing simulated benchmarks in this field are often too simple or too specialized to sufficiently validate the observed i… ▽ More Systematic validation is an essential part of algorithm development. The enormous dataset sizes and the complexity observed in many recent time-resolved 3D fluorescence microscopy imaging experiments, however, prohibit a comprehensive manual ground truth generation. Moreover, existing simulated benchmarks in this field are often too simple or too specialized to sufficiently validate the observed image analysis problems. We present a new semi-synthetic approach to generate realistic 3D+t benchmarks that combines challenging cellular movement dynamics of real embryos with simulated fluorescent nuclei and artificial image distortions including various parametrizable options like cell numbers, acquisition deficiencies or multiview simulations. We successfully applied the approach to simulate the development of a zebrafish embryo with thousands of cells over 14 hours of its early existence. △ Less

Submitted 17 April, 2016; originally announced April 2016.

Comments: Accepted publication at IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI), 2016

Robust Individual Circadian Parameter Estimation for Biosignal-based Personalisation of Cancer Chronotherapy

Authors: Benjamin Schott, Johannes Stegmaier, Alexandre Arbaud, Markus Reischl, Ralf Mikut, Francis Lévi

Abstract: In cancer treatment, chemotherapy is administered according a constant schedule. The chronotherapy approach, considering chronobiological drug delivery, adapts the chemotherapy profile to the circadian rhythms of the human organism. This reduces toxicity effects and at the same time enhances efficiency of chemotherapy. To personalize cancer treatment, chemotherapy profiles have to be further adapt… ▽ More In cancer treatment, chemotherapy is administered according a constant schedule. The chronotherapy approach, considering chronobiological drug delivery, adapts the chemotherapy profile to the circadian rhythms of the human organism. This reduces toxicity effects and at the same time enhances efficiency of chemotherapy. To personalize cancer treatment, chemotherapy profiles have to be further adapted to individual patients. Therefore, we present a new model to represent cycle phenomena in circadian rhythms. The model enables a more precise modelling of the underlying circadian rhythms. In comparison with the standard model, our model delivers better results in all defined quality indices. The new model can be used to adapt the chemotherapy profile efficiently to individual patients. The adaption to individual patients contributes to the aim of personalizing cancer therapy. △ Less

Submitted 11 April, 2016; originally announced April 2016.

Comments: Conference Biosig 2016, Berlin

Challenges of Integrating A Priori Information Efficiently in the Discovery of Spatio-Temporal Objects in Large Databases

Authors: Benjamin Schott, Johannes Stegmaier, Masanari Takamiya, Ralf Mikut

Abstract: Using the knowledge discovery framework, it is possible to explore object databases and extract groups of objects with highly heterogeneous movement behavior by efficiently integrating a priori knowledge through interacting with the framework. The whole process is modular expandable and is therefore adaptive to any problem formulation. Further, the flexible use of different information allocation… ▽ More Using the knowledge discovery framework, it is possible to explore object databases and extract groups of objects with highly heterogeneous movement behavior by efficiently integrating a priori knowledge through interacting with the framework. The whole process is modular expandable and is therefore adaptive to any problem formulation. Further, the flexible use of different information allocation processes reveal a great potential to efficiently incorporate the a priori knowledge of different users in different ways. Therefore, the stepwise knowledge discovery process embedded in the knowledge discovery framework is described in detail to point out the flexibility of such a system incorporating object databases from different applications. The described framework can be used to gain knowledge out of object databases in many different fields. This knowledge can be used to gain further insights and improve the understanding of underlying phenomena. The functionality of the proposed framework is exemplarily demonstrated using a benchmark database based on real biological object data. △ Less

Submitted 9 February, 2016; originally announced February 2016.

Comments: Proc., 25. Workshop Computational Intelligence, Dortmund, 2015

MSC Class: 92-08; 92C37; 92C55; 68U10

Design and performance of the South Pole Acoustic Test Setup

Authors: Yasser Abdou, Karl-Heinz Becker, Jens Berdermann, Martin Bissok, Christian Bohm, Sebastian Boeser, Martin Bothe, Michael Carson, Freija Descamps, Jan-Hendrik Fischer-Wolfarth, Leif Gustafsson, Allan Hallgren, Dirk Heinen, Klaus Helbing, Reinhart Heller, Stephan Hundertmark, Timo Karg, Kevin Krieger, Karim Laihem, Thomas Meures, Rolf Nahnhauer, Uwe Naumann, Filip Oberson, Larissa Paul, Mario Pohl , et al. (10 additional authors not shown)

Abstract: The South Pole Acoustic Test Setup (SPATS) was built to evaluate the acoustic characteristics of the South Pole ice in the 10 kHz to 100 kHz frequency range, for the purpose of assessing the feasibility of an acoustic neutrino detection array at the South Pole. The SPATS hardware consists of four vertical strings deployed in the upper 500 m of the South Pole ice cap. The strings form a trapezoidal… ▽ More The South Pole Acoustic Test Setup (SPATS) was built to evaluate the acoustic characteristics of the South Pole ice in the 10 kHz to 100 kHz frequency range, for the purpose of assessing the feasibility of an acoustic neutrino detection array at the South Pole. The SPATS hardware consists of four vertical strings deployed in the upper 500 m of the South Pole ice cap. The strings form a trapezoidal array with a maximum baseline of 543 m. Each string has 7 stages equipped with one transmitter and one sensor module. Sound is detected or generated by piezoelectric ceramic elements inside the modules. Analogue signals are sent to the surface on electric cables where they are digitized by a PC-based data acquisition system. The data from all strings are collected on a central computer in the IceCube Laboratory from where they are send to a central data storage facility via a satellite link or stored locally on tape. A technical overview of SPATS and its performance is presented. △ Less

Submitted 22 May, 2011; originally announced May 2011.

The Photodetector Array Camera and Spectrometer (PACS) on the Herschel Space Observatory

Authors: A. Poglitsch, C. Waelkens, N. Geis, H. Feuchtgruber, B. Vandenbussche, L. Rodriguez, O. Krause, E. Renotte, C. van Hoof, P. Saraceno, J. Cepa, F. Kerschbaum, P. Agnese, B. Ali, B. Altieri, P. Andreani, J. -L. Augueres, Z. Balog, L. Barl, O. H. Bauer, N. Belbachir, M. Benedettini, N. Billot, O. Boulade, H. Bischof , et al. (58 additional authors not shown)

Abstract: The Photodetector Array Camera and Spectrometer (PACS) is one of the three science instruments on ESA's far infrared and submillimetre observatory. It employs two Ge:Ga photoconductor arrays (stressed and unstressed) with 16x25 pixels, each, and two filled silicon bolometer arrays with 16x32 and 32x64 pixels, respectively, to perform integral-field spectroscopy and imaging photometry in the 60-210… ▽ More The Photodetector Array Camera and Spectrometer (PACS) is one of the three science instruments on ESA's far infrared and submillimetre observatory. It employs two Ge:Ga photoconductor arrays (stressed and unstressed) with 16x25 pixels, each, and two filled silicon bolometer arrays with 16x32 and 32x64 pixels, respectively, to perform integral-field spectroscopy and imaging photometry in the 60-210μ m wavelength regime. In photometry mode, it simultaneously images two bands, 60-85μ m or 85-125μ\m and 125-210μ m, over a field of view of ~1.75'x3.5', with close to Nyquist beam sampling in each band. In spectroscopy mode, it images a field of 47"x47", resolved into 5x5 pixels, with an instantaneous spectral coverage of ~1500km/s and a spectral resolution of ~175km/s. We summarise the design of the instrument, describe observing modes, calibration, and data analysis methods, and present our current assessment of the in-orbit performance of the instrument based on the Performance Verification tests. PACS is fully operational, and the achieved performance is close to or better than the pre-launch predictions. △ Less

Submitted 20 May, 2010; v1 submitted 10 May, 2010; originally announced May 2010.

Milliarcsecond angular resolution of reddened stellar sources in the vicinity of the Galactic Center

Authors: A. Richichi, O. Fors, E. Mason, J. Stegmaier, T. Chandrasekhar

Abstract: For the first time, the lunar occultation technique has been employed on a very large telescope in the near-IR with the aim of achieving systematically milliarcsecond resolution on stellar sources. We have demonstrated the burst mode of the ISAAC instrument, using a fast read-out on a small area of the detector to record many tens of seconds of data at a time on fields of few squared arcsec. W… ▽ More For the first time, the lunar occultation technique has been employed on a very large telescope in the near-IR with the aim of achieving systematically milliarcsecond resolution on stellar sources. We have demonstrated the burst mode of the ISAAC instrument, using a fast read-out on a small area of the detector to record many tens of seconds of data at a time on fields of few squared arcsec. We have used the opportunity to record a large number of LO events during a passage of the Moon close to the Galactic Center in March 2006. We have developed a data pipeline for the treatment of LO data, including the automated estimation of the main data analysis parameters using a wavelet-based method, and the preliminary fitting and plotting of all light curves. We recorded 51 LO events over about four hours. Of these, 30 resulted of sufficient quality to enable a detailed fitting. We detected two binaries with subarcsec projected separation and three stars with a marginally resolved angular diameter of about 2 mas. Two more SiO masers, were found to be resolved and in one case we could recover the brightness profile of the extended emission, which is well consistent with an optically thin shell. The remaining unresolved stars were used to characterize the performance of the method. The LO technique at a very large telescope is a powerful and efficient method to achieve angular resolution, sensitivity, and dynamic range that are among the best possible today with any technique. The selection of targets is naturally limited and LOs are fixed-time events, however each observation requires only a few minutes including overheads. As such, LOs are ideally suited to fill small gaps of idle time between standard observations. △ Less

Submitted 21 July, 2008; v1 submitted 16 July, 2008; originally announced July 2008.

Comments: A&A in press

Cold performance tests of blocked-impurity-band Si:As detectors developed for DARWIN

Authors: Stephan M. Birkmann, Jutta Stegmaier, Ulrich Groezinger, Oliver Krause, Tim Souverijns, Jan Putzeys, Deniz Sabuncuoglu Tezcan, Koen De Munck, Paolo Fiorini, Kiki Minoglou, Patrick Merken, Chris Van Hoof, Piet De Moor

Abstract: We report first results of laboratory tests of Si:As blocked-impurity-band (BIB) mid-infrared (4 to 28 um) detectors developed by IMEC. These prototypes feature 88 pixels hybridized on an integrated cryogenic readout electronics (CRE). They were developed as part of a technology demonstration program for the future DARWIN mission. In order to be able to separate detector and readout effects, a c… ▽ More We report first results of laboratory tests of Si:As blocked-impurity-band (BIB) mid-infrared (4 to 28 um) detectors developed by IMEC. These prototypes feature 88 pixels hybridized on an integrated cryogenic readout electronics (CRE). They were developed as part of a technology demonstration program for the future DARWIN mission. In order to be able to separate detector and readout effects, a custom build TIA circuitry was used to characterize additional single pixel detectors. We used a newly designed test setup at the MPIA to determine the relative spectral response, the quantum efficiency, and the dark current. All these properties were measured as a function of operating temperature and detector bias. In addition the effects of ionizing radiation on the detector were studied. For determining the relative spectral response we used a dual-grating monochromator and a bolometer with known response that was operated in parallel to the Si:As detectors. The quantum efficiency was measured by using a custom-build high-precision vacuum black body together with cold (T ~ 4 K) filters of known (measured) transmission. △ Less

Submitted 9 July, 2008; originally announced July 2008.

Comments: 11 pages, 8 figures, to appear in "High Energy, Optical, and Infrared Detectors for Astronomy" SPIE conference Proc. 7021, Marseille, 23-28 June 2008

Life on the fast lane: the burst mode at the VLT at present and in the future

Authors: A. Richichi, O. Fors, E. Mason, M. Delbo, J. Stegmaier, G. Finger

Abstract: The recent implementation of the high-speed burst mode at the ISAAC instrument on UT1, and its propagation to other ESO instruments, has opened the door to observational capabilities which hold the potential for a wealth of novel results. In the ELT era, when the accent will likely be on lengthy programs aimed at the best sensitivity and angular resolution, the VLT telescopes could continue to p… ▽ More The recent implementation of the high-speed burst mode at the ISAAC instrument on UT1, and its propagation to other ESO instruments, has opened the door to observational capabilities which hold the potential for a wealth of novel results. In the ELT era, when the accent will likely be on lengthy programs aimed at the best sensitivity and angular resolution, the VLT telescopes could continue to play a significant and largely unique role by performing routinely observations of transient events at high temporal resolution. In our contribution, we provide details on two such kinds of observations, namely lunar occultations of stars and of asteroids. For the first ones, we report on two passages of the Moon in regions with high stellar density as the Galactic Center. The VLT-UT1 telescope was used for the first time to record successfully 53 and 71 occultations on March 22 and August 6, 2006, with an angular resolution of 0.5-1 milliarcsecond and $K\sim12.5$ limiting magnitude. We note that the angular resolution is superior to that achieved at present by Adaptive Optics on any telescope, and also superior to that foreseen for the ELT at the same wavelength. LO are also very efficient in terms of telescope time. We present some of the results, including the discovery of close binaries, and the detection and study of compact circumstellar components of cool giants, AGB stars and embedded IR sources. Rest of the abstract follows at the paper △ Less

Submitted 7 December, 2007; originally announced December 2007.

Comments: Contribution to the "The VLT in the ELT era" ESO workshop. Garching, October 2007

VADER - A Satellite Mission Concept For High Precision Dark Energy Studies

Authors: Rene Fassbender, Jutta Stegmaier, Anne-Marie Weijmans, Stefan Koestner, Andreas Kruselburger, Cornelia Diethart, Peter Fertl, Elisabetta Valiante, Matthew Hayes, Peter Schuecker, Guenther Hasinger

Abstract: We present a satellite mission concept to measure the dark energy equation of state parameter w with percent-level precision. The Very Ambitious Dark Energy Research satellite (VADER) is a multi-wavelength survey mission joining X-ray, optical, and IR instruments for a simultaneous spectral coverage from 4microns (0.3eV) to 10keV over a field of view (FoV) of 1 square degree. VADER combines seve… ▽ More We present a satellite mission concept to measure the dark energy equation of state parameter w with percent-level precision. The Very Ambitious Dark Energy Research satellite (VADER) is a multi-wavelength survey mission joining X-ray, optical, and IR instruments for a simultaneous spectral coverage from 4microns (0.3eV) to 10keV over a field of view (FoV) of 1 square degree. VADER combines several clean methods for dark energy studies, the baryonic acoustic oscillations in the galaxy and galaxy cluster power spectrum and weak lensing, for a joint analysis over an unrivalled survey volume. The payload consists of two XMM-like X-ray telescopes with an effective area of 2,800cm^2 at 1.5keV and state-of-the-art wide field DEPFET pixel detectors (0.1-10keV) in a curved focal plane configuration to extend the FoV. The X-ray telescopes are complemented by a 1.5m optical/IR telescope with 8 instruments for simultaneous coverage of the same FoV from 0.3 to 4 microns. The 8 dichroic-separated bands (u,g,r,z,J,H,K,L) provide accurate photometric galaxy redshifts, whereas the diffraction-limited resolution of the central z-band allows precise shape measurements for cosmic shear analysis. The 5 year VADER survey will cover a contiguous sky area of 3,500 square degrees to a depth of z~2 and will yield accurate photometric redshifts and multi-wavelength object parameters for about 175,000 galaxy clusters, one billion galaxies, and 5 million AGN. VADER will not only provide unprecedented constraints on the nature of dark energy, but will additionally extend and trigger a multitude of cosmic evolution studies to very large (>10 Gyrs) look-back times. △ Less

Submitted 28 June, 2006; originally announced June 2006.

Comments: 14 pages, 7 figures, accepted for publication in the SPIE conference proceedings