-
PhenoBench: A Comprehensive Benchmark for Cell Phenotyping
Authors:
Jerome Luescher,
Nora Koreuber,
Jannik Franzen,
Fabian H. Reith,
Claudia Winklmayr,
Elias Baumann,
Christian M. Schuerch,
Dagmar Kainmueller,
Josef Lorenz Rumberger
Abstract:
Digital pathology has seen the advent of a wealth of foundational models (FM), yet to date their performance on cell phenotyping has not been benchmarked in a unified manner. We therefore propose PhenoBench: A comprehensive benchmark for cell phenotyping on Hematoxylin and Eosin (H&E) stained histopathology images. We provide both PhenoCell, a new H&E dataset featuring 14 granular cell types ident…
▽ More
Digital pathology has seen the advent of a wealth of foundational models (FM), yet to date their performance on cell phenotyping has not been benchmarked in a unified manner. We therefore propose PhenoBench: A comprehensive benchmark for cell phenotyping on Hematoxylin and Eosin (H&E) stained histopathology images. We provide both PhenoCell, a new H&E dataset featuring 14 granular cell types identified by using multiplexed imaging, and ready-to-use fine-tuning and benchmarking code that allows the systematic evaluation of multiple prominent pathology FMs in terms of dense cell phenotype predictions in different generalization scenarios. We perform extensive benchmarking of existing FMs, providing insights into their generalization behavior under technical vs. medical domain shifts. Furthermore, while FMs achieve macro F1 scores > 0.70 on previously established benchmarks such as Lizard and PanNuke, on PhenoCell, we observe scores as low as 0.20. This indicates a much more challenging task not captured by previous benchmarks, establishing PhenoCell as a prime asset for future benchmarking of FMs and supervised models alike. Code and data are available on GitHub.
△ Less
Submitted 8 July, 2025; v1 submitted 4 July, 2025;
originally announced July 2025.
-
FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures
Authors:
Lisa Mais,
Peter Hirsch,
Claire Managan,
Ramya Kandarpa,
Josef Lorenz Rumberger,
Annika Reinke,
Lena Maier-Hein,
Gudrun Ihrke,
Dagmar Kainmueller
Abstract:
Instance segmentation of neurons in volumetric light microscopy images of nervous systems enables groundbreaking research in neuroscience by facilitating joint functional and morphological analyses of neural circuits at cellular resolution. Yet said multi-neuron light microscopy data exhibits extremely challenging properties for the task of instance segmentation: Individual neurons have long-rangi…
▽ More
Instance segmentation of neurons in volumetric light microscopy images of nervous systems enables groundbreaking research in neuroscience by facilitating joint functional and morphological analyses of neural circuits at cellular resolution. Yet said multi-neuron light microscopy data exhibits extremely challenging properties for the task of instance segmentation: Individual neurons have long-ranging, thin filamentous and widely branching morphologies, multiple neurons are tightly inter-weaved, and partial volume effects, uneven illumination and noise inherent to light microscopy severely impede local disentangling as well as long-range tracing of individual neurons. These properties reflect a current key challenge in machine learning research, namely to effectively capture long-range dependencies in the data. While respective methodological research is buzzing, to date methods are typically benchmarked on synthetic datasets. To address this gap, we release the FlyLight Instance Segmentation Benchmark (FISBe) dataset, the first publicly available multi-neuron light microscopy dataset with pixel-wise annotations. In addition, we define a set of instance segmentation metrics for benchmarking that we designed to be meaningful with regard to downstream analyses. Lastly, we provide three baselines to kick off a competition that we envision to both advance the field of machine learning regarding methodology for capturing long-range data dependencies, and facilitate scientific discovery in basic neuroscience.
△ Less
Submitted 29 March, 2024;
originally announced April 2024.
-
CoNIC Challenge: Pushing the Frontiers of Nuclear Detection, Segmentation, Classification and Counting
Authors:
Simon Graham,
Quoc Dang Vu,
Mostafa Jahanifar,
Martin Weigert,
Uwe Schmidt,
Wenhua Zhang,
Jun Zhang,
Sen Yang,
Jinxi Xiang,
Xiyue Wang,
Josef Lorenz Rumberger,
Elias Baumann,
Peter Hirsch,
Lihao Liu,
Chenyang Hong,
Angelica I. Aviles-Rivero,
Ayushi Jain,
Heeyoung Ahn,
Yiyu Hong,
Hussam Azzuni,
Min Xu,
Mohammad Yaqub,
Marie-Claire Blache,
Benoît Piégu,
Bertrand Vernay
, et al. (64 additional authors not shown)
Abstract:
Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of repro…
▽ More
Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 whole-slide images of colon tissue. With around 700 million detected nuclei per model, associated features were used for dysplasia grading and survival analysis, where we demonstrated that the challenge's improvement over the previous state-of-the-art led to significant boosts in downstream performance. Our findings also suggest that eosinophils and neutrophils play an important role in the tumour microevironment. We release challenge models and WSI-level results to foster the development of further methods for biomarker discovery.
△ Less
Submitted 14 March, 2023; v1 submitted 10 March, 2023;
originally announced March 2023.
-
Panoptic segmentation with highly imbalanced semantic labels
Authors:
Josef Lorenz Rumberger,
Elias Baumann,
Peter Hirsch,
Andrew Janowczyk,
Inti Zlobec,
Dagmar Kainmueller
Abstract:
We describe here the panoptic segmentation method we devised for our participation in the CoNIC: Colon Nuclei Identification and Counting Challenge at ISBI 2022. Key features of our method are a weighted loss specifically engineered for semantic segmentation of highly imbalanced cell types, and a state-of-the art nuclei instance segmentation model, which we combine in a Hovernet-like architecture.
We describe here the panoptic segmentation method we devised for our participation in the CoNIC: Colon Nuclei Identification and Counting Challenge at ISBI 2022. Key features of our method are a weighted loss specifically engineered for semantic segmentation of highly imbalanced cell types, and a state-of-the art nuclei instance segmentation model, which we combine in a Hovernet-like architecture.
△ Less
Submitted 19 April, 2022; v1 submitted 3 March, 2022;
originally announced March 2022.
-
How Shift Equivariance Impacts Metric Learning for Instance Segmentation
Authors:
Josef Lorenz Rumberger,
Xiaoyan Yu,
Peter Hirsch,
Melanie Dohmen,
Vanessa Emanuela Guarino,
Ashkan Mokarian,
Lisa Mais,
Jan Funke,
Dagmar Kainmueller
Abstract:
Metric learning has received conflicting assessments concerning its suitability for solving instance segmentation tasks. It has been dismissed as theoretically flawed due to the shift equivariance of the employed CNNs and their respective inability to distinguish same-looking objects. Yet it has been shown to yield state of the art results for a variety of tasks, and practical issues have mainly b…
▽ More
Metric learning has received conflicting assessments concerning its suitability for solving instance segmentation tasks. It has been dismissed as theoretically flawed due to the shift equivariance of the employed CNNs and their respective inability to distinguish same-looking objects. Yet it has been shown to yield state of the art results for a variety of tasks, and practical issues have mainly been reported in the context of tile-and-stitch approaches, where discontinuities at tile boundaries have been observed. To date, neither of the reported issues have undergone thorough formal analysis. In our work, we contribute a comprehensive formal analysis of the shift equivariance properties of encoder-decoder-style CNNs, which yields a clear picture of what can and cannot be achieved with metric learning in the face of same-looking objects. In particular, we prove that a standard encoder-decoder network that takes $d$-dimensional images as input, with $l$ pooling layers and pooling factor $f$, has the capacity to distinguish at most $f^{dl}$ same-looking objects, and we show that this upper limit can be reached. Furthermore, we show that to avoid discontinuities in a tile-and-stitch approach, assuming standard batch size 1, it is necessary to employ valid convolutions in combination with a training output window size strictly greater than $f^l$, while at test-time it is necessary to crop tiles to size $n\cdot f^l$ before stitching, with $n\geq 1$. We complement these theoretical findings by discussing a number of insightful special cases for which we show empirical results on synthetic data.
△ Less
Submitted 19 August, 2021; v1 submitted 14 January, 2021;
originally announced January 2021.
-
Probabilistic Deep Learning for Instance Segmentation
Authors:
Josef Lorenz Rumberger,
Lisa Mais,
Dagmar Kainmueller
Abstract:
Probabilistic convolutional neural networks, which predict distributions of predictions instead of point estimates, led to recent advances in many areas of computer vision, from image reconstruction to semantic segmentation. Besides state of the art benchmark results, these networks made it possible to quantify local uncertainties in the predictions. These were used in active learning frameworks t…
▽ More
Probabilistic convolutional neural networks, which predict distributions of predictions instead of point estimates, led to recent advances in many areas of computer vision, from image reconstruction to semantic segmentation. Besides state of the art benchmark results, these networks made it possible to quantify local uncertainties in the predictions. These were used in active learning frameworks to target the labeling efforts of specialist annotators or to assess the quality of a prediction in a safety-critical environment. However, for instance segmentation problems these methods are not frequently used so far. We seek to close this gap by proposing a generic method to obtain model-inherent uncertainty estimates within proposal-free instance segmentation models. Furthermore, we analyze the quality of the uncertainty estimates with a metric adapted from semantic segmentation. We evaluate our method on the BBBC010 C.\ elegans dataset, where it yields competitive performance while also predicting uncertainty estimates that carry information about object-level inaccuracies like false splits and false merges. We perform a simulation to show the potential use of such uncertainty estimates in guided proofreading.
△ Less
Submitted 17 December, 2020; v1 submitted 24 August, 2020;
originally announced August 2020.
-
State of the Art in Fair ML: From Moral Philosophy and Legislation to Fair Classifiers
Authors:
Elias Baumann,
Josef Lorenz Rumberger
Abstract:
Machine learning is becoming an ever present part in our lives as many decisions, e.g. to lend a credit, are no longer made by humans but by machine learning algorithms. However those decisions are often unfair and discriminating individuals belonging to protected groups based on race or gender. With the recent General Data Protection Regulation (GDPR) coming into effect, new awareness has been ra…
▽ More
Machine learning is becoming an ever present part in our lives as many decisions, e.g. to lend a credit, are no longer made by humans but by machine learning algorithms. However those decisions are often unfair and discriminating individuals belonging to protected groups based on race or gender. With the recent General Data Protection Regulation (GDPR) coming into effect, new awareness has been raised for such issues and with computer scientists having such a large impact on peoples lives it is necessary that actions are taken to discover and prevent discrimination. This work aims to give an introduction into discrimination, legislative foundations to counter it and strategies to detect and prevent machine learning algorithms from showing such behavior.
△ Less
Submitted 26 May, 2024; v1 submitted 20 November, 2018;
originally announced November 2018.
