-
A Design Space for Visualization Transitions of 3D Spatial Data in Hybrid AR-Desktop Environments
Authors:
Yucheng Lu,
Tobias Rau,
Benjamin Lee,
Andreas Köhn,
Michael Sedlmair,
Christian Sandor,
Tobias Isenberg
Abstract:
We present a design space for animated transitions of the appearance of 3D spatial datasets in a hybrid Augmented Reality (AR)-desktop context. Such hybrid interfaces combine both traditional and immersive displays to facilitate the exploration of 2D and 3D data representations in the environment in which they are best displayed. One key aspect is to introduce transitional animations that change b…
▽ More
We present a design space for animated transitions of the appearance of 3D spatial datasets in a hybrid Augmented Reality (AR)-desktop context. Such hybrid interfaces combine both traditional and immersive displays to facilitate the exploration of 2D and 3D data representations in the environment in which they are best displayed. One key aspect is to introduce transitional animations that change between the different dimensionalities to illustrate the connection between the different representations and to reduce the potential cognitive load on the user. The specific transitions to be used depend on the type of data, the needs of the application domain, and other factors. We summarize these as a transition design space to simplify the decision-making process and provide inspiration for future designs. First, we discuss 3D visualizations from a spatial perspective: a spatial encoding pipeline, where 3D data sampled from the physical world goes through various transformations, being mapped to visual representations, and then being integrated into a hybrid AR-desktop environment. The transition design then focuses on interpolating between two spatial encoding pipelines to provide a smooth experience. To illustrate the use of our design space, we apply it to three case studies that focus on applications in astronomy, radiology, and chemistry; we then discuss lessons learned from these applications.
△ Less
Submitted 27 June, 2025;
originally announced June 2025.
-
Group Gaze-Sharing with Projection Displays
Authors:
Maurice Koch,
Tobias Rau,
Vladimir Mikheev,
Seyda Öney,
Michael Becher,
Xiangyu Wang,
Nelusa Pathmanathan,
Patrick Gralka,
Daniel Weiskopf,
Kuno Kurzhals
Abstract:
The eyes play an important role in human collaboration. Mutual and shared gaze help communicate visual attention to each other or to a specific object of interest. Shared gaze was typically investigated for pair collaborations in remote settings and with people in virtual and augmented reality. With our work, we expand this line of research by a new technique to communicate gaze between groups in…
▽ More
The eyes play an important role in human collaboration. Mutual and shared gaze help communicate visual attention to each other or to a specific object of interest. Shared gaze was typically investigated for pair collaborations in remote settings and with people in virtual and augmented reality. With our work, we expand this line of research by a new technique to communicate gaze between groups in tabletop workshop scenarios. To achieve this communication, we use an approach based on projection mapping to unify gaze data from multiple participants into a common visualization space on a tabletop. We showcase our approach with a collaborative puzzle-solving task that displays shared visual attention on individual pieces and provides hints to solve the problem at hand.
△ Less
Submitted 2 May, 2025;
originally announced May 2025.
-
Traversing Dual Realities: Investigating Techniques for Transitioning 3D Objects between Desktop and Augmented Reality Environments
Authors:
Tobias Rau,
Tobias Isenberg,
Andreas Köhn,
Michael Sedlmair,
Benjamin Lee
Abstract:
Desktop environments can integrate augmented reality (AR) head-worn devices to support 3D representations, visualizations, and interactions in a novel yet familiar setting. As users navigate across the dual realities -- desktop and AR -- a way to move 3D objects between them is needed. We devise three baseline transition techniques based on common approaches in the literature and evaluate their us…
▽ More
Desktop environments can integrate augmented reality (AR) head-worn devices to support 3D representations, visualizations, and interactions in a novel yet familiar setting. As users navigate across the dual realities -- desktop and AR -- a way to move 3D objects between them is needed. We devise three baseline transition techniques based on common approaches in the literature and evaluate their usability and practicality in an initial user study (N=18). After refining both our transition techniques and the surrounding technical setup, we validate the applicability of the overall concept for real-world activities in an expert user study (N=6). In it, computational chemists followed their usual desktop workflows to build, manipulate, and analyze 3D molecular structures, but now aided with the addition of AR and our transition techniques. Based on our findings from both user studies, we provide lessons learned and takeaways for the design of 3D object transition techniques in desktop + AR environments.
△ Less
Submitted 31 March, 2025;
originally announced April 2025.
-
Pathologist-like explainable AI for interpretable Gleason grading in prostate cancer
Authors:
Gesa Mittmann,
Sara Laiouar-Pedari,
Hendrik A. Mehrtens,
Sarah Haggenmüller,
Tabea-Clara Bucher,
Tirtha Chanda,
Nadine T. Gaisa,
Mathias Wagner,
Gilbert Georg Klamminger,
Tilman T. Rau,
Christina Neppl,
Eva Maria Compérat,
Andreas Gocht,
Monika Hämmerle,
Niels J. Rupp,
Jula Westhoff,
Irene Krücken,
Maximillian Seidl,
Christian M. Schürch,
Marcus Bauer,
Wiebke Solass,
Yu Chun Tam,
Florian Weber,
Rainer Grobholz,
Jaroslaw Augustyniak
, et al. (41 additional authors not shown)
Abstract:
The aggressiveness of prostate cancer, the most common cancer in men worldwide, is primarily assessed based on histopathological data using the Gleason scoring system. While artificial intelligence (AI) has shown promise in accurately predicting Gleason scores, these predictions often lack inherent explainability, potentially leading to distrust in human-machine interactions. To address this issue…
▽ More
The aggressiveness of prostate cancer, the most common cancer in men worldwide, is primarily assessed based on histopathological data using the Gleason scoring system. While artificial intelligence (AI) has shown promise in accurately predicting Gleason scores, these predictions often lack inherent explainability, potentially leading to distrust in human-machine interactions. To address this issue, we introduce a novel dataset of 1,015 tissue microarray core images, annotated by an international group of 54 pathologists. The annotations provide detailed localized pattern descriptions for Gleason grading in line with international guidelines. Utilizing this dataset, we develop an inherently explainable AI system based on a U-Net architecture that provides predictions leveraging pathologists' terminology. This approach circumvents post-hoc explainability methods while maintaining or exceeding the performance of methods trained directly for Gleason pattern segmentation (Dice score: 0.713 $\pm$ 0.003 trained on explanations vs. 0.691 $\pm$ 0.010 trained on Gleason patterns). By employing soft labels during training, we capture the intrinsic uncertainty in the data, yielding strong results in Gleason pattern segmentation even in the context of high interobserver variability. With the release of this dataset, we aim to encourage further research into segmentation in medical tasks with high levels of subjectivity and to advance the understanding of pathologists' reasoning processes.
△ Less
Submitted 19 October, 2024;
originally announced October 2024.
-
Quantifying Explainers of Graph Neural Networks in Computational Pathology
Authors:
Guillaume Jaume,
Pushpak Pati,
Behzad Bozorgtabar,
Antonio Foncubierta-Rodríguez,
Florinda Feroce,
Anna Maria Anniciello,
Tilman Rau,
Jean-Philippe Thiran,
Maria Gabrani,
Orcun Goksel
Abstract:
Explainability of deep learning methods is imperative to facilitate their clinical adoption in digital pathology. However, popular deep learning methods and explainability techniques (explainers) based on pixel-wise processing disregard biological entities' notion, thus complicating comprehension by pathologists. In this work, we address this by adopting biological entity-based graph processing an…
▽ More
Explainability of deep learning methods is imperative to facilitate their clinical adoption in digital pathology. However, popular deep learning methods and explainability techniques (explainers) based on pixel-wise processing disregard biological entities' notion, thus complicating comprehension by pathologists. In this work, we address this by adopting biological entity-based graph processing and graph explainers enabling explanations accessible to pathologists. In this context, a major challenge becomes to discern meaningful explainers, particularly in a standardized and quantifiable fashion. To this end, we propose herein a set of novel quantitative metrics based on statistics of class separability using pathologically measurable concepts to characterize graph explainers. We employ the proposed metrics to evaluate three types of graph explainers, namely the layer-wise relevance propagation, gradient-based saliency, and graph pruning approaches, to explain Cell-Graph representations for Breast Cancer Subtyping. The proposed metrics are also applicable in other domains by using domain-specific intuitive concepts. We validate the qualitative and quantitative findings on the BRACS dataset, a large cohort of breast cancer RoIs, by expert pathologists.
△ Less
Submitted 14 May, 2021; v1 submitted 25 November, 2020;
originally announced November 2020.
-
Towards realistic HPC models of the neuromuscular system
Authors:
Chris Bradley,
Nehzat Emamy,
Thomas Ertl,
Dominik Göddeke,
Andreas Hessenthaler,
Thomas Klotz,
Aaron Krämer,
Michael Krone,
Benjamin Maier,
Miriam Mehl,
Tobias Rau,
Oliver Röhrle
Abstract:
Realistic simulations of detailed, biophysics-based, multi-scale models require very high resolution and, thus, large-scale compute facilities. Existing simulation environments, especially for biomedical applications, are designed to allow for a high flexibility and generality in model development. Flexibility and model development, however, are often a limiting factor for large-scale simulations.…
▽ More
Realistic simulations of detailed, biophysics-based, multi-scale models require very high resolution and, thus, large-scale compute facilities. Existing simulation environments, especially for biomedical applications, are designed to allow for a high flexibility and generality in model development. Flexibility and model development, however, are often a limiting factor for large-scale simulations. Therefore, new models are typically tested and run on small-scale compute facilities. By using a detailed biophysics-based, chemo-electromechanical skeletal muscle model and the international open-source software library OpenCMISS as an example, we present an approach to upgrade an existing muscle simulation framework from a moderately parallel version towards a massively parallel one that scales both in terms of problem size and in terms of the number of parallel processes. For this purpose, we investigate different modeling, algorithmic and implementational aspects. We present improvements addressing both numerical and parallel scalability. In addition, our approach includes a novel visualization environment, which is based on the MegaMol environment capable of handling large amounts of simulated data. It offers a platform for fast visualization prototyping, distributed rendering, and advanced visualization techniques. We present results of a variety of scaling studies at the Tier-1 supercomputer HazelHen at the High Performance Computing Center Stuttgart (HLRS). We improve the overall runtime by a factor of up to 2.6 and achieved good scalability on up to 768 cores, where the previous implementation used only 4 cores.
△ Less
Submitted 9 February, 2018;
originally announced February 2018.
