-
Open Your Ears and Take a Look: A State-of-the-Art Report on the Integration of Sonification and Visualization
Authors:
Kajetan Enge,
Elias Elmquist,
Valentina Caiola,
Niklas Rönnberg,
Alexander Rind,
Michael Iber,
Sara Lenzi,
Fangfei Lan,
Robert Höldrich,
Wolfgang Aigner
Abstract:
The research communities studying visualization and sonification for data display and analysis share exceptionally similar goals, essentially making data of any kind interpretable to humans. One community does so by using visual representations of data, and the other community employs auditory (non-speech) representations of data. While the two communities have a lot in common, they developed most…
▽ More
The research communities studying visualization and sonification for data display and analysis share exceptionally similar goals, essentially making data of any kind interpretable to humans. One community does so by using visual representations of data, and the other community employs auditory (non-speech) representations of data. While the two communities have a lot in common, they developed mostly in parallel over the course of the last few decades. With this STAR, we discuss a collection of work that bridges the borders of the two communities, hence a collection of work that aims to integrate the two techniques into one form of audiovisual display, which we argue to be "more than the sum of the two."
We introduce and motivate a classification system applicable to such audiovisual displays and categorize a corpus of 57 academic publications that appeared between 2011 and 2023 in categories such as reading level, dataset type, or evaluation system, to mention a few. The corpus also enables a meta-analysis of the field, including regularly occurring design patterns such as type of visualization and sonification techniques, or the use of visual and auditory channels, showing an overall diverse field with different designs. An analysis of a co-author network of the field shows individual teams without many interconnections. The body of work covered in this STAR also relates to three adjacent topics: audiovisual monitoring, accessibility, and audiovisual data art. These three topics are discussed individually in addition to the systematically conducted part of this research. The findings of this report may be used by researchers from both fields to understand the potentials and challenges of such integrated designs while hopefully inspiring them to collaborate with experts from the respective other field.
△ Less
Submitted 24 April, 2024; v1 submitted 26 February, 2024;
originally announced February 2024.
-
Trustworthy Visual Analytics in Clinical Gait Analysis: A Case Study for Patients with Cerebral Palsy
Authors:
Alexander Rind,
Djordje Slijepčević,
Matthias Zeppelzauer,
Fabian Unglaube,
Andreas Kranzl,
Brian Horsak
Abstract:
Three-dimensional clinical gait analysis is essential for selecting optimal treatment interventions for patients with cerebral palsy (CP), but generates a large amount of time series data. For the automated analysis of these data, machine learning approaches yield promising results. However, due to their black-box nature, such approaches are often mistrusted by clinicians. We propose gaitXplorer,…
▽ More
Three-dimensional clinical gait analysis is essential for selecting optimal treatment interventions for patients with cerebral palsy (CP), but generates a large amount of time series data. For the automated analysis of these data, machine learning approaches yield promising results. However, due to their black-box nature, such approaches are often mistrusted by clinicians. We propose gaitXplorer, a visual analytics approach for the classification of CP-related gait patterns that integrates Grad-CAM, a well-established explainable artificial intelligence algorithm, for explanations of machine learning classifications. Regions of high relevance for classification are highlighted in the interactive visual interface. The approach is evaluated in a case study with two clinical gait experts. They inspected the explanations for a sample of eight patients using the visual interface and expressed which relevance scores they found trustworthy and which they found suspicious. Overall, the clinicians gave positive feedback on the approach as it allowed them a better understanding of which regions in the data were relevant for the classification.
△ Less
Submitted 19 December, 2022; v1 submitted 10 August, 2022;
originally announced August 2022.
-
Towards a Structural Framework for Explicit Domain Knowledge in Visual Analytics
Authors:
Alexander Rind,
Markus Wagner,
Wolfgang Aigner
Abstract:
Clinicians and other analysts working with healthcare data are in need for better support to cope with large and complex data. While an increasing number of visual analytics environments integrates explicit domain knowledge as a means to deliver a precise representation of the available data, theoretical work so far has focused on the role of knowledge in the visual analytics process. There has be…
▽ More
Clinicians and other analysts working with healthcare data are in need for better support to cope with large and complex data. While an increasing number of visual analytics environments integrates explicit domain knowledge as a means to deliver a precise representation of the available data, theoretical work so far has focused on the role of knowledge in the visual analytics process. There has been little discussion about how such explicit domain knowledge can be structured in a generalized framework. This paper collects desiderata for such a structural framework, proposes how to address these desiderata based on the model of linked data, and demonstrates the applicability in a visual analytics environment for physiotherapy.
△ Less
Submitted 6 January, 2020; v1 submitted 21 August, 2019;
originally announced August 2019.
-
Looking beyond the horizon: Evaluation of four compact visualization techniques for time series in a spatial context
Authors:
Manuel Dahnert,
Alexander Rind,
Wolfgang Aigner,
Johannes Kehrer
Abstract:
Visualizing time series in a dense spatial context such as a geographical map is a challenging task, which requires careful balance between the amount of depicted data and perceptual precision. Horizon graphs are a well-known technique for compactly representing time series data. They provide fine details while simultaneously giving an overview of the data where extrema are emphasized. Horizon gra…
▽ More
Visualizing time series in a dense spatial context such as a geographical map is a challenging task, which requires careful balance between the amount of depicted data and perceptual precision. Horizon graphs are a well-known technique for compactly representing time series data. They provide fine details while simultaneously giving an overview of the data where extrema are emphasized. Horizon graphs compress the vertical resolution of the individual line graphs, but they do not affect the horizontal resolution. We present two variations of a new visualization technique called collapsed horizon graphs which extend the idea of horizon graphs to two dimensions. Our main contribution is a quantitative evaluation that experimentally compares four visualization techniques with high visual information resolution (compact boxplots, horizon graphs, collapsed horizon graphs, and braided collapsed horizon graphs). The experiment investigates the performance of these techniques across tasks addressing both individual graphs as well as groups of adjacent graphs. Compact boxplots consistently provide good results for all tasks, horizon graphs excel, for instance, in maximum tasks but underperform in trend detection. Collapsed horizon graphs shine in certain tasks in which an increased horizontal resolution is beneficial. Moreover, our results indicate that the visual complexity of the techniques highly affects users' confidence and perceived task difficulty.
△ Less
Submitted 18 June, 2019;
originally announced June 2019.
-
KAVAGait: Knowledge-Assisted Visual Analytics for Clinical Gait Analysis
Authors:
Markus Wagner,
Djordje Slijepcevic,
Brian Horsak,
Alexander Rind,
Matthias Zeppelzauer,
Wolfgang Aigner
Abstract:
In 2014, more than 10 million people in the US were affected by an ambulatory disability. Thus, gait rehabilitation is a crucial part of health care systems. The quantification of human locomotion enables clinicians to describe and analyze a patient's gait performance in detail and allows them to base clinical decisions on objective data. These assessments generate a vast amount of complex data wh…
▽ More
In 2014, more than 10 million people in the US were affected by an ambulatory disability. Thus, gait rehabilitation is a crucial part of health care systems. The quantification of human locomotion enables clinicians to describe and analyze a patient's gait performance in detail and allows them to base clinical decisions on objective data. These assessments generate a vast amount of complex data which need to be interpreted in a short time period. We conducted a design study in cooperation with gait analysis experts to develop a novel Knowledge-Assisted Visual Analytics solution for clinical Gait analysis (KAVAGait). KAVAGait allows the clinician to store and inspect complex data derived during clinical gait analysis. The system incorporates innovative and interactive visual interface concepts, which were developed based on the needs of clinicians. Additionally, an explicit knowledge store (EKS) allows externalization and storage of implicit knowledge from clinicians. It makes this information available for others, supporting the process of data inspection and clinical decision making. We validated our system by conducting expert reviews, a user study, and a case study. Results suggest that KAVAGait is able to support a clinician during clinical practice by visualizing complex gait data and providing knowledge of other clinicians.
△ Less
Submitted 14 December, 2017; v1 submitted 19 July, 2017;
originally announced July 2017.
-
A Knowledge-Assisted Visual Malware Analysis System: Design, Validation, and Reflection of KAMAS
Authors:
Markus Wagner,
Alexander Rind,
Niklas Thür,
Wolfgang Aigner
Abstract:
IT-security experts engage in behavior-based malware analysis in order to learn about previously unknown samples of malicious software (malware) or malware families. For this, they need to find and categorize suspicious patterns from large collections of execution traces. Currently available systems do not meet the analysts' needs described as: visual access suitable for complex data structures, v…
▽ More
IT-security experts engage in behavior-based malware analysis in order to learn about previously unknown samples of malicious software (malware) or malware families. For this, they need to find and categorize suspicious patterns from large collections of execution traces. Currently available systems do not meet the analysts' needs described as: visual access suitable for complex data structures, visual representations appropriate for IT-security experts, provide work flow-specific interaction techniques, and the ability to externalize knowledge in the form of rules to ease analysis and for sharing with colleagues. To close this gap, we designed and developed KAMAS, a knowledge-assisted visualization system for behavior-based malware analysis. KAMAS supports malware analysts with visual analytics and knowledge externalization methods for the analysis process. The paper at hand is a design study that describes the design, implementation, and evaluation of the prototype. We report on the validation of KAMAS by expert reviews, a user study with domain experts, and focus group meetings with analysts from industry. Additionally, we reflect the gained insights of the design study and discuss the advantages and disadvantages of the applied visualization methods.
△ Less
Submitted 24 February, 2017; v1 submitted 19 December, 2016;
originally announced December 2016.
