-
Beware of "Explanations" of AI
Authors:
David Martens,
Galit Shmueli,
Theodoros Evgeniou,
Kevin Bauer,
Christian Janiesch,
Stefan Feuerriegel,
Sebastian Gabel,
Sofie Goethals,
Travis Greene,
Nadja Klein,
Mathias Kraus,
Niklas Kühl,
Claudia Perlich,
Wouter Verbeke,
Alona Zharova,
Patrick Zschech,
Foster Provost
Abstract:
Understanding the decisions made and actions taken by increasingly complex AI system remains a key challenge. This has led to an expanding field of research in explainable artificial intelligence (XAI), highlighting the potential of explanations to enhance trust, support adoption, and meet regulatory standards. However, the question of what constitutes a "good" explanation is dependent on the goal…
▽ More
Understanding the decisions made and actions taken by increasingly complex AI system remains a key challenge. This has led to an expanding field of research in explainable artificial intelligence (XAI), highlighting the potential of explanations to enhance trust, support adoption, and meet regulatory standards. However, the question of what constitutes a "good" explanation is dependent on the goals, stakeholders, and context. At a high level, psychological insights such as the concept of mental model alignment can offer guidance, but success in practice is challenging due to social and technical factors. As a result of this ill-defined nature of the problem, explanations can be of poor quality (e.g. unfaithful, irrelevant, or incoherent), potentially leading to substantial risks. Instead of fostering trust and safety, poorly designed explanations can actually cause harm, including wrong decisions, privacy violations, manipulation, and even reduced AI adoption. Therefore, we caution stakeholders to beware of explanations of AI: while they can be vital, they are not automatically a remedy for transparency or responsible AI adoption, and their misuse or limitations can exacerbate harm. Attention to these caveats can help guide future research to improve the quality and impact of AI explanations.
△ Less
Submitted 9 April, 2025;
originally announced April 2025.
-
CleanDIFT: Diffusion Features without Noise
Authors:
Nick Stracke,
Stefan Andreas Baumann,
Kolja Bauer,
Frank Fundel,
Björn Ommer
Abstract:
Internal features from large-scale pre-trained diffusion models have recently been established as powerful semantic descriptors for a wide range of downstream tasks. Works that use these features generally need to add noise to images before passing them through the model to obtain the semantic features, as the models do not offer the most useful features when given images with little to no noise.…
▽ More
Internal features from large-scale pre-trained diffusion models have recently been established as powerful semantic descriptors for a wide range of downstream tasks. Works that use these features generally need to add noise to images before passing them through the model to obtain the semantic features, as the models do not offer the most useful features when given images with little to no noise. We show that this noise has a critical impact on the usefulness of these features that cannot be remedied by ensembling with different random noises. We address this issue by introducing a lightweight, unsupervised fine-tuning method that enables diffusion backbones to provide high-quality, noise-free semantic features. We show that these features readily outperform previous diffusion features by a wide margin in a wide variety of extraction setups and downstream tasks, offering better performance than even ensemble-based methods at a fraction of the cost.
△ Less
Submitted 7 April, 2025; v1 submitted 4 December, 2024;
originally announced December 2024.
-
Marker-free Human Gait Analysis using a Smart Edge Sensor System
Authors:
Eva Katharina Bauer,
Simon Bultmann,
Sven Behnke
Abstract:
The human gait is a complex interplay between the neuronal and the muscular systems, reflecting an individual's neurological and physiological condition. This makes gait analysis a valuable tool for biomechanics and medical experts. Traditional observational gait analysis is cost-effective but lacks reliability and accuracy, while instrumented gait analysis, particularly using marker-based optical…
▽ More
The human gait is a complex interplay between the neuronal and the muscular systems, reflecting an individual's neurological and physiological condition. This makes gait analysis a valuable tool for biomechanics and medical experts. Traditional observational gait analysis is cost-effective but lacks reliability and accuracy, while instrumented gait analysis, particularly using marker-based optical systems, provides accurate data but is expensive and time-consuming. In this paper, we introduce a novel markerless approach for gait analysis using a multi-camera setup with smart edge sensors to estimate 3D body poses without fiducial markers. We propose a Siamese embedding network with triplet loss calculation to identify individuals by their gait pattern. This network effectively maps gait sequences to an embedding space that enables clustering sequences from the same individual or activity closely together while separating those of different ones. Our results demonstrate the potential of the proposed system for efficient automated gait analysis in diverse real-world environments, facilitating a wide range of applications.
△ Less
Submitted 14 November, 2024;
originally announced November 2024.
-
On the phase space of fourth-order fiber-orientation tensors
Authors:
Julian Karl Bauer,
Matti Schneider,
Thomas Böhlke
Abstract:
Fiber-orientation tensors describe the relevant features of the fiber-orientation distribution compactly and are thus ubiquitous in injection-molding simulations and subsequent mechanical analyses. In engineering applications to date, the second-order fiber-orientation tensor is the basic quantity of interest, and the fourth-order fiber-orientation tensor is obtained via a closure approximation. U…
▽ More
Fiber-orientation tensors describe the relevant features of the fiber-orientation distribution compactly and are thus ubiquitous in injection-molding simulations and subsequent mechanical analyses. In engineering applications to date, the second-order fiber-orientation tensor is the basic quantity of interest, and the fourth-order fiber-orientation tensor is obtained via a closure approximation. Unfortunately, such a description limits the predictive capabilities of the modeling process significantly, because the wealth of possible fourth-order fiber-orientation tensors is not exploited by such closures, and the restriction to second-order fiber-orientation tensors implies artifacts. Closures based on the second-order fiber-orientation tensor face a fundamental problem - which fourth-order fiber-orientation tensors can be realized? In the literature, only necessary conditions for a fiber-orientation tensor to be connected to a fiber-orientation distribution are found. In this article, we show that the typically considered necessary conditions, positive semidefiniteness and a trace condition, are also sufficient for being a fourth-order fiber-orientation tensor in the physically relevant case of two and three spatial dimensions. Moreover, we show that these conditions are not sufficient in higher dimensions. The argument is based on convex duality and a celebrated theorem of D. Hilbert (1888) on the decomposability of positive and homogeneous polynomials of degree four. The result has numerous implications for modeling the flow and the resulting microstructures of fiber-reinforced composites, in particular for the effective elastic constants of such materials.
△ Less
Submitted 15 November, 2022;
originally announced November 2022.
