-
Uncertainty Propagation in XAI: A Comparison of Analytical and Empirical Estimators
Authors:
Teodor Chiaburu,
Felix Bießmann,
Frank Haußer
Abstract:
Understanding uncertainty in Explainable AI (XAI) is crucial for building trust and ensuring reliable decision-making in Machine Learning models. This paper introduces a unified framework for quantifying and interpreting Uncertainty in XAI by defining a general explanation function $e_θ(x, f)$ that captures the propagation of uncertainty from key sources: perturbations in input data and model para…
▽ More
Understanding uncertainty in Explainable AI (XAI) is crucial for building trust and ensuring reliable decision-making in Machine Learning models. This paper introduces a unified framework for quantifying and interpreting Uncertainty in XAI by defining a general explanation function $e_θ(x, f)$ that captures the propagation of uncertainty from key sources: perturbations in input data and model parameters. By using both analytical and empirical estimates of explanation variance, we provide a systematic means of assessing the impact uncertainty on explanations. We illustrate the approach using a first-order uncertainty propagation as the analytical estimator. In a comprehensive evaluation across heterogeneous datasets, we compare analytical and empirical estimates of uncertainty propagation and evaluate their robustness. Extending previous work on inconsistencies in explanations, our experiments identify XAI methods that do not reliably capture and propagate uncertainty. Our findings underscore the importance of uncertainty-aware explanations in high-stakes applications and offer new insights into the limitations of current XAI methods. The code for the experiments can be found in our repository at https://github.com/TeodorChiaburu/UXAI
△ Less
Submitted 1 April, 2025;
originally announced April 2025.
-
Confident Teacher, Confident Student? A Novel User Study Design for Investigating the Didactic Potential of Explanations and their Impact on Uncertainty
Authors:
Teodor Chiaburu,
Frank Haußer,
Felix Bießmann
Abstract:
Evaluating the quality of explanations in Explainable Artificial Intelligence (XAI) is to this day a challenging problem, with ongoing debate in the research community. While some advocate for establishing standardized offline metrics, others emphasize the importance of human-in-the-loop (HIL) evaluation. Here we propose an experimental design to evaluate the potential of XAI in human-AI collabora…
▽ More
Evaluating the quality of explanations in Explainable Artificial Intelligence (XAI) is to this day a challenging problem, with ongoing debate in the research community. While some advocate for establishing standardized offline metrics, others emphasize the importance of human-in-the-loop (HIL) evaluation. Here we propose an experimental design to evaluate the potential of XAI in human-AI collaborative settings as well as the potential of XAI for didactics. In a user study with 1200 participants we investigate the impact of explanations on human performance on a challenging visual task - annotation of biological species in complex taxonomies. Our results demonstrate the potential of XAI in complex visual annotation tasks: users become more accurate in their annotations and demonstrate less uncertainty with AI assistance. The increase in accuracy was, however, not significantly different when users were shown the mere prediction of the model compared to when also providing an explanation. We also find negative effects of explanations: users tend to replicate the model's predictions more often when shown explanations, even when those predictions are wrong. When evaluating the didactic effects of explanations in collaborative human-AI settings, we find that users' annotations are not significantly better after performing annotation with AI assistance. This suggests that explanations in visual human-AI collaboration do not appear to induce lasting learning effects. All code and experimental data can be found in our GitHub repository: https://github.com/TeodorChiaburu/beexplainable.
△ Less
Submitted 10 September, 2024;
originally announced September 2024.
-
Multisensor Data Fusion for Automatized Insect Monitoring (KInsecta)
Authors:
Martin Tschaikner,
Danja Brandt,
Henning Schmidt,
Felix Bießmann,
Teodor Chiaburu,
Ilona Schrimpf,
Thomas Schrimpf,
Alexandra Stadel,
Frank Haußer,
Ingeborg Beckers
Abstract:
Insect populations are declining globally, making systematic monitoring essential for conservation. Most classical methods involve death traps and counter insect conservation. This paper presents a multisensor approach that uses AI-based data fusion for insect classification. The system is designed as low-cost setup and consists of a camera module and an optical wing beat sensor as well as environ…
▽ More
Insect populations are declining globally, making systematic monitoring essential for conservation. Most classical methods involve death traps and counter insect conservation. This paper presents a multisensor approach that uses AI-based data fusion for insect classification. The system is designed as low-cost setup and consists of a camera module and an optical wing beat sensor as well as environmental sensors to measure temperature, irradiance or daytime as prior information. The system has been tested in the laboratory and in the field. First tests on a small very unbalanced data set with 7 species show promising results for species classification. The multisensor system will support biodiversity and agriculture studies.
△ Less
Submitted 29 April, 2024;
originally announced April 2024.
-
Low Cost Machine Vision for Insect Classification
Authors:
Danja Brandt,
Martin Tschaikner,
Teodor Chiaburu,
Henning Schmidt,
Ilona Schrimpf,
Alexandra Stadel,
Ingeborg E. Beckers,
Frank Haußer
Abstract:
Preserving the number and diversity of insects is one of our society's most important goals in the area of environmental sustainability. A prerequisite for this is a systematic and up-scaled monitoring in order to detect correlations and identify countermeasures. Therefore, automatized monitoring using live traps is important, but so far there is no system that provides image data of sufficient de…
▽ More
Preserving the number and diversity of insects is one of our society's most important goals in the area of environmental sustainability. A prerequisite for this is a systematic and up-scaled monitoring in order to detect correlations and identify countermeasures. Therefore, automatized monitoring using live traps is important, but so far there is no system that provides image data of sufficient detailed information for entomological classification.
In this work, we present an imaging method as part of a multisensor system developed as a low-cost, scalable, open-source system that is adaptable to classical trap types. The image quality meets the requirements needed for classification in the taxonomic tree. Therefore, illumination and resolution have been optimized and motion artefacts have been suppressed. The system is evaluated exemplarily on a dataset consisting of 16 insect species of the same as well as different genus, family and order. We demonstrate that standard CNN-architectures like ResNet50 (pretrained on iNaturalist data) or MobileNet perform very well for the prediction task after re-training. Smaller custom made CNNs also lead to promising results. Classification accuracy of $>96\%$ has been achieved. Moreover, it was proved that image cropping of insects is necessary for classification of species with high inter-class similarity.
△ Less
Submitted 26 April, 2024;
originally announced April 2024.
-
CoProNN: Concept-based Prototypical Nearest Neighbors for Explaining Vision Models
Authors:
Teodor Chiaburu,
Frank Haußer,
Felix Bießmann
Abstract:
Mounting evidence in explainability for artificial intelligence (XAI) research suggests that good explanations should be tailored to individual tasks and should relate to concepts relevant to the task. However, building task specific explanations is time consuming and requires domain expertise which can be difficult to integrate into generic XAI methods. A promising approach towards designing usef…
▽ More
Mounting evidence in explainability for artificial intelligence (XAI) research suggests that good explanations should be tailored to individual tasks and should relate to concepts relevant to the task. However, building task specific explanations is time consuming and requires domain expertise which can be difficult to integrate into generic XAI methods. A promising approach towards designing useful task specific explanations with domain experts is based on compositionality of semantic concepts. Here, we present a novel approach that enables domain experts to quickly create concept-based explanations for computer vision tasks intuitively via natural language. Leveraging recent progress in deep generative methods we propose to generate visual concept-based prototypes via text-to-image methods. These prototypes are then used to explain predictions of computer vision models via a simple k-Nearest-Neighbors routine. The modular design of CoProNN is simple to implement, it is straightforward to adapt to novel tasks and allows for replacing the classification and text-to-image models as more powerful models are released. The approach can be evaluated offline against the ground-truth of predefined prototypes that can be easily communicated also to domain experts as they are based on visual concepts. We show that our strategy competes very well with other concept-based XAI approaches on coarse grained image classification tasks and may even outperform those methods on more demanding fine grained tasks. We demonstrate the effectiveness of our method for human-machine collaboration settings in qualitative and quantitative user studies. All code and experimental data can be found in our GitHub $\href{https://github.com/TeodorChiaburu/beexplainable}{repository}$.
△ Less
Submitted 23 April, 2024;
originally announced April 2024.
-
Towards ML Methods for Biodiversity: A Novel Wild Bee Dataset and Evaluations of XAI Methods for ML-Assisted Rare Species Annotations
Authors:
Teodor Chiaburu,
Felix Biessmann,
Frank Hausser
Abstract:
Insects are a crucial part of our ecosystem. Sadly, in the past few decades, their numbers have worryingly decreased. In an attempt to gain a better understanding of this process and monitor the insects populations, Deep Learning may offer viable solutions. However, given the breadth of their taxonomy and the typical hurdles of fine grained analysis, such as high intraclass variability compared to…
▽ More
Insects are a crucial part of our ecosystem. Sadly, in the past few decades, their numbers have worryingly decreased. In an attempt to gain a better understanding of this process and monitor the insects populations, Deep Learning may offer viable solutions. However, given the breadth of their taxonomy and the typical hurdles of fine grained analysis, such as high intraclass variability compared to low interclass variability, insect classification remains a challenging task. There are few benchmark datasets, which impedes rapid development of better AI models. The annotation of rare species training data, however, requires expert knowledge. Explainable Artificial Intelligence (XAI) could assist biologists in these annotation tasks, but choosing the optimal XAI method is difficult. Our contribution to these research challenges is threefold: 1) a dataset of thoroughly annotated images of wild bees sampled from the iNaturalist database, 2) a ResNet model trained on the wild bee dataset achieving classification scores comparable to similar state-of-the-art models trained on other fine-grained datasets and 3) an investigation of XAI methods to support biologists in annotation tasks.
△ Less
Submitted 15 June, 2022;
originally announced June 2022.
