-
Formally Exploring Time-Series Anomaly Detection Evaluation Metrics
Authors:
Dennis Wagner,
Arjun Nair,
Billy Joe Franks,
Justus Arweiler,
Aparna Muraleedharan,
Indra Jungjohann,
Fabian Hartung,
Mayank C. Ahuja,
Andriy Balinskyy,
Saurabh Varshneya,
Nabeel Hussain Syed,
Mayank Nagda,
Phillip Liznerski,
Steffen Reithermann,
Maja Rudolph,
Sebastian Vollmer,
Ralf Schulz,
Torsten Katz,
Stephan Mandt,
Michael Bortz,
Heike Leitte,
Daniel Neider,
Jakob Burger,
Fabian Jirasek,
Hans Hasse
, et al. (2 additional authors not shown)
Abstract:
Undetected anomalies in time series can trigger catastrophic failures in safety-critical systems, such as chemical plant explosions or power grid outages. Although many detection methods have been proposed, their performance remains unclear because current metrics capture only narrow aspects of the task and often yield misleading results. We address this issue by introducing verifiable properties…
▽ More
Undetected anomalies in time series can trigger catastrophic failures in safety-critical systems, such as chemical plant explosions or power grid outages. Although many detection methods have been proposed, their performance remains unclear because current metrics capture only narrow aspects of the task and often yield misleading results. We address this issue by introducing verifiable properties that formalize essential requirements for evaluating time-series anomaly detection. These properties enable a theoretical framework that supports principled evaluations and reliable comparisons. Analyzing 37 widely used metrics, we show that most satisfy only a few properties, and none satisfy all, explaining persistent inconsistencies in prior results. To close this gap, we propose LARM, a flexible metric that provably satisfies all properties, and extend it to ALARM, an advanced variant meeting stricter requirements.
△ Less
Submitted 20 October, 2025;
originally announced October 2025.
-
Interpretable Tensor Fusion
Authors:
Saurabh Varshneya,
Antoine Ledent,
Philipp Liznerski,
Andriy Balinskyy,
Purvanshi Mehta,
Waleed Mustafa,
Marius Kloft
Abstract:
Conventional machine learning methods are predominantly designed to predict outcomes based on a single data type. However, practical applications may encompass data of diverse types, such as text, images, and audio. We introduce interpretable tensor fusion (InTense), a multimodal learning method for training neural networks to simultaneously learn multimodal data representations and their interpre…
▽ More
Conventional machine learning methods are predominantly designed to predict outcomes based on a single data type. However, practical applications may encompass data of diverse types, such as text, images, and audio. We introduce interpretable tensor fusion (InTense), a multimodal learning method for training neural networks to simultaneously learn multimodal data representations and their interpretable fusion. InTense can separately capture both linear combinations and multiplicative interactions of diverse data types, thereby disentangling higher-order interactions from the individual effects of each modality. InTense provides interpretability out of the box by assigning relevance scores to modalities and their associations. The approach is theoretically grounded and yields meaningful relevance scores on multiple synthetic and real-world datasets. Experiments on six real-world datasets show that InTense outperforms existing state-of-the-art multimodal interpretable approaches in terms of accuracy and interpretability.
△ Less
Submitted 7 May, 2024;
originally announced May 2024.
-
Reimagining Anomalies: What If Anomalies Were Normal?
Authors:
Philipp Liznerski,
Saurabh Varshneya,
Ece Calikus,
Sophie Fellenz,
Marius Kloft
Abstract:
Deep learning-based methods have achieved a breakthrough in image anomaly detection, but their complexity introduces a considerable challenge to understanding why an instance is predicted to be anomalous. We introduce a novel explanation method that generates multiple counterfactual examples for each anomaly, capturing diverse concepts of anomalousness. A counterfactual example is a modification o…
▽ More
Deep learning-based methods have achieved a breakthrough in image anomaly detection, but their complexity introduces a considerable challenge to understanding why an instance is predicted to be anomalous. We introduce a novel explanation method that generates multiple counterfactual examples for each anomaly, capturing diverse concepts of anomalousness. A counterfactual example is a modification of the anomaly that is perceived as normal by the anomaly detector. The method provides a high-level semantic explanation of the mechanism that triggered the anomaly detector, allowing users to explore "what-if scenarios." Qualitative and quantitative analyses across various image datasets show that the method applied to state-of-the-art anomaly detectors can achieve high-quality semantic explanations of detectors.
△ Less
Submitted 22 February, 2024;
originally announced February 2024.
-
Learning Interpretable Concept Groups in CNNs
Authors:
Saurabh Varshneya,
Antoine Ledent,
Robert A. Vandermeulen,
Yunwen Lei,
Matthias Enders,
Damian Borth,
Marius Kloft
Abstract:
We propose a novel training methodology -- Concept Group Learning (CGL) -- that encourages training of interpretable CNN filters by partitioning filters in each layer into concept groups, each of which is trained to learn a single visual concept. We achieve this through a novel regularization strategy that forces filters in the same group to be active in similar image regions for a given layer. We…
▽ More
We propose a novel training methodology -- Concept Group Learning (CGL) -- that encourages training of interpretable CNN filters by partitioning filters in each layer into concept groups, each of which is trained to learn a single visual concept. We achieve this through a novel regularization strategy that forces filters in the same group to be active in similar image regions for a given layer. We additionally use a regularizer to encourage a sparse weighting of the concept groups in each layer so that a few concept groups can have greater importance than others. We quantitatively evaluate CGL's model interpretability using standard interpretability evaluation techniques and find that our method increases interpretability scores in most cases. Qualitatively we compare the image regions that are most active under filters learned using CGL versus filters learned without CGL and find that CGL activation regions more strongly concentrate around semantically relevant features.
△ Less
Submitted 21 September, 2021;
originally announced September 2021.
-
Recognizing Challenging Handwritten Annotations with Fully Convolutional Networks
Authors:
Andreas Kölsch,
Ashutosh Mishra,
Saurabh Varshneya,
Muhammad Zeshan Afzal,
Marcus Liwicki
Abstract:
This paper introduces a very challenging dataset of historic German documents and evaluates Fully Convolutional Neural Network (FCNN) based methods to locate handwritten annotations of any kind in these documents. The handwritten annotations can appear in form of underlines and text by using various writing instruments, e.g., the use of pencils makes the data more challenging. We train and evaluat…
▽ More
This paper introduces a very challenging dataset of historic German documents and evaluates Fully Convolutional Neural Network (FCNN) based methods to locate handwritten annotations of any kind in these documents. The handwritten annotations can appear in form of underlines and text by using various writing instruments, e.g., the use of pencils makes the data more challenging. We train and evaluate various end-to-end semantic segmentation approaches and report the results. The task is to classify the pixels of documents into two classes: background and handwritten annotation. The best model achieves a mean Intersection over Union (IoU) score of 95.6% on the test documents of the presented dataset. We also present a comparison of different strategies used for data augmentation and training on our presented dataset. For evaluation, we use the Layout Analysis Evaluator for the ICDAR 2017 Competition on Layout Analysis for Challenging Medieval Manuscripts.
△ Less
Submitted 22 June, 2018; v1 submitted 31 March, 2018;
originally announced April 2018.
-
A Novel Modified Apriori Approach for Web Document Clustering
Authors:
Rajendra Kumar Roul,
Saransh Varshneya,
Ashu Kalra,
Sanjay Kumar Sahay
Abstract:
The traditional apriori algorithm can be used for clustering the web documents based on the association technique of data mining. But this algorithm has several limitations due to repeated database scans and its weak association rule analysis. In modern world of large databases, efficiency of traditional apriori algorithm would reduce manifolds. In this paper, we proposed a new modified apriori ap…
▽ More
The traditional apriori algorithm can be used for clustering the web documents based on the association technique of data mining. But this algorithm has several limitations due to repeated database scans and its weak association rule analysis. In modern world of large databases, efficiency of traditional apriori algorithm would reduce manifolds. In this paper, we proposed a new modified apriori approach by cutting down the repeated database scans and improving association analysis of traditional apriori algorithm to cluster the web documents. Further we improve those clusters by applying Fuzzy C-Means (FCM), K-Means and Vector Space Model (VSM) techniques separately. For experimental purpose, we use Classic3 and Classic4 datasets of Cornell University having more than 10,000 documents and run both traditional apriori and our modified apriori approach on it. Experimental results show that our approach outperforms the traditional apriori algorithm in terms of database scan and improvement on association of analysis. We found out that FCM is better than K-Means and VSM in terms of F-measure of clusters of different sizes.
△ Less
Submitted 29 March, 2015;
originally announced March 2015.
