A Saliency-based Clustering Framework for Identifying Aberrant Predictions
Authors:
Aina Tersol Montserrat,
Alexander R. Loftus,
Yael Daihes
Abstract:
In machine learning, classification tasks serve as the cornerstone of a wide range of real-world applications. Reliable, trustworthy classification is particularly intricate in biomedical settings, where the ground truth is often inherently uncertain and relies on high degrees of human expertise for labeling. Traditional metrics such as precision and recall, while valuable, are insufficient for ca…
▽ More
In machine learning, classification tasks serve as the cornerstone of a wide range of real-world applications. Reliable, trustworthy classification is particularly intricate in biomedical settings, where the ground truth is often inherently uncertain and relies on high degrees of human expertise for labeling. Traditional metrics such as precision and recall, while valuable, are insufficient for capturing the nuances of these ambiguous scenarios. Here we introduce the concept of aberrant predictions, emphasizing that the nature of classification errors is as critical as their frequency. We propose a novel, efficient training methodology aimed at both reducing the misclassification rate and discerning aberrant predictions. Our framework demonstrates a substantial improvement in model performance, achieving a 20\% increase in precision. We apply this methodology to the less-explored domain of veterinary radiology, where the stakes are high but have not been as extensively studied compared to human medicine. By focusing on the identification and mitigation of aberrant predictions, we enhance the utility and trustworthiness of machine learning classifiers in high-stakes, real-world scenarios, including new applications in the veterinary world.
△ Less
Submitted 10 November, 2023;
originally announced November 2023.
MORTON: Detection of Malicious Routines in Large-Scale DNS Traffic
Authors:
Yael Daihes,
Hen Tzaban,
Asaf Nadler,
Asaf Shabtai
Abstract:
In this paper, we present MORTON, a method that identifies compromised devices in enterprise networks based on the existence of routine DNS communication between devices and disreputable host names. With its compact representation of the input data and use of efficient signal processing and a neural network for classification, MORTON is designed to be accurate, robust, and scalable. We evaluate MO…
▽ More
In this paper, we present MORTON, a method that identifies compromised devices in enterprise networks based on the existence of routine DNS communication between devices and disreputable host names. With its compact representation of the input data and use of efficient signal processing and a neural network for classification, MORTON is designed to be accurate, robust, and scalable. We evaluate MORTON using a large dataset of corporate DNS logs and compare it with two recently proposed beaconing detection methods aimed at detecting malware communication. The results demonstrate that while MORTON's accuracy in a synthetic experiment is comparable to that of the other methods, it outperforms those methods in terms of its ability to detect sophisticated bot communication techniques, such as multistage channels, as well as in its robustness and efficiency. In a real-world evaluation, which includes previously unreported threats, MORTON and the two compared methods were deployed to monitor the (unlabeled) DNS traffic of two global enterprises for a week-long period; this evaluation demonstrates the effectiveness of MORTON in real-world scenarios and showcases its superiority in terms of true and false positive rates.
△ Less
Submitted 21 January, 2021; v1 submitted 5 August, 2020;
originally announced August 2020.
