-
Breaking Dataset Boundaries: Class-Agnostic Targeted Adversarial Attacks
Authors:
Taïga Gonçalves,
Tomo Miyazaki,
Shinichiro Omachi
Abstract:
We present Cross-Domain Multi-Targeted Attack (CD-MTA), a method for generating adversarial examples that mislead image classifiers toward any target class, including those not seen during training. Traditional targeted attacks are limited to one class per model, requiring expensive retraining for each target. Multi-targeted attacks address this by introducing a perturbation generator with a condi…
▽ More
We present Cross-Domain Multi-Targeted Attack (CD-MTA), a method for generating adversarial examples that mislead image classifiers toward any target class, including those not seen during training. Traditional targeted attacks are limited to one class per model, requiring expensive retraining for each target. Multi-targeted attacks address this by introducing a perturbation generator with a conditional input to specify the target class. However, existing methods are constrained to classes observed during training and require access to the black-box model's training data--introducing a form of data leakage that undermines realistic evaluation in practical black-box scenarios. We identify overreliance on class embeddings as a key limitation, leading to overfitting and poor generalization to unseen classes. To address this, CD-MTA replaces class-level supervision with an image-based conditional input and introduces class-agnostic losses that align the perturbed and target images in the feature space. This design removes dependence on class semantics, thereby enabling generalization to unseen classes across datasets. Experiments on ImageNet and seven other datasets show that CD-MTA outperforms prior multi-targeted attacks in both standard and cross-domain settings--without accessing the black-box model's training data.
△ Less
Submitted 27 May, 2025;
originally announced May 2025.
-
Classification of Keratitis from Eye Corneal Photographs using Deep Learning
Authors:
Maria Miguel Beirão,
João Matos,
Tiago Gonçalves,
Camila Kase,
Luis Filipe Nakayama,
Denise de Freitas,
Jaime S. Cardoso
Abstract:
Keratitis is an inflammatory corneal condition responsible for 10% of visual impairment in low- and middle-income countries (LMICs), with bacteria, fungi, or amoeba as the most common infection etiologies. While an accurate and timely diagnosis is crucial for the selected treatment and the patients' sight outcomes, due to the high cost and limited availability of laboratory diagnostics in LMICs, d…
▽ More
Keratitis is an inflammatory corneal condition responsible for 10% of visual impairment in low- and middle-income countries (LMICs), with bacteria, fungi, or amoeba as the most common infection etiologies. While an accurate and timely diagnosis is crucial for the selected treatment and the patients' sight outcomes, due to the high cost and limited availability of laboratory diagnostics in LMICs, diagnosis is often made by clinical observation alone, despite its lower accuracy. In this study, we investigate and compare different deep learning approaches to diagnose the source of infection: 1) three separate binary models for infection type predictions; 2) a multitask model with a shared backbone and three parallel classification layers (Multitask V1); and, 3) a multitask model with a shared backbone and a multi-head classification layer (Multitask V2). We used a private Brazilian cornea dataset to conduct the empirical evaluation. We achieved the best results with Multitask V2, with an area under the receiver operating characteristic curve (AUROC) confidence intervals of 0.7413-0.7740 (bacteria), 0.8395-0.8725 (fungi), and 0.9448-0.9616 (amoeba). A statistical analysis of the impact of patient features on models' performance revealed that sex significantly affects amoeba infection prediction, and age seems to affect fungi and bacteria predictions.
△ Less
Submitted 13 November, 2024;
originally announced November 2024.
-
Evaluating the Impact of Pulse Oximetry Bias in Machine Learning under Counterfactual Thinking
Authors:
Inês Martins,
João Matos,
Tiago Gonçalves,
Leo A. Celi,
A. Ian Wong,
Jaime S. Cardoso
Abstract:
Algorithmic bias in healthcare mirrors existing data biases. However, the factors driving unfairness are not always known. Medical devices capture significant amounts of data but are prone to errors; for instance, pulse oximeters overestimate the arterial oxygen saturation of darker-skinned individuals, leading to worse outcomes. The impact of this bias in machine learning (ML) models remains uncl…
▽ More
Algorithmic bias in healthcare mirrors existing data biases. However, the factors driving unfairness are not always known. Medical devices capture significant amounts of data but are prone to errors; for instance, pulse oximeters overestimate the arterial oxygen saturation of darker-skinned individuals, leading to worse outcomes. The impact of this bias in machine learning (ML) models remains unclear. This study addresses the technical challenges of quantifying the impact of medical device bias in downstream ML. Our experiments compare a "perfect world", without pulse oximetry bias, using SaO2 (blood-gas), to the "actual world", with biased measurements, using SpO2 (pulse oximetry). Under this counterfactual design, two models are trained with identical data, features, and settings, except for the method of measuring oxygen saturation: models using SaO2 are a "control" and models using SpO2 a "treatment". The blood-gas oximetry linked dataset was a suitable test-bed, containing 163,396 nearly-simultaneous SpO2 - SaO2 paired measurements, aligned with a wide array of clinical features and outcomes. We studied three classification tasks: in-hospital mortality, respiratory SOFA score in the next 24 hours, and SOFA score increase by two points. Models using SaO2 instead of SpO2 generally showed better performance. Patients with overestimation of O2 by pulse oximetry of > 3% had significant decreases in mortality prediction recall, from 0.63 to 0.59, P < 0.001. This mirrors clinical processes where biased pulse oximetry readings provide clinicians with false reassurance of patients' oxygen levels. A similar degradation happened in ML models, with pulse oximetry biases leading to more false negatives in predicting adverse outcomes.
△ Less
Submitted 8 August, 2024;
originally announced August 2024.
-
Deep Learning-based Prediction of Breast Cancer Tumor and Immune Phenotypes from Histopathology
Authors:
Tiago Gonçalves,
Dagoberto Pulido-Arias,
Julian Willett,
Katharina V. Hoebel,
Mason Cleveland,
Syed Rakin Ahmed,
Elizabeth Gerstner,
Jayashree Kalpathy-Cramer,
Jaime S. Cardoso,
Christopher P. Bridge,
Albert E. Kim
Abstract:
The interactions between tumor cells and the tumor microenvironment (TME) dictate therapeutic efficacy of radiation and many systemic therapies in breast cancer. However, to date, there is not a widely available method to reproducibly measure tumor and immune phenotypes for each patient's tumor. Given this unmet clinical need, we applied multiple instance learning (MIL) algorithms to assess activi…
▽ More
The interactions between tumor cells and the tumor microenvironment (TME) dictate therapeutic efficacy of radiation and many systemic therapies in breast cancer. However, to date, there is not a widely available method to reproducibly measure tumor and immune phenotypes for each patient's tumor. Given this unmet clinical need, we applied multiple instance learning (MIL) algorithms to assess activity of ten biologically relevant pathways from the hematoxylin and eosin (H&E) slide of primary breast tumors. We employed different feature extraction approaches and state-of-the-art model architectures. Using binary classification, our models attained area under the receiver operating characteristic (AUROC) scores above 0.70 for nearly all gene expression pathways and on some cases, exceeded 0.80. Attention maps suggest that our trained models recognize biologically relevant spatial patterns of cell sub-populations from H&E. These efforts represent a first step towards developing computational H&E biomarkers that reflect facets of the TME and hold promise for augmenting precision oncology.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
Massively Annotated Datasets for Assessment of Synthetic and Real Data in Face Recognition
Authors:
Pedro C. Neto,
Rafael M. Mamede,
Carolina Albuquerque,
Tiago Gonçalves,
Ana F. Sequeira
Abstract:
Face recognition applications have grown in parallel with the size of datasets, complexity of deep learning models and computational power. However, while deep learning models evolve to become more capable and computational power keeps increasing, the datasets available are being retracted and removed from public access. Privacy and ethical concerns are relevant topics within these domains. Throug…
▽ More
Face recognition applications have grown in parallel with the size of datasets, complexity of deep learning models and computational power. However, while deep learning models evolve to become more capable and computational power keeps increasing, the datasets available are being retracted and removed from public access. Privacy and ethical concerns are relevant topics within these domains. Through generative artificial intelligence, researchers have put efforts into the development of completely synthetic datasets that can be used to train face recognition systems. Nonetheless, the recent advances have not been sufficient to achieve performance comparable to the state-of-the-art models trained on real data. To study the drift between the performance of models trained on real and synthetic datasets, we leverage a massive attribute classifier (MAC) to create annotations for four datasets: two real and two synthetic. From these annotations, we conduct studies on the distribution of each attribute within all four datasets. Additionally, we further inspect the differences between real and synthetic datasets on the attribute set. When comparing through the Kullback-Leibler divergence we have found differences between real and synthetic samples. Interestingly enough, we have verified that while real samples suffice to explain the synthetic distribution, the opposite could not be further from being true.
△ Less
Submitted 23 April, 2024;
originally announced April 2024.
-
Unveiling the Two-Faced Truth: Disentangling Morphed Identities for Face Morphing Detection
Authors:
Eduarda Caldeira,
Pedro C. Neto,
Tiago Gonçalves,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso
Abstract:
Morphing attacks keep threatening biometric systems, especially face recognition systems. Over time they have become simpler to perform and more realistic, as such, the usage of deep learning systems to detect these attacks has grown. At the same time, there is a constant concern regarding the lack of interpretability of deep learning models. Balancing performance and interpretability has been a d…
▽ More
Morphing attacks keep threatening biometric systems, especially face recognition systems. Over time they have become simpler to perform and more realistic, as such, the usage of deep learning systems to detect these attacks has grown. At the same time, there is a constant concern regarding the lack of interpretability of deep learning models. Balancing performance and interpretability has been a difficult task for scientists. However, by leveraging domain information and proving some constraints, we have been able to develop IDistill, an interpretable method with state-of-the-art performance that provides information on both the identity separation on morph samples and their contribution to the final prediction. The domain information is learnt by an autoencoder and distilled to a classifier system in order to teach it to separate identity information. When compared to other methods in the literature it outperforms them in three out of five databases and is competitive in the remaining.
△ Less
Submitted 5 June, 2023;
originally announced June 2023.
-
Causality-Inspired Taxonomy for Explainable Artificial Intelligence
Authors:
Pedro C. Neto,
Tiago Gonçalves,
João Ribeiro Pinto,
Wilson Silva,
Ana F. Sequeira,
Arun Ross,
Jaime S. Cardoso
Abstract:
As two sides of the same coin, causality and explainable artificial intelligence (xAI) were initially proposed and developed with different goals. However, the latter can only be complete when seen through the lens of the causality framework. As such, we propose a novel causality-inspired framework for xAI that creates an environment for the development of xAI approaches. To show its applicability…
▽ More
As two sides of the same coin, causality and explainable artificial intelligence (xAI) were initially proposed and developed with different goals. However, the latter can only be complete when seen through the lens of the causality framework. As such, we propose a novel causality-inspired framework for xAI that creates an environment for the development of xAI approaches. To show its applicability, biometrics was used as case study. For this, we have analysed 81 research papers on a myriad of biometric modalities and different tasks. We have categorised each of these methods according to our novel xAI Ladder and discussed the future directions of the field.
△ Less
Submitted 4 March, 2024; v1 submitted 19 August, 2022;
originally announced August 2022.
-
OrthoMAD: Morphing Attack Detection Through Orthogonal Identity Disentanglement
Authors:
Pedro C. Neto,
Tiago Gonçalves,
Marco Huber,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso
Abstract:
Morphing attacks are one of the many threats that are constantly affecting deep face recognition systems. It consists of selecting two faces from different individuals and fusing them into a final image that contains the identity information of both. In this work, we propose a novel regularisation term that takes into account the existent identity information in both and promotes the creation of t…
▽ More
Morphing attacks are one of the many threats that are constantly affecting deep face recognition systems. It consists of selecting two faces from different individuals and fusing them into a final image that contains the identity information of both. In this work, we propose a novel regularisation term that takes into account the existent identity information in both and promotes the creation of two orthogonal latent vectors. We evaluate our proposed method (OrthoMAD) in five different types of morphing in the FRLL dataset and evaluate the performance of our model when trained on five distinct datasets. With a small ResNet-18 as the backbone, we achieve state-of-the-art results in the majority of the experiments, and competitive results in the others. The code of this paper will be publicly available.
△ Less
Submitted 23 August, 2022; v1 submitted 16 August, 2022;
originally announced August 2022.
-
SYN-MAD 2022: Competition on Face Morphing Attack Detection Based on Privacy-aware Synthetic Training Data
Authors:
Marco Huber,
Fadi Boutros,
Anh Thi Luu,
Kiran Raja,
Raghavendra Ramachandra,
Naser Damer,
Pedro C. Neto,
Tiago Gonçalves,
Ana F. Sequeira,
Jaime S. Cardoso,
João Tremoço,
Miguel Lourenço,
Sergio Serra,
Eduardo Cermeño,
Marija Ivanovska,
Borut Batagelj,
Andrej Kronovšek,
Peter Peer,
Vitomir Štruc
Abstract:
This paper presents a summary of the Competition on Face Morphing Attack Detection Based on Privacy-aware Synthetic Training Data (SYN-MAD) held at the 2022 International Joint Conference on Biometrics (IJCB 2022). The competition attracted a total of 12 participating teams, both from academia and industry and present in 11 different countries. In the end, seven valid submissions were submitted by…
▽ More
This paper presents a summary of the Competition on Face Morphing Attack Detection Based on Privacy-aware Synthetic Training Data (SYN-MAD) held at the 2022 International Joint Conference on Biometrics (IJCB 2022). The competition attracted a total of 12 participating teams, both from academia and industry and present in 11 different countries. In the end, seven valid submissions were submitted by the participating teams and evaluated by the organizers. The competition was held to present and attract solutions that deal with detecting face morphing attacks while protecting people's privacy for ethical and legal reasons. To ensure this, the training data was limited to synthetic data provided by the organizers. The submitted solutions presented innovations that led to outperforming the considered baseline in many experimental settings. The evaluation benchmark is now available at: https://github.com/marcohuber/SYN-MAD-2022.
△ Less
Submitted 15 August, 2022;
originally announced August 2022.
-
A survey on attention mechanisms for medical applications: are we moving towards better algorithms?
Authors:
Tiago Gonçalves,
Isabel Rio-Torto,
Luís F. Teixeira,
Jaime S. Cardoso
Abstract:
The increasing popularity of attention mechanisms in deep learning algorithms for computer vision and natural language processing made these models attractive to other research domains. In healthcare, there is a strong need for tools that may improve the routines of the clinicians and the patients. Naturally, the use of attention-based algorithms for medical applications occurred smoothly. However…
▽ More
The increasing popularity of attention mechanisms in deep learning algorithms for computer vision and natural language processing made these models attractive to other research domains. In healthcare, there is a strong need for tools that may improve the routines of the clinicians and the patients. Naturally, the use of attention-based algorithms for medical applications occurred smoothly. However, being healthcare a domain that depends on high-stake decisions, the scientific community must ponder if these high-performing algorithms fit the needs of medical applications. With this motto, this paper extensively reviews the use of attention mechanisms in machine learning (including Transformers) for several medical applications. This work distinguishes itself from its predecessors by proposing a critical analysis of the claims and potentialities of attention mechanisms presented in the literature through an experimental case study on medical image classification with three different use cases. These experiments focus on the integrating process of attention mechanisms into established deep learning architectures, the analysis of their predictive power, and a visual assessment of their saliency maps generated by post-hoc explanation methods. This paper concludes with a critical analysis of the claims and potentialities presented in the literature about attention mechanisms and proposes future research lines in medical applications that may benefit from these frameworks.
△ Less
Submitted 26 April, 2022;
originally announced April 2022.
-
GuideBP: Guiding Backpropagation Through Weaker Pathways of Parallel Logits
Authors:
Bodhisatwa Mandal,
Swarnendu Ghosh,
Teresa Gonçalves,
Paulo Quaresma,
Mita Nasipuri,
Nibaran Das
Abstract:
Convolutional neural networks often generate multiple logits and use simple techniques like addition or averaging for loss computation. But this allows gradients to be distributed equally among all paths. The proposed approach guides the gradients of backpropagation along weakest concept representations. A weakness scores defines the class specific performance of individual pathways which is then…
▽ More
Convolutional neural networks often generate multiple logits and use simple techniques like addition or averaging for loss computation. But this allows gradients to be distributed equally among all paths. The proposed approach guides the gradients of backpropagation along weakest concept representations. A weakness scores defines the class specific performance of individual pathways which is then used to create a logit that would guide gradients along the weakest pathways. The proposed approach has been shown to perform better than traditional column merging techniques and can be used in several application scenarios. Not only can the proposed model be used as an efficient technique for training multiple instances of a model parallely, but also CNNs with multiple output branches have been shown to perform better with the proposed upgrade. Various experiments establish the flexibility of the learning technique which is simple yet effective in various multi-objective scenarios both empirically and statistically.
△ Less
Submitted 23 April, 2021;
originally announced April 2021.
-
A Requirements Engineering Technology for the IoT Software Systems
Authors:
Danyllo Valente da Silva,
Bruno Pedraça de Souza,
Taisa Guidini Gonçalves,
Guilherme Horta Travassos
Abstract:
Contemporary software systems (CSS), such as the internet of things (IoT) based software systems, incorporate new concerns and characteristics inherent to the network, software, hardware, context awareness, interoperability, and others, compared to conventional software systems. In this sense, requirements engineering (RE) plays a fundamental role in ensuring these software systems' correct develo…
▽ More
Contemporary software systems (CSS), such as the internet of things (IoT) based software systems, incorporate new concerns and characteristics inherent to the network, software, hardware, context awareness, interoperability, and others, compared to conventional software systems. In this sense, requirements engineering (RE) plays a fundamental role in ensuring these software systems' correct development looking for the business and end-user needs. Several software technologies supporting RE are available in the literature, but many do not cover all CSS specificities, notably those based on IoT. This research article presents RETIoT (Requirements Engineering Technology for the Internet of Things based software systems), aiming to provide methodological, technical, and tooling support to produce IoT software system requirements document. It is composed of an IoT scenario description technique, a checklist to verify IoT scenarios, construction processes, and templates for IoT software systems. A feasibility study was carried out in IoT system projects to observe its templates and identify improvement opportunities. The results indicate the feasibility of RETIoT templates' when used to capture IoT characteristics. However, further experimental studies represent research opportunities, strengthen confidence in its elements (construction process, techniques, and templates), and capture end-user perception.
△ Less
Submitted 26 March, 2021;
originally announced March 2021.
-
From Textual Information Sources to Linked Data in the Agatha Project
Authors:
Paulo Quaresma,
Vitor Beires Nogueira,
Kashyap Raiyani,
Roy Bayot,
Teresa Gonçalves
Abstract:
Automatic reasoning about textual information is a challenging task in modern Natural Language Processing (NLP) systems. In this work we describe our proposal for representing and reasoning about Portuguese documents by means of Linked Data like ontologies and thesauri. Our approach resorts to a specialized pipeline of natural language processing (part-of-speech tagger, named entity recognition, s…
▽ More
Automatic reasoning about textual information is a challenging task in modern Natural Language Processing (NLP) systems. In this work we describe our proposal for representing and reasoning about Portuguese documents by means of Linked Data like ontologies and thesauri. Our approach resorts to a specialized pipeline of natural language processing (part-of-speech tagger, named entity recognition, semantic role labeling) to populate an ontology for the domain of criminal investigations. The provided architecture and ontology are language independent. Although some of the NLP modules are language dependent, they can be built using adequate AI methodologies.
△ Less
Submitted 3 September, 2019;
originally announced September 2019.
