Showing 1–17 of 17 results for author: Torroni, P

Investigating the Effectiveness of Explainability Methods in Parkinson's Detection from Speech

Authors: Eleonora Mancini, Francesco Paissan, Paolo Torroni, Mirco Ravanelli, Cem Subakan

Abstract: Speech impairments in Parkinson's disease (PD) provide significant early indicators for diagnosis. While models for speech-based PD detection have shown strong performance, their interpretability remains underexplored. This study systematically evaluates several explainability methods to identify PD-specific speech features, aiming to support the development of accurate, interpretable models for c… ▽ More Speech impairments in Parkinson's disease (PD) provide significant early indicators for diagnosis. While models for speech-based PD detection have shown strong performance, their interpretability remains underexplored. This study systematically evaluates several explainability methods to identify PD-specific speech features, aiming to support the development of accurate, interpretable models for clinical decision-making in PD diagnosis and monitoring. Our methodology involves (i) obtaining attributions and saliency maps using mainstream interpretability techniques, (ii) quantitatively evaluating the faithfulness of these maps and their combinations obtained via union and intersection through a range of established metrics, and (iii) assessing the information conveyed by the saliency maps for PD detection from an auxiliary classifier. Our results reveal that, while explanations are aligned with the classifier, they often fail to provide valuable information for domain experts. △ Less

Submitted 13 November, 2024; v1 submitted 12 November, 2024; originally announced November 2024.

Comments: The first two authors contributed equally to this research: author order is alphabetical

A Chatbot for Asylum-Seeking Migrants in Europe

Authors: Bettina Fazzinga, Elena Palmieri, Margherita Vestoso, Luca Bolognini, Andrea Galassi, Filippo Furfaro, Paolo Torroni

Abstract: We present ACME: A Chatbot for asylum-seeking Migrants in Europe. ACME relies on computational argumentation and aims to help migrants identify the highest level of protection they can apply for. This would contribute to a more sustainable migration by reducing the load on territorial commissions, Courts, and humanitarian organizations supporting asylum applicants. We describe the background conte… ▽ More We present ACME: A Chatbot for asylum-seeking Migrants in Europe. ACME relies on computational argumentation and aims to help migrants identify the highest level of protection they can apply for. This would contribute to a more sustainable migration by reducing the load on territorial commissions, Courts, and humanitarian organizations supporting asylum applicants. We describe the background context, system architecture, underlying technologies, and a case study used to validate the tool with domain experts. △ Less

Submitted 31 January, 2025; v1 submitted 12 July, 2024; originally announced July 2024.

Comments: Copyright 2024 IEEE

Journal ref: 2024 IEEE 36th International Conference on Tools with Artificial Intelligence (ICTAI), pp. 702-707

Dynamic Few-Shot Learning for Knowledge Graph Question Answering

Authors: Jacopo D'Abramo, Andrea Zugarini, Paolo Torroni

Abstract: Large language models present opportunities for innovative Question Answering over Knowledge Graphs (KGQA). However, they are not inherently designed for query generation. To bridge this gap, solutions have been proposed that rely on fine-tuning or ad-hoc architectures, achieving good results but limited out-of-domain distribution generalization. In this study, we introduce a novel approach called… ▽ More Large language models present opportunities for innovative Question Answering over Knowledge Graphs (KGQA). However, they are not inherently designed for query generation. To bridge this gap, solutions have been proposed that rely on fine-tuning or ad-hoc architectures, achieving good results but limited out-of-domain distribution generalization. In this study, we introduce a novel approach called Dynamic Few-Shot Learning (DFSL). DFSL integrates the efficiency of in-context learning and semantic similarity and provides a generally applicable solution for KGQA with state-of-the-art performance. We run an extensive evaluation across multiple benchmark datasets and architecture configurations. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodology

Authors: Federico Ruggeri, Eleonora Misino, Arianna Muti, Katerina Korre, Paolo Torroni, Alberto Barrón-Cedeño

Abstract: We introduce the Guideline-Centered Annotation Methodology (GCAM), a novel data annotation methodology designed to report the annotation guidelines associated with each data sample. Our approach addresses three key limitations of the standard prescriptive annotation methodology by reducing the information loss during annotation and ensuring adherence to guidelines. Furthermore, GCAM enables the ef… ▽ More We introduce the Guideline-Centered Annotation Methodology (GCAM), a novel data annotation methodology designed to report the annotation guidelines associated with each data sample. Our approach addresses three key limitations of the standard prescriptive annotation methodology by reducing the information loss during annotation and ensuring adherence to guidelines. Furthermore, GCAM enables the efficient reuse of annotated data across multiple tasks. We evaluate GCAM in two ways: (i) through a human annotation study and (ii) an experimental evaluation with several machine learning models. Our results highlight the advantages of GCAM from multiple perspectives, demonstrating its potential to improve annotation quality and error analysis. △ Less

Submitted 10 December, 2024; v1 submitted 20 June, 2024; originally announced June 2024.

Promoting the Responsible Development of Speech Datasets for Mental Health and Neurological Disorders Research

Authors: Eleonora Mancini, Ana Tanevska, Andrea Galassi, Alessio Galatolo, Federico Ruggeri, Paolo Torroni

Abstract: Current research in machine learning and artificial intelligence is largely centered on modeling and performance evaluation, less so on data collection. However, recent research demonstrated that limitations and biases in data may negatively impact trustworthiness and reliability. These aspects are particularly impactful on sensitive domains such as mental health and neurological disorders, where… ▽ More Current research in machine learning and artificial intelligence is largely centered on modeling and performance evaluation, less so on data collection. However, recent research demonstrated that limitations and biases in data may negatively impact trustworthiness and reliability. These aspects are particularly impactful on sensitive domains such as mental health and neurological disorders, where speech data are used to develop AI applications for patients and healthcare providers. In this paper, we chart the landscape of available speech datasets for this domain, to highlight possible pitfalls and opportunities for improvement and promote fairness and diversity. We present a comprehensive list of desiderata for building speech datasets for mental health and neurological disorders and distill it into an actionable checklist focused on ethical concerns to foster more responsible research. △ Less

Submitted 17 February, 2025; v1 submitted 6 June, 2024; originally announced June 2024.

Comments: 36 pages

Journal ref: Journal of Artificial Intelligence Research (JAIR), vol 82 (2025), pp 937-972

TWOLAR: a TWO-step LLM-Augmented distillation method for passage Reranking

Authors: Davide Baldelli, Junfeng Jiang, Akiko Aizawa, Paolo Torroni

Abstract: In this paper, we present TWOLAR: a two-stage pipeline for passage reranking based on the distillation of knowledge from Large Language Models (LLM). TWOLAR introduces a new scoring strategy and a distillation process consisting in the creation of a novel and diverse training dataset. The dataset consists of 20K queries, each associated with a set of documents retrieved via four distinct retrieval… ▽ More In this paper, we present TWOLAR: a two-stage pipeline for passage reranking based on the distillation of knowledge from Large Language Models (LLM). TWOLAR introduces a new scoring strategy and a distillation process consisting in the creation of a novel and diverse training dataset. The dataset consists of 20K queries, each associated with a set of documents retrieved via four distinct retrieval methods to ensure diversity, and then reranked by exploiting the zero-shot reranking capabilities of an LLM. Our ablation studies demonstrate the contribution of each new component we introduced. Our experimental results show that TWOLAR significantly enhances the document reranking ability of the underlying model, matching and in some cases even outperforming state-of-the-art models with three orders of magnitude more parameters on the TREC-DL test sets and the zero-shot evaluation benchmark BEIR. To facilitate future work we release our data set, finetuned models, and code. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Fast Vocabulary Transfer for Language Model Compression

Authors: Leonidas Gee, Andrea Zugarini, Leonardo Rigutini, Paolo Torroni

Abstract: Real-world business applications require a trade-off between language model performance and size. We propose a new method for model compression that relies on vocabulary transfer. We evaluate the method on various vertical domains and downstream tasks. Our results indicate that vocabulary transfer can be effectively used in combination with other compression techniques, yielding a significant redu… ▽ More Real-world business applications require a trade-off between language model performance and size. We propose a new method for model compression that relies on vocabulary transfer. We evaluate the method on various vertical domains and downstream tasks. Our results indicate that vocabulary transfer can be effectively used in combination with other compression techniques, yielding a significant reduction in model size and inference time while marginally compromising on performance. △ Less

Submitted 15 February, 2024; originally announced February 2024.

Comments: The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022)

Journal ref: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022): Industry Track

Combining Transformers with Natural Language Explanations

Authors: Federico Ruggeri, Marco Lippi, Paolo Torroni

Abstract: Many NLP applications require models to be interpretable. However, many successful neural architectures, including transformers, still lack effective interpretation methods. A possible solution could rely on building explanations from domain knowledge, which is often available as plain, natural language text. We thus propose an extension to transformer models that makes use of external memories to… ▽ More Many NLP applications require models to be interpretable. However, many successful neural architectures, including transformers, still lack effective interpretation methods. A possible solution could rely on building explanations from domain knowledge, which is often available as plain, natural language text. We thus propose an extension to transformer models that makes use of external memories to store natural language explanations and use them to explain classification outputs. We conduct an experimental evaluation on two domains, legal text analysis and argument mining, to show that our approach can produce relevant explanations while retaining or even improving classification performance. △ Less

Submitted 3 April, 2024; v1 submitted 2 September, 2021; originally announced October 2021.

Tree-Constrained Graph Neural Networks For Argument Mining

Authors: Federico Ruggeri, Marco Lippi, Paolo Torroni

Abstract: We propose a novel architecture for Graph Neural Networks that is inspired by the idea behind Tree Kernels of measuring similarity between trees by taking into account their common substructures, named fragments. By imposing a series of regularization constraints to the learning problem, we exploit a pooling mechanism that incorporates such notion of fragments within the node soft assignment funct… ▽ More We propose a novel architecture for Graph Neural Networks that is inspired by the idea behind Tree Kernels of measuring similarity between trees by taking into account their common substructures, named fragments. By imposing a series of regularization constraints to the learning problem, we exploit a pooling mechanism that incorporates such notion of fragments within the node soft assignment function that produces the embeddings. We present an extensive experimental evaluation on a collection of sentence classification tasks conducted on several argument mining corpora, showing that the proposed approach performs well with respect to state-of-the-art techniques. △ Less

Submitted 2 September, 2021; originally announced October 2021.

An Argumentative Dialogue System for COVID-19 Vaccine Information

Authors: Bettina Fazzinga, Andrea Galassi, Paolo Torroni

Abstract: Dialogue systems are widely used in AI to support timely and interactive communication with users. We propose a general-purpose dialogue system architecture that leverages computational argumentation to perform reasoning and provide consistent and explainable answers. We illustrate the system using a COVID-19 vaccine information case study. Dialogue systems are widely used in AI to support timely and interactive communication with users. We propose a general-purpose dialogue system architecture that leverages computational argumentation to perform reasoning and provide consistent and explainable answers. We illustrate the system using a COVID-19 vaccine information case study. △ Less

Submitted 15 October, 2021; v1 submitted 26 July, 2021; originally announced July 2021.

Comments: 9 pages, 2 figures, Accepted at CLAR 2021

ACM Class: I.2.1; I.2.4

Journal ref: Logic and Argumentation (2021). Lecture Notes in Computer Science, vol 13040

Multi-Task Attentive Residual Networks for Argument Mining

Authors: Andrea Galassi, Marco Lippi, Paolo Torroni

Abstract: We explore the use of residual networks and neural attention for multiple argument mining tasks. We propose a residual architecture that exploits attention, multi-task learning, and makes use of ensemble, without any assumption on document or argument structure. We present an extensive experimental evaluation on five different corpora of user-generated comments, scientific publications, and persua… ▽ More We explore the use of residual networks and neural attention for multiple argument mining tasks. We propose a residual architecture that exploits attention, multi-task learning, and makes use of ensemble, without any assumption on document or argument structure. We present an extensive experimental evaluation on five different corpora of user-generated comments, scientific publications, and persuasive essays. Our results show that our approach is a strong competitor against state-of-the-art architectures with a higher computational footprint or corpus-specific design, representing an interesting compromise between generality, performance accuracy and reduced model size. △ Less

Submitted 25 May, 2023; v1 submitted 24 February, 2021; originally announced February 2021.

Comments: 16 pages, 3 figures

Journal ref: IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol 31, pp 1877-1892, 2023

Memory networks for consumer protection:unfairness exposed

Authors: Federico Ruggeri, Francesca Lagioia, Marco Lippi, Paolo Torroni

Abstract: Recent work has demonstrated how data-driven AI methods can leverage consumer protection by supporting the automated analysis of legal documents. However, a shortcoming of data-driven approaches is poor explainability. We posit that in this domain useful explanations of classifier outcomes can be provided by resorting to legal rationales. We thus consider several configurations of memory-augmented… ▽ More Recent work has demonstrated how data-driven AI methods can leverage consumer protection by supporting the automated analysis of legal documents. However, a shortcoming of data-driven approaches is poor explainability. We posit that in this domain useful explanations of classifier outcomes can be provided by resorting to legal rationales. We thus consider several configurations of memory-augmented neural networks where rationales are given a special role in the modeling of context knowledge. Our results show that rationales not only contribute to improve the classification accuracy, but are also able to offer meaningful, natural language explanations of otherwise opaque classifier outcomes. △ Less

Submitted 24 July, 2020; originally announced August 2020.

Parallelizing Machine Learning as a Service for the End-User

Authors: Daniela Loreti, Marco Lippi, Paolo Torroni

Abstract: As ML applications are becoming ever more pervasive, fully-trained systems are made increasingly available to a wide public, allowing end-users to submit queries with their own data, and to efficiently retrieve results. With increasingly sophisticated such services, a new challenge is how to scale up to evergrowing user bases. In this paper, we present a distributed architecture that could be expl… ▽ More As ML applications are becoming ever more pervasive, fully-trained systems are made increasingly available to a wide public, allowing end-users to submit queries with their own data, and to efficiently retrieve results. With increasingly sophisticated such services, a new challenge is how to scale up to evergrowing user bases. In this paper, we present a distributed architecture that could be exploited to parallelize a typical ML system pipeline. We propose a case study consisting of a text mining service and discuss how the method can be generalized to many similar applications. We demonstrate the significance of the computational gain boosted by the distributed architecture by way of an extensive experimental evaluation. △ Less

Submitted 29 May, 2020; v1 submitted 28 May, 2020; originally announced May 2020.

Journal ref: Future Generation Computer Systems 105 (2020) 275-286

Neural-Symbolic Argumentation Mining: an Argument in Favor of Deep Learning and Reasoning

Authors: Andrea Galassi, Kristian Kersting, Marco Lippi, Xiaoting Shao, Paolo Torroni

Abstract: Deep learning is bringing remarkable contributions to the field of argumentation mining, but the existing approaches still need to fill the gap toward performing advanced reasoning tasks. In this position paper, we posit that neural-symbolic and statistical relational learning could play a crucial role in the integration of symbolic and sub-symbolic methods to achieve this goal. Deep learning is bringing remarkable contributions to the field of argumentation mining, but the existing approaches still need to fill the gap toward performing advanced reasoning tasks. In this position paper, we posit that neural-symbolic and statistical relational learning could play a crucial role in the integration of symbolic and sub-symbolic methods to achieve this goal. △ Less

Submitted 28 January, 2020; v1 submitted 22 May, 2019; originally announced May 2019.

Journal ref: Frontiers in Big Data 2 (2020) 52

Attention in Natural Language Processing

Authors: Andrea Galassi, Marco Lippi, Paolo Torroni

Abstract: Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for attention architectures in natural language processing, with a focus on those desig… ▽ More Attention is an increasingly popular mechanism used in a wide range of neural architectures. The mechanism itself has been realized in a variety of formats. However, because of the fast-paced advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for attention architectures in natural language processing, with a focus on those designed to work with vector representations of the textual data. We propose a taxonomy of attention models according to four dimensions: the representation of the input, the compatibility function, the distribution function, and the multiplicity of the input and/or output. We present the examples of how prior information can be exploited in attention models and discuss ongoing research efforts and open challenges in the area, providing the first extensive categorization of the vast body of literature in this exciting domain. △ Less

Submitted 11 October, 2021; v1 submitted 4 February, 2019; originally announced February 2019.

Comments: 18 pages, 8 figures

MSC Class: 68T50; 68T05; 68T07 ACM Class: I.2; I.7

Journal ref: IEEE Transactions on Neural Networks and Learning Systems, vol 32, n 10, pp 4291-4308, 2021

Blocking time under basic priority inheritance: Polynomial bound and exact computation

Authors: Paolo Torroni, Zeynep Kiziltan, Eugenio Faldella

Abstract: The Priority Inheritance Protocol (PIP) is arguably the best-known protocol for resource sharing under real-time constraints. Its importance in modern applications is undisputed. Nevertheless, because jobs may be blocked under PIP for a variety of reasons, determining a job's maximum blocking time could be difficult, and thus far no exact method has been proposed that does it. Existing analysis me… ▽ More The Priority Inheritance Protocol (PIP) is arguably the best-known protocol for resource sharing under real-time constraints. Its importance in modern applications is undisputed. Nevertheless, because jobs may be blocked under PIP for a variety of reasons, determining a job's maximum blocking time could be difficult, and thus far no exact method has been proposed that does it. Existing analysis methods are inefficient, inaccurate, and of limited applicability. This article proposes a new characterization of the problem, thus allowing a polynomial method for bounding the blocking time, and an exact, optimally efficient method for blocking time computation under priority inheritance that have a general applicability. △ Less

Submitted 11 June, 2018; v1 submitted 5 June, 2018; originally announced June 2018.

CLAUDETTE: an Automated Detector of Potentially Unfair Clauses in Online Terms of Service

Authors: Marco Lippi, Przemyslaw Palka, Giuseppe Contissa, Francesca Lagioia, Hans-Wolfgang Micklitz, Giovanni Sartor, Paolo Torroni

Abstract: Terms of service of on-line platforms too often contain clauses that are potentially unfair to the consumer. We present an experimental study where machine learning is employed to automatically detect such potentially unfair clauses. Results show that the proposed system could provide a valuable tool for lawyers and consumers alike. Terms of service of on-line platforms too often contain clauses that are potentially unfair to the consumer. We present an experimental study where machine learning is employed to automatically detect such potentially unfair clauses. Results show that the proposed system could provide a valuable tool for lawyers and consumers alike. △ Less

Submitted 18 February, 2019; v1 submitted 3 May, 2018; originally announced May 2018.

Journal ref: Artif. Intell. Law 27 (2019) 117-139