-
Effectiveness of Counter-Speech against Abusive Content: A Multidimensional Annotation and Classification Study
Authors:
Greta Damo,
Elena Cabrio,
Serena Villata
Abstract:
Counter-speech (CS) is a key strategy for mitigating online Hate Speech (HS), yet defining the criteria to assess its effectiveness remains an open challenge. We propose a novel computational framework for CS effectiveness classification, grounded in social science concepts. Our framework defines six core dimensions - Clarity, Evidence, Emotional Appeal, Rebuttal, Audience Adaptation, and Fairness…
▽ More
Counter-speech (CS) is a key strategy for mitigating online Hate Speech (HS), yet defining the criteria to assess its effectiveness remains an open challenge. We propose a novel computational framework for CS effectiveness classification, grounded in social science concepts. Our framework defines six core dimensions - Clarity, Evidence, Emotional Appeal, Rebuttal, Audience Adaptation, and Fairness - which we use to annotate 4,214 CS instances from two benchmark datasets, resulting in a novel linguistic resource released to the community. In addition, we propose two classification strategies, multi-task and dependency-based, achieving strong results (0.94 and 0.96 average F1 respectively on both expert- and user-written CS), outperforming standard baselines, and revealing strong interdependence among dimensions.
△ Less
Submitted 13 June, 2025;
originally announced June 2025.
-
Merging Embedded Topics with Optimal Transport for Online Topic Modeling on Data Streams
Authors:
Federica Granese,
Benjamin Navet,
Serena Villata,
Charles Bouveyron
Abstract:
Topic modeling is a key component in unsupervised learning, employed to identify topics within a corpus of textual data. The rapid growth of social media generates an ever-growing volume of textual data daily, making online topic modeling methods essential for managing these data streams that continuously arrive over time. This paper introduces a novel approach to online topic modeling named Strea…
▽ More
Topic modeling is a key component in unsupervised learning, employed to identify topics within a corpus of textual data. The rapid growth of social media generates an ever-growing volume of textual data daily, making online topic modeling methods essential for managing these data streams that continuously arrive over time. This paper introduces a novel approach to online topic modeling named StreamETM. This approach builds on the Embedded Topic Model (ETM) to handle data streams by merging models learned on consecutive partial document batches using unbalanced optimal transport. Additionally, an online change point detection algorithm is employed to identify shifts in topics over time, enabling the identification of significant changes in the dynamics of text streams. Numerical experiments on simulated and real-world data show StreamETM outperforming competitors.
△ Less
Submitted 10 April, 2025;
originally announced April 2025.
-
CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures
Authors:
Ekaterina Sviridova,
Anar Yeginbergen,
Ainara Estarrona,
Elena Cabrio,
Serena Villata,
Rodrigo Agerri
Abstract:
Explaining Artificial Intelligence (AI) decisions is a major challenge nowadays in AI, in particular when applied to sensitive scenarios like medicine and law. However, the need to explain the rationale behind decisions is a main issue also for human-based deliberation as it is important to justify \textit{why} a certain decision has been taken. Resident medical doctors for instance are required n…
▽ More
Explaining Artificial Intelligence (AI) decisions is a major challenge nowadays in AI, in particular when applied to sensitive scenarios like medicine and law. However, the need to explain the rationale behind decisions is a main issue also for human-based deliberation as it is important to justify \textit{why} a certain decision has been taken. Resident medical doctors for instance are required not only to provide a (possibly correct) diagnosis, but also to explain how they reached a certain conclusion. Developing new tools to aid residents to train their explanation skills is therefore a central objective of AI in education. In this paper, we follow this direction, and we present, to the best of our knowledge, the first multilingual dataset for Medical Question Answering where correct and incorrect diagnoses for a clinical case are enriched with a natural language explanation written by doctors. These explanations have been manually annotated with argument components (i.e., premise, claim) and argument relations (i.e., attack, support), resulting in the Multilingual CasiMedicos-Arg dataset which consists of 558 clinical cases in four languages (English, Spanish, French, Italian) with explanations, where we annotated 5021 claims, 2313 premises, 2431 support relations, and 1106 attack relations. We conclude by showing how competitive baselines perform over this challenging dataset for the argument mining task.
△ Less
Submitted 8 October, 2024; v1 submitted 7 October, 2024;
originally announced October 2024.
-
Is Safer Better? The Impact of Guardrails on the Argumentative Strength of LLMs in Hate Speech Countering
Authors:
Helena Bonaldi,
Greta Damo,
Nicolás Benjamín Ocampo,
Elena Cabrio,
Serena Villata,
Marco Guerini
Abstract:
The potential effectiveness of counterspeech as a hate speech mitigation strategy is attracting increasing interest in the NLG research community, particularly towards the task of automatically producing it. However, automatically generated responses often lack the argumentative richness which characterises expert-produced counterspeech. In this work, we focus on two aspects of counterspeech gener…
▽ More
The potential effectiveness of counterspeech as a hate speech mitigation strategy is attracting increasing interest in the NLG research community, particularly towards the task of automatically producing it. However, automatically generated responses often lack the argumentative richness which characterises expert-produced counterspeech. In this work, we focus on two aspects of counterspeech generation to produce more cogent responses. First, by investigating the tension between helpfulness and harmlessness of LLMs, we test whether the presence of safety guardrails hinders the quality of the generations. Secondly, we assess whether attacking a specific component of the hate speech results in a more effective argumentative strategy to fight online hate. By conducting an extensive human and automatic evaluation, we show how the presence of safety guardrails can be detrimental also to a task that inherently aims at fostering positive social interactions. Moreover, our results show that attacking a specific component of the hate speech, and in particular its implicit negative stereotype and its hateful parts, leads to higher-quality generations.
△ Less
Submitted 4 October, 2024;
originally announced October 2024.
-
Medical mT5: An Open-Source Multilingual Text-to-Text LLM for The Medical Domain
Authors:
Iker García-Ferrero,
Rodrigo Agerri,
Aitziber Atutxa Salazar,
Elena Cabrio,
Iker de la Iglesia,
Alberto Lavelli,
Bernardo Magnini,
Benjamin Molinet,
Johana Ramirez-Romero,
German Rigau,
Jose Maria Villa-Gonzalez,
Serena Villata,
Andrea Zaninello
Abstract:
Research on language technology for the development of medical applications is currently a hot topic in Natural Language Understanding and Generation. Thus, a number of large language models (LLMs) have recently been adapted to the medical domain, so that they can be used as a tool for mediating in human-AI interaction. While these LLMs display competitive performance on automated medical texts be…
▽ More
Research on language technology for the development of medical applications is currently a hot topic in Natural Language Understanding and Generation. Thus, a number of large language models (LLMs) have recently been adapted to the medical domain, so that they can be used as a tool for mediating in human-AI interaction. While these LLMs display competitive performance on automated medical texts benchmarks, they have been pre-trained and evaluated with a focus on a single language (English mostly). This is particularly true of text-to-text models, which typically require large amounts of domain-specific pre-training data, often not easily accessible for many languages. In this paper, we address these shortcomings by compiling, to the best of our knowledge, the largest multilingual corpus for the medical domain in four languages, namely English, French, Italian and Spanish. This new corpus has been used to train Medical mT5, the first open-source text-to-text multilingual model for the medical domain. Additionally, we present two new evaluation benchmarks for all four languages with the aim of facilitating multilingual research in this domain. A comprehensive evaluation shows that Medical mT5 outperforms both encoders and similarly sized text-to-text models for the Spanish, French, and Italian benchmarks, while being competitive with current state-of-the-art LLMs in English.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Argument Quality Assessment in the Age of Instruction-Following Large Language Models
Authors:
Henning Wachsmuth,
Gabriella Lapesa,
Elena Cabrio,
Anne Lauscher,
Joonsuk Park,
Eva Maria Vecchi,
Serena Villata,
Timon Ziegenbein
Abstract:
The computational treatment of arguments on controversial issues has been subject to extensive NLP research, due to its envisioned impact on opinion formation, decision making, writing education, and the like. A critical task in any such application is the assessment of an argument's quality - but it is also particularly challenging. In this position paper, we start from a brief survey of argument…
▽ More
The computational treatment of arguments on controversial issues has been subject to extensive NLP research, due to its envisioned impact on opinion formation, decision making, writing education, and the like. A critical task in any such application is the assessment of an argument's quality - but it is also particularly challenging. In this position paper, we start from a brief survey of argument quality research, where we identify the diversity of quality notions and the subjectiveness of their perception as the main hurdles towards substantial progress on argument quality assessment. We argue that the capabilities of instruction-following large language models (LLMs) to leverage knowledge across contexts enable a much more reliable assessment. Rather than just fine-tuning LLMs towards leaderboard chasing on assessment tasks, they need to be instructed systematically with argumentation theories and scenarios as well as with ways to solve argument-related problems. We discuss the real-world opportunities and ethical issues emerging thereby.
△ Less
Submitted 24 March, 2024;
originally announced March 2024.
-
A Dataset Independent Set of Baselines for Relation Prediction in Argument Mining
Authors:
Oana Cocarascu,
Elena Cabrio,
Serena Villata,
Francesca Toni
Abstract:
Argument Mining is the research area which aims at extracting argument components and predicting argumentative relations (i.e.,support and attack) from text. In particular, numerous approaches have been proposed in the literature to predict the relations holding between the arguments, and application-specific annotated resources were built for this purpose. Despite the fact that these resources ha…
▽ More
Argument Mining is the research area which aims at extracting argument components and predicting argumentative relations (i.e.,support and attack) from text. In particular, numerous approaches have been proposed in the literature to predict the relations holding between the arguments, and application-specific annotated resources were built for this purpose. Despite the fact that these resources have been created to experiment on the same task, the definition of a single relation prediction method to be successfully applied to a significant portion of these datasets is an open research problem in Argument Mining. This means that none of the methods proposed in the literature can be easily ported from one resource to another. In this paper, we address this problem by proposing a set of dataset independent strong neural baselines which obtain homogeneous results on all the datasets proposed in the literature for the argumentative relation prediction task. Thus, our baselines can be employed by the Argument Mining community to compare more effectively how well a method performs on the argumentative relation prediction task.
△ Less
Submitted 14 February, 2020;
originally announced March 2020.
-
Graph Data on the Web: extend the pivot, don't reinvent the wheel
Authors:
Fabien Gandon,
Franck Michel,
Olivier Corby,
Michel Buffa,
Andrea Tettamanzi,
Catherine Faron Zucker,
Elena Cabrio,
Serena Villata
Abstract:
This article is a collective position paper from the Wimmics research team, expressing our vision of how Web graph data technologies should evolve in the future in order to ensure a high-level of interoperability between the many types of applications that produce and consume graph data. Wimmics stands for Web-Instrumented Man-Machine Interactions, Communities, and Semantics. We are a joint resear…
▽ More
This article is a collective position paper from the Wimmics research team, expressing our vision of how Web graph data technologies should evolve in the future in order to ensure a high-level of interoperability between the many types of applications that produce and consume graph data. Wimmics stands for Web-Instrumented Man-Machine Interactions, Communities, and Semantics. We are a joint research team between INRIA Sophia Antipolis-M{é}diterran{é}e and I3S (CNRS and Universit{é} C{ô}te d'Azur). Our challenge is to bridge formal semantics and social semantics on the web. Our research areas are graph-oriented knowledge representation, reasoning and operationalization to model and support actors, actions and interactions in web-based epistemic communities. The application of our research is supporting and fostering interactions in online communities and management of their resources. In this position paper, we emphasize the need to extend the semantic Web standard stack to address and fulfill new graph data needs, as well as the importance of remaining compatible with existing recommendations, in particular the RDF stack, to avoid the painful duplication of models, languages, frameworks, etc. The following sections group motivations for different directions of work and collect reasons for the creation of a working group on RDF 2.0 and other recommendations of the RDF family.
△ Less
Submitted 11 March, 2019;
originally announced March 2019.
-
Prioritized Norms in Formal Argumentation
Authors:
Beishui Liao,
Nir Oren,
Leendert van der Torre,
Serena Villata
Abstract:
To resolve conflicts among norms, various nonmonotonic formalisms can be used to perform prioritized normative reasoning. Meanwhile, formal argumentation provides a way to represent nonmonotonic logics. In this paper, we propose a representation of prioritized normative reasoning by argumentation. Using hierarchical abstract normative systems, we define three kinds of prioritized normative reasoni…
▽ More
To resolve conflicts among norms, various nonmonotonic formalisms can be used to perform prioritized normative reasoning. Meanwhile, formal argumentation provides a way to represent nonmonotonic logics. In this paper, we propose a representation of prioritized normative reasoning by argumentation. Using hierarchical abstract normative systems, we define three kinds of prioritized normative reasoning approaches, called Greedy, Reduction, and Optimization. Then, after formulating an argumentation theory for a hierarchical abstract normative system, we show that for a totally ordered hierarchical abstract normative system, Greedy and Reduction can be represented in argumentation by applying the weakest link and the last link principles respectively, and Optimization can be represented by introducing additional defeats capturing the idea that for each argument that contains a norm not belonging to the maximal obeyable set then this argument should be rejected.
△ Less
Submitted 28 February, 2018; v1 submitted 23 September, 2017;
originally announced September 2017.
-
System Descriptions of the First International Competition on Computational Models of Argumentation (ICCMA'15)
Authors:
Matthias Thimm,
Serena Villata
Abstract:
This volume contains the system description of the 18 solvers submitted to the First International Competition on Computational Models of Argumentation (ICCMA'15) and therefore gives an overview on state-of-the-art of computational approaches to abstract argumentation problems. Further information on the results of the competition and the performance of the individual solvers can be found on at ht…
▽ More
This volume contains the system description of the 18 solvers submitted to the First International Competition on Computational Models of Argumentation (ICCMA'15) and therefore gives an overview on state-of-the-art of computational approaches to abstract argumentation problems. Further information on the results of the competition and the performance of the individual solvers can be found on at http://argumentationcompetition.org/2015/.
△ Less
Submitted 19 October, 2015;
originally announced October 2015.
-
Challenges in Bridging Social Semantics and Formal Semantics on the Web
Authors:
Fabien Lucien Gandon,
Michel Buffa,
Elena Cabrio,
Catherine Faron-Zucker,
Alain Giboin,
Nhan Le Thanh,
Isabelle Mirbel,
Peter Sander,
Andrea G. B. Tettamanzi,
Serena Villata
Abstract:
This paper describes several results of Wimmics, a research lab which names stands for: web-instrumented man-machine interactions, communities, and semantics. The approaches introduced here rely on graph-oriented knowledge representation, reasoning and operationalization to model and support actors, actions and interactions in web-based epistemic communities. The re-search results are applied to s…
▽ More
This paper describes several results of Wimmics, a research lab which names stands for: web-instrumented man-machine interactions, communities, and semantics. The approaches introduced here rely on graph-oriented knowledge representation, reasoning and operationalization to model and support actors, actions and interactions in web-based epistemic communities. The re-search results are applied to support and foster interactions in online communities and manage their resources.
△ Less
Submitted 29 August, 2014;
originally announced August 2014.
-
Towards a Benchmark of Natural Language Arguments
Authors:
Elena Cabrio,
Serena Villata
Abstract:
The connections among natural language processing and argumentation theory are becoming stronger in the latest years, with a growing amount of works going in this direction, in different scenarios and applying heterogeneous techniques. In this paper, we present two datasets we built to cope with the combination of the Textual Entailment framework and bipolar abstract argumentation. In our approach…
▽ More
The connections among natural language processing and argumentation theory are becoming stronger in the latest years, with a growing amount of works going in this direction, in different scenarios and applying heterogeneous techniques. In this paper, we present two datasets we built to cope with the combination of the Textual Entailment framework and bipolar abstract argumentation. In our approach, such datasets are used to automatically identify through a Textual Entailment system the relations among the arguments (i.e., attack, support), and then the resulting bipolar argumentation graphs are analyzed to compute the accepted arguments.
△ Less
Submitted 5 May, 2014;
originally announced May 2014.
-
The blazar PKS 0528+134: new results from BeppoSAX observations
Authors:
G. Ghisellini,
L. Costamante,
G. Tagliaferri,
L. Maraschi,
A. Celotti,
G. Fossati,
E. Pian,
A. Comastri,
G. De Francesco,
L. Lanteri,
C. M. Raiteri,
G. Sobrito M. Villata,
I. S. Glass,
P. Grandi,
P. Padovani,
C. Perola,
A. Treves
Abstract:
BeppoSAX observed 8 times the gamma-ray bright blazar PKS 0528+134 in Feb. and Mar. 1997, during a multiwavelength campaign involving EGRET, onboard CGRO, and ground based telescopes. The source was in its faintest X-ray state observed so far, with an unabsorbed [2-10] keV flux of 2.5E-12 erg/cm2/s. The spectrum can be fitted with a power law, with energy index alpha_X=0.49+-0.07 between 0.1 and…
▽ More
BeppoSAX observed 8 times the gamma-ray bright blazar PKS 0528+134 in Feb. and Mar. 1997, during a multiwavelength campaign involving EGRET, onboard CGRO, and ground based telescopes. The source was in its faintest X-ray state observed so far, with an unabsorbed [2-10] keV flux of 2.5E-12 erg/cm2/s. The spectrum can be fitted with a power law, with energy index alpha_X=0.49+-0.07 between 0.1 and 10 keV. The high energy instrument PDS detected a flux in the 15-100 keV band which appears disconnected from the lower X-ray energies, most likely due to a contaminating source. We consider our findings in the context of the overall spectral energy distribution and discuss their implications for synchrotron and inverse Compton models.
△ Less
Submitted 9 June, 1999;
originally announced June 1999.
