Showing 1–12 of 12 results for author: Biswas, M R

An Annotated Corpus of Arabic Tweets for Hate Speech Analysis

Authors: Wajdi Zaghouani, Md. Rafiul Biswas

Abstract: Identifying hate speech content in the Arabic language is challenging due to the rich quality of dialectal variations. This study introduces a multilabel hate speech dataset in the Arabic language. We have collected 10000 Arabic tweets and annotated each tweet, whether it contains offensive content or not. If a text contains offensive content, we further classify it into different hate speech targ… ▽ More Identifying hate speech content in the Arabic language is challenging due to the rich quality of dialectal variations. This study introduces a multilabel hate speech dataset in the Arabic language. We have collected 10000 Arabic tweets and annotated each tweet, whether it contains offensive content or not. If a text contains offensive content, we further classify it into different hate speech targets such as religion, gender, politics, ethnicity, origin, and others. A text can contain either single or multiple targets. Multiple annotators are involved in the data annotation task. We calculated the inter-annotator agreement, which was reported to be 0.86 for offensive content and 0.71 for multiple hate speech targets. Finally, we evaluated the data annotation task by employing a different transformers-based model in which AraBERTv2 outperformed with a micro-F1 score of 0.7865 and an accuracy of 0.786. △ Less

Submitted 22 May, 2025; v1 submitted 17 May, 2025; originally announced May 2025.

EmoHopeSpeech: An Annotated Dataset of Emotions and Hope Speech in English and Arabic

Authors: Wajdi Zaghouani, Md. Rafiul Biswas

Abstract: This research introduces a bilingual dataset comprising 23,456 entries for Arabic and 10,036 entries for English, annotated for emotions and hope speech, addressing the scarcity of multi-emotion (Emotion and hope) datasets. The dataset provides comprehensive annotations capturing emotion intensity, complexity, and causes, alongside detailed classifications and subcategories for hope speech. To ens… ▽ More This research introduces a bilingual dataset comprising 23,456 entries for Arabic and 10,036 entries for English, annotated for emotions and hope speech, addressing the scarcity of multi-emotion (Emotion and hope) datasets. The dataset provides comprehensive annotations capturing emotion intensity, complexity, and causes, alongside detailed classifications and subcategories for hope speech. To ensure annotation reliability, Fleiss' Kappa was employed, revealing 0.75-0.85 agreement among annotators both for Arabic and English language. The evaluation metrics (micro-F1-Score=0.67) obtained from the baseline model (i.e., using a machine learning model) validate that the data annotations are worthy. This dataset offers a valuable resource for advancing natural language processing in underrepresented languages, fostering better cross-linguistic analysis of emotions and hope speech. △ Less

Submitted 21 May, 2025; v1 submitted 17 May, 2025; originally announced May 2025.

Propaganda to Hate: A Multimodal Analysis of Arabic Memes with Multi-Agent LLMs

Authors: Firoj Alam, Md. Rafiul Biswas, Uzair Shah, Wajdi Zaghouani, Georgios Mikros

Abstract: In the past decade, social media platforms have been used for information dissemination and consumption. While a major portion of the content is posted to promote citizen journalism and public awareness, some content is posted to mislead users. Among different content types such as text, images, and videos, memes (text overlaid on images) are particularly prevalent and can serve as powerful vehicl… ▽ More In the past decade, social media platforms have been used for information dissemination and consumption. While a major portion of the content is posted to promote citizen journalism and public awareness, some content is posted to mislead users. Among different content types such as text, images, and videos, memes (text overlaid on images) are particularly prevalent and can serve as powerful vehicles for propaganda, hate, and humor. In the current literature, there have been efforts to individually detect such content in memes. However, the study of their intersection is very limited. In this study, we explore the intersection between propaganda and hate in memes using a multi-agent LLM-based approach. We extend the propagandistic meme dataset with coarse and fine-grained hate labels. Our finding suggests that there is an association between propaganda and hate in memes. We provide detailed experimental results that can serve as a baseline for future studies. We will make the experimental resources publicly available to the community (https://github.com/firojalam/propaganda-and-hateful-memes). △ Less

Submitted 6 October, 2024; v1 submitted 11 September, 2024; originally announced September 2024.

Comments: propaganda, hate-speech, disinformation, misinformation, fake news, LLMs, GPT-4, multimodality, multimodal LLMs

MSC Class: 68T50 ACM Class: F.2.2; I.2.7

MemeMind at ArAIEval Shared Task: Spotting Persuasive Spans in Arabic Text with Persuasion Techniques Identification

Authors: Md Rafiul Biswas, Zubair Shah, Wajdi Zaghouani

Abstract: This paper focuses on detecting propagandistic spans and persuasion techniques in Arabic text from tweets and news paragraphs. Each entry in the dataset contains a text sample and corresponding labels that indicate the start and end positions of propaganda techniques within the text. Tokens falling within a labeled span were assigned "B" (Begin) or "I" (Inside), "O", corresponding to the specific… ▽ More This paper focuses on detecting propagandistic spans and persuasion techniques in Arabic text from tweets and news paragraphs. Each entry in the dataset contains a text sample and corresponding labels that indicate the start and end positions of propaganda techniques within the text. Tokens falling within a labeled span were assigned "B" (Begin) or "I" (Inside), "O", corresponding to the specific propaganda technique. Using attention masks, we created uniform lengths for each span and assigned BIO tags to each token based on the provided labels. Then, we used AraBERT-base pre-trained model for Arabic text tokenization and embeddings with a token classification layer to identify propaganda techniques. Our training process involves a two-phase fine-tuning approach. First, we train only the classification layer for a few epochs, followed by full model fine-tuning, updating all parameters. This methodology allows the model to adapt to the specific characteristics of the propaganda detection task while leveraging the knowledge captured by the pre-trained AraBERT model. Our approach achieved an F1 score of 0.2774, securing the 3rd position in the leaderboard of Task 1. △ Less

Submitted 8 August, 2024; originally announced August 2024.

Nullpointer at CheckThat! 2024: Identifying Subjectivity from Multilingual Text Sequence

Authors: Md. Rafiul Biswas, Abrar Tasneem Abir, Wajdi Zaghouani

Abstract: This study addresses a binary classification task to determine whether a text sequence, either a sentence or paragraph, is subjective or objective. The task spans five languages: Arabic, Bulgarian, English, German, and Italian, along with a multilingual category. Our approach involved several key techniques. Initially, we preprocessed the data through parts of speech (POS) tagging, identification… ▽ More This study addresses a binary classification task to determine whether a text sequence, either a sentence or paragraph, is subjective or objective. The task spans five languages: Arabic, Bulgarian, English, German, and Italian, along with a multilingual category. Our approach involved several key techniques. Initially, we preprocessed the data through parts of speech (POS) tagging, identification of question marks, and application of attention masks. We fine-tuned the sentiment-based Transformer model 'MarieAngeA13/Sentiment-Analysis-BERT' on our dataset. Given the imbalance with more objective data, we implemented a custom classifier that assigned greater weight to objective data. Additionally, we translated non-English data into English to maintain consistency across the dataset. Our model achieved notable results, scoring top marks for the multilingual dataset (Macro F1=0.7121) and German (Macro F1=0.7908). It ranked second for Arabic (Macro F1=0.4908) and Bulgarian (Macro F1=0.7169), third for Italian (Macro F1=0.7430), and ninth for English (Macro F1=0.6893). △ Less

Submitted 14 July, 2024; originally announced July 2024.

ArAIEval Shared Task: Propagandistic Techniques Detection in Unimodal and Multimodal Arabic Content

Authors: Maram Hasanain, Md. Arid Hasan, Fatema Ahmed, Reem Suwaileh, Md. Rafiul Biswas, Wajdi Zaghouani, Firoj Alam

Abstract: We present an overview of the second edition of the ArAIEval shared task, organized as part of the ArabicNLP 2024 conference co-located with ACL 2024. In this edition, ArAIEval offers two tasks: (i) detection of propagandistic textual spans with persuasion techniques identification in tweets and news articles, and (ii) distinguishing between propagandistic and non-propagandistic memes. A total of… ▽ More We present an overview of the second edition of the ArAIEval shared task, organized as part of the ArabicNLP 2024 conference co-located with ACL 2024. In this edition, ArAIEval offers two tasks: (i) detection of propagandistic textual spans with persuasion techniques identification in tweets and news articles, and (ii) distinguishing between propagandistic and non-propagandistic memes. A total of 14 teams participated in the final evaluation phase, with 6 and 9 teams participating in Tasks 1 and 2, respectively. Finally, 11 teams submitted system description papers. Across both tasks, we observed that fine-tuning transformer models such as AraBERT was at the core of the majority of the participating systems. We provide a description of the task setup, including a description of the dataset construction and the evaluation setup. We further provide a brief overview of the participating systems. All datasets and evaluation scripts are released to the research community (https://araieval.gitlab.io/). We hope this will enable further research on these important tasks in Arabic. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Comments: propaganda, span detection, disinformation, misinformation, fake news, LLMs, GPT-4, multimodality, multimodal LLMs

MSC Class: 68T50 ACM Class: F.2.2; I.2.7

An Early Investigation into the Utility of Multimodal Large Language Models in Medical Imaging

Authors: Sulaiman Khan, Md. Rafiul Biswas, Alina Murad, Hazrat Ali, Zubair Shah

Abstract: Recent developments in multimodal large language models (MLLMs) have spurred significant interest in their potential applications across various medical imaging domains. On the one hand, there is a temptation to use these generative models to synthesize realistic-looking medical image data, while on the other hand, the ability to identify synthetic image data in a pool of data is also significantl… ▽ More Recent developments in multimodal large language models (MLLMs) have spurred significant interest in their potential applications across various medical imaging domains. On the one hand, there is a temptation to use these generative models to synthesize realistic-looking medical image data, while on the other hand, the ability to identify synthetic image data in a pool of data is also significantly important. In this study, we explore the potential of the Gemini (\textit{gemini-1.0-pro-vision-latest}) and GPT-4V (gpt-4-vision-preview) models for medical image analysis using two modalities of medical image data. Utilizing synthetic and real imaging data, both Gemini AI and GPT-4V are first used to classify real versus synthetic images, followed by an interpretation and analysis of the input images. Experimental results demonstrate that both Gemini and GPT-4 could perform some interpretation of the input images. In this specific experiment, Gemini was able to perform slightly better than the GPT-4V on the classification task. In contrast, responses associated with GPT-4V were mostly generic in nature. Our early investigation presented in this work provides insights into the potential of MLLMs to assist with the classification and interpretation of retinal fundoscopy and lung X-ray images. We also identify key limitations associated with the early investigation study on MLLMs for specialized tasks in medical image analysis. △ Less

Submitted 2 June, 2024; originally announced June 2024.

Comments: Accepted in Fifth IEEE Workshop on Artificial Intelligence for HealthCare, IEEE 25th International Conference on Information Reuse and Integration for Data Science

Pushing Boundaries: Exploring Zero Shot Object Classification with Large Multimodal Models

Authors: Ashhadul Islam, Md. Rafiul Biswas, Wajdi Zaghouani, Samir Brahim Belhaouari, Zubair Shah

Abstract: $ $The synergy of language and vision models has given rise to Large Language and Vision Assistant models (LLVAs), designed to engage users in rich conversational experiences intertwined with image-based queries. These comprehensive multimodal models seamlessly integrate vision encoders with Large Language Models (LLMs), expanding their applications in general-purpose language and visual comprehen… ▽ More $ $The synergy of language and vision models has given rise to Large Language and Vision Assistant models (LLVAs), designed to engage users in rich conversational experiences intertwined with image-based queries. These comprehensive multimodal models seamlessly integrate vision encoders with Large Language Models (LLMs), expanding their applications in general-purpose language and visual comprehension. The advent of Large Multimodal Models (LMMs) heralds a new era in Artificial Intelligence (AI) assistance, extending the horizons of AI utilization. This paper takes a unique perspective on LMMs, exploring their efficacy in performing image classification tasks using tailored prompts designed for specific datasets. We also investigate the LLVAs zero-shot learning capabilities. Our study includes a benchmarking analysis across four diverse datasets: MNIST, Cats Vs. Dogs, Hymnoptera (Ants Vs. Bees), and an unconventional dataset comprising Pox Vs. Non-Pox skin images. The results of our experiments demonstrate the model's remarkable performance, achieving classification accuracies of 85\%, 100\%, 77\%, and 79\% for the respective datasets without any fine-tuning. To bolster our analysis, we assess the model's performance post fine-tuning for specific tasks. In one instance, fine-tuning is conducted over a dataset comprising images of faces of children with and without autism. Prior to fine-tuning, the model demonstrated a test accuracy of 55\%, which significantly improved to 83\% post fine-tuning. These results, coupled with our prior findings, underscore the transformative potential of LLVAs and their versatile applications in real-world scenarios. △ Less

Submitted 29 December, 2023; originally announced January 2024.

Comments: 5 pages,6 figures, 4 tables, Accepted on The International Symposium on Foundation and Large Language Models (FLLM2023)

Journal ref: https://fllm-conference.org/2023/

Potentials of ChatGPT for Annotating Vaccine Related Tweets

Authors: Md. Rafiul Biswas, Farida Mohsen, Zubair Shah, Wajdi Zaghouani

Abstract: This study evaluates ChatGPT's performance in annotating vaccine-related Arabic tweets by comparing its annotations with human annotations. A dataset of 2,100 tweets representing various factors contributing to vaccine hesitancy was examined. Two domain experts annotated the data, with a third resolving conflicts. ChatGPT was then employed to annotate the same dataset using specific prompts for ea… ▽ More This study evaluates ChatGPT's performance in annotating vaccine-related Arabic tweets by comparing its annotations with human annotations. A dataset of 2,100 tweets representing various factors contributing to vaccine hesitancy was examined. Two domain experts annotated the data, with a third resolving conflicts. ChatGPT was then employed to annotate the same dataset using specific prompts for each factor. The ChatGPT annotations were evaluated through zero-shot, one-shot, and few-shot learning tests, with an average accuracy of 82.14%, 83.85%, and 85.57%, respectively. Precision averaged around 86%, minimizing false positives. The average recall and F1-score ranged from 0.74 to 0.80 and 0.65 to 0.93, respectively. AUC for zero-shot, one-shot, and few-shot learning was 0.79, 0.80, and 0.83. In cases of ambiguity, both human annotators and ChatGPT faced challenges. These findings suggest that ChatGPT holds promise as a tool for annotating vaccine-related tweets. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: 6 pages, 5 figures, two tables, accepted on The International Symposium on Foundation and Large Language Models (FLLM2023)

Journal ref: https://fllm-conference.org/2023/

Can ChatGPT be Your Personal Medical Assistant?

Authors: Md. Rafiul Biswas, Ashhadul Islam, Zubair Shah, Wajdi Zaghouani, Samir Brahim Belhaouari

Abstract: The advanced large language model (LLM) ChatGPT has shown its potential in different domains and remains unbeaten due to its characteristics compared to other LLMs. This study aims to evaluate the potential of using a fine-tuned ChatGPT model as a personal medical assistant in the Arabic language. To do so, this study uses publicly available online questions and answering datasets in Arabic langua… ▽ More The advanced large language model (LLM) ChatGPT has shown its potential in different domains and remains unbeaten due to its characteristics compared to other LLMs. This study aims to evaluate the potential of using a fine-tuned ChatGPT model as a personal medical assistant in the Arabic language. To do so, this study uses publicly available online questions and answering datasets in Arabic language. There are almost 430K questions and answers for 20 disease-specific categories. GPT-3.5-turbo model was fine-tuned with a portion of this dataset. The performance of this fine-tuned model was evaluated through automated and human evaluation. The automated evaluations include perplexity, coherence, similarity, and token count. Native Arabic speakers with medical knowledge evaluated the generated text by calculating relevance, accuracy, precision, logic, and originality. The overall result shows that ChatGPT has a bright future in medical assistance. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Comments: 5 pages, 7 figures, two tables, Accepted on The International Symposium on Foundation and Large Language Models (FLLM2023)

Journal ref: The International Symposium on Foundation and Large Language Models (FLLM2023) https://fllm-conference.org/2023/

Influences of social media usage on public attitudes and behavior towards COVID-19 vaccine in the Arab world

Authors: Md. Rafiul Biswas, Hazrat Ali, Raian Ali, Zubair Shah

Abstract: Background: Vaccination programs are effective only when a significant percentage of people are vaccinated. However, vaccine acceptance varies among communities around the world. Social media usage is arguably one of the factors affecting public attitudes towards vaccines. Objective: This study aims to identify if the social media usages factors can be used to predict attitudes and behavior toward… ▽ More Background: Vaccination programs are effective only when a significant percentage of people are vaccinated. However, vaccine acceptance varies among communities around the world. Social media usage is arguably one of the factors affecting public attitudes towards vaccines. Objective: This study aims to identify if the social media usages factors can be used to predict attitudes and behavior towards the COVID-19 vaccines among the people in the Arab world. Methods: An online survey was conducted in the Arab countries and 217 Arab people participated in this study. Logistic regression was applied to identify what demographics and social media usage factors predict public attitudes and behavior towards the COVID-19 vaccines. Results: Of the 217 participants, 56.22% of them were willing to accept the vaccine and 41.47% of them were hesitant. This study shows that none of the social media usages factors were significant enough to predict the actual vaccine acceptance behavior. Whereas the analysis showed few of the social media usage factors can predict public attitudes towards the COVID-19 vaccines. For example, frequent social media users were 2.85 times more likely to agree that the risk of COVID-19 is being exaggerated (OR=2.85, 95% CI=0.86-9.45, p=0.046) than infrequent social media users. Whereas participants having more trust in vaccine information shared by their contacts are less likely to agree that decision-makers have verified that vaccines are safe (OR=0.528, 95% CI= 0.276-1.012, p=0.05). Conclusion: The use of social media and information shared on it may affect public attitudes towards COVID-19 vaccines. Therefore, disseminating correct and validated information about COVID-19 and other vaccines on social media is important for increasing public trust and countering the impact of incorrect and misinformation. △ Less

Submitted 8 June, 2022; v1 submitted 20 January, 2022; originally announced January 2022.

Characterising Communities of Twitter Users Who Posted Vaccines Related Tweets by Information Sources

Authors: Md. Rafiul Biswas

Abstract: Objective: We formed community structure based on web page credibility and we measured the types of information for characterizing communities of tweeter users who posted about tweets related to vaccine. Methods: We performed the experiment on only Twitter data (tweets) regarding vaccine. The duration of data collection was between 17 January 2017 and 14 March 2018. We formulated cluster based on… ▽ More Objective: We formed community structure based on web page credibility and we measured the types of information for characterizing communities of tweeter users who posted about tweets related to vaccine. Methods: We performed the experiment on only Twitter data (tweets) regarding vaccine. The duration of data collection was between 17 January 2017 and 14 March 2018. We formulated cluster based on the information on its contents and sources it resides (i.e., website domains). We only focused the topics which were related to vaccine. To detect the structure and network of community, we applied Louvain community algorithm along with Random walks called Info map method over vaccines related tweeter user. We defined the communities based on various measures derived from the information shared by Twitter users. Representations and visualizations of the communities based on these derived measures help the public health organization to make understand the possible cause of the rejecting the safety and efficacy of vaccines. Results: To analyse people perception over social media, we downloaded 6,591,566 tweets. We distinguished 1,860,662 users who were posting related to vaccine. We applied Louvain community detection algorithm along with DMM values and we found 192 communities. It also produced higher alignment values as long as the number of topics were low. With the increase of topics number, the alignment becomes lower. The total tweets were divided into two parts based on their characterizes. The first category contained 163,148 (57.16%) tweets which were based as evidence and advocacy and the second category contained 6244 (2.19%) tweets which were based on the experiences and opinion. We observed 4548 users were posting about experiential tweets about vaccine and of them 3449 users (75.84%) were posted evidence and advocacy related. △ Less

Submitted 21 March, 2021; originally announced March 2021.

Comments: 16 pages, 8 figures, 1 table

ACM Class: J.4