-
Palm: A Culturally Inclusive and Linguistically Diverse Dataset for Arabic LLMs
Authors:
Fakhraddin Alwajih,
Abdellah El Mekki,
Samar Mohamed Magdy,
Abdelrahim A. Elmadany,
Omer Nacar,
El Moatez Billah Nagoudi,
Reem Abdel-Salam,
Hanin Atwany,
Youssef Nafea,
Abdulfattah Mohammed Yahya,
Rahaf Alhamouri,
Hamzah A. Alsayadi,
Hiba Zayed,
Sara Shatnawi,
Serry Sibaee,
Yasir Ech-Chammakhy,
Walid Al-Dhabyani,
Marwa Mohamed Ali,
Imen Jarraya,
Ahmed Oumar El-Shangiti,
Aisha Alraeesi,
Mohammed Anwar Al-Ghrawi,
Abdulrahman S. Al-Batati,
Elgizouli Mohamed,
Noha Taha Elgindi
, et al. (19 additional authors not shown)
Abstract:
As large language models (LLMs) become increasingly integrated into daily life, ensuring their cultural sensitivity and inclusivity is paramount. We introduce our dataset, a year-long community-driven project covering all 22 Arab countries. The dataset includes instructions (input, response pairs) in both Modern Standard Arabic (MSA) and dialectal Arabic (DA), spanning 20 diverse topics. Built by…
▽ More
As large language models (LLMs) become increasingly integrated into daily life, ensuring their cultural sensitivity and inclusivity is paramount. We introduce our dataset, a year-long community-driven project covering all 22 Arab countries. The dataset includes instructions (input, response pairs) in both Modern Standard Arabic (MSA) and dialectal Arabic (DA), spanning 20 diverse topics. Built by a team of 44 researchers across the Arab world, all of whom are authors of this paper, our dataset offers a broad, inclusive perspective. We use our dataset to evaluate the cultural and dialectal capabilities of several frontier LLMs, revealing notable limitations. For instance, while closed-source LLMs generally exhibit strong performance, they are not without flaws, and smaller open-source models face greater challenges. Moreover, certain countries (e.g., Egypt, the UAE) appear better represented than others (e.g., Iraq, Mauritania, Yemen). Our annotation guidelines, code, and data for reproducibility are publicly available.
△ Less
Submitted 28 February, 2025;
originally announced March 2025.
-
Casablanca: Data and Models for Multidialectal Arabic Speech Recognition
Authors:
Bashar Talafha,
Karima Kadaoui,
Samar Mohamed Magdy,
Mariem Habiboullah,
Chafei Mohamed Chafei,
Ahmed Oumar El-Shangiti,
Hiba Zayed,
Mohamedou cheikh tourad,
Rahaf Alhamouri,
Rwaa Assi,
Aisha Alraeesi,
Hour Mohamed,
Fakhraddin Alwajih,
Abdelrahman Mohamed,
Abdellah El Mekki,
El Moatez Billah Nagoudi,
Benelhadj Djelloul Mama Saadia,
Hamzah A. Alsayadi,
Walid Al-Dhabyani,
Sara Shatnawi,
Yasir Ech-Chammakhy,
Amal Makouar,
Yousra Berrachedi,
Mustafa Jarrar,
Shady Shehata
, et al. (2 additional authors not shown)
Abstract:
In spite of the recent progress in speech processing, the majority of world languages and dialects remain uncovered. This situation only furthers an already wide technological divide, thereby hindering technological and socioeconomic inclusion. This challenge is largely due to the absence of datasets that can empower diverse speech systems. In this paper, we seek to mitigate this obstacle for a nu…
▽ More
In spite of the recent progress in speech processing, the majority of world languages and dialects remain uncovered. This situation only furthers an already wide technological divide, thereby hindering technological and socioeconomic inclusion. This challenge is largely due to the absence of datasets that can empower diverse speech systems. In this paper, we seek to mitigate this obstacle for a number of Arabic dialects by presenting Casablanca, a large-scale community-driven effort to collect and transcribe a multi-dialectal Arabic dataset. The dataset covers eight dialects: Algerian, Egyptian, Emirati, Jordanian, Mauritanian, Moroccan, Palestinian, and Yemeni, and includes annotations for transcription, gender, dialect, and code-switching. We also develop a number of strong baselines exploiting Casablanca. The project page for Casablanca is accessible at: www.dlnlp.ai/speech/casablanca.
△ Less
Submitted 6 October, 2024;
originally announced October 2024.
-
2D Face Recognition System Based on Selected Gabor Filters and Linear Discriminant Analysis LDA
Authors:
Samir F. Hafez,
Mazen M. Selim,
Hala H. Zayed
Abstract:
We present a new approach for face recognition system. The method is based on 2D face image features using subset of non-correlated and Orthogonal Gabor Filters instead of using the whole Gabor Filter Bank, then compressing the output feature vector using Linear Discriminant Analysis (LDA). The face image has been enhanced using multi stage image processing technique to normalize it and compensate…
▽ More
We present a new approach for face recognition system. The method is based on 2D face image features using subset of non-correlated and Orthogonal Gabor Filters instead of using the whole Gabor Filter Bank, then compressing the output feature vector using Linear Discriminant Analysis (LDA). The face image has been enhanced using multi stage image processing technique to normalize it and compensate for illumination variation. Experimental results show that the proposed system is effective for both dimension reduction and good recognition performance when compared to the complete Gabor filter bank. The system has been tested using CASIA, ORL and Cropped YaleB 2D face images Databases and achieved average recognition rate of 98.9 %.
△ Less
Submitted 12 March, 2015;
originally announced March 2015.