La Leaderboard: A Large Language Model Leaderboard for Spanish Varieties and Languages of Spain and Latin America
Authors:
María Grandury,
Javier Aula-Blasco,
Júlia Falcão,
Clémentine Fourrier,
Miguel González,
Gonzalo Martínez,
Gonzalo Santamaría,
Rodrigo Agerri,
Nuria Aldama,
Luis Chiruzzo,
Javier Conde,
Helena Gómez,
Marta Guerrero,
Guido Ivetta,
Natalia López,
Flor Miriam Plaza-del-Arco,
María Teresa Martín-Valdivia,
Helena Montoro,
Carmen Muñoz,
Pedro Reviriego,
Leire Rosado,
Alejandro Vaca,
María Estrella Vallecillo-Rodríguez,
Jorge Vallego,
Irune Zubiaga
Abstract:
Leaderboards showcase the current capabilities and limitations of Large Language Models (LLMs). To motivate the development of LLMs that represent the linguistic and cultural diversity of the Spanish-speaking community, we present La Leaderboard, the first open-source leaderboard to evaluate generative LLMs in languages and language varieties of Spain and Latin America. La Leaderboard is a communi…
▽ More
Leaderboards showcase the current capabilities and limitations of Large Language Models (LLMs). To motivate the development of LLMs that represent the linguistic and cultural diversity of the Spanish-speaking community, we present La Leaderboard, the first open-source leaderboard to evaluate generative LLMs in languages and language varieties of Spain and Latin America. La Leaderboard is a community-driven project that aims to establish an evaluation standard for everyone interested in developing LLMs for the Spanish-speaking community. This initial version combines 66 datasets in Basque, Catalan, Galician, and different Spanish varieties, showcasing the evaluation results of 50 models. To encourage community-driven development of leaderboards in other languages, we explain our methodology, including guidance on selecting the most suitable evaluation setup for each downstream task. In particular, we provide a rationale for using fewer few-shot examples than typically found in the literature, aiming to reduce environmental impact and facilitate access to reproducible results for a broader research community.
△ Less
Submitted 1 July, 2025;
originally announced July 2025.
