-
EuroLLM-9B: Technical Report
Authors:
Pedro Henrique Martins,
João Alves,
Patrick Fernandes,
Nuno M. Guerreiro,
Ricardo Rei,
Amin Farajian,
Mateusz Klimaszewski,
Duarte M. Alves,
José Pombal,
Manuel Faysse,
Pierre Colombo,
François Yvon,
Barry Haddow,
José G. C. de Souza,
Alexandra Birch,
André F. T. Martins
Abstract:
This report presents EuroLLM-9B, a large language model trained from scratch to support the needs of European citizens by covering all 24 official European Union languages and 11 additional languages. EuroLLM addresses the issue of European languages being underrepresented and underserved in existing open large language models. We provide a comprehensive overview of EuroLLM-9B's development, inclu…
▽ More
This report presents EuroLLM-9B, a large language model trained from scratch to support the needs of European citizens by covering all 24 official European Union languages and 11 additional languages. EuroLLM addresses the issue of European languages being underrepresented and underserved in existing open large language models. We provide a comprehensive overview of EuroLLM-9B's development, including tokenizer design, architectural specifications, data filtering, and training procedures. We describe the pre-training data collection and filtering pipeline, including the creation of EuroFilter, an AI-based multilingual filter, as well as the design of EuroBlocks-Synthetic, a novel synthetic dataset for post-training that enhances language coverage for European languages. Evaluation results demonstrate EuroLLM-9B's competitive performance on multilingual benchmarks and machine translation tasks, establishing it as the leading open European-made LLM of its size. To support open research and adoption, we release all major components of this work, including the base and instruction-tuned models, the EuroFilter classifier, and the synthetic post-training dataset.
△ Less
Submitted 4 June, 2025;
originally announced June 2025.
-
Findings of the WMT 2024 Shared Task on Chat Translation
Authors:
Wafaa Mohammed,
Sweta Agrawal,
M. Amin Farajian,
Vera Cabarrão,
Bryan Eikema,
Ana C. Farinha,
José G. C. de Souza
Abstract:
This paper presents the findings from the third edition of the Chat Translation Shared Task. As with previous editions, the task involved translating bilingual customer support conversations, specifically focusing on the impact of conversation context in translation quality and evaluation. We also include two new language pairs: English-Korean and English-Dutch, in addition to the set of language…
▽ More
This paper presents the findings from the third edition of the Chat Translation Shared Task. As with previous editions, the task involved translating bilingual customer support conversations, specifically focusing on the impact of conversation context in translation quality and evaluation. We also include two new language pairs: English-Korean and English-Dutch, in addition to the set of language pairs from previous editions: English-German, English-French, and English-Brazilian Portuguese. We received 22 primary submissions and 32 contrastive submissions from eight teams, with each language pair having participation from at least three teams. We evaluated the systems comprehensively using both automatic metrics and human judgments via a direct assessment framework. The official rankings for each language pair were determined based on human evaluation scores, considering performance in both translation directions--agent and customer. Our analysis shows that while the systems excelled at translating individual turns, there is room for improvement in overall conversation-level translation quality.
△ Less
Submitted 15 October, 2024;
originally announced October 2024.
-
EuroLLM: Multilingual Language Models for Europe
Authors:
Pedro Henrique Martins,
Patrick Fernandes,
João Alves,
Nuno M. Guerreiro,
Ricardo Rei,
Duarte M. Alves,
José Pombal,
Amin Farajian,
Manuel Faysse,
Mateusz Klimaszewski,
Pierre Colombo,
Barry Haddow,
José G. C. de Souza,
Alexandra Birch,
André F. T. Martins
Abstract:
The quality of open-weight LLMs has seen significant improvement, yet they remain predominantly focused on English. In this paper, we introduce the EuroLLM project, aimed at developing a suite of open-weight multilingual LLMs capable of understanding and generating text in all official European Union languages, as well as several additional relevant languages. We outline the progress made to date,…
▽ More
The quality of open-weight LLMs has seen significant improvement, yet they remain predominantly focused on English. In this paper, we introduce the EuroLLM project, aimed at developing a suite of open-weight multilingual LLMs capable of understanding and generating text in all official European Union languages, as well as several additional relevant languages. We outline the progress made to date, detailing our data collection and filtering process, the development of scaling laws, the creation of our multilingual tokenizer, and the data mix and modeling configurations. Additionally, we release our initial models: EuroLLM-1.7B and EuroLLM-1.7B-Instruct and report their performance on multilingual general benchmarks and machine translation.
△ Less
Submitted 24 September, 2024;
originally announced September 2024.
-
Is Context Helpful for Chat Translation Evaluation?
Authors:
Sweta Agrawal,
Amin Farajian,
Patrick Fernandes,
Ricardo Rei,
André F. T. Martins
Abstract:
Despite the recent success of automatic metrics for assessing translation quality, their application in evaluating the quality of machine-translated chats has been limited. Unlike more structured texts like news, chat conversations are often unstructured, short, and heavily reliant on contextual information. This poses questions about the reliability of existing sentence-level metrics in this doma…
▽ More
Despite the recent success of automatic metrics for assessing translation quality, their application in evaluating the quality of machine-translated chats has been limited. Unlike more structured texts like news, chat conversations are often unstructured, short, and heavily reliant on contextual information. This poses questions about the reliability of existing sentence-level metrics in this domain as well as the role of context in assessing the translation quality. Motivated by this, we conduct a meta-evaluation of existing sentence-level automatic metrics, primarily designed for structured domains such as news, to assess the quality of machine-translated chats. We find that reference-free metrics lag behind reference-based ones, especially when evaluating translation quality in out-of-English settings. We then investigate how incorporating conversational contextual information in these metrics affects their performance. Our findings show that augmenting neural learned metrics with contextual information helps improve correlation with human judgments in the reference-free scenario and when evaluating translations in out-of-English settings. Finally, we propose a new evaluation metric, Context-MQM, that utilizes bilingual context with a large language model (LLM) and further validate that adding context helps even for LLM-based evaluation metrics.
△ Less
Submitted 13 March, 2024;
originally announced March 2024.
-
Tower: An Open Multilingual Large Language Model for Translation-Related Tasks
Authors:
Duarte M. Alves,
José Pombal,
Nuno M. Guerreiro,
Pedro H. Martins,
João Alves,
Amin Farajian,
Ben Peters,
Ricardo Rei,
Patrick Fernandes,
Sweta Agrawal,
Pierre Colombo,
José G. C. de Souza,
André F. T. Martins
Abstract:
While general-purpose large language models (LLMs) demonstrate proficiency on multiple tasks within the domain of translation, approaches based on open LLMs are competitive only when specializing on a single task. In this paper, we propose a recipe for tailoring LLMs to multiple tasks present in translation workflows. We perform continued pretraining on a multilingual mixture of monolingual and pa…
▽ More
While general-purpose large language models (LLMs) demonstrate proficiency on multiple tasks within the domain of translation, approaches based on open LLMs are competitive only when specializing on a single task. In this paper, we propose a recipe for tailoring LLMs to multiple tasks present in translation workflows. We perform continued pretraining on a multilingual mixture of monolingual and parallel data, creating TowerBase, followed by finetuning on instructions relevant for translation processes, creating TowerInstruct. Our final model surpasses open alternatives on several tasks relevant to translation workflows and is competitive with general-purpose closed LLMs. To facilitate future research, we release the Tower models, our specialization dataset, an evaluation framework for LLMs focusing on the translation ecosystem, and a collection of model generations, including ours, on our benchmark.
△ Less
Submitted 27 February, 2024;
originally announced February 2024.
-
Unbabel's Participation in the WMT19 Translation Quality Estimation Shared Task
Authors:
Fabio Kepler,
Jonay Trénous,
Marcos Treviso,
Miguel Vera,
António Góis,
M. Amin Farajian,
António V. Lopes,
André F. T. Martins
Abstract:
We present the contribution of the Unbabel team to the WMT 2019 Shared Task on Quality Estimation. We participated on the word, sentence, and document-level tracks, encompassing 3 language pairs: English-German, English-Russian, and English-French. Our submissions build upon the recent OpenKiwi framework: we combine linear, neural, and predictor-estimator systems with new transfer learning approac…
▽ More
We present the contribution of the Unbabel team to the WMT 2019 Shared Task on Quality Estimation. We participated on the word, sentence, and document-level tracks, encompassing 3 language pairs: English-German, English-Russian, and English-French. Our submissions build upon the recent OpenKiwi framework: we combine linear, neural, and predictor-estimator systems with new transfer learning approaches using BERT and XLM pre-trained models. We compare systems individually and propose new ensemble techniques for word and sentence-level predictions. We also propose a simple technique for converting word labels into document-level predictions. Overall, our submitted systems achieve the best results on all tracks and language pairs by a considerable margin.
△ Less
Submitted 11 September, 2019; v1 submitted 24 July, 2019;
originally announced July 2019.
-
Unbabel's Submission to the WMT2019 APE Shared Task: BERT-based Encoder-Decoder for Automatic Post-Editing
Authors:
António V. Lopes,
M. Amin Farajian,
Gonçalo M. Correia,
Jonay Trenous,
André F. T. Martins
Abstract:
This paper describes Unbabel's submission to the WMT2019 APE Shared Task for the English-German language pair. Following the recent rise of large, powerful, pre-trained models, we adapt the BERT pretrained model to perform Automatic Post-Editing in an encoder-decoder framework. Analogously to dual-encoder architectures we develop a BERT-based encoder-decoder (BED) model in which a single pretraine…
▽ More
This paper describes Unbabel's submission to the WMT2019 APE Shared Task for the English-German language pair. Following the recent rise of large, powerful, pre-trained models, we adapt the BERT pretrained model to perform Automatic Post-Editing in an encoder-decoder framework. Analogously to dual-encoder architectures we develop a BERT-based encoder-decoder (BED) model in which a single pretrained BERT encoder receives both the source src and machine translation tgt strings. Furthermore, we explore a conservativeness factor to constrain the APE system to perform fewer edits. As the official results show, when trained on a weighted combination of in-domain and artificial training data, our BED system with the conservativeness penalty improves significantly the translations of a strong Neural Machine Translation system by $-0.78$ and $+1.23$ in terms of TER and BLEU, respectively. Finally, our submission achieves a new state-of-the-art, ex-aequo, in English-German APE of NMT.
△ Less
Submitted 29 June, 2019; v1 submitted 30 May, 2019;
originally announced May 2019.
-
LOCV calculation of the equation of state and properties of rapidly rotating neutron stars
Authors:
A. H. Farajian,
M. Bigdeli,
S. Belbasi
Abstract:
In this paper, we have investigated the structural properties of rotating neutron stars using the numerical RNS code and the equation of states which have been calculated within the lowest order constrained variational approach. In order to calculate the equation of state of nuclear matter, we have used UV$_{14}$ $+$TNI and AV$_{18}$ potentials. Here, we have computed the maximum mass of the neutr…
▽ More
In this paper, we have investigated the structural properties of rotating neutron stars using the numerical RNS code and the equation of states which have been calculated within the lowest order constrained variational approach. In order to calculate the equation of state of nuclear matter, we have used UV$_{14}$ $+$TNI and AV$_{18}$ potentials. Here, we have computed the maximum mass of the neutron star and the corresponding equatorial radius at different angular velocity. We have also computed the structural properties of Keplerian rotating neutron star for maximum mass configuration, $M_{K}$, $R_{K}$, $f_{K}$ and $j_{max}$.
△ Less
Submitted 11 May, 2018; v1 submitted 12 April, 2018;
originally announced April 2018.
-
Hydrogen Compounds of Group-IV Nanosheets
Authors:
L. C. Lew Yan Voon,
E. Sandberg,
R. S. Aga,
A. A. Farajian
Abstract:
The structural and electronic properties of the hydrides of silicene and germanene have been studied using ab initio calculations. The trend for the M-H (M=C, Si, Ge) bond lengths, and corresponding bond energies, is consistent with the atomic size trend, and comparable to those of MH_4 hydrides. Band structures were also obtained for the buckled configuration, which is the stable form for both si…
▽ More
The structural and electronic properties of the hydrides of silicene and germanene have been studied using ab initio calculations. The trend for the M-H (M=C, Si, Ge) bond lengths, and corresponding bond energies, is consistent with the atomic size trend, and comparable to those of MH_4 hydrides. Band structures were also obtained for the buckled configuration, which is the stable form for both silicene and germanene. Upon hydrogenation, both silicane (indirect gap) and germanane (direct gap) are semiconducting.
△ Less
Submitted 13 July, 2010;
originally announced July 2010.
-
Vacuum polarization in nanotubes
Authors:
K. Sasaki,
A. A. Farajian,
H. Mizuseki,
Y. Kawazoe
Abstract:
This paper has been withdrawn by the author due to a crucial error.
This paper has been withdrawn by the author due to a crucial error.
△ Less
Submitted 24 June, 2003; v1 submitted 5 July, 2002;
originally announced July 2002.
-
Effective Screening of Localized Charged Perturbations in Metallic Nanotubes: Roles of Massive Bands
Authors:
K. Sasaki,
A. A. Farajian,
H. Mizuseki,
Y. Kawazoe
Abstract:
The massive-band effects on screening behavior of metallic carbon nanotubes are theoretically investigated using two different methods; continuous and lattice quantum theories. Both approaches show screening of a localized external perturbation with an effective screening length of the order of the nanotube diameter. Calculating the nonlinear deformation of the local density of states near the c…
▽ More
The massive-band effects on screening behavior of metallic carbon nanotubes are theoretically investigated using two different methods; continuous and lattice quantum theories. Both approaches show screening of a localized external perturbation with an effective screening length of the order of the nanotube diameter. Calculating the nonlinear deformation of the local density of states near the charged perturbation, we show that the perturbative effects of the massive bands are effectively canceled by direct massive band interactions, such that a good agreement between the two methods can be achieved. The effective screening is important in nanoscale integration of nanotube-based electronic devices.
△ Less
Submitted 16 April, 2003; v1 submitted 30 May, 2002;
originally announced May 2002.
-
Nonlinear charging, and transport times in doped nanotubes junctions
Authors:
Keivan Esfarjani,
Amir A. Farajian,
Siu Tat Chui,
Yoshiyuki Kawazoe
Abstract:
The nonlinear capacitance in doped nanotube junctions is calculated self consistently. It decreases as a function of the applied bias when the latter becomes larger than the pseudogap of the nanotube. For this device, one can deduce a relaxation time of about 0.1 femtosecond. Because of its negative differential resistance (NDR), a switching time of less than a fs can also be deduced.
The nonlinear capacitance in doped nanotube junctions is calculated self consistently. It decreases as a function of the applied bias when the latter becomes larger than the pseudogap of the nanotube. For this device, one can deduce a relaxation time of about 0.1 femtosecond. Because of its negative differential resistance (NDR), a switching time of less than a fs can also be deduced.
△ Less
Submitted 10 September, 2003; v1 submitted 29 April, 2002;
originally announced April 2002.
