cs.CLNov 18, 2026

Large Language Models or AI Agents for English to Arabic Translation? An assessment on WikiEn-Ar190

Authors: Fatima Zahra Iguenfer, Yousra Chtouki

Organizations: Al Akhawayn University in Ifrane

Abstract

This research investigates whether a large language model or an autonomous AI Agent performs better at translating from English to Arabic, which is considered a low-resource language. The authors introduce WikiEn-Ar190, a new dataset of 190 carefully selected Wikipedia articles from 18 different areas, which were created and verified by two native Arabic speakers to ensure that they accurately reflect both meaning and style. Using this dataset, the study evaluates the translation quality of GPT-5-mini both as an independent neural translator and when used within a more complex process that includes self-assessment and improvement. As a standalone model, GPT-5-mini produced very fluent translations , achieving a COMET score of 0.8649 and receiving high human scores for adequacy, fluency, and style (averaging 4.53 out of 5). When used in a system that combined translation with self-assessment, it improved lexical accuracy by 28.76% (BLEU score of 22.83) while maintaining a similar level of semantic quality. These findings show that methods involving autonomous reasoning can improve translation consistency for language pairs with limited resources. Beyond the results, the approach offers a framework for evaluating and improving translations, potentially using them as assisting tools to expand the representation of low resource languages, such as Arabic, on the web.

Explore similar work

CardsList
  1. Current state of LLMs for Arabic dialectal machine translation

    Nov 1, 2026Josef Jon, Rawan Bondok, Ondřej BojarDialect IdentificationMachine Translation

  2. Evaluation of Small Language Models for Arabic Language Processing

    Jun 19, 2026Jumana Alsubhi, Ahmed Alhusayni, Abdulrahman Gharawi +5Arabic NLPArabic Language Model Evaluation