Automated Scoring of Arabic Text Using Large Language Models: A Literature Review
Organizations: Laboratoire d’Informatique et de Mathématiques LIM Amar Telidji University, Laghouat, Algeria
Abstract
In modern educational systems, Automatic Text Scoring (ATS) plays a central role by enabling scalable and consistent evaluation of learner responses without human intervention. Recently, the increased accessibility of LLMs and Arabic-specific datasets has sparked renewed interest in this area. In this work, we investigate LLM-Based approaches for the automated evaluation of Arabic texts, focusing on both short answer grading (ASAG) and essay scoring (AES). We further introduce a structured taxonomy comprising five dimensions: application domain, feedback generation capability, LLM architecture deployed, alignment with competency referential frameworks, and prompt engineering strategy. By applying this taxonomy, we conduct a comparative analysis of existing studies, examining their methodological approaches, datasets, evaluation metrics, and reported performance. The findings highlight the need for sustained and pedagogically grounded research efforts in Arabic ATS, given its significance for improving educational quality across Arabic-speaking communities.
Figures & tables
| Aspect | Short Answer Scoring (ASAG) | Essay Scoring (AES) |
| Text Length | Brief (sentences/paragraphs) | Extended (multi-paragraph) |
| Key Focus | Content accuracy, keyword overlap | Coherence, style, development |
| Challenges | Domain-specific meaning | Subjectivity in creativity |
| Dataset | Language | Annotation | Size | # Prompts | Trait-Specific Score | Linguistic Annotation | CEFR |
| Arabic Essay Scoring | |||||||
| ZAEBUC ( Habash and Palfreyman, 2022 ) | Bilingual (AR–EN) | Manual | 214 essays | 3 | × | ✓(POS, morph, lemma) | A2–C1 |
| QAES ( Bashendy et al., 2024 ) | Arabic | Manual | 195 essays | 2 | ✓(7 traits) | × | × |
| ZaQQ ( Elsayed et al., 2025 ) | Arabic | Manual & Automatic | 1021 essays | 5 | ✓ | × | × |
| AR-AES ( Ghazawi and Simpson, 2024 ) | Arabic | Manual | 2046 essays | 12 | × | × | × |
| TAQEEM ( Bashendy et al., 2025a ) | Arabic | Manual | 1,265 essays | Multiple | ✓ | × | × |
| Approach | Application Domain | Feedback Generation | LLM / Model | Competency Framework | Prompt Engineering | Score |
| Nael et al. (2022) | General (ASAP-SAS) | ✗ | AraBERT, ELECTRA, Bi-LSTM, RNN | ✗ | ✗ | QWK = 0.78 |
| Badry et al. (2023) | Cybercrime Course (AR-ASAG) | ✗ | Local weight-based LSA | ✗ | ✗ | F1 = 82.82% |
| Soulimani et al. (2024) | Islamic Edu. (Moroccan pupils) | ✗ | AraBERT, LSTM | ✗ | ✗ | Acc = 71.31% |
| Noaman et al. (2025) | Cybercrime Course (AR-ASAG) | ✗ | AraBERT (Siamese network) | ✗ | ✗ | Pearson = 0.79 |
| Nabil et al. (2025) | Science (ESD) | ✓ | GPT-4, Llama 3, DeepSeek-V3 | ✗ | ✓ | QWK = 0.83 |
| Approach | Application Domain / Used Dataset | Feedback Generation | LLM / Model | Competency Framework | Prompt Engineering | Score (QWK) |
| Ghazawi and Simpson (2024) | Info. Science, Chemistry, Biotechnology (AR-AES) | ✗ | AraBERT (fine-tuned) | ✗ | ✗ | |
| Ghazawi and Simpson (2025) | Info. Science, Chemistry, Biotechnology (AR-AES) | ✗ | ChatGPT, Llama, Aya, Jais, ACEGPT | ✗ | Mixed-language prompting | |
| Almarwani et al. (2025) | General (TAQEEM) | ✗ | GPT-4o | ✗ | ✓ Few-shot + grading rubric | |
| Alnajjar et al. (2025) | General (TAQEEM) | ✗ | GPT-4.1 | ✗ | ✓ Few-shot + grading rubric | |
| Bashendy et al. (2025b) | General (LAILA) | ✗ | Fanar, ALLaM, R7B Arabic | ✗ | Prompt-specific strategies |
Automated Grading
Automated grading uses machine learning and language models to score student work against answer keys or rubrics and provide feedback.
See all 88 papers