Large language models have shown strong potential for Arabic medical text generation; however, traditional fine-tuning objectives treat all medical cases uniformly, ignoring differences in clinical severity. This limitation is particularly critical in healthcare settings, where errors in severe cases contain higher clinical risk. In this work, we propose a severity-aware weighted loss for fine-tuning Arabic language models on medical complaint-response data. The method depends on soft severity probabilities to dynamically scale token-level loss contributions during optimization, thereby prioritizing clinically critical interactions without modifying model architectures. Experiments are conducted using the MAQA dataset, which provides Arabic medical complaints and trusted human responses. Severity labels and probabilistic scores are automatically derived using a fine-tuned AraBERT-based classifier and incorporated exclusively at the loss level. The proposed approach is evaluated across ten Arabic large language models of varying architectures and parameter scales. While standard cross-entropy fine-tuning yields only modest improvements, severity-aware optimization consistently achieves larger gains. Using a balanced weighting configuration, performance improves from 54.04% to 66.14% for AraGPT2-Base, from 59.16% to 67.18% for AraGPT2-Medium, and from 57.83% to 66.86% for Qwen2.5-0.5B, with peak performance reaching 67.18%. Overall, severity-aware fine-tuning delivers improvements of up to 12.10% over non-fine-tuned baselines, demonstrating robust and architecture-consistent gains.
Figures & tables
Question
Answer
Severity
Non-Critical
Neutral
Critical
ورم في الرقبة كيف أتعامل معه هل يستدعي جراحة
على حسب مكانه ونوعه، راجع طبيب أورام
حرج
32%
32%
36%
A neck lump, how should I deal with it? Does it require surgery?
It depends on its location and type; consult an oncologist.
Critical
32%
32%
36%
أعاني من انتفاخ الخد نتيجة تورم اللثة بسبب تسوس الأسنان الأمامية
يحتاج الأمر إلى مضاد حيوي وعلاج عصب
غير حرج
37%
35%
28%
I suffer from cheek swelling due to gum inflammation caused by front tooth decay.
The condition requires antibiotic treatment and a root canal procedure.
Non-Critical
37%
35%
28%
هل الكركم يساهم في ارتفاع ضغط الدم وما هي فوائده وكيفية استعماله
على العكس، كما هو معروف، يساعد على خفض الضغط
متوسط
35%
36%
29%
Does turmeric contribute to high blood pressure? What are its benefits and how is it used?
On the contrary, it helps lower blood pressure.
Neutral
35%
36%
29%
Table 1: Example instances from the severity-aware MAQA dataset in English translations.
Model
Base
Cross-Entropy
SA (Mild)
SA (Strong)
SA (Balanced)
AraGPT2-Base
54.04%
59.71%
59.75%
65.91%
66.14%
AraGPT2-Medium
59.16%
60.14%
60.48%
60.55%
67.18%
Bloomz-560M
63.59%
63.65%
63.91%
63.79%
65.52%
Al-Atlas-0.5B
61.03%
61.52%
62.03%
65.02%
66.83%
GPT2-Small-Arabic
56.40%
58.88%
62.01%
60.92%
63.06%
LFM2-350M
63.34%
63.72%
65.24%
64.08%
66.60%
Table 2: Performance comparison across baseline, standard cross-entropy fine-tuning, and severity-aware weighting configurations.
Arabic medical text generation is increasingly needed to help users interpret symptoms and access general health guidance in their native language. Nevertheless, many existing methods assume uniform importance across training samples, overlooking differences in clinical severity. This simplification can hinder the model's ability to properly capture complex or high-risk cases. To overcome this issue, this work introduces a Severity-based Curriculum Learning Strategy for Arabic Medical Text Generation, where the training process is structured to move gradually from less severe to more critical medical conditions. The approach divides the dataset into ordered stages based on severity and incrementally exposes the model to more challenging cases during fine-tuning, allowing it to first learn basic medical patterns before addressing more complex scenarios. The proposed method is evaluated on a subset of the Medical Arabic Question Answering (MAQA) dataset, which includes Arabic medical questions describing symptoms alongside corresponding responses. In addition, the dataset is annotated with three severity levels (Mild, Moderate, and Critical) using a rule-based method developed in this study. The results demonstrate that incorporating severity-aware curriculum learning leads to consistent performance improvements across all tested models, with gains of around +4% to +7% over baseline models and +3% to +6% compared with conventional fine-tuning approaches.
Ahmed Alansary, Molham Mohamed, Ali Hamdi
October University for Modern Sciences & Arts, Egypt
Telehealth systems have become increasingly important for delivering accessible and timely medical information. Existing large language models often struggle to provide consistent and contextually appropriate medical responses across varying levels of case severity. This limitation highlights the need for models that can effectively adapt to the progressive complexity in medical queries. To address this challenge, we introduce a severity-aware multi-model framework that integrates curriculum training strategy with relevance-based response selection. The proposed framework employs a three-stage curriculum learning strategy, where each model is trained sequentially on mild, moderate, and critical cases to progressively acquire domain knowledge. The approach uses five large language models, each trained independently under the same curriculum. During inference, all models generate candidate responses, and the response with highest BERTScore is selected as the final output. The framework is trained and evaluated on the MAQA dataset, which provides annotated medical question-answer pairs. Experimental results evaluated using BERTScore demonstrate that the proposed method achieves superior performance compared to both baseline and fine-tuned models, attaining 86.71% in the baseline setting and 90.30% after fine-tuning. These results highlight the effectiveness of combining curriculum learning with multi-model response selection in improving response quality and relevance in medical text generation.
Ahmed Alansary, Molham Mohamed, Ali Hamdi
Faculty of Computer Science MSA University Giza, Egypt
Large Language Models (LLMs) perform strongly in English medical tasks but degrade substantially in Arabic, a gap widely attributed to limited training data. We systematically investigate this assumption via tuned lens probing and causal activation patching, and find that Arabic medical knowledge is present in intermediate model representations but fails to surface at the output. This mechanistic insight motivates a targeted adaptation strategy: rather than fine-tuning the full network, we propose Targeted Low-Rank Adaptation (TLoRA), restricted to the layer window where cross-lingual representations diverge, upstream of the output layers where the failure manifests. We evaluate TLoRA on multiple-choice medical QA, where our approach outperforms full-network LoRA, zero-shot, and few-shot baselines. We further evaluate it on short-answer generation and multi-turn clinical dialogue, where it performs competitively without the need for task-specific finetuning. We additionally introduce AraClinicDialog, a clinician-constructed Arabic medical dialogue benchmark in MSA with validated variants across four Arabic dialects. Together, these contributions demonstrate that mechanistic diagnosis can serve as a practical guide for targeted adaptation in underrepresented-language medical LLMs.
Chaimae Abouzahir, Musa Khan, Hala Ali-Hassan +7
New York University Abu Dhabi · Cleveland Clinic Abu Dhabi