A Parallel Multi-Dialect Arabic Dependency Treebank

Noor Abo Mokh; Mohamed Adel; Jamila El-Gizuli; Go Inoue; Ahmed Elshabrawy; Nizar Habash

cs.CLOct 20, 2026

A Parallel Multi-Dialect Arabic Dependency Treebank

Authors: Noor Abo Mokh, Mohamed Adel, Jamila El-Gizuli, Go Inoue, Ahmed Elshabrawy, Nizar Habash

Organizations: Mohamed bin Zayed University of Artificial Intelligence · Computational Approaches to Modeling Language Lab, New York University Abu Dhabi

Abstract

Dependency parsing resource development for Arabic has largely focused on Modern Standard Arabic, leaving dialectal Arabic comparatively under-resourced. We introduce MADARTB, the first large-scale parallel dependency treebank spanning five city dialects: Beirut, Cairo, Doha, Rabat and Tunis, and extending an existing parallel MSA version. In total MADARTB consists of 5,000 sentences annotated using the CATiB dependency scheme. We analyze syntactic similarity across all city dialects and MSA. Although global construction profiles exhibit high similarity across varieties, local syntactic structures frequently diverge, revealing substantial structural variation across dialects. We further establish a cross-dialect dependency parsing comparison using a supervised parser and four LLMs. The supervised parser achieves 87.5% LAS, while the best LLM setup trails at 85.7% LAS across all six varieties on the test set. We release our annotations and models publicly.

Figures & tables

Figure 1: CATiB-style Habash et al. (2009) parallel dependency annotations in MSA and five city dialects. English gloss: ‘Which bus goes to the city center?’. Transliterations in the Buckwalter Scheme Buckwalter (2002) .

	Train			Development			Test			Total
Variety	Sent.	Words	Tokens	Sent.	Words	Tokens	Sent.	Words	Tokens	Sent.	Words	Tokens
Beirut	700	3,642	5,843	150	803	1,312	150	840	1,307	1,000	5,285	8,462
Cairo	700	3,987	5,973	150	898	1,342	150	899	1,328	1,000	5,784	8,643
Doha	700	3,625	5,369	150	802	1,204	150	809	1,195	1,000	5,236	7,768
Rabat	700	4,125	6,185	150	900	1,367	150	925	1,373	1,000	5,950	8,925
Tunis	700	3,851	5,845	150	844	1,322	150	864	1,302	1,000	5,559	8,469

Table 1: Numbers of sentences, words, and tokens in the train, development, and test splits. The six varieties are parallel at the sentence level.

Table 3

Model	Training Data / Prompting	Beirut	Cairo	Doha	Rabat	Tunis	MSA	Dialects	All
Supervised	MadarTB Beirut	81.1	77.8	78.7	73.0	75.0	72.7	77.1	76.3
Supervised	MadarTB Cairo	73.2	82.0	82.2	68.0	74.8	75.6	75.9	75.8
Supervised	MadarTB Doha	73.4	78.9	85.4	70.2	76.5	73.7	76.7	76.1
Supervised	MadarTB Rabat	66.5	72.9	76.2	83.9	75.1	70.1	75.0	74.1
Supervised	MadarTB Tunis	66.0	73.7	77.4	71.0	83.4	71.9	74.2	73.8
Supervised	MadarTB MSA	64.6	67.2	70.8	66.6	67.9	83.2	67.4	70.3

Table 4: Development-set parsing results (LAS) for supervised parsers and prompted LLMs. All Supervised rows use the CAMeLBERT-Mix Biaffine model and projective multi-root decoding; their configurations differ only in training data. All Treebanks denotes PATB, CamelTB, ARZTB, Gumar, and MadarTB . Bold indicates the best result overall, and underlining indicates the best LLM result.

Model	Training Data / Prompting	Beirut	Cairo	Doha	Rabat	Tunis	MSA	Dialects	All
Supervised	All treebanks	86.9	87.4	90.7	85.4	87.9	86.9	87.6	87.5
Gemini	10-shot	85.8	83.3	86.9	83.8	88.1	86.4	85.5	85.7
GPT	10-shot	84.5	81.9	84.5	82.4	85.0	85.1	83.6	83.9
Fanar	10-shot	67.6	70.0	72.3	72.6	72.5	73.3	71.0	71.4
Llama	10-shot	63.4	66.9	70.2	67.5	69.0	69.4	67.3	67.7

Table 5: Test-set parsing results (LAS) for the best supervised configuration and the best setup for each LLM, selected on development data. “All treebanks” denotes PATB, CamelTB, ARZTB, Gumar, and the proposed MadarTB . Bold indicates the best result overall, and underlining indicates the best LLM result. The complete test matrix appears in Appendix Table 10 .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

System	Parameters	Role and access
CAMeLBERT-Mix biaffine parser	110,936,282	Locally trained supervised parser used for the reported All Treebanks results; the count includes the CAMeLBERT-Mix encoder and SuPar parsing head.
CAMeLBERT-Mix (SimAlign)	109.1M	Frozen pretrained encoder used to obtain contextual token representations; SimAlign requires no task-specific training.
CAMeLBERT-Mix (Awesome-Align)	132.7M	Locally fine-tuned masked-language-model checkpoint, including its alignment-training heads.
Fanar-2-27B-Instruct	27B	Open-weight model served locally with vLLM.
Llama-4-Scout	109B total; 17B active	Open-weight mixture-of-experts model served locally with vLLM.
Gemini; GPT	Not disclosed	Proprietary models accessed through their providers’ Batch APIs.

Appendix

Table 6: Model sizes and access methods.

Parameter	Value	Parameter	Value
Parser	Biaffine	Epochs	20
BERT layers mixed	4	BERT pooling	Mean
Arc MLP	500	Relation MLP	100
Encoder dropout	0.1	MLP dropout	0.33
Learning rate	$5\times 10^{-5}$	BERT LR rate	10
Batch token budget	5,000	Update steps	4

Appendix

Table 7: Hyperparameters for the reported CAMeLBERT-Mix supervised parsers.

System/stage	Configuration
SimAlign	Frozen CAMeLBERT-Mix encoder; BPE tokenization; layer 8; intersection of the bidirectional argmax alignments. No alignment-model training.
Awesome-Align training	CAMeLBERT-Mix initialization; MLM, TLM, full bidirectional TLM, SO, and PSI objectives; contrastive objective disabled; 5 epochs (1,645 optimizer updates); batch size 64; gradient accumulation 1; maximum block length 128; learning rate $2\times 10^{-5}$ ; weight decay 0.01; 165 warmup updates; maximum gradient norm 1.0; seed 42; one A100 GPU with TF32.
Awesome-Align extraction	Layer 8; bidirectional softmax intersection; threshold 0.001; batch size 512; 8 data workers.
MGIZA++ training	Both directions for every pair; 5 IBM Model 1, 5 HMM, 3 Model 3, and 3 Model 4 EM iterations; Models 2 and 5 disabled; 50 word classes; 4 threads per direction and 8 concurrent pair jobs. Directional links were combined with grow-diag-final-and.

Appendix

Table 8: Training and extraction settings for the alignment experiments.

System	Time (min)	Hardware/access	API cost (USD)
Gemini-3.6-Flash	41.7	Hosted Batch API	$30.92
GPT-5.6-Sol	29.3	Hosted Batch API	$35.31
Fanar-2-27B-Instruct	14.0	4 $\times$ A100 80GB	–
Llama-4-Scout	27.4	4 $\times$ A100 80GB	–
All Treebanks parser	0.21	1 $\times$ A100 80GB	–

Appendix

Table 9: Inference runtime on the evaluated data and hosted API cost. The parser measurement uses projective multi-root decoding and excludes model loading.

Model	Training Data / Prompting	Beirut	Cairo	Doha	Rabat	Tunis	MSA	Dialects	All
Supervised	MadarTB Beirut	79.6	76.1	80.0	69.3	75.1	69.8	75.9	74.8
Supervised	MadarTB Cairo	70.9	80.8	78.7	66.2	77.3	72.4	74.7	74.3
Supervised	MadarTB Doha	71.2	77.0	82.9	66.9	75.8	74.3	74.5	74.5
Supervised	MadarTB Rabat	63.7	72.1	73.1	79.4	76.3	70.2	73.0	72.5
Supervised	MadarTB Tunis	64.5	72.3	74.9	69.6	83.0	71.5	72.8	72.6
Supervised	MadarTB MSA	59.8	68.9	69.4	62.9	68.0	82.5	65.7	68.7

Appendix

Table 10: test-set parsing results (LAS) for supervised parsers and prompted LLMs. All Supervised rows use the CAMeLBERT-Mix Biaffine model and projective multi-root decoding; their configurations differ only in training data. All Treebanks denotes PATB, CamelTB, ARZTB, Gumar, and MadarTB . Bold indicates the best result overall, and underlining indicates the best LLM result.

Claim	System A	System B	Set	$\Delta$ LAS	95% CI
Development set
Dialect pooling	MadarTB Dialects	MadarTB Beirut	Dialects	9.9	[8.6, 11.2]
	MadarTB Dialects	MadarTB Cairo	Dialects	11.1	[9.8, 12.4]
	MadarTB Dialects	MadarTB Doha	Dialects	10.3	[9.0, 11.6]
	MadarTB Dialects	MadarTB Rabat	Dialects	12.0	[10.5, 13.5]
	MadarTB Dialects	MadarTB Tunis	Dialects	12.7	[11.2, 14.3]

Appendix

Table 11: Paired sentence-ID-clustered percentile bootstrap intervals for principal development- and test-set LAS comparisons (10,000 replicates per comparison).

\Delta

LAS is System A minus System B in absolute percentage points. Within each split, every replicate resamples all 150 underlying sentence IDs with replacement and retains every relevant parallel variety realization.

Model	Training Data	Decoding	Gumar	ARZTB	CamelTB	PATB	MadarTB All
CamelParser2.0	PATB + CamelTB	Single-root	72.7	73.2	87.5	91.3	76.0
CAMeLParser-Dialects	PATB + CamelTB + ARZTB + Gumar	Single-root	80.3	84.2	87.2	90.8	81.2
Ours	All Treebanks	Single-root	80.4	84.1	87.5	90.7	86.5
Ours	All Treebanks	Multi-root	88.0	84.7	89.2	91.2	87.5

Appendix

Table 12: Test-set LAS on established Arabic dependency treebanks and MadarTB . The published baselines are CamelParser2.0 ( Elshabrawy et al., 2023 ) and CAMeLParser-Dialects ( Elshabrawy et al., 2026 ) . “All Treebanks” denotes PATB, CamelTB, ARZTB, Gumar, and MadarTB . Bold indicates the best result for each test set.

	Corpus size		Lexicon and length		Tree structure		Roots
Variety	Sent.	Tokens	Unique	Sent. len.	Max depth	Dep. dist.	Roots/sent.	Multi-root (%)
Beirut	1,000	8,462	1,948	8.46	3.03	1.97	1.21	16.9
Cairo	1,000	8,643	2,005	8.64	3.27	2.01	1.14	10.9
Doha	1,000	7,768	1,846	7.77	2.97	1.94	1.20	16.7
Rabat	1,000	8,925	2,089	8.93	3.26	2.02	1.19	15.7
Tunis	1,000	8,469	2,020	8.47	3.21	2.01	1.19	16.3

Appendix

Table 13: Descriptive statistics for the six parallel treebanks. Unique forms are distinct token forms. Mean maximum depth assigns roots depth zero. Mean dependency distance is the absolute difference between the token positions of a non-root dependent and its head, averaged first within each sentence and then across sentences. Roots/sent. is the mean number of tokens with HEAD=0 per sentence; multi-root is the percentage of sentences containing more than one such token.

POS	Beirut	Cairo	Doha	Rabat	Tunis	MSA	All
NOM	50.5	53.3	49.9	50.8	50.2	49.2	50.7
PNX	17.1	17.3	19.2	17.0	17.7	15.6	17.3
PRT	19.0	16.2	13.8	16.7	14.5	22.1	17.2
VRB	11.2	10.8	14.5	13.4	15.4	11.0	12.7
PROP	2.2	2.4	2.6	2.0	2.1	1.7	2.1
VRB-PASS	0.0	0.0	0.0	0.0	0.0	0.5	0.1

Appendix

Table 14: POS distributions (%) over all data splits.

Relation	Beirut	Cairo	Doha	Rabat	Tunis	MSA	All
MOD	45.1	44.4	42.9	49.5	44.7	43.8	45.1
OBJ	21.4	19.8	21.3	19.9	22.9	21.3	21.1
—	15.1	13.9	16.5	14.2	14.8	13.8	14.7
IDF	10.3	11.3	9.5	8.7	9.9	11.4	10.2
SBJ	6.9	8.9	8.5	6.4	6.6	7.4	7.4
PRD	0.8	1.3	0.9	0.7	0.8	1.9	1.1

Appendix

Table 15: Dependency-relation distributions (%) over all data splits.

Variety pair		All tokens	Aligned token pairs		Head-aligned token pairs
Pair		Aligned	Token POS	Head also	Token+head POS	Relation	Direction	Complete
		(%)	match (%)	aligned (%)	match (%)	match (%)	match (%)	match (%)
Beirut	Cairo	85.6	90.8	67.1	86.4	93.4	98.3	81.6
Beirut	Doha	87.3	89.7	71.4	79.2	93.9	99.2	76.0
Beirut	MSA	82.5	87.3	58.6	75.3	90.7	99.1	70.7
Beirut	Rabat	82.8	88.0	65.6	74.6	92.6	99.0	70.8

Appendix

Table 16: Alignment-level structural agreement from the selected Awesome-Align links by unordered variety pair; all values are percentages. The columns follow the successive evaluation subsets shown in the header. Aligned is the mean of the two varieties’ token-coverage percentages. For aligned token pairs, we report token POS match and the percentage whose heads also align. Both POS requires agreement in token and head POS. The final four measures are evaluated on this head-aligned subset; complete match requires both POS, relation, and direction to all match.

Manual evaluation data	Sentence pairs	Source tokens	Target tokens	Manual links
Complete annotated sample	300	2,545	2,615	2,172
Aligner	Predicted links	Precision	Recall	F 1
SimAlign	2,036	91.9	86.1	88.9
Awesome-Align	2,227	91.1	93.4	92.2
MGIZA++	2,215	79.0	80.6	79.8

Appendix

Table 17: Manual one-to-one alignment data and system evaluation. The upper block describes the complete sample annotated by two annotators; the lower block reports each system’s predicted links and micro-averaged precision, recall, and F 1 . These metrics compare each predicted link set with the manually annotated non-null links.

Dependency Parsing

Dependency parsing identifies grammatical relationships between words and represents sentence structure as a tree.

See all 24 papers

EDRAC: Benchmarking Arabic Dialect Reading Comprehension
Sep 1, 2026Noor Abo Mokh, Kirill Chirkunov, Teresa Lynn +15LLM Evaluation Arabic NLP
Mohamed bin Zayed University of Artificial Intelligence · New York University Abu Dhabi · IBM Research AI +1
Aladdin-FTI @ AMIYA Three Wishes for Arabic NLP: Fidelity, Diglossia, and Multidialectal Generation
Feb 18, 2026Jonathan Mutal, Perla Al Almaoui, Simon Hengchen +1Arabic NLP Arabic Dialects
Faculté de traduction et d’interprétation, Université de Genève
Arabic Morphosyntactic Tagging and Dependency Parsing with Large Language Models
Mar 17, 2026Mohamed Adel, Bashar Alhafni, Nizar HabashLLM Evaluation Arabic NLP
Computational Approaches to Modeling Language Lab 1New York University Abu Dhabi · 2Mohamed bin Zayed University of Artificial Intelligence

Sep 1, 2026J/K move · Enter open · S save

EDRAC: Benchmarking Arabic Dialect Reading Comprehension

Noor Abo Mokh, Kirill Chirkunov, Teresa Lynn, Nizar Habash+14

Mohamed bin Zayed University of Artificial Intelligence · New York University Abu Dhabi · IBM Research AI+1

LLM Evaluation Arabic NLP Question Answering Low-Resource Language Processing Arabic Dialects

Open paper AskPDF arXiv

Dialectal Arabic (DA) remains under-resourced compared to Modern Standard Arabic (MSA), particularly for machine reading comprehension (MRC) and question answering (QA). Existing Arabic QA benchmarks primarily focus on formal written MSA or multiple-choice QA, with limited coverage of naturally spoken dialects. Here, we aim to bridge this gap. We introduce EDRAC, the first large-scale benchmark for dialectal Arabic machine reading comprehension (MRC) and generative QA, covering five major dialects: Egyptian, Moroccan, Emirati, Syrian, and Saudi Arabic. EDRAC contains 499 passages derived from naturally occurring spoken interactions and 4,977 corresponding QA pairs generated through a human--LLM collaborative pipeline combining iterative generation, LLM-as-a-judge evaluation, and human verification. We benchmark Arabic-centric and multilingual LLMs on EDRAC using lexical and semantic metrics. Our results reveal substantial gaps between semantic answer quality and dialectal fidelity, highlighting the limitations of existing evaluation metrics for dialectal Arabic generation. EDRAC provides a realistic and challenging MRC benchmark for future research on dialectal Arabic NLP.

Abstract

Dependency Parsing

Explore similar work

EDRAC: Benchmarking Arabic Dialect Reading Comprehension

Aladdin-FTI @ AMIYA Three Wishes for Arabic NLP: Fidelity, Diglossia, and Multidialectal Generation

Arabic Morphosyntactic Tagging and Dependency Parsing with Large Language Models