Figurative language is deeply culturally embedded; fluent use requires not just linguistic competence but cultural immersion. We ask whether LLMs can learn this link: does fine-tuning on cultural data improve figurative language understanding, and vice versa? We conduct a systematic study across four models (ALLaM-7B, Fanar-1-9B, Qwen3-8B, Llama-3.1-8B) and six Arabic datasets spanning cultural commonsense, proverbs, and poetry across diverse dialects and regions. Fine-tuning on poetry improves idiom comprehension (+2.33%, p<0.05), a gain our ArabicMMLU control does not reproduce, indicating that it stems from figurative content rather than Arabic language adaptation and pointing to a sensitivity to non-literal meaning that transfers across figurative types. Cultural fine-tuning, by contrast, lowers proverb-interpretation accuracy in both Arabic-centric models. Transfer between the two domains is otherwise indistinguishable from noise, with Arabic models frequently regressing after fine-tuning, suggesting prior saturation of relevant knowledge, while multilingual models show greater adaptation headroom. Error analysis further reveals that fine-tuning reinforces experiential cultural knowledge while destabilizing historically grounded factual knowledge. Our findings suggest that the relationship between culture and figurative language, though conceptually natural, is not straightforwardly captured through fine-tuning alone.
Figures & tables
Figure 1: Scene from the Egyptian TV show Nelly and Sherihan.
Poem–explanation pairs capturing poetic preference and aesthetic judgment across 14 genres, 12 eras
6,984
Arabic poetry
Train
Jawaher magdy-etal-2025-jawaher
Arabic proverb understanding and interpretation
800 (train), 198 (test)
20 Arabic varieties
Train, test
Kinayat attia-etal-2026-beyond
Egyptian Arabic idiom–explanation pairs
150
Egyptian Arabic
Test
Table 1: Datasets used for training and evaluation, covering figurative language and cultural knowledge across Arabic varieties and regions. Cultural datasets (AraDiCE-Culture, ArabCulture, Palm) and figurative language datasets (FannOrFlop, Jawaher, Kinayat) are visually distinguished; ArabicMMLU serves as a control.
Figure 2: Average performance difference.
Model
Jawaher
Kinayat
AraDiCE
Base
ALLaM-7B-Instruct
0.8990
0.8400
0.7537
Qwen3-8B
0.8131
0.6800
0.5037
Fanar-1-9B-Instruct
0.8906
0.7667
0.6870
Llama-3.1-8B-Instruct
0.6886
0.5956
0.5204
Fine-tuned on Palm Subset
ALLaM-7B-Instruct
0.8620
0.8333
0.7167
Qwen3-8B
0.8300
0.7178
0.5204
Table 2: Average evaluation results across three runs on Jawaher, Kinayat, and AraDiCE datasets. Results show accuracy scores ( ↑ ) for base models and models fine-tuned on different subsets.
Model
Train Set
Avg Δ %
Impr
Regr
Net
ALLaM
Jawaher
-1.30
21
28
-7
FannOrFlop
-1.87
15
25
-10
Fanar
Jawaher
+0.37
22
20
+2
FannOrFlop
+0.73
23
19
+4
Llama
Jawaher
-0.77
38
42
-4
FannOrFlop
+0.37
43
41
+2
Table 3: Aggregate performance changes on culture by model and fine-tuning dataset.
Impr
Regr
Net
By Country
Jordan
50
18
+32
Lebanon
30
29
+1
Qatar
26
28
−2
Palestine
30
35
−5
Egypt
22
34
−12
Table 4: Performance changes on cultural evaluation by country and topic category, aggregated across all models and fine-tuning configurations.
Task
Impr
Regr
Net
Avg Δ%
Cultural Fine-tuning
Idioms
195
159
+36
+1.00
Proverbs
149
162
−13
−0.27
Poetry Fine-tuning
Idioms
100
58
+42
+2.33
Proverbs
97
73
+24
+1.01
Table 5: Overall fine-tuning effect on idiom and proverb interpretation, aggregated across all models. Cultural fine-tuning uses ArabCulture and Palm; Poetry fine-tuning uses FannOrFlop.
Idioms
Proverbs
Model
Train Set
Avg Δ %
Net
Avg Δ %
Net
ALLaM
ArabCulture
−1.33
−6
−0.84
−5
Palm
−0.67
−3
−3.70
−22
Poetry
−1.11
−5
−1.01
−6
Fanar
ArabCulture
−1.33
−6
−1.01
−6
Palm
−2.00
−9
−3.03
−18
Table 6: Aggregate fine-tuning results by model and fine-tuning dataset for both idiom and proverb interpretation. Cultural fine-tuning datasets (ArabCulture, Palm) and Poetry fine-tuning (FannOrFlop) are visually distinguished. Net refers to the total net improved predictions across all three seeds.
Appendix figures & tables33 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Prompt used for the MCQ understanding task.
Figure 4: Prompt template used for AraDiCE-Culture evaluation.
Figure 5: Prompt used for fine-tuning on FannOrFlop poetry explanation.
Figure 6: Prompt used for fine-tuning on the Jawaher proverb explanation task.
Figure 7: Prompt used for fine-tuning on ArabCulture open-ended completion.
Figure 8: Prompt used for fine-tuning on the Palm instruction-following task. No task-specific preamble is used; the instruction field carries full context.
Run 1 (seed=0)
Run 2 (seed=42)
Run 3 (seed=21)
Model
Jawaher
Kinayat
AraDiCE
Jawaher
Kinayat
AraDiCE
Jawaher
Kinayat
AraDiCE
Base
ALLaM-7B-Instruct
0.8889
0.8600
0.7667
0.9091
0.8267
0.7667
0.8990
0.8333
0.7278
Qwen3-8B
0.8030
0.6733
0.5278
0.8333
0.6867
0.5056
0.8030
0.6800
0.4778
Fanar-1-9B-Instruct
0.8788
0.7733
0.7222
0.9040
0.7267
0.7000
0.8889
0.8000
0.6389
Llama-3.1-8B-Instruct
0.7020
0.5933
0.5056
0.6919
0.5933
0.5167
0.6717
0.6000
0.5389
Fine-tuned on Palm Subset
ALLaM-7B-Instruct
0.8687
0.8533
0.7333
0.8737
0.8133
0.7111
0.8434
0.8333
0.7056
Appendix
Table 7: Zero-shot evaluation results across three runs with different random seeds on Jawaher, Kinayat, and AraDiCE datasets. Results show accuracy scores ( ↑ ) for base models and models fine-tuned on different subsets.
Figure 9: Performance difference on different datasets for models fine-tuned on the ArabCulture dataset (diff = fine-tuned accuracy - base accuracy).
Figure 10: Performance difference on different datasets for models fine-tuned on the FannOrFlop dataset (diff = fine-tuned accuracy - base accuracy).
Figure 11: Performance difference on different datasets for models fine-tuned on the Jawaher dataset (diff = fine-tuned accuracy - base accuracy).
Figure 12: Performance difference on different datasets for models fine-tuned on the Palm dataset (diff = fine-tuned accuracy - base accuracy).
Figure 13: Performance difference on different datasets for models fine-tuned on the ArabicMMLU baseline dataset (diff = fine-tuned accuracy - base accuracy).
Figure 14: Average performance difference between models fine-tuned on the full FannOrFlop dataset vs. FannOrFlop subset.
Figure 15: Average performance difference between models fine-tuned on the full ArabCulture dataset vs. ArabCulture subset.
Run 1 (seed=0)
Run 2 (seed=42)
Run 3 (seed=21)
Model
Jawaher
Kinayat
AraDiCE
Jawaher
Kinayat
AraDiCE
Jawaher
Kinayat
AraDiCE
Fine-tuned on FannOrFlop
ALLaM-7B-Instruct
0.8990
0.8600
0.7889
0.9343
0.8333
0.7167
0.8990
0.8467
0.7222
Qwen3-8B
0.8333
0.6867
0.5556
0.8586
0.7333
0.4944
0.8586
0.7200
0.5278
Fanar-1-9B-Instruct
0.8687
0.7533
0.7167
0.8636
0.7267
0.6889
0.8586
0.7867
0.6278
Llama-3.1-8B-Instruct
0.7424
0.5733
0.5167
0.6919
0.5467
0.4889
0.6970
0.6200
0.5556
Fine-tuned on ArabCulture
ALLaM-7B-Instruct
0.8990
0.8467
0.7722
0.9040
0.8333
0.7444
0.8838
0.8333
0.7389
Appendix
Table 8: Zero-shot evaluation results across three runs with different random seeds on Jawaher, Kinayat, and AraDiCE datasets. Results show accuracy scores ( ↑ ) for models finetuned on FannOrFlop and ArabCulture full datasets.
Configuration
Subset
Seed
Jawaher
Kinayat
AraDiCE
Default
Palm
0
0.8687
0.8533
0.7333
42
0.8737
0.8133
0.7111
21
0.8434
0.8333
0.7056
FannOrFlop
0
0.8838
0.8667
0.7778
42
0.9040
0.8133
0.7167
21
0.8788
0.8067
0.7111
Appendix
Table 9: Zero-shot evaluation results for ALLaM-7B-Instruct on Jawaher, Kinayat, and AraDiCE datasets. Results show accuracy scores ( ↑ ) comparing the default LoRA configuration against two ablation settings.
Configuration
Dataset
Jawaher
Δ Jawaher
Kinayat
Δ Kinayat
AraDiCE
Δ AraDiCE
r=4, α =8, lr=5e-5
Palm Subset
0.8620
—
0.8333
—
0.7167
—
FannOrFlop
0.8889
—
0.8289
—
0.7352
—
r=16, α =32, lr=1e-5
Palm Subset
0.8737
+0.0118
0.8222
− 0.0111
0.7370
+0.0204
r=16, α =32, lr=5e-4
Palm Subset
0.8182
− 0.0438
0.8222
− 0.0111
0.7426
+0.0259
FannOrFlop
0.8788
− 0.0101
0.7756
− 0.0533
0.6870
− 0.0481
Appendix
Table 10: Average zero-shot evaluation results for ALLaM-7B-Instruct across three runs on Jawaher, Kinayat, and AraDiCE datasets. Δ values indicate the difference relative to the default configuration for the corresponding finetuning dataset. Results show accuracy scores ( ↑ ).
Task
Finetuned on
Δ
95% CI
p
ICC
Sig.
AraDiCE
Jawaher
−0.19
[−1.76,+1.39]
0.8101
−0.018
AraDiCE
Poetry
+0.00
[−1.48,+1.48]
0.9940
−0.006
Kinayat
ArabCulture
+0.78
[−1.22,+2.72]
0.4353
−0.011
Kinayat
Palm
+1.22
[−1.00,+3.44]
0.2881
−0.027
Kinayat
Poetry
+2.33
[+0.39,+4.28]
0.0207
−0.009
∗
Jawaher
ArabCulture
−0.08
[−1.56,+1.39]
0.9195
−0.078
Appendix
Table 11: Clustered aggregate effects, pooled across all four models (items resampled jointly). ICC is the intraclass correlation coefficient. Δ and CI bounds in percentage points. ∗ denotes p<0.05 .
Model
Finetuned on
Task
Seed
Δ
95% CI
p
ALLaM
Poetry
AraDiCE
42
−5.00
[−8.89,−1.11]
0.0225
Qwen
Palm
Idioms
42
+4.67
[+1.33,+8.00]
0.0156
Llama
Poetry
Idioms
21
+10.00
[+3.33,+16.67]
0.0081
ALLaM
Palm
Proverbs
21
−5.56
[−9.60,−1.52]
0.0127
Fanar
Palm
Proverbs
42
−3.54
[−6.57,−1.01]
0.0391
Appendix
Table 12: Per-run results significant by exact McNemar’s test ( p<0.05 ).
Count
Country
Question
Top 10 Most Frequently Improved
9
Jordan
؟ندرألاب ناضمر راهنب راطفإلاب رهجلا ةبوقع وش
7
Jordan
؟ندرألاب جاوزلا ديلاقتو تاداع يه وش
6
Syria
؟ايروس لهأ اهيف لفتحيب يللا دايعألا يه وش
6
Jordan
؟ندرألاب لافطألل ةيبعشلا باعلألا يه وش
6
Jordan
؟ندرألاب ةروهشم راجشأو تاتابن رتكأ وش
Appendix
Table 13: Top 10 most frequently improved and regressed questions from the AraDiCE dataset, with counts across all models fine-tuned on figurative language (proverbs and poetry).
Idiom (Arabic)
Impr
Regr
ْهشِو ْلَكَأ
9
3
ْهَدْلِجْلا ىَلَع
8
4
نِرِي ْهاَّلَخ
2
7
ْهَضاَخَمْلا ِّسَج
2
5
ْبلْقِتِو ْبَرْضِتِب اَيْنُّدلا
4
5
بياس هفك
6
2
Appendix
Table 14: Representative subset of the 21 unstable idioms (improved in some seeds, regressed in others), with counts across all fine-tuned models.
Proverb (Arabic)
Dialect
Impr
Regr
لمج عسوي بابلا
Kuwaiti
1
8
اهنب نم هسفن لماع
Egyptian
2
7
اودعا بيجت ام نطبلا
Libyan
1
6
صوردلا وارعني ام ژاژغتلا تعاس
Mauritanian
5
3
مادك نم ديأو هرو نم ديأ عجر
Iraqi
4
2
اشاب نايمعلا ىلع روعألا
Omani
1
2
Appendix
Table 15: Representative subset of the 15 unstable proverbs (improved in some seeds, regressed in others).
Idiom (Arabic)
Impr
Regr
ْهشِو ْلَكَأ
3
3
نِرِي ْهاَّلَخ
1
4
ْهَدْلِجْلا ىَلَع
4
2
ْبلْقِتِو ْبَرْضِتِب اَيْنُّدلا
4
1
ْهَضاَخَمْلا ِّسَج
1
2
يِدْنه ْرْمَت ْنَبَل ْكَمَس
1
2
Appendix
Table 16: Representative subset of the 15 unstable idioms (improved in some seeds, regressed in others) under poetry fine-tuning, with counts across all models.
Proverb (Arabic)
Dialect
Impr
Regr
حازم ةودعلا
Algerian
3
2
اهنب نم هسفن لماع
Egyptian
1
3
لبج لوقت تناو لمج لوقا انا
Omani
1
1
نيَعلا يحَتسِت مَفلا مَعطِا
Palestinian
2
1
يراوشلا زنخت ةدحو ةتوح
Moroccan
2
1
مادك نم ديأو هرو نم ديأ عجر
Iraqi
2
1
Appendix
Table 17: The 7 unstable proverbs (improved in some seeds, regressed in others) under poetry fine-tuning.
Dialect
Impr
Regr
Net
Mauritanian
30
10
+20
Yemeni
15
1
+14
Iraqi
7
2
+5
Tunisian
7
2
+5
Jordanian
11
7
+4
Kuwaiti
12
8
+4
Appendix
Table 18: Net fine-tuning effect by Arabic dialect variety on the proverb task.
Model / Dataset
Base%
FT%
Δ%
95% CI
p
Impr
Regr
ALLaM
ArabCulture (seed 0)
86.0
84.7
−1.33
[−5.33,+2.67]
0.754
4
6
ArabCulture (seed 21)
83.3
82.0
−1.33
[−6.00,+3.33]
0.774
5
7
ArabCulture (seed 42)
82.7
81.3
−1.33
[−5.33,+2.67]
0.754
4
6
Palm (seed 0)
86.0
85.3
−0.67
[−5.33,+4.00]
1.000
6
7
Palm (seed 21)
83.3
83.3
±0.00
[−4.67,+4.67]
1.000
6
6
Appendix
Table 19: Improvement and regression breakdown on Kinayat idioms across cultural fine-tuning (ArabCulture and Palm) and poetry fine-tuning (FannOrFlop). Base% and FT% are accuracy before and after fine-tuning; Δ% is the percentage-point change, reported with 95% paired-bootstrap confidence intervals over test items and exact McNemar p -values; Impr and Regr are the number of individual predictions improved or worsened. ∗ marks p<0.05 .
Model / Dataset
Base%
FT%
Δ%
95% CI
p
Impr
Regr
ALLaM
ArabCulture (seed 0)
88.9
88.9
±0.00
[−2.53,+2.53]
1.000
3
3
ArabCulture (seed 21)
89.9
87.4
−2.53
[−5.05,+0.00]
0.125
1
6
ArabCulture (seed 42)
90.9
90.9
±0.00
[−3.03,+3.03]
1.000
5
5
Palm (seed 0)
88.9
86.9
−2.02
[−5.56,+1.52]
0.388
4
8
Palm (seed 21)
89.9
84.3
−5.56∗
[−9.60,−1.52]
0.013
3
14
Appendix
Table 20: Improvement and regression breakdown on Jawaher proverbs across cultural fine-tuning (ArabCulture and Palm) and poetry fine-tuning (FannOrFlop). Base% and FT% are accuracy before and after fine-tuning; Δ% is the percentage-point change, reported with 95% paired-bootstrap confidence intervals over test items and exact McNemar p -values; Impr and Regr are the number of individual predictions improved or worsened. ∗ marks p<0.05 .
Model / Dataset
Δ%
95% CI
Impr
Regr
ALLaM
Jawaher (seed 0)
+1.11
[−3.33,+5.56]
10
8
Jawaher (seed 21)
−1.11
[−5.56,+3.33]
8
10
Jawaher (seed 42)
−3.89
[−7.78,+0.00]
3
10
FannOrFlop (seed 0)
+1.11
[−3.33,+5.56]
9
7
FannOrFlop (seed 21)
−1.67
[−5.56,+1.67]
4
7
Appendix
Table 21: Improvement and regression breakdown on AraDiCE-Culture across figurative fine-tuning (Jawaher) and poetry fine-tuning (FannOrFlop). Δ% is the percentage-point change from base to fine-tuned accuracy, reported with 95% paired-bootstrap confidence intervals over test items; Impr and Regr are the number of individual predictions improved or worsened. ∗ marks p<0.05 under an exact McNemar test.
Count
Idiom
Top 15 Most Frequently Improved
16
ْدوُع ىَلَع ْدوُد
9
ْهَديِدَحْلا ىَلَع
9
ْهشِو ْلَكَأ
8
ْهَدْلِجْلا ىَلَع
6
ْعاَرِّدلاو ْعاَبْلاِب
Appendix
Table 22: Top 15 most frequently improved and regressed idioms across all models fine-tuned on cultural data (ArabCulture and Palm).
Count
Dialect
Proverb
Top 15 Most Frequently Improved
10
Mauritanian
.ْكلْكَّلَعْي ال ْكاعْما ْدِراو ُهاَمْلا
9
Mauritanian
لازغا وما نيعفا سونخب لك
6
Yemeni
قزري كبرو قزخ
6
Lebanese
سبايلاو رَضْخألا لكأ
6
Kuwaiti
.رحب هطقو ريخ وس
Appendix
Table 23: Top 15 most frequently improved and regressed proverbs across all fine-tuned models, with counts and dialect variety.
Dialect
Impr
Regr
Net
Yemeni
10
2
+8
Mauritanian
11
3
+8
Jordanian
8
1
+7
Kuwaiti
6
0
+6
Tunisian
5
1
+4
MSA
4
0
+4
Appendix
Table 24: Net poetry fine-tuning effect by Arabic dialect variety on the proverb task.
Count
Idiom
Top 15 Most Frequently Improved
8
ْدوُع ىَلَع ْدوُد
6
ّْبَد ْنِمْو ِّبَه ْنِم
4
ْهَدْلِجْلا ىَلَع
4
ْبلْقِتِو ْبَرْضِتِب اَيْنُّدلا
3
ْهَديِدَحْلا ىَلَع
Appendix
Table 25: Top 15 most frequently improved and regressed idioms across all models fine-tuned on poetry data (FannOrFlop).
Table 26: Top 15 most frequently improved and regressed proverbs across all models fine-tuned on poetry data (FannOrFlop), with counts and dialect variety.
Posts and Telecommunications Institute of Technology (PTIT), Vietnam · School of Computer Science and Information Technology, University College Cork, Ireland