cs.CLOct 18, 2026
SaveAligning Cultural Behaviors in Arabic Language Models via Preferences
Organizations: Mohamed bin Zayed University of Artificial Intelligence · IBM Research AI
Abstract
Arabic LLMs have improved rapidly, but aligning them with country-specific cultural behavioral preferences remains underexplored. Previous work has focused primarily on instruction-tuning for factual knowledge, overlooking nuanced cultural behaviors. We propose ArabPref, which to our knowledge is the first large-scale pan-Arab preference data set for the alignment of cultural behavior conditioned on the country. Our contribution includes two resources in English and Modern Standard Arabic: a culturally grounded preference dataset for training and evaluation, and a multiple-choice benchmark designed to assess culturally aligned behavior across nations. All test data are validated by native speakers to ensure authenticity. To optimize cultural alignment, we experiment with post-training methods across multilingual and Arabic-centric language models, evaluating performance on both generation tasks and our cultural multiple-choice benchmark. By releasing an evaluation benchmark, we provide a foundation for systematically studying and improving culturally aligned behavior in language models.
Figures & tables
Figure 1: An Overview of our ArabPref dataset, covering preference and MCQ data across 22 Arab League countries, spanning multiple categories and evaluation aspects, and available in English and Arabic. The upper part demonstrates two preference dataset samples in English and Arabic from Tunisia and Oman, respectively, while the bottom part demonstrates two MCQs from Saudi Arabia and Syria.
| Dataset | Arab-Specific | Topic Granularity | Prefs | Eval (MCQ) | Human Valid. | Langs | Scale |
|---|---|---|---|---|---|---|---|
| CARE | Partial | – | Yes | No | Yes | Multi (incl. Ar) | 3.5K Q / 32K prefs |
| Cidar | Yes | 17 topics | No | No | Yes | Ar (MSA) | 10K instr. |
| Palm | Yes | 20 (coarse) | No | No | Yes | Ar (MSA + Dial.) | 17.4K instr. |
| ArabCulture | Yes | 12 domains | No | Yes | Yes | Ar (MSA) | 3.5K MCQs |
| ArabPref (Our) | Yes | 200 (fine) | Yes | Yes | Yes | Ar (MSA), En | 31K train / 6K test / 2K MCQ |
Table 1: Comparison of cultural datasets for analyzing and aligning LLM behavior. Existing resources focus on instruction tuning ( Cidar , Palm ), evaluation ( ArabCulture ), or limited preference annotation ( CARE ). ArabPref uniquely combines pan-Arab coverage, fine-grained topics, and explicit behavioral preferences, enabling controlled preference-based alignment and assessment. Unlike Cidar and Palm , ArabPref supports preference optimization; unlike ArabCulture , it supports both training and evaluation; unlike CARE , it provides comprehensive pan-Arab coverage with fine-grained country-level topics and a unified MCQ benchmark.
Figure 2: Overview of the data curation and validation pipeline. Our process begins with the manual collection of cultural behavioral descriptions from diverse online sources and URLs, supplemented by CultureBank. After automated semantic filtering and paraphrasing, the data undergoes a rigorous Human-in-the-loop (HITL) validation phase. Annotators perform an inspection to ensure cultural authenticity. For the MCQ generation, a Quality Control (QC) loop is implemented where samples with low accuracy are re-annotated. Finally, the data is translated into MSA and verified to produce the final bilingual Preference and MCQ datasets.
| Country | Pref Train | Pref Test | MCQ | |||
|---|---|---|---|---|---|---|
| En | Ar | En | Ar | En | Ar | |
| Egypt | 1219 | 1219 | 329 | 329 | 62 | 62 |
| Morocco | 1114 | 1114 | 226 | 159 | 59 | 59 |
| Saudi Arabia | 972 | 972 | 141 | 141 | 52 | 52 |
| Algeria | 958 | 958 | 165 | 131 | 50 | 50 |
| Jordan | 924 | 924 | 254 | 202 | 72 | 72 |
Table 2: Statistics of the ArabPref dataset, including English and Arabic preference data and MCQ coverage across 22 Arab League countries. Where the test preferences and MCQs are verified by human annotators.
| Model & Method | English (En) | Arabic (Ar) | ||||||
|---|---|---|---|---|---|---|---|---|
| Cultural | General | Literacy | Avg. | Cultural | General | Literacy | Avg. | |
| Llama-3.1-8B-instruct | ||||||||
| Vanilla | 3.23 | 3.35 | 3.63 | 3.40 | 3.13 | 3.24 | 3.07 | 3.15 |
| w/ Palm (SFT) | 3.74* | 3.92* | 4.08 * | 3.91* | 3.36* | 3.49* | 3.88 * | 3.58* |
| w/ CARE (DPO) | 3.85* | 4.08* | 3.83* | 3.92* | 3.24* | 3.38* | 3.27* | 3.30* |
| w/ CARE (KTO) | 3.66* | 3.82* | 3.69* | 3.72* | 3.17 | 3.31* | 3.08 | 3.19 |
Table 3: Preference-alignment results in English and Arabic, evaluated using an LLM-as-a-judge framework along three dimensions: Cultural Appropriateness, General Quality, and Literacy. All values are scores on a 1–5 Likert scale. Cell colors indicate changes relative to the corresponding Vanilla model: green denotes improvement and red denotes degradation, with darker shades representing larger absolute changes. The best result for each model is highlighted in bold . Asterisks on the changes indicate statistically significant differences from the corresponding vanilla model ( ) through two-sided paired permutation tests. Proprietary models are evaluated in a zero-shot setting.
| Model & Method | English (En) | Arabic (Ar) | ||
|---|---|---|---|---|
| Acc. | Acc. | |||
| Jais-adapted-7B-chat | ||||
| Vanilla | 49.6 | – | 39.4 | – |
| w/ Palm (SFT) | 40.4 | 37.1 | ||
| w/ CARE (DPO) | 51.0 | 45.6 | ||
| w/ CARE (KTO) | 50.0 | 40.1 | ||
Table 4: MCQ accuracy (%) on the English and Arabic datasets. denotes the absolute change relative to the corresponding Vanilla model. For each open-source model, the best result is shown in bold and the second-best result is underlined. Proprietary models are evaluated in the zero-shot setting.
Figure 3: Scaling behavior on the ArabPref MCQ benchmark. The upper and lower panels report accuracy on the English (En) and Arabic (Ar) test sets, respectively. Results are shown for Qwen2.5-Instruct, Qwen3, and Jais-adapted models across different model sizes. Each language contains 992 questions.
Figure 4: Head-to-head win rate comparisons evaluated by GPT-4o as judge. Each bar shows the win percentage of the respective system relative to the 50% midpoint. (a) DPO training: our merged dataset ( ArabPref + CARE ) vs. CARE -only across three base models and two languages (Ar/En). (b) SFT training: merged dataset vs. Palm . Results show that the merged dataset consistently achieves competitive or superior win rates, demonstrating that the observed gains generalise beyond vanilla baselines.
| Model | Language | Training Data | ||
|---|---|---|---|---|
| English | Arabic | Whole | ||
| Llama | En | 3.53 | 3.47 | 3.78 |
| Ar | 3.17 | 3.08 | 3.59 | |
| Qwen | En | 4.56 | 4.63 | 4.41 |
| Ar | 4.10 | 3.96 | 4.02 | |
| Jais | En | 4.34 | 4.30 | 4.53 |
Table 5: Model performance comparison with monolingual versus whole-dataset training in preference generation. Cell colors compare English and Arabic training against the corresponding Whole-training score. Green indicates an improvement, red indicates a decrease, and darker shades indicate larger absolute differences.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Country | Sources |
|---|---|
| Saudi Arabia | S1 , S2 , S3 , S4 , S5 |
| United Arab Emirates | UAE1 , UAE2 , UAE3 , UAE4 , UAE5 , UAE6 , UAE7 |
| Yemen | Y1 , Y2 , Y3 , Y4 , Y5 , Y6 |
| Jordan | J1 , J2 , J3 , J4 , J5 , J6 |
| Lebanon | L1 , L2 , L3 , L4 , L5 , L6 , L7 |
| Syria | SY1 , SY2 , SY3 , SY4 , SY5 , SY6 , SY7 |
Table 6: List of travel etiquette and travel-advisory sources used in our work.
Figure 5: Statistics of the number of aspects for each cultural category in ArabPref .
| Country | Train | Test (Raw) | Test (Filtered) |
|---|---|---|---|
| Egypt | 1,219 | 329 | 329 |
| Morocco | 1,114 | 237 | 226 |
| Saudi Arabia | 972 | 160 | 141 |
| Algeria | 958 | 225 | 165 |
| Jordan | 924 | 257 | 254 |
| Lebanon | 907 | 207 | 199 |
Table 7: Statistics of the English preference dataset across 22 countries, detailing the sample counts for training, raw test, and human-filtered test splits.
| Category | Train | Test (Raw) | Test (Filtered) | MCQ |
|---|---|---|---|---|
| Social Etiquette | 3327 | 716 | 605 | 458 |
| Religion & Rituals | 1647 | 440 | 336 | 172 |
| Dining & Food | 1553 | 369 | 283 | 214 |
| Hospitality & Home Life | 1497 | 386 | 325 | 246 |
| Gender & Travel | 745 | 193 | 159 | 98 |
| Personal Appearance | 707 | 170 | 155 | 90 |
Table 8: Top 10 categories by sample count in the preference dataset across the training, raw test, and human-filtered test splits, along with the sample count in the MCQ dataset.
| Category (Number of unique aspects) | Aspects |
|---|---|
| Social Etiquette (42) | Greetings, Names and Titles, Handshakes, Eye Contact, Introductions, Personal Space, Gestures & Hand Gestures, Expressing Thanks, Expressing Refusal, Seating, Joking & Humor, Public Affection, Conversation / Asking Questions, Language Use, Meeting Time / Punctuality, Public Behavior / Street Interactions, Queuing / Queue Etiquette, Communication Style, Using Arabic/French, Touching Others, Greeting with “Salaam Alaikum”, Flexible Time / Punctuality, Urban/Rural Dialect Switching, Respect for Elders, Showing Soles of Feet, Right-Hand Use for Giving/Receiving, Avoiding Pointing with Index Finger, Silence and Pauses in Conversation, Indirect Communication / Saving Face, Honor and Shame Sensitivity, Compliments & “Mashallah” Responses, Asking About Family (Rapport-Building), Taboo Topics (politics/sectarian issues), Public Displays of Anger, Loud Laughter in Public, Queue Courtesy for Families/Elders, Greeting Order When Entering a Room, Door Etiquette (who enters first), Avoid Passing Objects Over Someone’s Head, Accommodating Left-Handed Guests, Respectful Address to Domestic Workers, Sports Events Etiquette (football matches) |
| Hospitality & Home Life (25) | Visiting Homes, Shoes in Homes, Invitations, Gifts, Hospitality, Thanking Host, Inviting to Eat, Leaving a Gathering, Hospitality to Strangers, Gift-Giving of Sweets, Home Hospitality Rituals, Role of Host in Welcoming, Visiting the Sick (Get-Well Etiquette), Condolences / Mourning Visits, Serving / Seating Elders First, Guest of Honor Seating, Polite Gift Refusal and Re-offer Ritual, Bringing Dates or Fruit to Hosts, Women’s Gatherings (Hareem Majlis), Men’s Majlis Etiquette, Incense (Bukhoor) for Guests, Gift Wrapping & Color Preferences, Reciprocity / Return-Gift Norms, Hosting Unannounced Visitors, Sharing Food with Neighbors |
| Religion & Rituals (24) | Religious Sites, Religious Items, Ramadan, Prayer Times / Friday Observance / Friday Prayers, Public Worship Spaces, Festivals / Holidays, Wedding Etiquette, Friday (Jumu’ah) Observance, Henna Traditions, Saying “Bismillah” Before Eating, Eid al-Fitr Etiquette (Zakat al-Fitr, greetings), Eid al-Adha Etiquette (Qurbani sharing), Removing Shoes in Mosques, Adhan (Call to Prayer) Respect (pausing music), Using “Inshallah / Alhamdulillah” Appropriately, Visiting Shrines / Saints’ Tombs, Ramadan: Non-Muslim Daytime Eating Etiquette, Prayer Rooms in Malls/Airports Etiquette, Wudu (Ablution) Facility Etiquette, Visiting Cemeteries Etiquette, Using Honorifics (Sheikh, Hajji), Addressing by Kunya (Abu/Umm), Name Order & “bin/bint”, Funeral Attire and Participation |
| Dining & Food (25) | Table Manners / Dining Etiquette, Bread Etiquette, Food (Dietary Norms), Finishing Food, Tea / Coffee Ritual, Refusing Food / Tea, Street Food, Serving & Drinking Arabic Coffee (Qahwah), Communal Eating from a Shared Dish, Bread is Sacred, Suhoor & Iftar Hosting, Offering Dates Before Meals, Eating with the Right Hand, Using Bread as an Utensil, Serving Elders First (Meals), Accepting Second Helpings, Avoiding Food Waste, Shisha / Hookah Etiquette, Street Coffee/Tea Kiosk Etiquette, Spice/Heat Consideration for Guests, Family Sections in Restaurants, Coffee vs Tea Preference by Region, Fruit Plate Ritual After Meals, Alcohol, Cafés & Coffee Culture |
| Business & Work (14) | Business Etiquette / Business Cards / Business Attire, Dealing with Officials, Business Gift-Giving, Relationship-First Negotiation Style, Wasta (Networking & Intermediation), Decision-Making Hierarchy, Scheduling Around Prayer/Friday, Polite Persistence in Follow-ups, Business Entertaining (non-alcoholic venues), Acceptable Small Talk Topics, Timing of Gifts in Business (after meeting), Business Card Handling (Arabic side up), Office Visiting Hours & Drop-ins, Formal Greetings in Email/Messaging |
| Modern & Technology (9) | Mobile Phones, Texting/Messaging, Social Media Etiquette, Smoking Etiquette, Photography of Children, Saving Contact Names with Titles (phones), WhatsApp Voice Notes Etiquette, Livestreaming in Public Etiquette, Sharing Religious Content Online Etiquette |
Table 9: Cultural categories and aspects of ArabPref. The taxonomy contains 16 categories and 200 unique aspects. Duplicate aspect entries in the mapping were counted once.
| Aspect | Country | Preferred Norm (paraphrased) | Stance |
| Alcohol | Saudi Arabia | Strictly prohibited; can lead to severe penalties. | Prohibited |
| Kuwait | Strictly prohibited; avoid bringing any alcohol. | Prohibited | |
| Libya | Illegal; avoid bringing or offering alcohol. | Prohibited | |
| Algeria | Safest to avoid possession entirely. | Restricted | |
| UAE | Restricted to licensed venues; drink responsibly. | Licensed only | |
| Jordan | Acceptable at licensed restaurants and hotels. | Licensed only |
Table 10: Examples of divergent preferred norms across Arab nations for four cultural aspects in ArabPref . Preferred responses are condensed from dataset entries. The Stance column summarizes the normative position along the observed spectrum.
| Score | Cultural Appropriateness | General Quality | Literacy |
|---|---|---|---|
| 1 | Completely misaligned with the cultural context, containing offensive or highly inappropriate references. | Very incomplete, with critical information missing, making the response unusable. | Very poor fluency, riddled with errors, making the text nearly unreadable or incoherent. |
| 2 | Major cultural misunderstandings or insensitive references that impact the overall understanding. | Significant omissions that make the instruction or question difficult to interpret or incomplete. | Multiple errors that hinder readability or cause significant confusion for the reader. |
| 3 | Noticeable cultural inconsistencies or slight misinterpretations, but the general meaning remains clear. | Noticeable gaps in information or coverage that leave the instruction or question lacking. | Noticeable errors in grammar, spelling, or structure, but the text is still understandable. |
| 4 | Minor cultural inaccuracies that do not significantly affect the overall understanding. | Slightly incomplete, with minor missing details that do not impact the overall understanding. | Minor grammatical or stylistic errors that do not significantly affect readability. |
| 5 | Fully aligned with the cultural context, showing deep respect for cultural norms and values. | Fully complete, with no missing information, thoroughly covering the required context. | Perfect fluency, with no errors in grammar, spelling, or structure. The text reads naturally. |
Table 11: Evaluation criterion for human annotation and LLM-as-a-judge.
Figure 6: Sample efficiency of preference data alignment on Qwen2.5-7B-Instruct, evaluated on MCQs. Qwen2.5-7B-Instruct performance with SFT, DPO, and DITTO (10–100 samples). Dotted lines show zero-shot baselines.
Figure 7: Sample efficiency of alignment methods on Arabic and English cultural preference MCQ evaluation. Qwen2.5-7B-Instruct performance across SFT, DPO, and KTO with 10–100% of training data. Baselines (dotted) show zero-shot performance.
Figure 8: Scale comparison of alignment methods on Arabic cultural preference MCQ evaluation. Qwen2.5-7B-Instruct performance in small-scale (top; 10–100 samples with SFT, DPO, DITTO) and large-scale (bottom; 10–100% data with SFT, DPO, KTO) settings. Baselines (dotted) show zero-shot performance.
Figure 9: Scale comparison of alignment methods on English cultural preference MCQ evaluation. Qwen2.5-7B-Instruct performance in small-scale (top; 10–100 samples with SFT, DPO, DITTO) and large-scale (bottom; 10–100% data with SFT, DPO, KTO) settings. Baselines (dotted) show zero-shot performance.
| Method | English | Arabic | ||||||
|---|---|---|---|---|---|---|---|---|
| Cultural | General | Literacy | Avg. | Cultural | General | Literacy | Avg. | |
| Vanilla | 3.41 | 3.58 | 4.07 | 3.69 | 3.89 | 4.08 | 3.98 | 3.98 |
| Palm (SFT) | 4.02 | 4.28 | 4.57 | 4.29 | 3.36 | 3.53 | 3.68 | 3.52 |
| CARE (DPO) | 4.28 | 4.55 | 4.06 | 4.30 | 4.21 | 4.46 | 3.66 | 4.11 |
| CARE (KTO) | 4.11 | 4.37 | 4.00 | 4.16 | 4.19 | 4.44 | 3.57 | 4.07 |
| ArabPref (DPO) | 4.02 | 4.26 | 4.33 | 4.20 | 3.87 | 4.14 | 4.09 | 4.03 |
Table 12: Scale-controlled sampled-data ablation on Qwen-2.5-7B-Instruct. We construct ArabPref by sampling 30 preference pairs per country across diverse aspects, in order to reduce the data-volume advantage of combining ArabPref with CARE . Under this controlled setting, ArabPref + CARE with DPO achieves the best English Cultural score and English average score, outperforming the CARE -only DPO baseline. This indicates that ArabPref contributes complementary cultural preference signals beyond merely increasing the amount of training data.
| Cultural | General | Literacy | |
|---|---|---|---|
| En | 0.7140 | 0.8312 | 0.8950 |
| Ar | 0.6707 | 0.7801 | 0.7398 |
Table 13: Spearman’s values for different categories (Cultural, General, Literacy) for both English (En) and Arabic (Ar).
Figure 10: Annotation for stage 1 (Cultural Preference Quality Check) and stage 2 (Cultural Preference MSA Verification), for creating preference data of ArabPref . (Page 1)
Figure 11: Annotation for stage 1 (Cultural Preference Quality Check) and stage 2 (Cultural Preference MSA Verification), for creating preference data of ArabPref . (Page 2)
Figure 12: Annotation for stage 1 (Cultural Preference Quality Check) and stage 2 (Cultural Preference MSA Verification), for creating preference data of ArabPref . (Page 3)
Figure 13: Annotation for stage 3 (MCQ answering) and stage 4 (MCQ verification), for creating MCQ data of ArabPref . (Page 1)
Figure 14: Annotation for stage 3 (MCQ answering) and stage 4 (MCQ verification), for creating MCQ data of ArabPref . (Page 2)
Figure 15: Annotation for stage 3 (MCQ answering) and stage 4 (MCQ verification), for creating MCQ data of ArabPref . (Page 3)