Toward Culturally Grounded Natural Language Processing

Sina Bagheri Nezhad

cs.CLMar 27, 2026

Toward Culturally Grounded Natural Language Processing

Authors: Sina Bagheri Nezhad

Organizations: Independent Researcher, Seattle, WA, USA

Abstract

Multilingual NLP is often treated as a route to global inclusion, but linguistic coverage and cultural competence frequently diverge. This paper synthesizes over 50 papers spanning multilingual performance inequality, cross-lingual transfer, culture-aware evaluation, cultural alignment, multimodal benchmarks, benchmark-design critique, and community-grounded data practices. Across this literature, training data coverage remains important, but outcomes are also shaped by tokenization, prompt language, translated benchmark design, culturally grounded supervision, modality, and who authors or validates evaluation data. We argue that culturally grounded NLP should move beyond treating languages as isolated rows in benchmark tables and instead model communicative ecologies: the institutions, scripts, domains, modalities, and communities through which language is used. We propose a layered evaluation and reporting agenda centered on representation audits, mixed elicitation, ecological validity, community validation, adaptation provenance, within-language variation, and maintenance of living cultural resources.

Figures & tables

Panel of Figure 1 — Figure 1: From multilingual coverage to culturally grounded NLP. The first three stages reflect common multilingual framing: increasing language coverage, explaining cross-lingual transfer, and approximating culture through indirect proxies. The paper argues for a further shift toward communicative ecologies, where language use is situated in communities, institutions, domains, and modalities, and where evaluation and adaptation are grounded in slice-based analysis, community validation, and explicit reporting.

Theme	Representative literature	Main implication
Coverage and disparity	( Joshi et al., 2020 ; Blasi et al., 2022 ; Adelani et al., 2024 ; Team et al., 2022 )	Aggregate multilingual progress still leaves many languages and communities poorly served.
Transfer factors	( Philippy et al., 2023 ; Limisiewicz et al., 2023 ; Hämmerl et al., 2025 ; Bagheri Nezhad and Agrawal, 2024 ; Bagheri Nezhad et al., 2025 )	Pretraining, script, and tokenizer behavior shape transfer, but they do not by themselves measure cultural fitness.
Culture definitions and surveys	( Adilazuarda et al., 2024 ; Liu et al., 2025a ; Pawar et al., 2025 ; Zhou et al., 2025 )	“Culture” is multi-faceted and usually operationalized through incomplete proxies.
Text and value-oriented evaluation	( Wang et al., 2024 ; Zhao et al., 2024 ; Singh et al., 2025 ; Kim and Lee, 2025 ; Chiu et al., 2025 ; Rystrøm et al., 2025 )	Model rankings and failure modes change on culturally sensitive, locally authored, or culturally situated subsets.
Alignment and adaptation	( AlKhamissi et al., 2024 ; Masoud et al., 2025 ; Xu et al., 2025 ; Guo et al., 2025 ; Liu et al., 2025a ; Feng et al., 2025 )	Cultural behavior can shift with prompts or targeted supervision, but gains depend on grounded data and task framing.
Multimodal, interactional, and local tasks	( Nayak et al., 2024 ; Schneider et al., 2025 ; Yang et al., 2025 ; Belay et al., 2025 ; Maji et al., 2025 ; Winata et al., 2025 ; Villa-Cueva et al., 2025 ; Nyandwi et al., 2025 ; Havaldar et al., 2025 ; Ochieng et al., 2025 ; Aji and Cohn, 2025 ; Ferawati et al., 2024 ; Pranida et al., 2025 ; Naous et al., 2024 ; Naous and Xu, 2025 )	Local knowledge, interactional norms, region-specific entities, and within-language variation remain difficult across settings and modalities.

Table 1: Representative lines of work that jointly motivate a culture-grounded view of multilingual NLP.

Protocol layer	Common shortcut	Stronger practice	Representative literature
Representation audit	Country/language labels only	Report authorship, language variety, translation pipeline, and whether majority/minority or subcultural groups are represented	( Wu et al., 2025 ; Alkhamissi et al., 2026 ; Faisal et al., 2024 )
Elicitation diversity	Multiple-choice or Likert only	Combine closed items with open generation, pairwise judgments, and qualitative error analysis	( Oh et al., 2025 ; Kabir et al., 2025 ; Havaldar et al., 2025 )
Ecological validity	Static text-only QA	Add conversation, web-agent, image, video, and region-specific task slices	( Qiu et al., 2025 ; Shafique et al., 2025 ; Nayak et al., 2024 )
Community validation	Expert-only or automatic scoring	Include native-speaker review, disagreement analysis, and participatory co-design	( Müller-Eberstein et al., 2025 ; Ivetta et al., 2025 ; Verma et al., 2026 )
Adaptation reporting	“Culture-tuned” as a black box	Publish supervision provenance, target population, and trade-offs across groups and tasks	( Guo et al., 2025 ; Pham et al., 2025 ; Stepanyan et al., 2026 )

Table 2: A practical protocol for evaluating cultural competence beyond language-only leaderboards.

Cross-Cultural Language Model Evaluation

Cross-cultural language model evaluation tests whether AI systems understand and respond appropriately across languages, communities, and cultural contexts.

See all 109 papers

CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks
Apr 21, 2026Peiqin Lin, Chenyang Lyu, Wenjiang Luo +22Multilingual Language Model Evaluation LLM Evaluation
Alibaba Group · Beijing Language and Culture University · LMU Munich +6
The GaoYao Benchmark: A Comprehensive Framework for Evaluating Multilingual and Multicultural Abilities of Large Language Models
Apr 22, 2026Yilun Liu, Chunguang Zhao, Mengyao Piao +14Multilingual Language Model Evaluation LLM Evaluation
Huawei, China · Fudan University, China
Lost in the Tower of Babel: The Adverse Effects of Incidental Multilingualism in LLMs
May 2, 2026Anjishnu Mukherjee, Chutong Meng, Antonios AnastasopoulosMultilingual Language Models Multilingual Language Model Evaluation
Department of Computer Science George Mason University

Apr 21, 2026J/K move · Enter open · S save

CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks

Peiqin Lin, Chenyang Lyu, Wenjiang Luo, Haotian Ye+21

Alibaba Group · Beijing Language and Culture University · LMU Munich+6

Multilingual Language Model Evaluation LLM Evaluation LLM Grounding Evidence-Grounded Reasoning Cross-Cultural Language Model Evaluation Cultural Knowledge in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) are now deployed worldwide, inspiring a surge of benchmarks that measure their multilingual and multicultural abilities. However, these benchmarks prioritize generic language understanding or superficial cultural trivia, leaving the evaluation of grounded tasks -- where models must reason within real-world, context-rich scenarios -- largely unaddressed. To fill this gap, we present CulturALL, a comprehensive and challenging benchmark to assess LLMs' multilingual and multicultural competence on grounded tasks. CulturALL is built via a human--AI collaborative framework: expert annotators ensure appropriate difficulty and factual accuracy, while LLMs lighten the manual workload. By incorporating diverse sources, CulturALL ensures comprehensive scenario coverage. Each item is carefully designed to present a high level of difficulty, making CulturALL challenging. CulturALL contains 2,610 samples in 14 languages from 51 regions, distributed across 16 topics to capture the full breadth of grounded tasks. Experiments show that the best LLM achieves 44.48% accuracy on CulturALL, underscoring substantial room for improvement.

Abstract

Cross-Cultural Language Model Evaluation

Explore similar work

CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks

The GaoYao Benchmark: A Comprehensive Framework for Evaluating Multilingual and Multicultural Abilities of Large Language Models

Lost in the Tower of Babel: The Adverse Effects of Incidental Multilingualism in LLMs