Cultural Knowledge in Language Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 44% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 77

All topics

Language Unalignability: Why Some Concepts Resist Cross-Cultural Benchmark Evaluation
Oct 6, 2026Shu-Kai Hsieh, Da-Chen LianMultilingual Language Model Evaluation LLM Evaluation
National Taiwan University
Artificial Intelligence and the New Science of Culture
Oct 5, 2026Douglas R. Guilbeault, Bhargav Srinivasa Desikan, James A. EvansCultural Evolution Computational Social Science
Graduate School of Business, Stanford University, U.S. · Oxford Internet Institute, University of Oxford, U.K. · Sociology & Data Science, University of Chicago, U.S.
Knowing the Rules, Applying the Rules: Evaluating Language Models on Traditional Chinese Bazi
Oct 5, 2026Jiulin Li, Ping HuangLLM Evaluation Cultural Knowledge in Language Models
Beijing Liuyi Guanhua Technology Co., Ltd. · State Key Laboratory of General Artificial Intelligence, BIGAI
CineSubBench: Evaluating LLMs on Long-Form Narrative and Cultural Understanding from Multilingual Movie Subtitles
Sep 28, 2026Mir Tafseer Nayeem, Susmoy Chakraborty, Davood RafieiMultilingual Language Model Evaluation Long-Context Language Model Evaluation
University of Alberta · Independent Researcher
AraDynFact: Dynamic Evaluation of Factual Knowledge in Arabic
Sep 28, 2026Ignacio Iacobacci, Faroq Altam, Zhaozhi Qian +1LLM Evaluation Arabic NLP
Elm Company
CRISP: Cultural Reward Modeling for Implicit Situated Propriety
Sep 28, 2026Zekun Yuan, Yangfan Ye, Baohang Li +6Reward Modeling RL for Language Models
Harbin Institute of Technology · Huawei Technologies Co., Ltd · Peng Cheng Laboratory
Recognized but Not Produced: A Generation Benchmark for Culturally Specific Kinship Terms
Sep 22, 2026Sahil Pardasani, Madhusudan SinghMultilingual Language Model Evaluation Language Model Generation Evaluation
Blockchain Data Intelligence Lab The Pennsylvania State University University Park, PA, USA
FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing
Sep 21, 2026Yusser Al Ghussin, Eva Gavaller, Cristina España-Bonet +2Multilingual Language Model Evaluation LLM Auditing
Saarland University · German Research Center for Artificial Intelligence (DFKI) · Barcelona Supercomputing Center (BSC-CNS), Barcelona, Catalonia, Spain
MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education
Sep 16, 2026Luyao Zhu, Xun Wei Yee, Wei Li +2VLM Evaluation Large Vision-Language Models
AI Singapore, National University of Singapore, Singapore · Institute of Advanced Intelligence and Computing, A*STAR · School of Computing, National University of Singapore, Singapore
Cultural Competence in Context: A Large Language Model Passes the Turing Test in Finland
Sep 16, 2026Otto Segersven, Pentti HenttonenMultilingual Language Model Evaluation LLM Evaluation
Department of Computer Science, University of Helsinki · Faculty of Social Sciences, University of Tampere
Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering
Sep 16, 2026Pablo Poulenard, Yannis Karmim, Valentin BarrièreRetrieval-Augmented Generation Graph Retrieval-Augmented Generation
DCC, Universidad de Chile, Santiago, Chile · École Polytechnique, Palaiseau, France · Inria, Almanach, Paris, France +2
Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture
Sep 14, 2026Cody Kommers, Mingrui Ye, Evelyn Gius +4Pragmatic Reasoning in Language Models Multimodal Large Language Models
King’s College London London, United Kingdom · University of Edinburgh Edinburgh, United Kingdom
Enemray: Toward Capable Language Models for Hassaniya
Sep 13, 2026Cheikh AhmedLanguage Modeling Cultural Knowledge in Language Models
Multilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu
Sep 12, 2026Farah Adeeba, Abdul Rafae Khan, Rajesh Bhatt +1Multilingual Language Model Evaluation Cultural Knowledge in Language Models
University of Konstanz, Germany · University of Massachusetts Amherst, USA · Malaysia School of Information Technology, Monash University Malaysia +1
When Metrics Reward the Worst Translations: Internalizing Cultural Reasoning for Social Media Translation Evaluation
Sep 8, 2026Yiwen Qiu, Linjuan Wu, Dingming Li +7Reinforcement Learning Cultural Knowledge in Language Models
Zhejiang University · Xiaohongshu Inc.
BanglaMemeX: Advancing Cultural Metaphoric Image Interpretation in Bangla with a Multimodal Explainable Dataset
Sep 7, 2026Md. Sadman Sakib, Zisan Mahmud, Md. Fahim Arefin +1Figurative Language Understanding Multilingual VLM Evaluation
University of Dhaka · University of Texas at Dallas
CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning
Sep 3, 2026Bo Zeng, Linfeng Gao, Peiqin Lin +9Multimodal Reasoning Multilingual VLM Evaluation
Alibaba Group · Xiamen University · University of Hamburg
Accurate in space, unreliable in time: how LLMs represent national cultural change
Sep 1, 2026Yalda Daryani, Miranda Bogen, Madeleine I. G. DaeppLLM Evaluation Cultural Evolution
Department of Psychology, University of Southern California · Center for Democracy and Technology
VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages
Sep 1, 2026Usneek Singh, Poorvaja Veera Balaji Kumar, Parth Nanda +4Multilingual Language Model Evaluation LLM Evaluation
Georgia Institute of Technology · University of Texas at Austin
MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models
Sep 1, 2026Tawsif Tashwar Dipto, Mehedi Ahamed, Radib Bin Kabir +5VLM Evaluation Multilingual VLM Evaluation
1Islamic University of Technology · 2South East University · 3Vector Institute +3
WorldBench: Culturally Grounded Benchmark for Multilingual Agents
Sep 1, 2026Leonardo Ranaldi, Sherrie Shen, Jushi Kai +1Cross-Cultural Language Model Evaluation LLM Agent Evaluation
(•) ILCC, School of Informatics, University of Edinburgh · (•,◦) ILCC, School of Informatics, University of Edinburgh · (◦) School of Artificial Intelligence, Shanghai Jiao Tong University
MemeBridge: A Dataset for Benchmarking and Mitigating the Bidirectional Cultural Gap in Meme Interpretation
Aug 31, 2026Hangxiao Zhu, Suliu Qin, Zhuoyan Li +3Cultural Bias in Language Models Cultural Knowledge in Language Models
Texas A&M University · College Station, USA · Independent Researcher +4
CM2: Multimodal Cultural Reasoning via an Integrated Multi-Agent Framework
Aug 31, 2026Qi Li, Zhaojie Kang, Yingjie He +6Multi-Agent LLM Systems Multimodal Reasoning
Lanzhou University, Lanzhou, China · Peking University, Beijing, China · Taiyuan University of Technology, Taiyuan, China +1
Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models
Aug 30, 2026Ghassan Al-Sumaidaee, Sajjad Abdoli, Ahmed Rashad +1Multilingual Language Model Evaluation LLM Evaluation
1Perle
BavGround: A Benchmark for Regional Cultural Grounding and Dialect Competence in Bavarian
Aug 13, 2026Jophin John, Michael Hoffmann, Jan Fillies +2Multilingual Language Model Evaluation LLM Grounding
Leibniz Supercomputing Centre (LRZ), Garching, Germany · Stanford University, Stanford, California, USA · Freie Universität Berlin, Germany +2
Easy to Complete, Hard to Choose: Investigating LLM Performance on the ProverbIT Benchmark
Aug 5, 2026Enrico Mensa, Lorenzo Zane, Calogero Jerik Scozzaro +3Figurative Language Understanding LLM Evaluation
Department of Computer Science, University of Turin, Turin, Italy · Independent Researcher
Cultural Awareness is Represented but Not Decoded: Tracing Mythological Knowledge across 18 Open-Source LLMs
Aug 3, 2026Iaroslav Chelombitko, Ekaterina Chelombitko, Mika HämäläinenMultilingual Language Models LLM Interpretability
1DataSpike · 2Metropolia University of Applied Sciences, Helsinki, Finland · 3Neapolis University Pafos, Paphos, Cyprus
MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes
Jul 30, 2026Weihang Wang, Kainan Tu, Jielei Zhang +9VLM Evaluation Vision-Language Models
1Bilibili · 2Fudan University · 3Peking University
IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems
Jul 25, 2026Shwetha Singaravelu, Gayathri Muruganantham, Lakshmi Rajendran +1LLM Fine-Tuning Cultural Knowledge in Language Models
Intrinsic Lab, Centre for Sensors, Instrumentation and Cyber-Physical System Engineering (Centre for SeNSE), Indian Institute of Technology Delhi, New Delhi 110016, India · RSL Quantum, FITT, IIT Delhi, New Delhi 110016, India
When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMs
Jul 23, 2026Anna Mosolova, Djamé SeddahMultilingual Language Model Evaluation LLM Evaluation
INRIA Paris

Oct 6, 2026, cs.CLJ/K move · Enter open · S save

Language Unalignability: Why Some Concepts Resist Cross-Cultural Benchmark Evaluation

Shu-Kai Hsieh, Da-Chen Lian

National Taiwan University

Multilingual Language Model Evaluation LLM Evaluation Cross-Cultural Language Model Evaluation Cross-Lingual Representation Alignment Cultural Knowledge in Language Models

Open paper AskPDF arXiv

Current evaluation of multilingual Large Language Models (LLMs) rests on an implicit Translation-Isomorphism Assumption (TIA): that semantic structures across languages are congruent and mutually mappable without loss of information. We argue that this assumption is not merely violated in practice, but ill-posed in principle for a typologically identifiable class of concepts, including pragmatic markers, honorifics, and diachronically stratified terms. We formalize this failure using a usage-cloud framework, representing concepts as point sets of contextualized embeddings. We define

α

-unalignability as the impossibility of any mapping that simultaneously preserves lexical faithfulness (centroid correspondence) and structural faithfulness (local neighborhood topology). We provide three layers of evidence. Behaviorally, we show that FLORES-200 translation failures are predicted by language family and resource class but not by script, and that LOBSTER reasoning scores vary by family. Mechanistically, we report a Representation-Intervention Gap (RIG) in a nine-model case study on Yami: the models' activations encode a regularity along which Yami groups with other low-resource and Austronesian languages, yet interventions on language-specific neurons show no demonstrated advantage over random masks: the regularity is visible but not usable by this intervention. Finally, we operationalize these findings into a multidimensional diagnostic profile: Cycle-Consistency, Pragmatic-Load Disagreement, Manifold-Curvature Mismatch, and RIG. We argue that collapsing cultural competence into a single scalar incentivizes "probabilistic flattening," and that recognizing the unalignable class is a precondition for AI that respects, rather than erases, cultural divergence. This suggests that multilingual alignment is not a single well-defined objective, but a set of mutually incompatible projections.

Momentum

Related topics

Latest papers 77