Cultural Knowledge in Language Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 44% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 77

All topics

Language Unalignability: Why Some Concepts Resist Cross-Cultural Benchmark Evaluation
Oct 6, 2026Shu-Kai Hsieh, Da-Chen LianMultilingual Language Model Evaluation LLM Evaluation
National Taiwan University
Artificial Intelligence and the New Science of Culture
Oct 5, 2026Douglas R. Guilbeault, Bhargav Srinivasa Desikan, James A. EvansCultural Evolution Computational Social Science
Graduate School of Business, Stanford University, U.S. · Oxford Internet Institute, University of Oxford, U.K. · Sociology & Data Science, University of Chicago, U.S.
Knowing the Rules, Applying the Rules: Evaluating Language Models on Traditional Chinese Bazi
Oct 5, 2026Jiulin Li, Ping HuangLLM Evaluation Cultural Knowledge in Language Models
Beijing Liuyi Guanhua Technology Co., Ltd. · State Key Laboratory of General Artificial Intelligence, BIGAI
CineSubBench: Evaluating LLMs on Long-Form Narrative and Cultural Understanding from Multilingual Movie Subtitles
Sep 28, 2026Mir Tafseer Nayeem, Susmoy Chakraborty, Davood RafieiMultilingual Language Model Evaluation Long-Context Language Model Evaluation
University of Alberta · Independent Researcher
AraDynFact: Dynamic Evaluation of Factual Knowledge in Arabic
Sep 28, 2026Ignacio Iacobacci, Faroq Altam, Zhaozhi Qian +1LLM Evaluation Arabic NLP
Elm Company
CRISP: Cultural Reward Modeling for Implicit Situated Propriety
Sep 28, 2026Zekun Yuan, Yangfan Ye, Baohang Li +6Reward Modeling RL for Language Models
Harbin Institute of Technology · Huawei Technologies Co., Ltd · Peng Cheng Laboratory
Recognized but Not Produced: A Generation Benchmark for Culturally Specific Kinship Terms
Sep 22, 2026Sahil Pardasani, Madhusudan SinghMultilingual Language Model Evaluation Language Model Generation Evaluation
Blockchain Data Intelligence Lab The Pennsylvania State University University Park, PA, USA
FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing
Sep 21, 2026Yusser Al Ghussin, Eva Gavaller, Cristina España-Bonet +2Multilingual Language Model Evaluation LLM Auditing
Saarland University · German Research Center for Artificial Intelligence (DFKI) · Barcelona Supercomputing Center (BSC-CNS), Barcelona, Catalonia, Spain
MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education
Sep 16, 2026Luyao Zhu, Xun Wei Yee, Wei Li +2VLM Evaluation Large Vision-Language Models
AI Singapore, National University of Singapore, Singapore · Institute of Advanced Intelligence and Computing, A*STAR · School of Computing, National University of Singapore, Singapore
Cultural Competence in Context: A Large Language Model Passes the Turing Test in Finland
Sep 16, 2026Otto Segersven, Pentti HenttonenMultilingual Language Model Evaluation LLM Evaluation
Department of Computer Science, University of Helsinki · Faculty of Social Sciences, University of Tampere
Knowledge-Graph Based Augmentation versus Retrieval Augmented Generation for Cultural-Related Question Answering
Sep 16, 2026Pablo Poulenard, Yannis Karmim, Valentin BarrièreRetrieval-Augmented Generation Graph Retrieval-Augmented Generation
DCC, Universidad de Chile, Santiago, Chile · École Polytechnique, Palaiseau, France · Inria, Almanach, Paris, France +2
Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture
Sep 14, 2026Cody Kommers, Mingrui Ye, Evelyn Gius +4Pragmatic Reasoning in Language Models Multimodal Large Language Models
King’s College London London, United Kingdom · University of Edinburgh Edinburgh, United Kingdom
Enemray: Toward Capable Language Models for Hassaniya
Sep 13, 2026Cheikh AhmedLanguage Modeling Cultural Knowledge in Language Models
Multilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu
Sep 12, 2026Farah Adeeba, Abdul Rafae Khan, Rajesh Bhatt +1Multilingual Language Model Evaluation Cultural Knowledge in Language Models
University of Konstanz, Germany · University of Massachusetts Amherst, USA · Malaysia School of Information Technology, Monash University Malaysia +1
When Metrics Reward the Worst Translations: Internalizing Cultural Reasoning for Social Media Translation Evaluation
Sep 8, 2026Yiwen Qiu, Linjuan Wu, Dingming Li +7Reinforcement Learning Cultural Knowledge in Language Models
Zhejiang University · Xiaohongshu Inc.
BanglaMemeX: Advancing Cultural Metaphoric Image Interpretation in Bangla with a Multimodal Explainable Dataset
Sep 7, 2026Md. Sadman Sakib, Zisan Mahmud, Md. Fahim Arefin +1Figurative Language Understanding Multilingual VLM Evaluation
University of Dhaka · University of Texas at Dallas
CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning
Sep 3, 2026Bo Zeng, Linfeng Gao, Peiqin Lin +9Multimodal Reasoning Multilingual VLM Evaluation
Alibaba Group · Xiamen University · University of Hamburg
Accurate in space, unreliable in time: how LLMs represent national cultural change
Sep 1, 2026Yalda Daryani, Miranda Bogen, Madeleine I. G. DaeppLLM Evaluation Cultural Evolution
Department of Psychology, University of Southern California · Center for Democracy and Technology
VakyArth: Evaluating Pragmatic Competence in LLMs across Indic Languages
Sep 1, 2026Usneek Singh, Poorvaja Veera Balaji Kumar, Parth Nanda +4Multilingual Language Model Evaluation LLM Evaluation
Georgia Institute of Technology · University of Texas at Austin
MemeCULT-1K: Benchmarking South Asian Cultural Context and Humor Understanding of Multimodal Models
Sep 1, 2026Tawsif Tashwar Dipto, Mehedi Ahamed, Radib Bin Kabir +5VLM Evaluation Multilingual VLM Evaluation
1Islamic University of Technology · 2South East University · 3Vector Institute +3
WorldBench: Culturally Grounded Benchmark for Multilingual Agents
Sep 1, 2026Leonardo Ranaldi, Sherrie Shen, Jushi Kai +1Cross-Cultural Language Model Evaluation LLM Agent Evaluation
(•) ILCC, School of Informatics, University of Edinburgh · (•,◦) ILCC, School of Informatics, University of Edinburgh · (◦) School of Artificial Intelligence, Shanghai Jiao Tong University
MemeBridge: A Dataset for Benchmarking and Mitigating the Bidirectional Cultural Gap in Meme Interpretation
Aug 31, 2026Hangxiao Zhu, Suliu Qin, Zhuoyan Li +3Cultural Bias in Language Models Cultural Knowledge in Language Models
Texas A&M University · College Station, USA · Independent Researcher +4
CM2: Multimodal Cultural Reasoning via an Integrated Multi-Agent Framework
Aug 31, 2026Qi Li, Zhaojie Kang, Yingjie He +6Multi-Agent LLM Systems Multimodal Reasoning
Lanzhou University, Lanzhou, China · Peking University, Beijing, China · Taiyuan University of Technology, Taiyuan, China +1
Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models
Aug 30, 2026Ghassan Al-Sumaidaee, Sajjad Abdoli, Ahmed Rashad +1Multilingual Language Model Evaluation LLM Evaluation
1Perle
BavGround: A Benchmark for Regional Cultural Grounding and Dialect Competence in Bavarian
Aug 13, 2026Jophin John, Michael Hoffmann, Jan Fillies +2Multilingual Language Model Evaluation LLM Grounding
Leibniz Supercomputing Centre (LRZ), Garching, Germany · Stanford University, Stanford, California, USA · Freie Universität Berlin, Germany +2
Easy to Complete, Hard to Choose: Investigating LLM Performance on the ProverbIT Benchmark
Aug 5, 2026Enrico Mensa, Lorenzo Zane, Calogero Jerik Scozzaro +3Figurative Language Understanding LLM Evaluation
Department of Computer Science, University of Turin, Turin, Italy · Independent Researcher
Cultural Awareness is Represented but Not Decoded: Tracing Mythological Knowledge across 18 Open-Source LLMs
Aug 3, 2026Iaroslav Chelombitko, Ekaterina Chelombitko, Mika HämäläinenMultilingual Language Models LLM Interpretability
1DataSpike · 2Metropolia University of Applied Sciences, Helsinki, Finland · 3Neapolis University Pafos, Paphos, Cyprus
MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes
Jul 30, 2026Weihang Wang, Kainan Tu, Jielei Zhang +9VLM Evaluation Vision-Language Models
1Bilibili · 2Fudan University · 3Peking University
IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems
Jul 25, 2026Shwetha Singaravelu, Gayathri Muruganantham, Lakshmi Rajendran +1LLM Fine-Tuning Cultural Knowledge in Language Models
Intrinsic Lab, Centre for Sensors, Instrumentation and Cyber-Physical System Engineering (Centre for SeNSE), Indian Institute of Technology Delhi, New Delhi 110016, India · RSL Quantum, FITT, IIT Delhi, New Delhi 110016, India
When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMs
Jul 23, 2026Anna Mosolova, Djamé SeddahMultilingual Language Model Evaluation LLM Evaluation
INRIA Paris
CultureTalk-ID: A Multi-Task Dialogue Benchmark for Cultural Commonsense in Indonesian Local Languages
Jul 23, 2026Muhammad Dehan Al Kautsar, Salsabila Pranida, Bilal Elbouardi +1Multilingual Language Model Evaluation Multi-Turn Dialogue Evaluation
Mohamed bin Zayed University of Artificial Intelligence
LKValues: Aligning Large Language Models with Sri Lankan Societal Values
Jul 22, 2026Nethmi Muthugala, Supryadi, Surangika Ranathunga +7Multilingual Language Model Evaluation LLM Alignment
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, Tianjin, China · School of Mathematical and Computational Sciences, Massey University, Auckland, New Zealand · Department of Computer Science & Engineering, University of Moratuwa, Sri Lanka +2
When a Name Is Not a Name: A Benchmark Dataset and Distilled Reasoning for Culturally Entangled Bangla Homographs in Low-Resource LLMs
Jul 20, 2026Md. Asaduzzaman ShuvoCultural Bias in Language Models Language Model Bias Evaluation
United International University, Bangladesh
CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations
Jul 17, 2026Suzan Awinat, Alfonso Ortega del PuenteFigurative Language Understanding Arabic NLP
Department of Computer Engineering Autonomous University of Madrid (UAM) School of Science and Technology IE University Madrid, Spain · Department of Computer Science Oviedo University Asturias, Spain
Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
Jul 7, 2026Alicia Parrish, Rajat Shinde, Sanket Badhe +57VLM Evaluation Cross-Cultural Language Model Evaluation
MSQA: A Natively Sourced Multilingual and Multicultural SimpleQA Benchmark
Jul 1, 2026Xianru Chen, Yukai Huang, Mingxiang Chen +6Multilingual Language Model Evaluation LLM Evaluation
ByteDance Seed · Nanjing University · M-A-P
Benchmarking Frontier LLMs on Arabic Cultural and Sociolinguistic Knowledge: A Cross-Evaluation Framework with Human SME Ground Truth
Jun 30, 2026Sajjad Abdoli, Ghassan Al-Sumaidaee, Ahmad ElShiekh +2Multilingual Language Model Evaluation LLM-as-a-Judge
1Perle AI
Same question, different history: language, national identity, and credit in large language models
Jun 22, 2026William Guey, Pierrick Bougault, Wei Zhang +2Social Bias in Language Models Cultural Bias in Language Models
Department of Industrial Engineering, Tsinghua University, Beijing, China · School of Social Sciences, Tsinghua University, Beijing, China · Department of Industrial Engineering, Federal University of Rio de Janeiro, Brazil
CulMind: Benchmarking Multimodal Understanding and Reasoning in Chinese Cultural Heritage
Jun 19, 2026Zhangwei Cao, Shuhan Fan, Yuting Wei +5Benchmark Construction Reasoning Evaluation
University of International Relations · 2Kedge Business School · 3Peking University +3
NRITYAM: Language Models Meet Art and Heritage of Dance
Jun 18, 2026Punit Kumar Singh, Niladri Ghosh, Advait Joshiınst +3Multilingual Language Model Evaluation LLM Evaluation
Shenzhen Technology University, 518118 Shenzhen, China · Indian Institute of Technology, India · Ramakrishna Mission Vivekananda Educational and Research Institute, India +4
LLMs Infer Cultural Context but Fail to Apply It When Responding
Jun 16, 2026Yisong Miao, Jian Zhu, Vered ShwartzPragmatic Reasoning in Language Models Cultural Bias in Language Models
National University of Singapore · University of British Columbia · Canada CIFAR AI Chair, Vector Institute
The Culture Funnel: You Can't Align What isn't in the Data
Jun 11, 2026Ananya Sahu, Mehrnaz Mofakhami, Daniel D'Souza +3Cultural Alignment Cultural Bias in Language Models
CohereCorrespondingLabs
Who Brought Easter Eggs to Eid? Auditing LLM-Generated Cultural Translation of Math Word Problems Across Languages and Regions
Jun 9, 2026Parisa Suchdev, Juniper LovatoMultilingual Language Model Evaluation Cultural Bias in Language Models
Computational Ethics Lab · Department of Computer Science, University of Vermont · Vermont Complex Systems Center, University of Vermont +1
CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queries
Jun 8, 2026Vasudha Varadarajan, Akhila Yerukola, Mona T. Diab +1LLM Evaluation Cultural Bias in Language Models
Carnegie Mellon University Pittsburgh, PA 15213, USA
ChinaHeritaQA: A Culturally-Grounded Visual Question Answering Dataset for World Heritage Sites in China
Jun 8, 2026Yi Zhang, Bolei Ma, Yong Cao +3VLM Evaluation Visual Question Answering
1LMU Munich · 2FAU Erlangen-Nuremberg · 3Munich Center for Machine Learning +4
The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs
Jun 5, 2026Yang Zhang, Xiao Fei, Amr Mohamed +6Multilingual Language Model Evaluation LLM Evaluation
Ecole Polytechnique · MBZUAI · ENS-PSL +1
Korean Culture into LLM Alignment: Toward Cultural Coherence
Jun 5, 2026MinJae Jung, Minwoo KimLLM Fine-Tuning LLM Safety Alignment
DATUMO Inc.
SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding
Jun 2, 2026Peerawat Chomphooyod, Jian Gang Ngui, Yosephine Susanto +5Multilingual Language Model Evaluation Cross-Cultural Language Model Evaluation
1Chulalongkorn University · 2AI Singapore · 3MBZUAI +1
CARTE: A Benchmark for Mapping Language Model Knowledge Across France
Jun 1, 2026Sarah Almeida Carneiro, Christos Xypolopoulos, Xiao Fei +2LLM Evaluation Factual Knowledge in Language Models
École Polytechnique, Paris · École Polytechnique, Institut Polytechnique de Paris, France · École Polytechnique, Paris, NTUA +3
CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs
Jun 1, 2026Yangfan Ye, Xiaocheng Feng, Jialong Tang +5LLM Evaluation Cultural Bias in Language Models
Harbin Institute of Technology · Harvard University · The University of Hong Kong
When English Rewrites Local Knowledge: Global Narrative Dominance in Large Language Models
May 28, 2026Md Arid Hasan, Ruwad Naswan, Farhan Samir +2Multilingual Language Model Evaluation LLM Grounding
University of Toronto · BUET · University of Illinois Urbana-Champaign
ExCAM: Explainable Cultural Awareness Metrics
May 28, 2026Christoph Leiter, Haiyue Song, Hour Kaing +4LLM Evaluation Language Model Generation Evaluation
University of Mannheim, Germany · National Institute of Information and Communications Technology · University of Technology Nuremberg, Germany
Cultural Binding Heads in Language Models
May 27, 2026Avrile Floro, Luca BenedettoLLM Interpretability Cultural Bias in Language Models
Institut Polytechnique de Paris · Télécom SudParis, Institut Polytechnique de Paris
Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low-Resource Cultural Reasoning in Multilingual Question Answering
May 26, 2026Hadi Bayrami Asl Tekanlou, Mahdi Bakhtiyarzadeh, Jafar RazmaraRetrieval-Augmented Generation Cross-Lingual Reasoning in Language Models
University of Tabriz, Tabriz, Iran
JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors
May 26, 2026Jiho Jin, Junho Myung, Juhyun Oh +5Multilingual Language Model Evaluation LLM-as-a-Judge
1KAIST · 3Universitas Gadjah Mada · 2Google
DFKI-MLT at SemEval-2026 TASK 7: Steering Multilingual Models Towards Cultural Knowledge
May 21, 2026Yusser Al Ghussin, Daniil Gurgurov, Yasser Hamidullah +3Multilingual Language Models Multilingual Language Model Evaluation
German Research Center for Artificial Intelligence (DFKI GmbH) · Saarland Informatics Campus, Saarbrücken, Germany · Barcelona Supercomputing Center (BSC-CNS), Barcelona, Catalonia, Spain
Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency
May 21, 2026Andrew Ivan Soegeng, Patrick Sutanto, Tan Sang NguyenCultural Bias in Language Models Cultural Knowledge in Language Models
SAP · School of Computing, National University of Singapore
CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs
May 7, 2026Zhen Zeng, Leijiang Gu, Feng Li +2Knowledge Augmentation for Language Models Multimodal Large Language Models
1Hefei University of Technology · 2Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE, Minzu University of China · School of Information Engineering, Minzu University of China
XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity
May 7, 2026Dasol Choi, Eugenia Kim, Jaewon Noh +14LLM Safety Benchmarks Multilingual Language Model Evaluation
AIM Intelligence · Microsoft · Korea AISI +7
SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures
May 4, 2026Nedjma Ousidhoum, Junho Myung, Carla Perez-Almendros +27Multilingual Language Model Evaluation LLM Evaluation
Cardiff University · KAIST · MBZUAI +13

Oct 6, 2026, cs.CLJ/K move · Enter open · S save

Language Unalignability: Why Some Concepts Resist Cross-Cultural Benchmark Evaluation

Shu-Kai Hsieh, Da-Chen Lian

National Taiwan University

Multilingual Language Model Evaluation LLM Evaluation Cross-Cultural Language Model Evaluation Cross-Lingual Representation Alignment Cultural Knowledge in Language Models

Open paper AskPDF arXiv

Current evaluation of multilingual Large Language Models (LLMs) rests on an implicit Translation-Isomorphism Assumption (TIA): that semantic structures across languages are congruent and mutually mappable without loss of information. We argue that this assumption is not merely violated in practice, but ill-posed in principle for a typologically identifiable class of concepts, including pragmatic markers, honorifics, and diachronically stratified terms. We formalize this failure using a usage-cloud framework, representing concepts as point sets of contextualized embeddings. We define

α

-unalignability as the impossibility of any mapping that simultaneously preserves lexical faithfulness (centroid correspondence) and structural faithfulness (local neighborhood topology). We provide three layers of evidence. Behaviorally, we show that FLORES-200 translation failures are predicted by language family and resource class but not by script, and that LOBSTER reasoning scores vary by family. Mechanistically, we report a Representation-Intervention Gap (RIG) in a nine-model case study on Yami: the models' activations encode a regularity along which Yami groups with other low-resource and Austronesian languages, yet interventions on language-specific neurons show no demonstrated advantage over random masks: the regularity is visible but not usable by this intervention. Finally, we operationalize these findings into a multidimensional diagnostic profile: Cycle-Consistency, Pragmatic-Load Disagreement, Manifold-Curvature Mismatch, and RIG. We argue that collapsing cultural competence into a single scalar incentivizes "probabilistic flattening," and that recognizing the unalignable class is a precondition for AI that respects, rather than erases, cultural divergence. This suggests that multilingual alignment is not a single well-defined objective, but a set of mutually incompatible projections.

Momentum

Related topics

Latest papers 77