Cultural Knowledge in Language Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 44% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 77

All topics

CultureTalk-ID: A Multi-Task Dialogue Benchmark for Cultural Commonsense in Indonesian Local Languages
Jul 23, 2026Muhammad Dehan Al Kautsar, Salsabila Pranida, Bilal Elbouardi +1Multilingual Language Model Evaluation Multi-Turn Dialogue Evaluation
Mohamed bin Zayed University of Artificial Intelligence
LKValues: Aligning Large Language Models with Sri Lankan Societal Values
Jul 22, 2026Nethmi Muthugala, Supryadi, Surangika Ranathunga +7Multilingual Language Model Evaluation LLM Alignment
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, Tianjin, China · School of Mathematical and Computational Sciences, Massey University, Auckland, New Zealand · Department of Computer Science & Engineering, University of Moratuwa, Sri Lanka +2
When a Name Is Not a Name: A Benchmark Dataset and Distilled Reasoning for Culturally Entangled Bangla Homographs in Low-Resource LLMs
Jul 20, 2026Md. Asaduzzaman ShuvoCultural Bias in Language Models Language Model Bias Evaluation
United International University, Bangladesh
CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations
Jul 17, 2026Suzan Awinat, Alfonso Ortega del PuenteFigurative Language Understanding Arabic NLP
Department of Computer Engineering Autonomous University of Madrid (UAM) School of Science and Technology IE University Madrid, Spain · Department of Computer Science Oviedo University Asturias, Spain
Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
Jul 7, 2026Alicia Parrish, Rajat Shinde, Sanket Badhe +57VLM Evaluation Cross-Cultural Language Model Evaluation
MSQA: A Natively Sourced Multilingual and Multicultural SimpleQA Benchmark
Jul 1, 2026Xianru Chen, Yukai Huang, Mingxiang Chen +6Multilingual Language Model Evaluation LLM Evaluation
ByteDance Seed · Nanjing University · M-A-P
Benchmarking Frontier LLMs on Arabic Cultural and Sociolinguistic Knowledge: A Cross-Evaluation Framework with Human SME Ground Truth
Jun 30, 2026Sajjad Abdoli, Ghassan Al-Sumaidaee, Ahmad ElShiekh +2Multilingual Language Model Evaluation LLM-as-a-Judge
1Perle AI
Same question, different history: language, national identity, and credit in large language models
Jun 22, 2026William Guey, Pierrick Bougault, Wei Zhang +2Social Bias in Language Models Cultural Bias in Language Models
Department of Industrial Engineering, Tsinghua University, Beijing, China · School of Social Sciences, Tsinghua University, Beijing, China · Department of Industrial Engineering, Federal University of Rio de Janeiro, Brazil
CulMind: Benchmarking Multimodal Understanding and Reasoning in Chinese Cultural Heritage
Jun 19, 2026Zhangwei Cao, Shuhan Fan, Yuting Wei +5Benchmark Construction Reasoning Evaluation
University of International Relations · 2Kedge Business School · 3Peking University +3
NRITYAM: Language Models Meet Art and Heritage of Dance
Jun 18, 2026Punit Kumar Singh, Niladri Ghosh, Advait Joshiınst +3Multilingual Language Model Evaluation LLM Evaluation
Shenzhen Technology University, 518118 Shenzhen, China · Indian Institute of Technology, India · Ramakrishna Mission Vivekananda Educational and Research Institute, India +4
LLMs Infer Cultural Context but Fail to Apply It When Responding
Jun 16, 2026Yisong Miao, Jian Zhu, Vered ShwartzPragmatic Reasoning in Language Models Cultural Bias in Language Models
National University of Singapore · University of British Columbia · Canada CIFAR AI Chair, Vector Institute
The Culture Funnel: You Can't Align What isn't in the Data
Jun 11, 2026Ananya Sahu, Mehrnaz Mofakhami, Daniel D'Souza +3Cultural Alignment Cultural Bias in Language Models
CohereCorrespondingLabs
Who Brought Easter Eggs to Eid? Auditing LLM-Generated Cultural Translation of Math Word Problems Across Languages and Regions
Jun 9, 2026Parisa Suchdev, Juniper LovatoMultilingual Language Model Evaluation Cultural Bias in Language Models
Computational Ethics Lab · Department of Computer Science, University of Vermont · Vermont Complex Systems Center, University of Vermont +1
CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queries
Jun 8, 2026Vasudha Varadarajan, Akhila Yerukola, Mona T. Diab +1LLM Evaluation Cultural Bias in Language Models
Carnegie Mellon University Pittsburgh, PA 15213, USA
ChinaHeritaQA: A Culturally-Grounded Visual Question Answering Dataset for World Heritage Sites in China
Jun 8, 2026Yi Zhang, Bolei Ma, Yong Cao +3VLM Evaluation Visual Question Answering
1LMU Munich · 2FAU Erlangen-Nuremberg · 3Munich Center for Machine Learning +4
The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs
Jun 5, 2026Yang Zhang, Xiao Fei, Amr Mohamed +6Multilingual Language Model Evaluation LLM Evaluation
Ecole Polytechnique · MBZUAI · ENS-PSL +1
Korean Culture into LLM Alignment: Toward Cultural Coherence
Jun 5, 2026MinJae Jung, Minwoo KimLLM Fine-Tuning LLM Safety Alignment
DATUMO Inc.
SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding
Jun 2, 2026Peerawat Chomphooyod, Jian Gang Ngui, Yosephine Susanto +5Multilingual Language Model Evaluation Cross-Cultural Language Model Evaluation
1Chulalongkorn University · 2AI Singapore · 3MBZUAI +1
CARTE: A Benchmark for Mapping Language Model Knowledge Across France
Jun 1, 2026Sarah Almeida Carneiro, Christos Xypolopoulos, Xiao Fei +2LLM Evaluation Factual Knowledge in Language Models
École Polytechnique, Paris · École Polytechnique, Institut Polytechnique de Paris, France · École Polytechnique, Paris, NTUA +3
CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs
Jun 1, 2026Yangfan Ye, Xiaocheng Feng, Jialong Tang +5LLM Evaluation Cultural Bias in Language Models
Harbin Institute of Technology · Harvard University · The University of Hong Kong
When English Rewrites Local Knowledge: Global Narrative Dominance in Large Language Models
May 28, 2026Md Arid Hasan, Ruwad Naswan, Farhan Samir +2Multilingual Language Model Evaluation LLM Grounding
University of Toronto · BUET · University of Illinois Urbana-Champaign
ExCAM: Explainable Cultural Awareness Metrics
May 28, 2026Christoph Leiter, Haiyue Song, Hour Kaing +4LLM Evaluation Language Model Generation Evaluation
University of Mannheim, Germany · National Institute of Information and Communications Technology · University of Technology Nuremberg, Germany
Cultural Binding Heads in Language Models
May 27, 2026Avrile Floro, Luca BenedettoLLM Interpretability Cultural Bias in Language Models
Institut Polytechnique de Paris · Télécom SudParis, Institut Polytechnique de Paris
Simorgh at SemEval-2026 task 7: Region-Aware Hybrid Retrieval for Low-Resource Cultural Reasoning in Multilingual Question Answering
May 26, 2026Hadi Bayrami Asl Tekanlou, Mahdi Bakhtiyarzadeh, Jafar RazmaraRetrieval-Augmented Generation Cross-Lingual Reasoning in Language Models
University of Tabriz, Tabriz, Iran
JuICE: A Benchmark for Evaluating LLM-Judge in Identifying Cultural Errors
May 26, 2026Jiho Jin, Junho Myung, Juhyun Oh +5Multilingual Language Model Evaluation LLM-as-a-Judge
1KAIST · 3Universitas Gadjah Mada · 2Google
DFKI-MLT at SemEval-2026 TASK 7: Steering Multilingual Models Towards Cultural Knowledge
May 21, 2026Yusser Al Ghussin, Daniil Gurgurov, Yasser Hamidullah +3Multilingual Language Models Multilingual Language Model Evaluation
German Research Center for Artificial Intelligence (DFKI GmbH) · Saarland Informatics Campus, Saarbrücken, Germany · Barcelona Supercomputing Center (BSC-CNS), Barcelona, Catalonia, Spain
Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency
May 21, 2026Andrew Ivan Soegeng, Patrick Sutanto, Tan Sang NguyenCultural Bias in Language Models Cultural Knowledge in Language Models
SAP · School of Computing, National University of Singapore
CrossCult-KIBench: A Benchmark for Cross-Cultural Knowledge Insertion in MLLMs
May 7, 2026Zhen Zeng, Leijiang Gu, Feng Li +2Knowledge Augmentation for Language Models Multimodal Large Language Models
1Hefei University of Technology · 2Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE, Minzu University of China · School of Information Engineering, Minzu University of China
XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity
May 7, 2026Dasol Choi, Eugenia Kim, Jaewon Noh +14LLM Safety Benchmarks Multilingual Language Model Evaluation
AIM Intelligence · Microsoft · Korea AISI +7
SemEval-2026 Task 7: Everyday Knowledge Across Diverse Languages and Cultures
May 4, 2026Nedjma Ousidhoum, Junho Myung, Carla Perez-Almendros +27Multilingual Language Model Evaluation LLM Evaluation
Cardiff University · KAIST · MBZUAI +13

Jul 23, 2026, cs.CLJ/K move · Enter open · S save

CultureTalk-ID: A Multi-Task Dialogue Benchmark for Cultural Commonsense in Indonesian Local Languages

Muhammad Dehan Al Kautsar, Salsabila Pranida, Bilal Elbouardi, Fajri Koto

Mohamed bin Zayed University of Artificial Intelligence

Multilingual Language Model Evaluation Multi-Turn Dialogue Evaluation Cultural Knowledge in Language Models Machine Translation

Open paper AskPDF arXiv

Culture is lived through conversation, yet existing Indonesian cultural commonsense benchmarks evaluate LLMs on short and isolated prompts, stripping away the dialogic context in which cultural nuances actually surface. We introduce CultureTalk-ID, the first dialogue-based benchmark for cultural commonsense in Indonesian and its local languages, comprising 4,496 culturally grounded dialogues across 11 languages and 13 culturally salient topics, curated through a multi-stage human pipeline with native speakers to ensure authenticity. CultureTalk-ID introduces three complementary tasks, namely dialogue-based multiple-choice cultural commonsense reasoning, culturally faithful machine translation, and language steering, which jointly probe whether LLMs can understand, transfer, and generate culturally grounded language.

Momentum

Related topics

Latest papers 77