Metaphors We Compute By: A Computational Audit of Cultural Translation vs. Thinking in LLMs

Yuan Chang; Jiaming Qu; Zhu Li

cs.CLApr 6, 2026

Metaphors We Compute By: A Computational Audit of Cultural Translation vs. Thinking in LLMs

Authors: Yuan Chang, Jiaming Qu, Zhu Li

Organizations: Meta, Burlingame, CA · Amazon, Seattle, WA

Abstract

Large language models (LLMs) are often described as multilingual because they can understand and respond in many languages. However, speaking a language is not the same as reasoning within a culture. This distinction motivates a critical question: do LLMs truly conduct culture-aware reasoning? This paper presents a preliminary computational audit of cultural inclusivity in a creative writing task. We empirically examine whether LLMs act as culturally diverse creative partners or merely as cultural translators that leverage a dominant conceptual framework with localized expressions. Using a metaphor generation task spanning five cultural settings and several abstract concepts as a case study, we find that the model exhibits stereotyped metaphor usage for certain settings, as well as Western defaultism. These findings suggest that merely prompting an LLM with a cultural identity does not guarantee culturally grounded reasoning.

Figures & tables

Figure 1: Intra-cultural semantic diversity of metaphors. Each cell shows the average pairwise cosine distance among 20 generated metaphors for a given (concept, culture) pair. Higher values indicate greater semantic diversity, while lower values indicate representational collapse.

Figure 2: Conceptual geometry of metaphor embeddings across cultures. Each panel shows a t-SNE projection of metaphor embeddings for one cultural condition. Colors indicate distinct abstract concepts.

Concept	Japan	China	India	Brazil
Time	–	–	$\uparrow^{**}$	$\uparrow^{***}$
Death	–	$\uparrow^{***}$	$\uparrow^{***}$	$\uparrow^{***}$
Success	$\downarrow^{*}$	–	–	–
Family	–	–	–	–
Freedom	–	–	$\uparrow^{*}$	$\downarrow^{*}$

Table 1: Directional results of the Western defaultism test. Arrows indicate the direction of statistically significant differences in cosine distance between the Default condition and the US condition versus another culture.

\uparrow

indicates that Default is significantly closer to U.S. than to the corresponding culture (evidence of Western defaultism), while

\downarrow

indicates that Default is significantly closer to the corresponding culture than to U.S. We used Fisher randomization test for significance testing:

{}^{*}p<0.05

{}^{**}p<0.01

{}^{***}p<0.001

Cultural Bias in Language Models

Cultural bias in language models is the uneven representation of cultural knowledge, values, identities, and norms across groups and languages.

See all 129 papers

The Culture Within: Towards Mechanistic Understanding of Cultural Reasoning in LLMs
Oct 19, 2026Samee Arif, Furkan Danisman, Angana Borah +6Cultural Bias in Language Models Cultural Knowledge in Language Models
LLMs Infer Cultural Context but Fail to Apply It When Responding
Jun 16, 2026Yisong Miao, Jian Zhu, Vered ShwartzPragmatic Reasoning in Language Models Cultural Bias in Language Models
National University of Singapore · University of British Columbia · Canada CIFAR AI Chair, Vector Institute
Cultural (In)Competence: Investigating Culture-Specific Stereotype Detection Capabilities of LLMs
Oct 15, 2026Maximilian Spliethöver, Jannes Lötfering, Dominik Bachmann +4Social Bias in Language Models Cultural Bias in Language Models
Institute of Artificial Intelligence & L3S Research Center, Leibniz University Hannover · Institute for Logic, Language & Computation, University of Amsterdam · Department of Experimental Psychology, Utrecht University

Oct 19, 2026J/K move · Enter open · S save

The Culture Within: Towards Mechanistic Understanding of Cultural Reasoning in LLMs

Samee Arif, Furkan Danisman, Angana Borah, Honglak Lee+5

Cultural Bias in Language Models Cultural Knowledge in Language Models Cross-Cultural Language Model Evaluation Knowledge Conflicts in Language Models Mechanistic Interpretability

Open paper AskPDF arXiv

Large Language Models (LLMs) increasingly operate across culturally diverse settings, yet existing evaluations largely measure cultural behavior at the output level. We introduce CultureLens, a reference-grounded benchmark and mechanistic study of how LLMs use cultural information. CultureLens contains 9,261 scenarios across 75 countries and systematically varies country, value, true-norm, and contradictory-norm evidence. Across Llama-3.1-8B, Gemma-3-12B, and Qwen-3.5-27B, stating a contradictory norm lowers aligned answers by 32-47 percentage points, even when the model answers correctly from the country alone. Mechanistically, models default to Anglophone norms when no country is given. The failure under a contradictory norm is not one of missing knowledge, as blocking the norm inside the model restores the correct answer in most cases. Models accept a stated norm according to how it is presented and not whether it is true, which leaves them most exposed to errors in retrieved content. Supervised fine-tuning with contradictory-norm examples raises aligned answers under a contradictory norm to 85-93%, including on unseen countries, at a small cost under a true norm.

Abstract

Cultural Bias in Language Models

Explore similar work

The Culture Within: Towards Mechanistic Understanding of Cultural Reasoning in LLMs

LLMs Infer Cultural Context but Fail to Apply It When Responding

Cultural (In)Competence: Investigating Culture-Specific Stereotype Detection Capabilities of LLMs