Improving policymaking is a central concern in public administration. Prior human subject studies reveal substantial cross-cultural differences in citizens' emotional responses to red tape during policy implementation. While LLM agents offer opportunities to simulate human-like responses and reduce experimental costs, their ability to generate culturally appropriate emotional responses to red tape remains unverified. To address this gap, we propose an evaluation framework for assessing LLMs' emotional responses to red tape across diverse cultural contexts. As a pilot study, we apply this framework to a single red-tape scenario. Our results show that all models exhibit limited alignment with human emotional responses, with notably weaker performance in Eastern cultures. Cultural prompting strategies prove largely ineffective in improving alignment. We further introduce RAMO, an interactive interface for simulating citizens' emotional responses to red tape and for collecting human data to improve models. The interface is publicly available at https://ramo-chi.ivia.ch.
Figures & tables
Model
Default
Cultural-aware
Overlap@3
SAS
Overlap@3
SAS ↑
GPT-4o
-
-0.12
-
-0.25
GPT-5-2025-08-07
-
-0.50
-
-0.12
Gemini-3-pro
-
-0.11
-
0.00
Qwen3-max-2026-01-23
-
-0.13
-
-0.25
Table 1. Alignment between LLM-simulated and human emotions in Hong Kong SAR. For culture-aware agents, higher SAS means more alignment with human results in significance.
Model
Default
Cultural-aware
Overlap@3
SAS
Overlap@3 ↑
SAS ↑
GPT-4o
0.16
−0.0038
0.18
-0.0038
GPT-5-2025-08-07
0.20
−0.1338
0.20
−0.1263
Gemini-3-pro
0.42
−0.1263
0.10
−0.2463
Qwen3-max-2026-01-23
0.20
−0.1338
0.20
−0.1263
Table 2. Alignment between LLM-predicted and human emotions in Mainland China. Higher Overlap@3 and higher Significance Alignment Score (SAS) indicate better alignment with human results. (For Mainland China, SAS favors an intermediate significance outcome; see Appendix A.2 .)
Model
Default
Cultural-aware
Overlap@3
SAS
Overlap@3 ↑
SAS ↑
GPT-4o
1.00
0.22
1.00
0.22
GPT-5-2025-08-07
0.93
0.56
1.00
0.56
Gemini-3-pro
0.25
0.00
0.50
0.00
Qwen3-max-2026-01-23
0.50
0.33
0.50
0.11
Table 3. Alignment between LLM-predicted and human emotions in Germany. Higher Overlap@3 and SAS indicate better alignment.
Figure 1. Overview of RAMO interface design, detailed in Section 5 . Major components are enlarged for visibility and labelled with circled letters for reference. The interface is designed for policymakers to simulate potential emotional responses to various red-tape scenarios across cultures.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 2. Cultural factor scores of Mainland China, Germany and Hong Kong SAR. Bar chart (or table-style graphic) comparing Hofstede cultural factor scores for Mainland China, Germany, and Hong Kong SAR across multiple dimensions.
The expression of emotions that serve social purposes, such as asserting independence or fostering interdependence, is central to human interactions and varies systematically across cultures. As LLMs are increasingly used to simulate human behavior in culturally nuanced interactions, it is important to understand whether they faithfully capture human patterns of social emotion expression. When LLM responses are not culturally aligned, their utility is compromised -- particularly when users assume they are interacting with a culturally attuned interlocutor, and may act on advice that proves inappropriate in their cultural context. We present a psychologically informed evaluation framework of cross-cultural social emotion expression in LLMs. Using a human study comparing European American and Latin American participants' expression of engaging and disengaging emotions, we evaluate six frontier LLMs on their ability to reflect culturally differentiated patterns for expressing social emotions. We find systematic misalignment between model and human behavior: all models express engaging emotions more than disengaging ones, with particularly stark differences observed for the generally well-represented European American persona. We further highlight that LLM responses are highly concentrated and deterministic, failing to capture the diversity of human responses in expressing social emotions. Our ablation analyses reveal that these patterns are robust to sampling temperatures, partially sensitive to prompt language, and dependent on the response elicitation format. Together, our findings highlight limitations in how current LLMs represent the interaction of cultural and emotional axes, particularly when expressing social emotions, with direct implications for their deployment in cross-cultural affective contexts.
Sree Bhattacharyya, Manas Mehta, Leona Chen +4
The Pennsylvania State University · Duke University · Northeastern University +1
Large language models (LLMs) are increasingly used to simulate human opinions and survey responses, but their ability to reproduce population responses across cultures remains limited. Existing persona-based prompting methods typically rely on sociodemographic or personality traits, which are only indirect proxies for the values that shape human responses. We propose a value-based persona construction method that derives textual descriptors from survey responses capturing core cultural dimensions. By sampling value profiles from target populations and aggregating LLM responses across personas, we obtain population-level predictions grounded in observed value distributions. We further introduce a calibration procedure that improves response diversity while preserving estimated opinions. We show that our approach reduces prediction error across countries, with the largest improvements observed in underrepresented populations. This substantially narrows the performance gap between countries aligned with dominant LLM priors and those that are less represented in training data, while also yielding response distributions that closely match human diversity.
Axel Abels, Elias Fernandez Domingos, Apurva Shah +1
1Machine Learning Group, Universit´e Libre de Bruxelles · 2AI Lab, Vrije Universiteit Brussel · 3FARI Institute, Universit´e Libre de Bruxelles - Vrije Universiteit Brussel +2
Recent work has shown that LLMs overrepresent dominant cultures, particularly Western ones, while marginalizing others. We investigate whether this affects models' ability to generate culturally adapted responses by evaluating their use of local measurement units based on the user's perceived cultural background. We introduce Cultural and Pragmatic Response Inference (CAPRI), a dataset of conversations with varying levels of cultural cues. Experiments with state-of-the-art LLMs show that models can infer cultural background and recall relevant conventions, but often fail to utilize the information to adapt their answers to the relevant cultural conventions, unless explicitly prompted to perform the tasks sequentially. We further evaluate adaptation to the interpretation of time and quantity expressions, two subjective language grounding dimensions that are affected by culture. We find that models increasingly adapt their answers as cultural cues accumulate, but their priors are not culture-neutral, sometimes aligning with the model's country of origin. Overall, CAPRI provides a resource for future research aimed at narrowing the gap between cultural knowledge and culturally adaptive language generation.
Yisong Miao, Jian Zhu, Vered Shwartz
National University of Singapore · University of British Columbia · Canada CIFAR AI Chair, Vector Institute