Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation

Qishuai Zhong; Zongmin Li; Siqi Fan; Aixin Sun

cs.CLSep 5, 2026

Alignment by Stereotyping: How LLMs Sacrifice Individual Distinctiveness for Cultural Adaptation

Authors: Qishuai Zhong, Zongmin Li, Siqi Fan, Aixin Sun

Organizations: Nanyang Technological University, Singapore · University of Electronic Science and Technology of China

Abstract

Large language models are increasingly deployed for personalized interaction, and demographic conditioning via user profiles is a widely adopted strategy for cultural adaptation. We ask whether this approach genuinely serves individual users or achieves accuracy by erasing individual distinctiveness. Studying seven models including frontier GPT-5.1 on the World Values Survey, we find that demographic profiles improve value alignment accuracy for most models, but at a systematic cost to individuality. That is, models pull responses toward demographic group centroids rather than preserving individual differences, a behavioral pattern we term alignment by stereotyping. Permutation tests (10,000 permutations, six demographic attributes, seven models) certify that top-performing models compress individuals far above the human baseline; within-family scaling amplifies this tradeoff while degrading intrinsic cultural understanding. Using a synthetic dialogue dataset validated on real human-chatbot conversations from PRISM (Kirk et al., 2024), we further show that distributing demographic signals across conversational turns partially suppresses prototype retrieval compared to compact demographic labels, a finding validated on real conversations via PRISM but requiring replication at larger scale.

Figures & tables

Figure 1: Three-condition evaluation framework. None (grey): model answers with no user context, establishing a default-culture baseline (VAA: 0.34–0.60; Homog Rate:

\sim

50%). Profile (green): an explicit demographic profile is prepended; VAA rises (+0.03 to +0.27) but models collapse responses toward the group centroid (Homog Rate: 28–83%; 55–83% for certified models), confirmed as demographically driven by permutation test (

p<0.001

z\geq 4.67

, 5/7 models). Dialogue (blue): a multi-turn chat history in place of demographics; homogenization partially reverses toward the human baseline (Homog Rate: 20–72%), validated on 80 real PRISM conversations (de-homogenization in 3/4 models, bootstrap CI excludes zero).

Figure 2: Agentic dialogue generation pipeline. Five roles coordinate each turn: the User Simulator generates a profile-conditioned query; the Out-of-Context Detector checks whether it stays on topic; the QA LLM responds, or the Rewriter revises off-topic queries; and the Reviewer decides whether to continue or end the dialogue. For each topic, 1,000 dialogues are generated based on WVS user profiles.

Metric	Formula
VAA	$\frac{1}{\|U\|}\sum_{u\in U}\rho(S_{u}^{\text{model}},H_{u})$
Homog Rate	$\frac{1}{\|U\|}\sum_{u}\mathbb{I}\!\bigl(\\|S_{u}^{\text{m}}{-}S_{g}^{\text{h}}\\|_{2}<\\|S_{u}^{\text{h}}{-}S_{g}^{\text{h}}\\|_{2}\bigr)$ Human baseline ${\approx}50\%$ (95% CI [45, 55]; split-half resampling).
Centroid Pull (CP)	$\overline{(\\|S_{u}^{\text{m}}{-}S_{g}^{\text{h}}\\|_{2}-\\|S_{u}^{\text{h}}{-}S_{g}^{\text{h}}\\|_{2})}$ ; negative = model closer to centroid.
Var. Ratio (VR)	$\text{std}(S_{g}^{\text{model}})\,/\,\text{std}(S_{g}^{\text{human}})$
Bias Amp. (BA)	$L_{1}(S_{g}^{\text{model}},S_{g^{\prime}}^{\text{model}})\,/\,L_{1}(S_{g}^{\text{human}},S_{g^{\prime}}^{\text{human}})$

Table 1: Metric definitions. VAA measures alignment quality; the others measure individuality cost.

Model	None	Profile	Career	Invest.
Llama-3.1-8B	0.510	0.494	0.486	0.500
Llama-3.1-70B	0.336	0.605	0.527	0.552
Qwen2.5-7B	0.599	0.550	0.593	0.577
DeepSeek-V3	0.547	0.606	0.532	0.532
QwQ-32B	0.400	0.614	0.575	0.587
GPT-5.1	0.584	0.613	0.539	0.574

Table 2: Value Alignment Accuracy (VAA; Pearson

r

, higher is better) across four conditions. None = no user information; Profile = explicit demographic profile; Career/Invest. = dialogue-based conditions. Bold = highest in column.

Model	None	Profile	$\Delta$	Eff.
Llama-3.1-70B	0.336	0.605	$+$ 0.269	✓
QwQ-32B	0.400	0.614	$+$ 0.215	✓
Llama-3.1-8B	0.510	0.494	$-$ 0.016	✗
DeepSeek-V3	0.547	0.606	$+$ 0.059	✓
Qwen2.5-72B	0.558	0.617	$+$ 0.059	✓
GPT-5.1	0.584	0.613	$+$ 0.029	✓

Table 3: No-profile baseline ( None ) vs. explicit-profile ( Profile ) per-user VAA, sorted by None .

\Delta=

Profile

-

None . ✓ = profile helps; ✗ = profile hurts. Examined in Section 4.2 .

Figure 3: Personalization spectrum across conditions. Models sorted by None VAA (left = weakest default). The gap between None and Profile bars represents demographic dependence: larger gaps indicate models relying more on stereotyping to achieve alignment. Dialogue-based VAA (Career) is comparable or slightly below Profile for most models.

Model	VAA	Prof HR%	Sig/6	Mean z	Dial HR%
Llama-3.1-8B	0.494	28.0	1/6	0.73	19.5
Llama-3.1-70B	0.605	55.3	6/6	4.67	43.8
Qwen2.5-7B	0.550	69.4	3/6	1.66	46.7
QwQ-32B	0.614	66.9	6/6	8.09	52.4
DeepSeek-V3	0.606	79.8	6/6	4.78	40.0
GPT-5.1	0.613	80.8	6/6	5.43	32.4

Table 4: Alignment-homogenization tradeoff. VAA under Profile ; Prof HR = mean homogenization rate over 6 demographic attributes (10,000 permutations); Dial HR = raw observed rate averaged over 6 attributes

\times

2 domains (career, investment), not permutation-tested (validated via PRISM, Table 5 ); Sig/6 = attributes with

p<0.05

; Mean z = mean

z

-score. All seven models show de-homogenization (Prof HR

>

Dial HR); five show permutation-certified stereotyping under Profile across all 6 attributes.

Figure 4: The alignment-homogenization paradox. Models achieving higher VAA (x-axis) tend to do so through greater homogenization (y-axis). The human baseline (dashed line at 50%) marks the natural clustering rate. All models with high VAA (

>

0.60) substantially exceed the 50% human baseline. QwQ-32B sits below the trend, suggesting reasoning partially mitigates the trade-off.

Figure 5: Within-family scaling effects. Larger model variants show higher homogenization rates (left) but weaker default culture quality (right), confirming that scaling improves demographic stereotype exploitation rather than intrinsic cultural knowledge.

Model	User dist	Dial dist	$\Delta$ dist	95% CI	Homog Rate
GPT-5.1	1.775	1.979	$+$ 0.204 ∗	$[{+}0.18,{+}0.26]$	80.8%
Qwen2.5-72B	1.830	2.065	$+$ 0.235 ∗	$[{+}0.22,{+}0.29]$	83.2%
DeepSeek-V3	1.969	2.275	$+$ 0.306 ∗	$[{+}0.30,{+}0.37]$	79.8%
Llama-3.1-70B	2.190	2.234	$+$ 0.044	$[-0.00,{+}0.10]$	55.3%

Table 5: PRISM cross-dataset validation on 80 real human–chatbot conversations. User dist / Dial dist = mean L2 distance from WVS demographic group centroids under Profile / Dialogue ; higher distance indicates greater individuation from the group centroid. Centroids for distance computation are per-question group means; the Homog Rate column uses the median-based centroid defined in Section 3 .

\Delta

dist = de-homogenization from profile to genuine dialogue; 95% CI from 5,000-sample bootstrap on per-user

\Delta

dist ( ∗ CI excludes zero). Last column: main-dataset homogenization rate for reference.

Figure 6: Pairwise L2 response distances between all 10 age-group pairs under Profile conditioning, plotted against age span (years). Each point is one pair; lines are OLS regression fits. GPT-5.1 distances increase with age span (Spearman

\rho=0.64

, dashed red line), consistent with demographic prototype retrieval whose shift magnitude scales with group prototype distance. Llama-3.1-8B (Homog Rate 28.0%, solid blue line) shows no gradient, confirming its weak stereotyping does not amplify group differences. Human ground truth (grey) provides a reference baseline. Two pairs sharing a 10-year span are jittered

\pm

1.5 yr for visibility.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Datasets and Inputs
Notation	Description
$U$	The complete set of 1,000 user profiles
$u,u^{\prime}$	Distinct user profiles from $U$
$D_{c},D_{i}$	Synthetic dialogue datasets (career, investment)
$D$	General notation for either dataset
$d,d^{\prime},d^{\prime\prime}$	Distinct dialogue instances from $D$

Appendix

Table 6: Summary of mathematical notation used throughout the paper.

Criterion	Career ( $D_{c}$ )		Investment ( $D_{i}$ )
Criterion	Mean	Std	Mean	Std
Coverage	4.99	0.09	4.98	0.13
Correctness	4.99	0.09	4.98	0.13
Diversity	4.19	0.57	4.31	0.52
Relevance	5.00	0.00	5.00	0.05
Naturalness	4.05	0.88	3.91	0.95

Appendix

Table 7: LLM-based validation of generated dialogue datasets (1,000 dialogues, GPT-4.1 , 3 seeds each). Coverage, Correctness, and Relevance are near-ceiling; Diversity and Naturalness show meaningful variance reflecting genuine variation in conversational richness.

Criterion	Career ( $D_{c}$ )		Investment ( $D_{i}$ )
Criterion	Mean	Std	Mean	Std
Coverage	4.50	0.34	3.79	0.74
Correctness	4.44	0.35	3.52	0.70
Diversity	3.04	0.81	3.48	0.74
Relevance	4.12	0.63	3.84	0.81
Naturalness	4.50	0.56	3.47	0.81

Appendix

Table 8: Human annotation on 50 randomly sampled dialogues per topic (4 annotators each). All criteria exceed the scale midpoint. Coverage and Correctness (

\geq 3.52

) confirm demographic embedding fidelity critical for downstream experiments.

Criterion	Career		Investment
Criterion	Pearson $r$	Kripp. $\alpha$	Pearson $r$	Kripp. $\alpha$
Coverage	0.01	$-$ 0.02	0.07	$-$ 0.04
Correctness	0.00	$-$ 0.07	$-$ 0.03	$-$ 0.15
Diversity	$-$ 0.09	0.02	0.22	0.11
Relevance	0.04	$-$ 0.09	0.24	0.07
Naturalness	0.13	0.01	0.27	0.22

Appendix

Table 9: Inter-annotator agreement. Near-zero and occasionally negative Krippendorff’s

\alpha

values indicate annotator responses are less consistent than chance at the item level, beyond what scale-calibration differences alone can explain ( Amidei et al., 2019 ) . Because every individual annotator rates every criterion above the scale midpoint on average, the study confirms overall quality acceptability; it cannot, however, reliably differentiate quality between individual dialogues. Importantly, the downstream de-homogenization finding (Section 4 ) does not depend on this annotation study: it is grounded independently on the PRISM real-conversation cross-validation (Table 5 ), which requires no annotation.

Model	Career		Investment
Model	Ratio	p-val	Ratio	p-val
Llama-3.1-8B	0.996	$<$ 0.001	0.975	$<$ 0.001
Llama-3.1-70B	0.946	$<$ 0.001	0.899	$<$ 0.001
Qwen2.5-7B	0.996	$<$ 0.001	0.987	$<$ 0.001
Qwen2.5-72B	0.886	$<$ 0.001	0.850	$<$ 0.001
DeepSeek-V3	0.923	$<$ 0.001	0.889	$<$ 0.001

Appendix

Table 10: Response-level consistency (lower ratio = more consistent across formats). QwQ-32B achieves best response consistency, yet shows largest preservation instability.

Model	$\rho$	$p$	$n$
GPT-5.1	0.142	0.300	54
Llama-3.1-70B	0.007	0.960	54
QwQ-32B	0.216	0.300	24
Qwen2.5-72B	0.154	0.260	54
DeepSeek-V3	0.164	0.230	54

Appendix

Table 11: Spearman correlation between demographic keyword density and centroid distance in PRISM conversations. No model shows a significant positive correlation, ruling out signal quantity as the driver of de-homogenization.

Model	Age	Cont.	Immig.	Educ.	Occ.
GPT-5.1	1.89\|2.05	1.56\|1.80	1.77\|1.97	1.82\|2.03	1.84\|2.05
Qwen-72B	1.95\|2.16	1.63\|1.90	1.79\|2.02	1.86\|2.07	1.92\|2.18
DeepSeek	2.06\|2.31	1.80\|2.14	1.93\|2.25	1.98\|2.30	2.08\|2.37
Llama-70B	2.26\|2.30	2.07\|2.13	2.14\|2.19	2.22\|2.25	2.26\|2.31

Appendix

Table 12: PRISM per-attribute centroid distances ( Profile | Dialogue ). Smaller = more stereotyped. All models show de-homogenization from profile to dialogue across all attributes.

Model	Condition	Turns	Homog Rate (95% CI)
GPT-5.1	Profile (label)	0	81.3% [77.3, 85.2]
	Dialogue	1	85.1% [81.8, 88.3]
	Dialogue	3	80.5% [76.6, 84.0]
	Dialogue	5 (untrunc.)	23.6% [18.8, 28.4]
DeepSeek-V3	Profile (label)	0	78.7% [74.6, 82.5]
	Dialogue	1	61.6% [56.5, 66.2]

Appendix

Table 13: Homogenization Rate by dialogue length, career domain (95% bootstrap CI; human baseline

{\approx}50\%

). Three qualitatively distinct profiles: GPT-5.1 shows a threshold (de-homogenization only at K=5); DeepSeek-V3 shows a gradient (de-homogenization from K=1); Qwen2.5-72B shows persistence (stays 73–84% across all K, above the human baseline). All models:

n=200

Turns $K$	Demo KW	User words	Density	% of total demo
1	1.22	25	0.049	71%
3	1.58	94	0.017	88%
5	1.88	172	0.011	100%

Appendix

Table 14: Demographic content across the first

K

user turns of the synthetic career dialogues (mean over 1,000 dialogues). Demographic keywords are front-loaded; density falls with length as individuating content accumulates.

	Profile			Dialogue
Model	CP	VR	BA	CP	VR	BA
Llama-3.1-8B	$-3.37$	0.581	0.521	$-3.76$	0.568	0.483
Llama-3.1-70B	$-5.42$	0.749	1.033	$-6.93$	0.522	0.597
Qwen2.5-7B	$-9.20$	0.485	1.397	$-8.27$	0.444	0.472
QwQ-32B	$-4.46$	1.037	1.770	$-6.87$	0.716	1.249
DeepSeek-V3	$-8.02$	0.573	1.399	$-8.60$	0.472	0.944

Appendix

Table 15: Corroborating metrics under Profile and Dialogue , averaged over 6 demographic attributes. CP = Centroid Pull: mean signed deviation

(\text{model dist to centroid})-(\text{human dist to centroid})

; negative values indicate pulling toward the group centroid. VR = Variance Ratio: model within-group std / human within-group std; below 1.0 indicates compression of individual variation. BA = Bias Amplification: ratio of model inter-group signal strength to human inter-group signal strength; above 1.0 indicates exaggeration of group differences. Dialogue values are averaged over career and investment domains.

Model	$\Delta$ VAA	95% CI	Cohen $d$
Llama-3.1-70B	$+0.269$	$[+0.258,\,+0.281]$	$1.43$
QwQ-32B	$+0.215$	$[+0.203,\,+0.227]$	$1.12$
DeepSeek-V3	$+0.059$	$[+0.053,\,+0.064]$	$0.65$
Qwen2.5-72B	$+0.059$	$[+0.053,\,+0.066]$	$0.57$
GPT-5.1	$+0.029$	$[+0.023,\,+0.034]$	$0.33$
Qwen2.5-7B	$-0.049$	$[-0.054,\,-0.043]$	$-0.55$

Appendix

Table 16: Per-model

\Delta

VAA (PROFILE

-

NONE), 95% bootstrap CI (

n=1{,}000

, 5,000 resamples), and paired Cohen’s

d

. All CIs exclude zero.

Personalized Language Model Alignment

Personalized language model alignment adapts model behavior to individual preferences while preserving safety, accuracy, and other shared constraints.

See all 61 papers

Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models
Jun 5, 2026Angana Borah, Isabelle Augenstein, Rada MihalceaLLM Alignment Normative Conformity in Language Models
University of Michigan - Ann Arbor · University of Copenhagen
Cultural Misalignment in Large Language Models: Detection, Measurement, and Mitigation Through Targeted Fine-Tuning
Sep 3, 2026Antoni Czolgowski, Abel IyaseleCultural Bias in Language Models Social Bias in Language Models
Masters in Data Science Program University of Colorado Boulder Boulder, CO 80309, USA · Department of Information Science University of Colorado Boulder Boulder, CO 80309, USA
Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit
May 29, 2026Jiwoo Choi, Seonwoo Ahn, Tongxin Zhang +1Gender Bias in Language Models Multilingual Language Model Evaluation
School of Digital Humanities and Computational Social Sciences, KAIST, South Korea

Jun 5, 2026J/K move · Enter open · S save

Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models

Angana Borah, Isabelle Augenstein, Rada Mihalcea

University of Michigan - Ann Arbor · University of Copenhagen

LLM Alignment Normative Conformity in Language Models Human Preference Modeling Cultural Alignment Language Model Bias Evaluation Personalized Language Model Alignment

Open paper AskPDF arXiv

Large language models are increasingly used for social decision-making situations that require balancing cultural norms with personal preferences. For example, a user preferring honesty might ask whether to correct a coworker publicly when local norms favor indirect feedback. Yet existing research studies cultural alignment and personalization largely separately. We introduce PACT, the Personal-Preference and Cultural-Norm Trade-off framework, which evaluates whether models choose to follow a cultural norm or allow personal preferences. We find that LLMs vary in how rigidly they enforce cultural norms, with behavior shifted more by country context (7.8%) than age (1%) and gender (0.7%) and shifting non-uniformly after instruction tuning. Furthermore, our five-country human study on PACT shows that culture-following in humans is mainly driven by scenario country, with the lowest agreement when participants judge their own cultural contexts, showing within-culture pluralism. Finally, human-LLM alignment experiments show that models can match majority choices, but fail to capture response distributions and uncertainty (with best correlations reaching only 0.24). Together, these findings motivate alignment evaluations that go beyond majority to capture cultural pluralism and disagreement in social judgment.

Model	Age	Cont.	Immig.	Educ.	Occ.
GPT-5.1	1.89\|2.05	1.56\|1.80	1.77\|1.97	1.82\|2.03	1.84\|2.05
Qwen-72B	1.95\|2.16	1.63\|1.90	1.79\|2.02	1.86\|2.07	1.92\|2.18
DeepSeek	2.06\|2.31	1.80\|2.14	1.93\|2.25	1.98\|2.30	2.08\|2.37
Llama-70B	2.26\|2.30	2.07\|2.13	2.14\|2.19	2.22\|2.25	2.26\|2.31

Abstract

Personalized Language Model Alignment

Explore similar work

Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models

Cultural Misalignment in Large Language Models: Detection, Measurement, and Mitigation Through Targeted Fine-Tuning

Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit