CoCoA: Context-Conditional Cultural Alignment for Large Language Models

Kyungdon Lee; Wei Xu; Alan Ritter; Dong-Ho Lee; JinYeong Bak

cs.CLAug 30, 2026

CoCoA: Context-Conditional Cultural Alignment for Large Language Models

Authors: Kyungdon Lee, Wei Xu, Alan Ritter, Dong-Ho Lee, JinYeong Bak

Organizations: Sungkyunkwan University, Suwon, South Korea · Georgia Institute of Technology, GA, USA · University of Southern California, CA, USA

Abstract

Large Language Models (LLMs) often favor Western-associated entities across cultural contexts. Conventional debiasing methods aim for uniform neutrality, but cultural bias mitigation demands context-conditional behavior, preferring culturally appropriate entities when cultural cues are present and remaining neutral when they are absent. We propose CoCoA (Context-Conditional Cultural Alignment), a framework that learns this behavior through dual-context training on the same entity pairs under contexts with and without cultural cues. CoCoA combines a contrastive alignment objective with calibration and drift regularization, optimized through goal-aware gradient reconciliation. We evaluate CoCoA on CAMeL and Camellia, two entity-centric cultural bias benchmarks, across ten language settings and four LLMs. CoCoA reduces the Cultural Bias Score from 43 to 24 on average while maintaining near-neutral preferences at 50.2, with minimal impact on general performance across five standard benchmarks. These findings highlight that effective cultural alignment requires context-conditional modeling rather than uniform debiasing, and establish a new direction for mitigating entity-centric cultural bias in LLMs.

Figures & tables

Figure 1: Illustration of cultural bias and cultural homogenization. (a) Cultural Bias. Given a context with an explicit Korean cultural cue, the model should prefer Korean entities over Western ones, but in practice produces the opposite ranking. (b) Cultural Homogenization. A context-insensitive neutralization approach equalizes entity preferences uniformly across all contexts.

Figure 2: Overview of the CoCoA framework. Each training instance pairs a local entity $e_{l}$ and an other-culture entity $e_{o}$ with a grounded context $c_{g}$ and a neutral context $c_{n}$ , producing four prior-adjusted scores. The grounded loss $\mathcal{L}_{g}$ encourages local preference, the neutral loss $\mathcal{L}_{n}$ targets balanced preferences, and the drift regularizer $\mathcal{L}_{d}$ stabilizes neutral-context behavior during joint optimization. Weighted gradient directions $\mathbf{v}_{g}$ and $\mathbf{v}_{n}$ are reconciled via goal-aware PCGrad into the final update direction $\mathbf{v}_{f}$ before updating the LoRA adapters $\phi$ .

Llama-3.1-8B

Korean

Japanese

Vietnamese

Hindi

Arabic

Avg

Method

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

Vanilla

47.0

\pm

2.8

–

49.4

\pm

2.1

2.4

36.1

\pm

2.4

–

42.7

\pm

3.6

6.6

12.8

\pm

1.8

–

19.7

\pm

1.7

6.9

32.2

\pm

3.7

–

41.0

\pm

1.7

8.8

37.5

\pm

1.9

–

41.9

\pm

2.8

4.4

33.1

\pm

5.6

–

38.9

\pm

4.9

5.8

BiasUnlearn

47.0

\pm

2.9

+0.0

49.3

\pm

2.2

2.3

40.6

\pm

4.5

+4.5

48.1

\pm

6.8

7.5

21.6

\pm

3.3

+8.8

30.2

\pm

3.3

8.6

36.4

\pm

5.3

+4.2

47.7

\pm

3.3

11.3

42.3

\pm

3.2

+4.8

48.4

\pm

4.5

6.1

37.6

\pm

4.3

+4.5

44.7

\pm

3.6

7.2

BiasEdit

20.8

\pm

3.8

-

26.2

21.8

\pm

3.9

1.0

27.6

\pm

2.2

-

8.5

28.0

\pm

1.7

0.4

32.4

\pm

2.7

+19.6

36.4

\pm

3.3

4.0

46.4

\pm

2.6

+14.2

46.6

\pm

2.6

0.2

39.2

\pm

2.8

+1.7

38.9

\pm

2.7

0.3

33.3

\pm

4.4

+0.2

34.3

\pm

4.2

1.2

CoCoA

3.6

\pm

0.5

-

43.4

51.3

\pm

1.9

47.7

6.3

\pm

1.1

-

29.8

49.4

\pm

5.6

43.1

7.6

\pm

1.9

-

5.2

36.0

\pm

4.2

28.4

15.4

\pm

3.0

-

16.8

52.2

\pm

3.7

36.8

12.2

\pm

2.0

-

25.3

52.3

\pm

3.0

40.1

9.0

\pm

2.1

-

24.1

48.2

\pm

3.0

39.2

Table 1: Cultural Bias Scores on five language settings across four backbone models (5-fold CV, mean

\pm

SE). CBS g measures Western preference in grounded contexts, CBS n measures balance in neutral contexts, and CS

=|

CBS

{}_{n}-

CBS

{}_{g}|

summarizes context sensitivity.

\Delta g

is the change in CBS g relative to the vanilla model. Bold marks the best value per column, underline the second best. The remaining five language settings are in Table 15 .

Objective	Weight	CBS ${}_{g}\!\downarrow$	CBS ${}_{n}\!\to\!50$
Grounded	$w_{g}=0$	$54.6{\pm}1.3$	$60.7{\pm}1.4$
	$w_{g}=0.5$	$15.8{\pm}1.4$	$53.6{\pm}1.4$
	$w_{g}=1.0$	$15.0{\pm}1.3$	$53.7{\pm}1.4$
	$w_{g}=1.5$	$14.7{\pm}1.3$	$52.8{\pm}1.4$
	$w_{g}=2.0$	$14.5{\pm}1.3$	$52.9{\pm}1.4$
Neutral	$w_{n}=0$	$13.4{\pm}1.1$	$37.0{\pm}1.1$

Table 2: Objective weight sensitivity on Llama-3.1-8B, averaged across 10 language settings (5-fold CV). Each block is an independent sweep of one weight with the others fixed at

w_{g}{=}1.0

w_{n}{=}2.0

w_{d}{=}1.0

Figure 3: Per-pair score gap of local–Western entities under the vanilla model (x-axis) vs. CoCoA (y-axis) on Llama-3.1-8B. The diagonal marks no change.

Figure 4: Per-entity-type $\Delta\mathrm{CBS}_{g}$ on Llama-3.1-8B. Negative values indicate reduced Western preference in grounded contexts.

Method	MMLU	HSwag	ARC-C	WGrande	TQA
Method	(5-shot)	(10-shot)	(25-shot)	(5-shot)	(0-shot)
Vanilla	65.34	82.24	58.19	78.37	44.16
CoCoA	65.22	82.04	57.81	78.06	44.14
$\Delta$	$-$ 0.12	$-$ 0.20	$-$ 0.38	$-$ 0.31	$-$ 0.02

Table 3: General benchmark performance on Llama-3.1-8B. CoCoA scores are averaged across all 50 checkpoints (10 language settings

\times

5 folds).

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Language	$w_{n}$	$w_{d}$
Llama	KO	2.0	20.0
	VI	2.0	0.5
	HI, MR	2.0	3.0
Qwen	KO	1.0	3.0
Mistral	KO	1.0	30.0
Mistral	HI	2.0	5.0

Appendix

Table 4: Non-default hyperparameters. All unlisted pairs use

w_{g}{=}1.0

w_{n}{=}2.0

w_{d}{=}1.0

. Gemma uses defaults for all languages.

		$\mathrm{CBS}_{n}$
Language	$w_{d}$	Default	Selected
KO	20.0	69.1	51.3
JA	1.0	49.4	49.4
ZH	1.0	49.3	49.3
VI	0.5	33.8	36.0
UR	1.0	52.4	52.4

Appendix

Table 5: Neutral-context behavior under the default weights and under the selected weights, on Llama-3.1-8B over 5-fold cross-validation. Per-setting entries are fold means. Settings with

w_{d}{=}1.0

keep the default and are unchanged between the two columns.

CSP

You are a person with {culture} cultural background. You will be performing a sentence completion task. Please directly complete the sentence.

CCT

You are a person with {culture} cultural background. You will be performing a sentence completion task. Please directly complete the sentence. Before you respond, think about how {culture} culture is different from Western cultures.

Appendix

Table 6: Instruction prefixes for the two prompting baselines. {culture} is filled with the cultural group of the setting. Both are prepended to the benchmark context, which is otherwise unchanged.

Backbone	Method	$\mathrm{CBS}_{g}\downarrow$	$\mathrm{CBS}_{n}\rightarrow 50$
Llama-3.1-8B	Vanilla	36.4 $\pm$ 1.4	43.4 $\pm$ 1.3
	CSP	36.2 $\pm$ 1.5	40.5 $\pm$ 1.4
	CCT	36.9 $\pm$ 1.5	42.1 $\pm$ 1.6
	CoCoA	15.2 $\pm$ 1.3	50.3 $\pm$ 0.9
Qwen3-8B	Vanilla	41.7 $\pm$ 1.2	45.8 $\pm$ 1.2
	CSP	41.1 $\pm$ 1.2	43.3 $\pm$ 1.1

Appendix

Table 7: Culture-aware prompting against CoCoA, macro-averaged over the ten language settings under 5-fold cross-validation and reported as mean

\pm

SE over the 50 setting–fold runs.

Training scheme	$\mathrm{CBS}_{g}\downarrow$	$\mathrm{CBS}_{n}\rightarrow 50$
Setting-specific	15.2 $\pm$ 1.3	50.3 $\pm$ 0.9
Single joint	17.2 $\pm$ 1.4	56.2 $\pm$ 1.7
Vanilla	36.4 $\pm$ 1.4	43.4 $\pm$ 1.3

Appendix

Table 8: One adapter trained on the pooled data of all ten language settings, against the setting-specific adapters used elsewhere in the paper. Llama-3.1-8B, macro-averaged over the ten settings under 5-fold cross-validation, reported as mean

\pm

SE over the 50 setting–fold runs.

Method	CrowS-Pairs	StereoSet
Method	Stereo. Pref. $\to 50$	SS $\to 50$	LMS $\uparrow$	ICAT $\uparrow$
Vanilla	65.8	66.40	92.53	62.19
CoCoA	65.3	66.23	92.20	62.27
$\Delta$	$-$ 0.5	$-$ 0.17	$-$ 0.33	$+$ 0.08

Appendix

Table 9: Social bias evaluation on Llama-3.1-8B. CrowS-Pairs reports stereotypical preference rate, and StereoSet reports Stereotype Score (SS), Language Model Score (LMS), and ICAT. For Stereo. Pref. and SS, 50% indicates no systematic stereotypical preference.

Group	Symbol	Description	Value
LoRA	$r$	Rank	$16$
	$\alpha_{\mathrm{LoRA}}$	LoRA scaling factor	$32$
	$p_{\mathrm{drop}}$	Dropout	$0.05$
Optimization	$\eta$	Peak learning rate (AdamW)	$1\!\times\!10^{-5}$
	–	Weight decay	$0.01$
	–	Warmup ratio (cosine schedule)	$0.1$

Appendix

Table 10: Default CoCoA hyperparameters (Llama-3.1-8B). LoRA targets attention projections (

W_{Q},W_{K},W_{V},W_{O}

) across all layers. The vanilla model is kept as a fixed snapshot. Language-specific adjustments are in Table 4 .

			CBS g ( $\downarrow$ )		$\|$ CBS ${}_{n}-50\|$ ( $\downarrow$ )
Model	Language	Mean $\|\Delta\|$	vanilla	CoCoA	vanilla	CoCoA
Llama-3.1-8B	KO	0.11	3.6	3.4	1.46	1.71
	JA	0.51	6.3	5.0	4.74	4.40
	ZH	0.99	14.1	14.4	2.67	4.87
	VI	0.52	7.6	5.8	14.06	20.24
	UR	0.24	11.6	11.2	2.81	3.38

Appendix

Table 11: Prior drift and its impact on PMI-based CBS evaluation (5-fold average). Mean

|\Delta|

measures the average absolute change in entity log-priors after LoRA fine-tuning. CBS columns compare evaluation with vanilla priors (used in all main experiments) against re-estimated CoCoA fine-tuned priors.

Gradient combination	$\mathrm{CBS}_{g}\downarrow$	$\mathrm{CBS}_{n}\rightarrow 50$
Goal-aware projection	15.2 $\pm$ 1.3	50.3 $\pm$ 0.9
Weighted sum	15.3 $\pm$ 1.3	49.9 $\pm$ 0.9
Vanilla	36.4 $\pm$ 1.4	43.4 $\pm$ 1.3

Appendix

Table 12: Gradient combination on Llama-3.1-8B, macro-averaged over the ten language settings under 5-fold cross-validation, reported as mean

\pm

SE over the 50 setting–fold runs. The two CoCoA rows use the same objective weights and differ only in whether the projection is applied.

Context	$\Delta$ local	$\Delta$ other-culture	$\Delta$ gap
Grounded	$+0.21$	$-6.87$	$+7.08$
Neutral	$+0.38$	$+1.26$	$-0.88$

Appendix

Table 13: Change in prior-adjusted score from the vanilla model to CoCoA, in log-probability units, on Llama-3.1-8B over the ten language settings and five folds. Positive values mean the entity is scored higher after training.

Held-out entity-level result	Value
Pairs with $\Delta_{g}>0$	93.8 $\pm$ 0.8%
Pairs with $\Delta_{g}>\Delta_{n}$	95.6 $\pm$ 0.6%
Entities with $\Delta_{g}>\Delta_{n}$ for $\geq$ 80% of counterparts	83.1%

Appendix

Table 14: Context sensitivity measured over held-out local entities and all their Western counterparts, on Llama-3.1-8B, with mean

\pm

SE over the 50 setting–fold runs.

Llama-3.1-8B

Chinese

Urdu

Malayalam

Marathi

Gujarati

Avg

Method

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

\mathrm{CBS}_{\!g}

\downarrow

\Delta g

\mathrm{CBS}_{\!n}

\!\to\!50

\uparrow

Vanilla

36.2

\pm

3.7

–

43.1

\pm

1.8

6.9

29.5

\pm

4.2

–

44.6

\pm

3.1

15.1

44.5

\pm

2.8

–

51.5

\pm

5.5

7.0

43.6

\pm

1.1

–

51.2

\pm

0.6

7.6

44.1

\pm

1.9

–

49.4

\pm

2.6

5.3

39.6

\pm

2.7

–

48.0

\pm

1.7

8.4

BiasUnlearn

39.5

\pm

3.1

+3.3

47.6

\pm

3.0

8.1

31.4

\pm

3.6

+1.9

49.0

\pm

3.6

17.6

44.4

\pm

3.3

-

0.1

51.2

\pm

7.4

6.8

43.6

\pm

1.1

+0.0

51.1

\pm

0.5

7.5

44.3

\pm

1.8

+0.2

49.5

\pm

2.9

5.2

40.6

\pm

2.3

+1.0

49.7

\pm

0.6

9.0

BiasEdit

26.2

\pm

2.0

-

10.0

26.7

\pm

1.6

0.5

41.5

\pm

5.3

+12.0

40.8

\pm

3.6

0.7

46.3

\pm

2.3

+1.8

42.8

\pm

4.6

3.5

44.1

\pm

1.7

+0.5

43.5

\pm

1.9

0.6

44.5

\pm

1.4

+0.4

41.6

\pm

2.6

2.9

40.5

\pm

3.6

+0.9

39.1

\pm

2.9

1.6

CoCoA

14.1

\pm

3.9

-

22.1

49.3

\pm

3.2

35.2

11.6

\pm

4.0

-

17.9

52.4

\pm

4.1

40.8

28.6

\pm

3.0

-

15.9

52.0

\pm

7.1

23.4

22.4

\pm

2.9

-

21.2

54.7

\pm

4.5

32.3

30.3

\pm

3.6

-

13.8

53.6

\pm

3.3

23.3

21.4

\pm

3.8

-

18.2

52.4

\pm

0.9

31.0

Appendix

Table 15: Results for the remaining five language settings (ZH, UR, ML, MR, GU). Same evaluation protocol and notation as Table 1 . Avg is computed over these five settings only.

		Auth		Bev		Food		Loc		Na(M)		Na(F)		Sport		Total
Dataset	Language	G	N	G	N	G	N	G	N	G	N	G	N	G	N	G	N
Camellia	Korean (KO)	31	42	30	52	34	64	26	24	29	37	29	37	29	26	179	245
	Japanese (JA)	32	33	26	27	29	29	27	27	19	19	19	19	23	24	156	159
	Chinese (ZH)	25	21	27	27	31	31	27	27	27	24	27	24	21	20	158	150
	Vietnamese (VI)	18	32	14	39	67	46	33	44	16	14	21	7	10	16	179	198
	Urdu (UR)	10	10	10	10	10	10	10	10	10	10	10	10	10	10	70	70

Appendix

Table 16: Number of grounded (G) and neutral (N) contexts per language and entity type. For KO, JA, and ZH, Names contexts are shared across male and female entity types. Indian cultures (HI, ML, MR, GU) share context sets with minor variations.

		Local Entities per Type							Total
Dataset	Language	Auth	Bev	Food	Loc	Na(M)	Na(F)	Sport	Local	Other
Camellia	Korean (KO)	602	107	416	1,260	899	886	266	4,436	3,709
	Japanese (JA)	260	115	635	817	503	523	354	3,207	3,699
	Chinese (ZH)	165	189	415	1,000	906	1,123	1,578	5,376	3,709
	Vietnamese (VI)	24	89	376	92	251	151	51	1,034	3,709
	Urdu (UR)	43	10	74	196	334	163	17	837	2,939

Appendix

Table 17: Number of local entities per language and entity type. Other-culture (Western) entities are shared within each benchmark. Indian cultures (HI, ML, MR, GU) share entity sets with minor variations.

Artifact	Type	License	Source
Llama-3.1-8B	Model	Llama 3.1 Community	HuggingFace
Qwen3-8B	Model	Apache 2.0	HuggingFace
Mistral-7B-v0.3	Model	Apache 2.0	HuggingFace
Gemma-3-12B-pt	Model	Gemma Terms of Use	HuggingFace
Camellia	Dataset	MIT	GitHub
CAMeL	Dataset	MIT	GitHub

Appendix

Table 18: Licenses and sources for all artifacts used in this work. All artifacts are publicly available and used consistently with their intended research purposes.

Symbol	Description
Contexts and entities
$c_{g},\ c_{n}$	Grounded and neutral context strings
$\mathcal{C}_{g},\ \mathcal{C}_{n}$	Sets of grounded and neutral contexts
$e_{l},\ e_{o}$	Local entity and other-culture entity
$(e_{l},e_{o})$	Culturally contrasting entity pair
Scoring

Appendix

Table 19: Mathematical notation used throughout the paper.

Cultural Bias in Language Models

Cultural bias in language models is the uneven representation of cultural knowledge, values, identities, and norms across groups and languages.

See all 125 papers

AlignCultura: Towards Culturally Aligned Large Language Models?
Apr 21, 2026Gautam Siddharth Kashyap, Mark Dras, Usman NaseemCross-Cultural Language Model Evaluation Cultural Alignment
School of Computing, Macquarie University, Australia
Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages
Oct 6, 2025Tarek Naous, Anagha Savit, Carlos Rafael Catalan +17Multilingual Language Model Evaluation Cultural Bias in Language Models
Georgia Institute of Technology · Samsung R&D Institute Philippines · Sungkyunkwan University +6
Prompt Programming for Cultural Bias and Alignment of Large Language Models
Mar 17, 2026Maksim Eren, Eric Michalak, Brian Cook +1LLM Alignment Cultural Alignment
Advanced Research in Cyber Systems, Los Alamos National Laboratory, Los Alamos, New Mexico, USA · Analytics Division, Los Alamos National Laboratory, Los Alamos, New Mexico, USA

Apr 21, 2026J/K move · Enter open · S save

AlignCultura: Towards Culturally Aligned Large Language Models?

Gautam Siddharth Kashyap, Mark Dras, Usman Naseem

School of Computing, Macquarie University, Australia

Cross-Cultural Language Model Evaluation Cultural Alignment Cultural Bias in Language Models Language Model Bias Evaluation Cultural Knowledge in Language Models

Open paper AskPDF arXiv

Cultural alignment in Large Language Models (LLMs) is essential for producing contextually aware, respectful, and trustworthy outputs. Without it, models risk generating stereotyped, insensitive, or misleading responses that fail to reflect cultural diversity w.r.t Helpful, Harmless, and Honest (HHH) paradigm. Existing benchmarks represent early steps toward cultural alignment; yet, no benchmarks currently enables systematic evaluation of cultural alignment in line with UNESCO's principles of cultural diversity w.r.t HHH paradigm. Therefore, to address this gap, we built Align-Cultura, two-stage pipeline for cultural alignment. Stage I constructs CULTURAX, the HHH-English dataset grounded in the UNESCO cultural taxonomy, through Query Construction, which reclassifies prompts, expands underrepresented domains (or labels), and prevents data leakage with SimHash. Then, Response Generation pairs prompts with culturally grounded responses via two-stage rejection sampling. The final dataset contains 1,500 samples spanning 30 subdomains of tangible and intangible cultural forms. Stage II benchmarks CULTURAX on general-purpose models, culturally fine-tuned models, and open-weight LLMs (Qwen3-8B and DeepSeek-R1-Distill-Qwen-7B). Empirically, culturally fine-tuned models improve joint HHH by 4%-6%, reduce cultural failures by 18%, achieve 10%-12% efficiency gains, and limit leakage to 0.3%.

Abstract

Cultural Bias in Language Models

Explore similar work

AlignCultura: Towards Culturally Aligned Large Language Models?

Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages

Prompt Programming for Cultural Bias and Alignment of Large Language Models