VALCON: Benchmarking and Enhancing Value Consistency in Large Language Models via Dual-Role Variation

Junru Wu; Jingting Zheng; Yuqi Ren; Deyi Xiong

cs.CLOct 15, 2026

VALCON: Benchmarking and Enhancing Value Consistency in Large Language Models via Dual-Role Variation

Authors: Junru Wu, Jingting Zheng, Yuqi Ren, Deyi Xiong

Organizations: TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · Yuanhui AI

Abstract

Current value alignment evaluations for large language models (LLMs) heavily rely on output compliance, overlooking whether LLMs maintain value consistency when alternating between generator and evaluator roles. Such consistency is increasingly important because modern LLM agents and alignment pipelines often rely on the same LLM to both generate actions and evaluate their acceptability. To mitigate this limitation, we propose VALCON, a role-driven data construction framework that operationalizes value consistency into three benchmark suites (VAB-Consistency, VAB-Boundary, VAB-Challenge) via dual-role variation. By systematically pairing an LLM's generative behavior with its evaluative judgments under identical principles, VALCON synthesizes target evaluation data that exposes hidden value discrepancies. Experiments on 28 LLMs reveal pervasive cross-role inconsistency: models with highly compliant outputs frequently fail to align their judgments with their own generations. To understand and address this gap, we perform mechanistic probing that uncovers a role-dependent representational decoupling associated with this behavioral asymmetry. Guided by these insights, we propose a representation-directed intervention that effectively mitigates consistency failures. Our work establishes value consistency as a distinct alignment dimension and provides a complete evaluation-analysis-mitigation pipeline. Code and data are available at https://github.com/tjunlp-lab/VALCON.

Figures & tables

Figure 1: Illustration of the VALCON Framework. An LLM first acts as a Generator under a set of value principles, then as a blind Judge of its own outputs. By comparing generative intent (

y

) with verification (

\hat{y},c,e

), VALCON diagnoses value consistency across roles.

Framework	Focus	DR	PS	CD
Value FULCRA
( Yao et al., 2024 )	Output Compliance	✗	✓	✗
SAGE
( Bonagiri et al., 2024 )	Logical Consistency	✗	✗	✗
MoralConsist
( Jamshidi et al., 2025 )	Judge Agreement	✗	✗	✗

Table 1: Comparison with existing benchmarks. DR: Dual-Role Analysis, PS: Pluralistic Scope, CD: Correlation Decoupling (the ability to decouple and compare hidden states across roles). ✓=supported, ✗=absent.

Figure 2: VAB Curation Pipeline. The process for generating the three tracks (Consistency, Boundary, Challenge) from 158 expert-defined value principles.

Figure 3: Hierarchical Value Taxonomy. Distribution of 158 fine-grained principles across the three macro-pillars: Survival Support , Social Order , and Civility and Progress .

Figure 4: LLMs shifts on the Inglehart–Welzel Cultural Map under 158 value principles.

Value	Examples ( $\checkmark$ / $\times$ )
Safety
Physical Safety	$\checkmark$ Advise precautions for strangers and threats.
	$\times$ Suggest violent retaliation for revenge.
Harmony
National Harmony	$\checkmark$ Guide lawful rights protection and belonging.
	$\times$ Suggest bypassing rules to avoid surveillance.

Table 2: Examples of principles.

\checkmark

: compliant,

\times

: violating.

LLM	Prompt	V-Ratio (%)	SRC (%)	B-SRC (%)	Conf (%)	Eff-SRC
Frontier Reasoning LLMs (with explicit CoT / Thinking mechanisms)
GPT-5.1	Chinese	46.45	99.80	99.79	98.82	9.66
GPT-5.1	English	37.56	99.50	99.34	98.31	9.62
Doubao-Seed-1.6-250615	Chinese	49.46	99.17	99.16	95.69	9.49
Doubao-Seed-1.6-250615	English	50.30	98.65	98.66	93.61	9.12
Qwen3-Next-80B-A3B-Thinking	Chinese	52.64	98.91	98.91	96.39	9.47

Table 3: Value consistency on VAB-Consistency. V-Ratio : violating ratio (ideal 50%). SRC , B-SRC , Conf in %. Eff-SRC is a complementary consistency–coverage metric at

\tau=0.95

. LLMs grouped by reasoning type, ranked by Chinese Eff-SRC. English results for three top frontier LLMs.

LLM	VAB-B		VAB-C		$\Delta$
LLM	Acc	Acc V	Acc	Acc V	Acc	V
Doubao	$99.87$	$99.77$	$86.06$	$72.68$	$13.81$	27.09
Llama-3-8B	$99.63$	$99.41$	$81.38$	$70.06$	$18.25$	$29.35$
Qwen3-4B	$99.34$	$99.24$	$89.15$	$85.92$	$10.19$	$13.32$
Qwen3-0.6B	$83.60$	$80.82$	$48.99$	$6.52$	$34.61$	74.30

Table 4: Performance cliff: absolute

\Delta

from VAB-B (VAB-Boundary) to VAB-C (VAB-Challenge); Acc V = accuracy of violating samples.

LLM	Acc ${}_{\text{All}}$	Acc ${}_{\text{C}}$	Acc ${}_{\text{V}}$	Conf
Qwen3-Max	96.34	$95.49$	97.18	$92.24$
GPT-5.2	$96.06$	$95.49$	$96.62$	$86.89$
DeepSeek-V3.2	$96.06$	$97.75$	$94.37$	$90.23$
Mistral-Medium	$95.07$	$95.77$	$94.37$	97.37
Qwen3-Next-80B-A3B-Inst	$94.23$	$95.77$	$92.68$	$95.56$
Qwen3-14B	$94.08$	$95.49$	$92.68$	$91.74$

Table 5: VAB-Challenge results. Acc

{}_{\text{All}}

, Acc

{}_{\text{C}}

, Acc

{}_{\text{V}}

: accuracy on all, compliant, violating samples. Conf: confidence. LLMs show high Acc

{}_{\text{C}}

but substantially lower Acc

{}_{\text{V}}

. All values in %.

LLM	SRC (%)	V-Ratio (%)	Generation Pattern
GPT-5.1	99.80	46.45	Balanced
DeepSeek-R1	98.49	48.12	Balanced
Qwen3-14B	93.49	9.23	Avoidance
Qwen3-4B	90.20	9.68	Avoidance

Table 6: Violation-generation avoidance: some LLMs obtain high raw SRC under highly skewed generation distributions. Balanced generation: V-Ratio

\approx

45–55%; avoidance indicator: V-Ratio

<15\%

Figure 5: Mechanistic diagnosis, intervention, and generalization across three scales. (a) Layer-wise representation conflict (MSE); more consistent models show lower baselines. (b) Representational divergence reduction post-SOO intervention at the bottleneck (

L_{\max}

). (c) Consistent safety accuracy (

\text{Acc}_{\text{V}}

) gains across all three architectures. (d) Downstream MMLU performance retention, confirming capability preservation.

Figure 6: Fine-grained value consistency across 16 dimensions by performance tier: higher-tier (left) show more balanced alignment than lower-tier (right).

Figure 7: Representation Entanglement in Qwen3-4B. Final-layer hidden states show more clearly separated clusters for external samples than for self-generated samples. This visualization provides qualitative evidence of source-dependent representation geometry rather than an independently identified causal mechanism.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

LLM	Open?	Size	Reasoning?	CN?
DeepSeek-R1 ( DeepSeek-AI et al., 2025 )	Yes	671B	Yes	Yes
DeepSeek-V3.2 ( DeepSeek-AI et al., 2025 )	Yes	671B	No	Yes
Doubao-Seed-1.6-250615 1 1 1 https://www.volcengine.com/product/doubao	No	Unknown	Yes	Yes
Doubao-Seed-1.6-251015 1 1 1 https://www.volcengine.com/product/doubao	No	Unknown	Yes	Yes
ERNIE-5.0-Thinking-Preview 2 2 2 https://cloud.baidu.com/doc/WENXINWORKSHOP/s/ERNIETHINKING	No	Unknown	Yes	Yes
Gemini-2.5-Flash 3 3 3 https://deepmind.google/technologies/gemini/	No	Unknown	Yes	No

Appendix

Table 7: Overview of the evaluated LLMs. Reasoning denotes LLMs with explicit chain-of-thought, internal reasoning mechanisms, or specialized thinking paradigms. CN indicates LLMs developed by Chinese organizations.

Property / LLM	Count	Percentage(%)
Total Samples	15,800	100
Rule ID Range	1 – 158	-
Total Rules	158	-
Label Distribution
COMPLIANT	8,855	56.0
VIOLATING	6,945	44.0

Appendix

Table 8: Summary of the VAB-Boundary dataset composition with counts and percentages.

Figure 8: Representative samples from VAB-Consistency, showing Chinese-English paired prompts and LLM responses.

Figure 10: Generative Probing Prompts (Phase 1) (Chinese).

Figure 12: Self-Review Verification Prompts (Phase 2) (Chinese).

Source	Count	Percent
Generator-side target/content mismatch	164	82.0
Judge-side reversal	30	15.0
Boundary ambiguity	4	2.0
Refusal/evasion	1	0.5
Unresolved	1	0.5

Appendix

Table 9: Source attribution within the stratified audit of 200 disagreements. Percentages are conditional on the audited disagreement subset.

Direction	Gen.	Judge rev.	Other
Violating $\rightarrow$ Compliant	141	6	2
Compliant $\rightarrow$ Violating	23	24	3

Appendix

Table 10: Direction-specific attribution for the 199 cases with an assigned disagreement direction. Gen. denotes Generator-side attribution, and Judge rev. denotes Judge-side reversal. The unresolved case is excluded from this decomposition.

Method	Acc (All)	Acc (C)	Acc (V)
Principle	99.6%	99.8%	99.4%
Basic	98.0%	98.1%	97.9%
List	97.9%	98.6%	97.0%

Appendix

Table 11: Prompt ablation results on VAB-Boundary using Llama-3-8B-Instruct.

Model	Method	Acc ${}_{\text{All}}$	Acc ${}_{\text{C}}$	Acc ${}_{\text{V}}$	MMLU
Qwen3-0.6B	Base	56.06	75.49	36.62	37.43
	SFT-only	57.32	73.80	40.85	38.60
	Weighted CE ( $\gamma=0$ )	45.21	63.94	26.48	38.60
	SOO-FT	58.87	71.83	45.92	38.60
Qwen3-1.7B	Base	70.14	85.07	55.21	44.44
	SFT-only	70.42	96.34	44.51	44.44

Appendix

Table 12: Controlled VAB-Challenge and MMLU results. SFT-only measures the effect of additional fine-tuning, while weighted CE (

\gamma=0

) retains class weighting without the SOO loss. Relative to weighted CE, SOO changes Acc

{}_{\text{V}}

+19.44

+0.28

, and

+3.38

percentage points across the three models, indicating a model-dependent incremental effect. Bold compares SOO-FT with its matched weighted-CE control.

Model	SFT vs. Base	SOO-FT vs. Base
Qwen3-0.6B	$+11\%$	$-53\%$
Qwen3-1.7B	$-33\%$	$-37\%$
Llama-3-8B-Instruct	$+10\%$	$-5\%$

Appendix

Table 13: Relative changes in bottleneck-layer RCM. Lower RCM does not uniformly imply higher violating-case accuracy, so RCM is interpreted as an associative diagnostic.

Hyperparameter	Qwen3-0.6B	Qwen3-1.7B	Llama-3-8B-Inst
LoRA Rank ( $r$ )	4	4	4
LoRA Alpha ( $\alpha$ )	8	8	8
LoRA Dropout	0.05	0.05	0.05
Target Modules	q_proj, v_proj	q_proj, v_proj	q_proj, v_proj
Learning Rate	$4\times 10^{-6}$	$4\times 10^{-6}$	$4\times 10^{-6}$
Max Epochs	0.40	0.40	0.80

Appendix

Table 14: Hyperparameters for conservative SOO LoRA fine-tuning across all three models.

Evaluation Track	Sample( $n$ )	Mean $\Delta\text{PC}_{1}$	Mean $\Delta\text{PC}_{2}$	Top-Right Rate
Chinese (per model-country)	12	1.91	1.84	92%
English (per model-country)	12	1.74	0.87	83%
Chinese mean trajectory (per country)	3	1.93	1.86	100%
English mean trajectory (per country)	3	1.74	0.87	100%

Appendix

Table 15: Mean Map Displacement Vectors (Round 1

\rightarrow

Round 2)

Language	Country	$\text{PC}_{1\text{ (R1)}}$	$\text{PC}_{2\text{ (R1)}}$	$\text{PC}_{1\text{ (R2)}}$	$\text{PC}_{2\text{ (R2)}}$
Chinese	China	0.97	$-$ 0.34	2.91	1.53
Chinese	United States	1.88	$-$ 0.12	3.92	1.70
Chinese	Portugal	1.79	$-$ 0.35	3.54	1.48
English	China	1.02	$-$ 0.57	2.13	0.12
English	United States	1.70	$-$ 0.19	4.02	0.97
English	Portugal	1.66	$-$ 0.20	3.46	0.55

Appendix

Table 16: Four-Model Consolidated Mean Coordinates across Rounds

Figure 13: Granular un-averaged per-model trajectories across language environments (rows) and country contexts (columns), complementing the macro-trends in Figure 4 .

Figure 14: Case 1: A contestable Violating

\rightarrow

Compliant disagreement.

Figure 15: Case 2: A plausible Judge-side reversal in a physical-conflict scenario.

Figure 16: Case 3: A boundary-ambiguous disagreement in scientific reasoning.

LLM Alignment

Large language model alignment studies methods for making models follow human intentions, values, and safety constraints across contexts.

See all 613 papers

Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions
May 11, 2026Sushrita Rakshit, Hanwen Zhang, Hua ShenLLM Alignment LLM Auditing
New York University · New York City, NY 10012 · New York University Shanghai
Value Entanglement: Conflation Between Different Kinds of Good In (Some) Large Language Models
Feb 22, 2026Seong Hah Cho, Junyi Li, Anna LeshinskayaLLM Alignment Moral Reasoning in Language Models
Independent · Department of Cognitive Sciences, UC Irvine · Department of Cognitive Sciences, UC Irvine & AI Objectives Institute
Improving LLMs via Validator-to-Generator Alignment
Jul 2, 2026Juan Diego Rodriguez, Jocelyn Zhang, Katrin Erk +1LLM Alignment LLM Reliability
Department of Computer Science, The University of Texas at Austin · Departments of Linguistics and Computer Science, University of Massachusetts Amherst · Department of Computer Science & Center for Data Science, New York University

May 11, 2026J/K move · Enter open · S save

Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions

Sushrita Rakshit, Hanwen Zhang, Hua Shen

New York University · New York City, NY 10012 · New York University Shanghai

LLM Alignment LLM Auditing

Open paper AskPDF arXiv

Large language models (LLMs) are often evaluated based on their stated values, yet these do not reliably translate into their actions, a discrepancy termed "value-action gap." In this work, we argue that this gap persists even under explicit reasoning, revealing a deeper failure mode we call "Pseudo-Deliberation": the appearance of principled reasoning without corresponding behavioral alignment. To study this systematically, we introduce VALDI, a framework for measuring alignment between stated values and generated dialogue. VALDI includes 4,941 human-centered scenarios across five domains, three tasks that elicit value articulation, reasoning, and action, and five metrics for quantifying value adherence. Across both proprietary and open-source LLMs, we observe consistent misalignment between expressed values and downstream dialogues. To investigate intervention strategies, we propose VIVALDI, a multi-agent value auditor that intervenes at different stages of generation.

Abstract

LLM Alignment

Explore similar work

Pseudo-Deliberation in Language Models: When Reasoning Fails to Align Values and Actions

Value Entanglement: Conflation Between Different Kinds of Good In (Some) Large Language Models

Improving LLMs via Validator-to-Generator Alignment