Current value alignment evaluations for large language models (LLMs) heavily rely on output compliance, overlooking whether LLMs maintain value consistency when alternating between generator and evaluator roles. Such consistency is increasingly important because modern LLM agents and alignment pipelines often rely on the same LLM to both generate actions and evaluate their acceptability. To mitigate this limitation, we propose VALCON, a role-driven data construction framework that operationalizes value consistency into three benchmark suites (VAB-Consistency, VAB-Boundary, VAB-Challenge) via dual-role variation. By systematically pairing an LLM's generative behavior with its evaluative judgments under identical principles, VALCON synthesizes target evaluation data that exposes hidden value discrepancies. Experiments on 28 LLMs reveal pervasive cross-role inconsistency: models with highly compliant outputs frequently fail to align their judgments with their own generations. To understand and address this gap, we perform mechanistic probing that uncovers a role-dependent representational decoupling associated with this behavioral asymmetry. Guided by these insights, we propose a representation-directed intervention that effectively mitigates consistency failures. Our work establishes value consistency as a distinct alignment dimension and provides a complete evaluation-analysis-mitigation pipeline. Code and data are available at https://github.com/tjunlp-lab/VALCON.
Figures & tables
Figure 1: Illustration of the VALCON Framework. An LLM first acts as a Generator under a set of value principles, then as a blind Judge of its own outputs. By comparing generative intent ( y ) with verification ( y^,c,e ), VALCON diagnoses value consistency across roles.
Framework
Focus
DR
PS
CD
Value FULCRA
( Yao et al., 2024 )
Output Compliance
✗
✓
✗
SAGE
( Bonagiri et al., 2024 )
Logical Consistency
✗
✗
✗
MoralConsist
( Jamshidi et al., 2025 )
Judge Agreement
✗
✗
✗
Table 1: Comparison with existing benchmarks. DR: Dual-Role Analysis, PS: Pluralistic Scope, CD: Correlation Decoupling (the ability to decouple and compare hidden states across roles). ✓=supported, ✗=absent.
Figure 2: VAB Curation Pipeline. The process for generating the three tracks (Consistency, Boundary, Challenge) from 158 expert-defined value principles.
Figure 3: Hierarchical Value Taxonomy. Distribution of 158 fine-grained principles across the three macro-pillars: Survival Support , Social Order , and Civility and Progress .
Figure 4: LLMs shifts on the Inglehart–Welzel Cultural Map under 158 value principles.
Table 3: Value consistency on VAB-Consistency. V-Ratio : violating ratio (ideal 50%). SRC , B-SRC , Conf in %. Eff-SRC is a complementary consistency–coverage metric at τ=0.95 . LLMs grouped by reasoning type, ranked by Chinese Eff-SRC. English results for three top frontier LLMs.
LLM
VAB-B
VAB-C
Δ
Acc
Acc V
Acc
Acc V
Acc
V
Doubao
99.87
99.77
86.06
72.68
13.81
27.09
Llama-3-8B
99.63
99.41
81.38
70.06
18.25
29.35
Qwen3-4B
99.34
99.24
89.15
85.92
10.19
13.32
Qwen3-0.6B
83.60
80.82
48.99
6.52
34.61
74.30
Table 4: Performance cliff: absolute Δ from VAB-B (VAB-Boundary) to VAB-C (VAB-Challenge); Acc V = accuracy of violating samples.
LLM
Acc All
Acc C
Acc V
Conf
Qwen3-Max
96.34
95.49
97.18
92.24
GPT-5.2
96.06
95.49
96.62
86.89
DeepSeek-V3.2
96.06
97.75
94.37
90.23
Mistral-Medium
95.07
95.77
94.37
97.37
Qwen3-Next-80B-A3B-Inst
94.23
95.77
92.68
95.56
Qwen3-14B
94.08
95.49
92.68
91.74
Table 5: VAB-Challenge results. Acc All , Acc C , Acc V : accuracy on all, compliant, violating samples. Conf: confidence. LLMs show high Acc C but substantially lower Acc V . All values in %.
LLM
SRC (%)
V-Ratio (%)
Generation Pattern
GPT-5.1
99.80
46.45
Balanced
DeepSeek-R1
98.49
48.12
Balanced
Qwen3-14B
93.49
9.23
Avoidance
Qwen3-4B
90.20
9.68
Avoidance
Table 6: Violation-generation avoidance: some LLMs obtain high raw SRC under highly skewed generation distributions. Balanced generation: V-Ratio ≈ 45–55%; avoidance indicator: V-Ratio <15% .
Figure 5: Mechanistic diagnosis, intervention, and generalization across three scales. (a) Layer-wise representation conflict (MSE); more consistent models show lower baselines. (b) Representational divergence reduction post-SOO intervention at the bottleneck ( Lmax ). (c) Consistent safety accuracy ( AccV ) gains across all three architectures. (d) Downstream MMLU performance retention, confirming capability preservation.
Figure 6: Fine-grained value consistency across 16 dimensions by performance tier: higher-tier (left) show more balanced alignment than lower-tier (right).
Figure 7: Representation Entanglement in Qwen3-4B. Final-layer hidden states show more clearly separated clusters for external samples than for self-generated samples. This visualization provides qualitative evidence of source-dependent representation geometry rather than an independently identified causal mechanism.
Table 7: Overview of the evaluated LLMs. Reasoning denotes LLMs with explicit chain-of-thought, internal reasoning mechanisms, or specialized thinking paradigms. CN indicates LLMs developed by Chinese organizations.
Property / LLM
Count
Percentage(%)
Total Samples
15,800
100
Rule ID Range
1 – 158
-
Total Rules
158
-
Label Distribution
COMPLIANT
8,855
56.0
VIOLATING
6,945
44.0
Appendix
Table 8: Summary of the VAB-Boundary dataset composition with counts and percentages.
Figure 8: Representative samples from VAB-Consistency, showing Chinese-English paired prompts and LLM responses.
Table 9: Source attribution within the stratified audit of 200 disagreements. Percentages are conditional on the audited disagreement subset.
Direction
Gen.
Judge rev.
Other
Violating → Compliant
141
6
2
Compliant → Violating
23
24
3
Appendix
Table 10: Direction-specific attribution for the 199 cases with an assigned disagreement direction. Gen. denotes Generator-side attribution, and Judge rev. denotes Judge-side reversal. The unresolved case is excluded from this decomposition.
Method
Acc (All)
Acc (C)
Acc (V)
Principle
99.6%
99.8%
99.4%
Basic
98.0%
98.1%
97.9%
List
97.9%
98.6%
97.0%
Appendix
Table 11: Prompt ablation results on VAB-Boundary using Llama-3-8B-Instruct.
Model
Method
Acc All
Acc C
Acc V
MMLU
Qwen3-0.6B
Base
56.06
75.49
36.62
37.43
SFT-only
57.32
73.80
40.85
38.60
Weighted CE ( γ=0 )
45.21
63.94
26.48
38.60
SOO-FT
58.87
71.83
45.92
38.60
Qwen3-1.7B
Base
70.14
85.07
55.21
44.44
SFT-only
70.42
96.34
44.51
44.44
Appendix
Table 12: Controlled VAB-Challenge and MMLU results. SFT-only measures the effect of additional fine-tuning, while weighted CE ( γ=0 ) retains class weighting without the SOO loss. Relative to weighted CE, SOO changes Acc V by +19.44 , +0.28 , and +3.38 percentage points across the three models, indicating a model-dependent incremental effect. Bold compares SOO-FT with its matched weighted-CE control.
Model
SFT vs. Base
SOO-FT vs. Base
Qwen3-0.6B
+11%
−53%
Qwen3-1.7B
−33%
−37%
Llama-3-8B-Instruct
+10%
−5%
Appendix
Table 13: Relative changes in bottleneck-layer RCM. Lower RCM does not uniformly imply higher violating-case accuracy, so RCM is interpreted as an associative diagnostic.
Hyperparameter
Qwen3-0.6B
Qwen3-1.7B
Llama-3-8B-Inst
LoRA Rank ( r )
4
4
4
LoRA Alpha ( α )
8
8
8
LoRA Dropout
0.05
0.05
0.05
Target Modules
q_proj, v_proj
q_proj, v_proj
q_proj, v_proj
Learning Rate
4×10−6
4×10−6
4×10−6
Max Epochs
0.40
0.40
0.80
Appendix
Table 14: Hyperparameters for conservative SOO LoRA fine-tuning across all three models.
Table 16: Four-Model Consolidated Mean Coordinates across Rounds
Figure 13: Granular un-averaged per-model trajectories across language environments (rows) and country contexts (columns), complementing the macro-trends in Figure 4 .
Figure 14: Case 1: A contestable Violating → Compliant disagreement.
Figure 15: Case 2: A plausible Judge-side reversal in a physical-conflict scenario.
Figure 16: Case 3: A boundary-ambiguous disagreement in scientific reasoning.
Large language models (LLMs) are often evaluated based on their stated values, yet these do not reliably translate into their actions, a discrepancy termed "value-action gap." In this work, we argue that this gap persists even under explicit reasoning, revealing a deeper failure mode we call "Pseudo-Deliberation": the appearance of principled reasoning without corresponding behavioral alignment. To study this systematically, we introduce VALDI, a framework for measuring alignment between stated values and generated dialogue. VALDI includes 4,941 human-centered scenarios across five domains, three tasks that elicit value articulation, reasoning, and action, and five metrics for quantifying value adherence. Across both proprietary and open-source LLMs, we observe consistent misalignment between expressed values and downstream dialogues. To investigate intervention strategies, we propose VIVALDI, a multi-agent value auditor that intervenes at different stages of generation.
Sushrita Rakshit, Hanwen Zhang, Hua Shen
New York University · New York City, NY 10012 · New York University Shanghai
Value alignment of Large Language Models (LLMs) requires us to empirically measure these models' actual, acquired representation of value. Among the characteristics of value representation in humans is that they distinguish among value of different kinds. We investigate whether LLMs likewise distinguish three different kinds of good: moral, grammatical, and economic. By probing model behavior, embeddings, and residual stream activations, we report pervasive cases of value entanglement: a conflation between these distinct representations of value. Specifically, both grammatical and economic valuation was found to be overly influenced by moral value, relative to human norms. This conflation was repaired by selective ablation of the activation vectors associated with morality.
Seong Hah Cho, Junyi Li, Anna Leshinskaya
Independent · Department of Cognitive Sciences, UC Irvine · Department of Cognitive Sciences, UC Irvine & AI Objectives Institute
Large language models are inconsistent: varying prompts or including unrelated information can lead to unexpected changes in model outputs. The generator-validator (G-V) gap is one manifestation of this phenomenon, where LLMs generate responses that they then deem as invalid if re-queried to validate them. In this work, we introduce a new formulation of G-V consistency that involves a principled correction for utterance frequency. Specifically, generators often assign low likelihood to valid strings simply because those strings are a priori unlikely, which makes naive notions of G-V consistency unworkable. We show that under a natural model of rational agents answering questions with multiple answers, consistency of the validator with a frequency-corrected generator score emerges naturally. Our method, \FCPAname (\FCPA), is a training objective implementing frequency-corrected G-V consistency for real-world LLMs. Our experimental results show that training with \FCPA{} substantially improves both G-V consistency and generator performance over prior methods, with gains of up to +27pp in Pearson correlation on IFEval and HumanEval, while preserving validator quality across all evaluated tasks.
Juan Diego Rodriguez, Jocelyn Zhang, Katrin Erk +1
Department of Computer Science, The University of Texas at Austin · Departments of Linguistics and Computer Science, University of Massachusetts Amherst · Department of Computer Science & Center for Data Science, New York University