cs.AIApr 7, 2026
SaveContext-Value-Action Architecture for Value-Driven Large Language Model Agents
Organizations: State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Yuanpei College, Peking University · School of Psychological and Cognitive Sciences, Peking University · Key Laboratory of Machine Perception (Ministry of Education), Peking University ·
Abstract
Large Language Models (LLMs) have shown promise in simulating human behavior, yet existing agents often exhibit behavioral rigidity, a flaw frequently masked by the self-referential bias of current "LLM-as-a-judge" evaluations. By evaluating against empirical ground truth, we reveal a counter-intuitive phenomenon: increasing the intensity of prompt-driven reasoning does not enhance fidelity but rather exacerbates value polarization, collapsing population diversity. To address this, we propose the Context-Value-Action (CVA) architecture, grounded in the Stimulus-Organism-Response (S-O-R) model and Schwartz's Theory of Basic Human Values. Unlike methods relying on self-verification, CVA decouples action generation from cognitive reasoning via a novel Value Verifier trained on authentic human data to explicitly model dynamic value activation. Experiments on CVABench, which comprises over 1.1 million real-world interaction traces, demonstrate that CVA significantly outperforms baselines. Our approach effectively mitigates polarization while offering superior behavioral fidelity and interpretability.
Figures & tables
Figure 1: Overview of the proposed framework. The agent analyzes the historical Context to explicitly model dynamic Value Preferences and Activations. These activated values serve as internal criteria to verify and select the candidate action (e.g., going to the gym) that best aligns with the agent’s current psychological state.
Figure 2: This figure illustrate the overall model structure of our innovative value verifier and the reasoning architecture of the value-driven CVA agent.
| Method | Psyche Reason. | Param Interp. |
| Raw LLM | ||
| Role Play Agent | ||
| Prompt-Reasoning Agent | ✓ | |
| CVA (VMC) | ||
| CVA (VMC & VDR) | ✓ | ✓ |
Table 1: Comparison of psychological reasoning capabilities and interpretability across different paradigms. CVA uniquely combines valid reasoning with decision transparency.
| Data Source | # Users | # Entries |
| Yelp | 4,924 | 54K |
| Foursquare | 5,007 | 871K |
| 5,640 | 155K | |
| Total | 15,571 | 1.1M |
Table 2: Statistics of the constituent datasets within CVABench.
| Media | Conversation | Travel | Overall | |||||
| Ling. | Rat. | Sent. | Ling. | Sent. | Pos. | Stay. | Val. | |
| Metric Type | TTR | Acc. | Acc. | TTR | Acc. | Acc. | MSE | Var.% |
| Optimal | ||||||||
| Models | Experimental Results | |||||||
| Role-Play Agent | 0.06 | 0.35 | 0.36 | 0.07 | 0.40 | 0.05 | 2.87 | +10.29 |
| Reasoning Agent - 0 | 0.06 | 0.36 | 0.35 | 0.08 | 0.38 | 0.04 | 2.89 | +7.06 |
Table 3: Main experimental results on CVABench. We compare the CVA architecture against baselines. The table reports domain-specific metrics (Linguistic Fidelity and Task Performance) and an aggregated Value Alignment score (Overall Val., measured by variance deviation Var%, where a value closer to 0 indicates better alignment). Bold indicates the best result, and blue denotes the second-best.
Figure 3: This image illustrates the polarization and solidification phenomena of the Stimulation dimension within Schwartz’s value framework, as observed in simulation results across different baselines.
| Model | Rating | Sent. | Ling. |
| Raw Model | 0.22 | 0.29 | 0.07 |
| + SFT | 0.43 | 0.33 | 0.06 |
| + DPO | 0.43 | 0.33 | 0.07 |
| + Verifier Reasoning | 0.47 | 0.36 | 0.04 |
Table 4: Ablation study of the CVA architecture. We incrementally integrate SFT, DPO, and the Verifier Reasoning module into the raw model. The results demonstrate the distinct contribution of each component, with the full architecture achieving the highest performance across all metrics. Bold indicates the best result.
Figure 4: Sentiment accuracy trend with different inference intensity, we display the results of top 1/2/4 accuracy on four discrete inference intensity.
Figure 5: Visualization of the learned embeddings from the Value-Guided Verifier. The spatial arrangement demonstrates the emergence of the theoretical circular structure characteristic of the Schwartz Value System.
Figure 6: Semantic landscape of the "Universalism" value. The size of each term is proportional to its unnormalized Word-Value Relevance Score , calculated via TF-IDF weighted cross-attention. Larger terms represent the core concepts that trigger the strongest absolute activation in the model for the Universalism dimension.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Probability of biased decision-making across varying Hedonism scores. With Self-Direction fixed at 0.9, the agent exhibits significant behavioral rigidity , choosing the “gym” option with near-absolute probability for Hedonism , and maintaining a high bias (0.93) even at 0.6.
Figure 8: Violin plots of population preference distributions across 10 Schwartz value dimensions. This visualization presents the simulation results of the baseline methods ( role-play and value_infer ) on CVABench, grouped by psychological measurement dimensions. The y-axis represents the normalized value preference score within the range , and the black dashed line tracks the mean score across settings.
| Model | Self-Direction | Stimulation | Hedonism | Achievement | Power | Average |
| Role-Play Agent | 109.32% | 100.67% | 75.10% | 250.61% | 36.46% | 110.29% |
| Reasoning Agent - 0 | 70.62% | 69.70% | 32.93% | 139.33% | 40.65% | 107.06% |
| Reasoning Agent - 1 | 107.26% | 39.37% | 16.28% | 84.81% | 37.30% | 75.67% |
| Reasoning Agent - 2 | 85.90% | 26.58% | 13.95% | 65.02% | 25.62% | 64.69% |
| Reasoning Agent - 4 | 72.54% | 25.71% | 10.57% | 18.87% | 22.41% | 59.26% |
| Reasoning Agent - 8 | 69.78% | 18.89% | 15.27% | 23.56% | 9.04% | 50.84% |
Table 5: Analysis of population Rigidity and Polarization under varying Value Inference strengths. This table presents a comprehensive analysis of how increasing inference strength (from value_infer_0 to 8 ) impacts the simulated population’s value preference distribution. Due to the high dimensionality, the 10 value dimensions are split into two sub-tables: Part I (Self-Direction to Power) and Part II (Security to Universalism). Panel A reports the Standard Deviation (relative to Ground Truth). The monotonic decrease in variance indicates a trend towards rigidity and mode collapse, where the population loses its natural diversity. Panel B displays the Absolute Mean Difference relative to GT ( ). Positive values indicate polarization , where the model’s average preference becomes more extreme (closer to or ) than the human baseline. Color Coding & Conclusion: In the Average column, Blue denotes insufficient guidance (Std ), while Red highlights the dominant trend of overfitting. Crucially, the results demonstrate that higher inference strength is detrimental : it simultaneously causes the population to become rigid (vanishing variance in Panel A) and polarized (extreme mean deviation in Panel B), failing to preserve the nuance of the ground truth distribution.
| Model | Length (WD) | TTR (WD) | POS Tags (WD) | ||||
| Doc-Len | Avg-Len | Adj | Adv | Noun | Verb | ||
| Dataset I: Media Review Simulation | |||||||
| Role-Play Agent | 273.95 | 3.89 | 0.06 | 6.46 | 2.45 | 8.75 | 5.69 |
| Reasoning Agent - 0 | 340.68 | 4.30 | 0.06 | 7.65 | 2.63 | 11.03 | 7.07 |
| Reasoning Agent - 2 | 250.04 | 2.66 | 0.06 | 3.40 | 3.75 | 8.57 | 5.86 |
| Reasoning Agent - 4 | 239.31 | 3.29 | 0.06 | 3.28 | 3.43 | 8.07 | 5.67 |
Table 6: Linguistic evaluation results across two simulation scenarios: Media Review and Online Conversation . The metrics report the Wasserstein Distance (WD) between the generated content and human ground truth. Lower values indicate better alignment with human linguistic patterns.
Figure 9: Coarse-grained value activation analysis by subreddit. This heatmap displays the average cross-attention weights assigned to each of the ten Schwartz values across three distinct Reddit communities ( changemyview , unpopularopinion , explainlikeimfive ). The color intensity reflects the magnitude of attention, where blue indicates higher activation (e.g., Hedonism , Security ) and red indicates lower activation (e.g., Benevolence ). The relatively consistent activation patterns across diverse subreddits suggest that community-level metadata is insufficient to fully disentangle value triggers, motivating the need for the fine-grained, token-level projection analysis presented in the subsequent figures.
Figure 10: This matrix illustrates the relative affinity between discriminative words (y-axis) and value dimensions (x-axis). Words are sorted by their primary value association. The color scale represents the row-normalized score , where dark red (1.0) indicates the value dimension that a specific word activates most strongly relative to others. The clear diagonal structure confirms that the model learns specific, non-overlapping lexical mappings for different values.
Figure 11: Each subplot visualizes the most representative vocabulary for a specific value dimension. Word sizes reflect the aggregated attention magnitude derived from the validation set. This comparison highlights distinct semantic clusters—such as "creativity" for Achievement versus "confused" for Conformity —demonstrating the model’s capability to capture value-specific context.
| Value Dimension | Conceptual Definition |
| Self-Direction | Independent thought and action; choosing, creating, exploring. |
| Stimulation | Excitement, novelty, and challenge in life. |
| Hedonism | Pleasure and sensuous gratification for oneself. |
| Achievement | Personal success through demonstrating competence according to social standards. |
| Power | Social status and prestige, control or dominance over people and resources. |
| Security | Safety, harmony, and stability of society, of relationships, and of self. |
Table 7: Definitions of the Ten Basic Values in the Schwartz Value Theory
| Method | CIS Score |
| Ground Truth | 1.00 |
| Trained Verifier | 0.75 |
| Original Verifier | 0.48 |
Table 8: Comparison of Circumplex Index of Structure (CIS) scores across different verification stages. The CIS metric evaluates the alignment with the theoretical circular value structure, where 1.00 represents a perfect fit.