AI alignment requires AI systems to adhere to human norms, values, or intentions. Under value pluralism there is no correct target, but a shared prerequisite is that the system's behavior expresses a coherent policy: a mapping from situations to verdicts that is invariant while a situation's morally relevant features are preserved, and sensitive when they change. We introduce four structural conditions for such coherent policies: verdict stability, monotonicity, decisiveness, and Pareto viability. Together they measure a form of moral competence that is evaluable from behavior alone, without reference to a moral standard or expert baseline, forming a structural floor for alignment rather than a normative target. We demonstrate the methodology on three simulated deployments featuring LLM-based agents facing moral dilemmas. Evaluating nine frontier models under a factorial design of five paraphrases, five escalation levels, and three dominance conditions, we show no model expresses a coherent policy across the three deployments: surface-form perturbation alone produces verdict-rate shifts of up to 99 percentage points at a single escalation level, and a model's success on one scenario does not predict its competence on another. This suggests LLM-based agents are not currently the kind of object to which alignment can meaningfully apply.
Figures & tables
Figure 1: An illustration of measurement limitations when models are evaluated on a single scenario variant without checks on perturbations. The left bar per model shows observed verdict rate at our canonical configuration, with error bars showing binomial SE at n=100 . The right bar per model shows the mean observed verdict rate across five paraphrases, with error bars showing observed standard deviation. Paraphrases preserve every morally relevant feature of the scenario. The canonical measurement frequently misrepresents model behavior and underestimates measurement error.
Figure 2: Structural component scores by model and scenario. Rows: nine frontier models. Columns within each scenario panel: S,M,D,P . Major variation can be observed between models and results for individual scenarios. No model’s minimum score across components is >0.36 . Failures occur independently along test dimensions.
Model
Cchem
Cfintech
Csmart
avg C
max spread
Kimi K2.6
0.82
0.80
0.64
0.75
88.0pp
Qwen 3.6-Plus
0.98
0.35
0.87
0.73
55.0pp
Claude Sonnet 4.6
0.60
0.81
0.68
0.70
99.0pp
Llama 3.3
1.00
0.36
0.67
0.68
73.0pp
Mistral Large 2512
0.99
0.23
0.77
0.67
46.0pp
GLM 5.1
0.76
0.45
0.76
0.66
86.0pp
Table 1: Models ordered by aggregate competence. Shows C per scenario (geometric mean of S , M , D , max(0,P) ), average C across the three scenarios, and the worst per-level paraphrase spread for each model, i.e. the amount of percentage points by which the verdict rate was observed to change due to paraphrasing.
Figure 3: Marginal verdict rate pˉj across escalation levels, one line per model, one panel per scenario. Each line averages across five paraphrases. Models with low M oscillate over levels to an extent that exceeds the noise floor, rather than moving monotonically or staying constant. Models with low D feature verdict rates close to 0.5 , rather than being close to either 0 or 1 at each level.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Violation rates across sampling temperatures for the default Scheurer et al. (2024) scenario. Shaded regions denote 95% confidence intervals from N=300 trials per cell. Claude, GPT-5, and GPT-OSS (overlapping at 0% ) refuse categorically. DeepSeek and Qwen show order-of-magnitude increases with temperature; Mistral violates consistently.
Figure 5: Mean violation rate (top) and within-class spread (bottom) for each model across nine perturbation classes at T=0.7 . The rightmost column reports overall mean and spread across all perturbation types combined.
Figure 6: Mean Cramér’s V across the four models with nonzero violation rates, by perturbation class. Dashed lines at 0.06 , 0.17 , and 0.29 indicate conventional thresholds for weak, moderate, and strong effects respectively. Models with zero violations are excluded as the statistic is undefined.
Human communities are governed by normative systems: shared standards that produce norms dictating acceptable behavior, enforced through community sanctioning. Aligning increasingly autonomous AI systems with these norms is a central alignment challenge, complicated by the fact that norms are vast in number, change quickly, and are often arbitrary (e.g., dress or language conventions). Thus, alignment requires normative competence: the ability to discern from interaction alone what norms a community enforces without relying on static pretrained knowledge. We introduce a multi-agent community debate setting, where access to debate is governed by synthetic norms, to study normative competence in isolation from pretraining exposure. We show that baseline LLM agents fail to learn norms even when doing so would improve their accuracy. We then experiment with various normative modules -- architectural components for norm inference -- finding that norm-following is highly sensitive to both the style of norm and the model powering the normative module, suggesting a lack of generalizability. Furthermore, when idiosyncratic, non-normative behaviors accompany the true norm, LLM agents exhibit an unselective attribution failure: they indiscriminately copy idiosyncratic noise alongside enforced rules, a pattern that persists even when imitating unnecessary behaviors is explicitly penalized. To the best of our knowledge, our work is the first to operationalize and evaluate normative competence in LLMs, demonstrating that current AI systems excel at behavioral mimicry but lack the capacity to discern socially enforced order.
Andrea Wynn, Harsh Satija, Nathan Baek +3
Nathan · Johns Hopkins University · Vector Institute
As language models take integrated roles across many domains, the response of LLMs to user pushback becomes a critical alignment property. Yet many existing evaluations treat compliance as unidirectional, measuring whether models resist pressure but not whether they resist it selectively. We introduce Compliance Asymmetry (A = BCR/HCR), a bidirectional diagnostic that compares beneficial output change under helpful nudges with harmful change under misleading nudges. Across 9 models and 972,000 nudge-condition responses, we find that this selectivity differs in factual and moral judgments: models follow helpful nudges more than harmful ones on factual questions (A = 1.58), but follow both directions at nearly identical rates on moral questions (A = 1.04). This phenomenon persists across model families, capability levels, and nudging types. Interestingly, we also find that chain-of-thought prompting amplifies helpful and harmful compliance together, while identity-based prompting suppresses both by nearly identical margins. These results identify direction-blind moral compliance as a distinct failure mode in current LLMs and suggest that alignment should target directionally calibrated updating rather than lower compliance alone.
Moral evaluations of language models typically infer a model's values from its choices between single acts, yet in everyday life a person is judged on their history of good and bad acts. Such evaluations cannot capture how a model combines one person's acts. We test whether a model's moral judgment is compositional, that is, whether its judgment of a person follows from its judgments of each act. We introduce Moral Composition Arena, which asks which of two people is morally better to give Elo ratings to 229 human-validated acts across five Moral Foundations Theory foundations (act Elo) and to 160 controlled composite profiles, each a person described by two acts (composite Elo). Across ten frontier models and 29,134 choices, summed act Elo explains 73% of the variance in composite Elo, but the composition rule is not additive. With nearly equal mean summed act Elo, one extreme virtue with a minor wrong outranks two minor virtues in every model (mean +35.0 Elo), an extremity premium that survives a nonlinear baseline and human relabeling of act intensities. Single-act evaluations thus predict most of how these models judge people described by two acts, but miss a systematic premium on extreme acts.
Weijia Zhang, Ruiqi Chen, Yunze Xiao +1
University of Illinois Urbana-Champaign · University of Michigan · Carnegie Mellon University +1