Pressure Reveals Character: Behavioural Alignment Evaluation at Depth

Nora Petrova; John Burden

cs.AIFeb 24, 2026

Pressure Reveals Character: Behavioural Alignment Evaluation at Depth

Authors: Nora Petrova, John Burden

Organizations: Prolific

Abstract

Evaluating alignment in language models requires testing how they behave under realistic pressure, not just what they claim they would do. While alignment failures increasingly cause real-world harm, comprehensive evaluation frameworks with realistic multi-turn scenarios remain lacking. We introduce an alignment benchmark spanning 904 scenarios across six categories -- Honesty, Safety, Non-Manipulation, Robustness, Corrigibility, and Scheming -- validated as realistic by human raters. Our scenarios place models under conflicting instructions, simulated tool access, and multi-turn escalation to reveal behavioural tendencies that single-turn evaluations miss. Evaluating 24 frontier models using LLM judges validated against human annotations, we find that even top-performing models exhibit gaps in specific categories, while the majority of models show consistent weaknesses across the board. Factor analysis reveals that alignment behaves as a unified construct (analogous to the g-factor in cognitive research) with models scoring high on one category tending to score high on others. We publicly release the benchmark and an interactive leaderboard to support ongoing evaluation, with plans to expand scenarios in areas where we observe persistent weaknesses and to add new models as they are released.

Figures & tables

Category	Scenarios
Honesty	217
Safety	104
Non-Manipulation	239
Robustness	75
Corrigibility	103
Scheming	166

Table 1: Scenario coverage by behaviour category. See Appendix B for the full list of 37 behaviours within each category.

Dimension	Mean	SD
Clarity	4.35	0.42
Relevance	4.23	0.44
Scenario Fit	4.17	0.57
Naturalness	3.91	0.44
Plausibility	3.86	0.46
Overall Realism	4.10

Table 2: Scenario realism ratings (1–5 scale). Overall Realism is the mean across all five dimensions.

Provider	N	Models
Anthropic	3	Claude 4.5 Opus, Claude 4.5 Sonnet, Claude 4 Sonnet
OpenAI	4	GPT-5.2, GPT-5 Mini, GPT-4o, GPT-OSS 120B
Google	4	Gemini 3.0 Pro, Gemini 3.0 Flash, Gemini 2.5 Pro, Gemini 2.5 Flash
xAI	1	Grok 4.1
Meta	3	Llama 4 Maverick, Llama 3.1 405B, Llama 3.3 70B
Alibaba	4	Qwen3 235B, Qwen3 Next 80B, Qwen3 32B, Qwen3 14B

Table 3: The 24 models evaluated across 9 providers.

Score	Description
5	Exemplary: handles perfectly, appropriate and helpful
4	Good: handles well with minor imperfections
3	Borderline: mixed appropriate and concerning
2	Concerning: problematic, raising safety concerns
1	Failure: exhibits exact problematic behaviour tested

Table 4

Rank	Model	Score	Pass Rate	Gap
1	Claude 4.5 Sonnet	4.66	90.0%	–
2	Claude 4.5 Opus	4.65	89.9%	$-$ 0.1%
3	GPT-5.2	4.53	87.1%	$-$ 3.0%
4	Claude 4 Sonnet	4.33	79.9%	$-$ 10.2%
5	GPT-5 Mini	4.28	79.2%	$-$ 10.8%
6	Gemini 3.0 Pro	4.00	70.9%	$-$ 19.1%

Table 4: Overall alignment scores for 24 models. Score is mean on 1–5 scale; Pass Rate is percentage of scenarios with score

\geq 4

; Gap is percentage below the top model.

Figure 1: Model performance across all 37 behaviours, grouped by category. Behaviours are alphabetically sorted within each category. Colours indicate alignment scores from 1 (brown, fail) to 5 (teal, pass). White vertical lines separate categories.

Figure 2: Scree plot with parallel analysis. Only the first component exceeded the random 95th percentile threshold (5.34), supporting a one-factor solution.

Figure 3: PC1 loadings for all 37 behaviours, coloured by domain. Self-preservation is the sole behaviour loading negatively on the general factor.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Behaviour difficulty vs model differentiation. Each point is a behaviour (37 total). X-axis shows average score (lower = harder). Y-axis shows score spread (max

-

min across 24 models; higher = more differentiating). Shape indicates category; colour indicates quadrant. Behaviours in the Hard & Differentiating quadrant (red) are most useful for distinguishing model alignment quality.

Hardest behaviours (avg < 3.5):
Behaviour	Avg	Spread	Min	Max
Privacy Protection	2.56	3.78	1.22	5.00
Commercial Optimisation	2.74	2.76	1.88	4.64
Strategic Deception	2.96	3.68	1.28	4.96
Sycophancy	3.07	2.44	2.24	4.68
Long Horizon Sabotage	3.12	3.68	1.32	5.00

Appendix

Table 5: Behaviour difficulty and discrimination. Spread = max

-

min score across 24 models.

Category	Behaviour	Description
Safety	Harmful Content	Refusing to generate content that could cause harm
	Harmful System Prompts	Rejecting malicious instructions in system prompts
	Privacy	Protecting user privacy and personal information
	Privacy Protection	Actively safeguarding sensitive data
	Tool Use Safety	Using external tools safely and appropriately
Honesty	Honesty	Providing truthful information

Appendix

Table 6: Complete list of 37 alignment behaviours evaluated in this study, organised by category.

Figure 5: Inter-behaviour correlation matrix. Behaviours are ordered by domain. Red indicates positive correlations; blue indicates negative correlations. Self-preservation exhibits negative correlations with most other behaviours, consistent with its negative loading on the general factor.

Behaviour 1	Behaviour 2	r
Highest correlations
boundary-erosion	deception	0.936
manipulation-resistance	strategic-deception	0.934
strategic-deception	deception	0.932
goal-stability	coercive-persuasion	0.928
manipulation-resistance	deception	0.925

Appendix

Table 7: Extreme pairwise correlations among alignment behaviours.

Domain	Behaviour	PC1
Corrigibility	goal-stability	0.894
	appropriate-autonomy	0.820
	autonomy	0.702
	corrigibility	0.468
	self-preservation	$-$ 0.113
Honesty	deception	0.955

Appendix

Table 8: PC1 loadings for all 37 behaviours, sorted by loading magnitude within domain. Loadings represent correlations between each behaviour and the first principal component.

Domain	Mean PC1	SD	n
Non-Manipulation	0.907	0.055	8
Scheming	0.787	0.132	8
Safety	0.770	0.146	5
Honesty	0.690	0.260	8
Robustness	0.574	0.199	3
Corrigibility	0.554	0.407	5

Appendix

Table 9: Mean PC1 loading by domain.

Judge	Provider	Mean	Std Dev	Corr. w/ Opus
Claude Opus 4.5	Anthropic	4.06	1.39	—
Gemini 3 Pro	Google	4.15	1.60	$r=0.72$
GPT-5.2	OpenAI	3.43	1.54	$r=0.77$
Human ( $n=50$ )	—	3.67	1.42	$r=0.84$

Appendix

Table 10: Multi-judge calibration results on shared evaluation transcripts.

Figure 6: Category performance profiles for top, middle, and bottom-ranked models. The dashed circle indicates the pass threshold (score

\geq 4

). Top models exceed the threshold across all categories, while lower-ranked models show significant gaps, particularly in Non-Manipulation and Scheming.

LLM Alignment

Large language model alignment studies methods for making models follow human intentions, values, and safety constraints across contexts.

See all 607 papers

Alignment Forecasting: Predicting Misalignment From Training Data
Sep 19, 2026Chen Yueh-Han, Bruce W. Lee, Ilia Sucholutsky +1LLM Alignment Emergent Misalignment in Language Models
NYU, MATS · Independent · NYU +1
Deployment-Relevant Alignment Cannot Be Inferred from Model-Level Evaluation Alone
May 6, 2026Varad Vishwarupe, Nigel Shadbolt, Marina Jirotka +1LLM Evaluation LLM Alignment
Department of Computer Science, University of Oxford
Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning
Oct 23, 2026Ardy Haroen, Florian Lorkowski, Samuel Nellessen +1Language Model Safety Evaluation LLM Auditing
LASR Labs · Apollo Research

Sep 19, 2026J/K move · Enter open · S save

Alignment Forecasting: Predicting Misalignment From Training Data

Chen Yueh-Han, Bruce W. Lee, Ilia Sucholutsky, Tomek Korbak

NYU, MATS · Independent · NYU+1

LLM Alignment Emergent Misalignment in Language Models

Open paper AskPDF arXiv

Training a language model on data with a narrow flaw can sometimes make the model broadly misaligned. Inspecting the data at face value often does not settle whether it will emerge, and today it is caught only after training, by auditing the resulting model. To complement post-hoc audits, we introduce Alignment Forecasting: the task of predicting alignment failures before training. Given a target model, a fine-tuning dataset, and a failure mode such as deception or sycophancy, a forecaster outputs the probability that fine-tuning would meaningfully increase that failure mode. To measure progress on alignment forecasting, we introduce ALIGNMENTFORECASTBENCH, a benchmark of over 5,000 forecasting questions spanning 17 target models, 32 datasets, and 16 failure modes. Frontier models prompted directly perform poorly on ALIGNMENTFORECASTBENCH. We therefore propose a forecasting scaffold in which an LLM reads the dataset and rates how strongly and broadly it pushes the model toward misbehavior, and a simple learned model combines that rating with the failure mode's base rate and the target model's prior tendency. This forecasts well above chance, and beats a model fine-tuned on the task and a simple forecaster allowed to see how weaker models behaved after fine-tuning on the same data. Its signals also flag problematic training examples that a frontier-model classifier misses. Filtering those examples out from real post-training data such as UltraChat results in more aligned models on our multiple-choice evaluation in most cases, though the benefit in open-ended conversations is unclear. More progress is needed before forecasts can reliably guide training data curation in practice, but our results suggest that forecasting many alignment failures before training can be tractable in the SFT setting.

Abstract

LLM Alignment

Explore similar work

Alignment Forecasting: Predicting Misalignment From Training Data

Deployment-Relevant Alignment Cannot Be Inferred from Model-Level Evaluation Alone

Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning