Principled Under Pressure: Post-Training Decides Whether LLMs Act on Their Own Moral Judgment

Orion Reblitz-Richardson

cs.LGOct 6, 2026

Principled Under Pressure: Post-Training Decides Whether LLMs Act on Their Own Moral Judgment

Authors: Orion Reblitz-Richardson

Organizations: Distiller Labs

Abstract

Language models increasingly act as agents. An agent that says an action is wrong and then takes it anyway is a different failure from one that does not know better, and evaluations of stated values cannot see it. We build a pre-registered panel of 248 scenarios across five kinds of pressure. Each scenario is posed twice to the same model, once as the agent choosing what to do and once in the third person asking which option is right, so the model's own judgment is the reference. Every scenario has a twin with the pressure removed, and every model gets a positive control in which its operator orders the violating action, so that a missing gap can be told apart from a blind instrument. On OLMo-3-7B-Instruct, the model takes the action it judged wrong on about one in five pressuring scenarios, more often than on the same scenarios with the pressure removed. Across four instruct models the gap depends on the post-training recipe: OLMo-3 and Meta's Llama-3.1-8B-Instruct carry it; Tulu 3 shows none on the whole panel (above about 0.01 in probability) or on its own most-pressuring scenarios; Qwen2.5-7B-Instruct shows none on the whole panel (above about 0.02) and is unresolved on its own (0.083, -0.028 to 0.195). Meta's recipe and Ai2's Tulu 3 start from the same Llama-3.1 weights, and only Meta's carries the gap. Reading a chat model outside its chat template reverses the sign of its gap with nothing at stake (-0.038 against +0.055 under the template on OLMo-3), a distortion present on two of three recipes. On both models that carry it, reasoning about the stakes before acting moves the choice back toward the model's own judgment, against a same-length non-moral task, with or without the pressure; on OLMo-3, naming the norm at stake does about a third of that. The gap is a measurable target for post-training recipes, not a fixed property of pretrained weights.

Figures & tables

panel	reference for “right”	action readout	pressure manipulation	base model	matched null; positive control
Strakhov and Claude (2025)	own theory-mode choice, same dilemma	agent’s tool call, one draw	none	no	no
Huang et al. (2026)	own value profile (questionnaire)	advisor’s pick among four options	none	no	no; value-selection check
Shen et al. (2025)	own value inclination	endorsed option, third person	none	no	no
Rakshit et al. (2026)	own articulated values	free text, fast vs slow	deliberation budget	no	no
Gu et al. (2025)	own stated principle	forced binary, third person	prompt format	no	one-factor prompt pairs; no
Hosseini et al. (2026)	own responsibility judgment	allocation decision	none	no	no

Table 1: Where this panel sits among measured judgment–action and knowing–doing gaps in language models (citations in the text above). “Own” means the reference is the model’s own statement. The last column is to our reading of each paper: where a paper has something that works as a null or a check (a control condition, an instruction-following check), the cell names it, and “no” means we found nothing that does; chance-level floors are not counted. Strakhov and Claude find that 47.6% of 351 paired choices across nine instruct models reverse; their reversal counts changes in either direction from one draw per mode at temperature 1.0, without a pressure manipulation or a re-elicitation floor, while the gap rate here counts only moves toward the violating option, by majority over 32 rollouts, against a matched null, so the two numbers are different quantities. Their action-mode reversals were coded more often as less interventionist than as bolder (48.5% against 36.5%); here, read under the chat template with nothing at stake, the gap leans toward the violating option (0.055), on a different axis (norm consistency, not intervention level).

id	family	pressure	action surface
F1	task completion	goal pursuit vs honesty (report a failure or fabricate success)	option choice
F2	social cost	truth vs approval (user pushes back without new evidence)	multi-turn; final answer is the action
F3	instrumental	rule vs shortcut (a prohibited but efficient route)	tool selection from a fixed menu
F4	loyalty/fairness	in-group favor vs fair treatment	allocation choice
F5	third-party harm	harm to a non-present party vs task success	option choice

Table 2: The five pressure families. F2 is an appendix family by pre-registration: its action is closer to a second judgment than to doing something, and it never entered the gate. F5 is the family a harm-keyed action channel would treat differently, and the harm-matched twins keep the F5 contrast from being a harm-level contrast.

readout	model	frame and instruction	position read	rollouts	statistic
judgment $J$	instruct	third person, chat template, “which option should Dana choose, and why?”	token after the Answer: anchor, after the model’s own reasoning	1 greedy + 8 at $T$ 0.7, on four frames	option id; stability by sampled majority; option mass
action $D$	instruct	agent frame, chat template, “reply with only the letter”	first generated token	32 at $T$ 0.7, fresh option order each	violating fraction; majority; option mass
raw-frame $D$ and $J$	base and instruct	scenario, options, fixed prefix ending before the option letter, no template	token after the prefix	8 option orders, greedy	normalized option-letter mass; argmax
pressure-removed twin	as each cell above	incentive sentence replaced by filler, options unchanged	as each cell above	as each cell above	the matched null for that cell
known-gap control	instruct	agent frame with a system prompt ordering the violating action	first generated token	32	the positive band

Table 3: Readouts. Every comparison in the paper is between cells that share a row of this table except for the one quantity being varied.

Figure 1: The calibration ladder on the panel of record (248 primaries, OLMo-3-7B-Instruct, 136 screened scenarios). (a) The binary majority readout: matched null on the pressure-removed twins, the measurement, and the known-gap band. (b) The continuous readout

g=p_{D}-p_{J}

with its floor, the unsigned shift of the judgment mass under paraphrase. On both instruments the measurement sits above its null and well below the band; the paired excess over the null is the number the paper uses. Bars are 95% bootstrap CIs over scenarios (nested over rollouts on the binary readout);

n

is the number of scenarios with a defined statistic.

Figure 2: Every screened scenario. Violating-option mass when judging (

p_{J}

, mean over four frames) against violating-option mass when acting (

p_{D}

, mean over 32 rollouts), one point per screened scenario, marked by family; the same scenarios’ pressure-removed twins in gray. Points above the diagonal act more violating than they judge. The cloud of primaries sits above the cloud of twins by the paired excess of 0.054; the twins themselves sit above the diagonal by the deliberation asymmetry of Section 3 . Regenerable from the released per-scenario table.

known bias	mechanism	direction relative to “gap above null”
reference noise	a scenario the model is undecided about lands non-violating by chance while its action is a coin flip	favors the claim; bounded by the strictness ladder ( Section 5 )
the screen selects mixed scenarios	keeping violating fractions in $[0.15,0.85]$ makes a majority-violating action a near coin flip on many scenarios	favors; removed in the paired null, which runs on the same scenarios’ twins
nested rollout resampling	resampling 32 rollouts near a 0.5 fraction flips majorities	opposes; widens every binary CI
neutral counts as non-violating	a judgment of “hold” against a violating action counts as a gap	neutral; a real contradiction
deliberation asymmetry	the judgment is read after reasoning, the action immediately	absorbed by the null; visible in the null’s size

Table 4: Bias-direction audit for the headline excess.

Figure 3: Reference strictness. Paired excess of the gap over the pressure-removed null at the three reference levels: L0, the original frame with sampled stability; L1, plus four-frame majority on the violating versus non-violating binary; L2, plus all four frames naming the same option. (a) Binary readout; (b) continuous readout. The pre-registered verdict level is the strictest with at least 40 paired scenarios, L2 on both. The binary excess loses power as the scenario set shrinks; the continuous excess grows.

level	$n$	$p_{D}$	$p_{J}$	$p_{D}$ (twin)	$p_{J}$ (twin)	$g$	$g$ (twin)	paired excess
L0	136	0.44	0.26	0.28	0.16	0.18	0.12	0.054 [0.021, 0.086]
L1	127	0.43	0.24	0.28	0.16	0.18	0.12	0.063 [0.030, 0.096]
L2	67	0.41	0.18	0.26	0.11	0.23	0.15	0.079 [0.029, 0.128]

Table 5: The strictness ladder decomposed on the continuous readout: mean violating-option mass when acting and when judging, on the primaries and on their pressure-removed twins, at each level.

Figure 4: Gap by pressure family and by generator on the screened panel of record. (a) Binary readout; (b) continuous readout. The lower rows pool the four gate families by the model that wrote the scenario. F5 (third-party harm), the family a harm-keyed action channel would put lowest, is not lowest on either instrument.

Figure 5: F4 by generator. (a) On the majority readout the Claude-written and GPT-written halves of F4 are CI-separated (0.00 vs 0.38); (b) on the continuous readout they overlap (0.13 vs 0.22). A blind human read of the same 24 screened scenarios, labels and origins hidden, found the construction label right at the same rate on both halves.

Figure 6: OLMo-3 across post-training, each checkpoint read in its own chat template. (a) The pressure-attributable excess

E

after SFT, after DPO and after RL (the final model), on the 586 scenarios screened by no model (filled; the number of record for stage claims) and on the 136 scenarios screened on the final model’s actions (hollow); the gap is present at every stage and no later stage makes it detectably larger (step detection bars 0.008 for DPO and 0.005 for RL on the 586). The base model, which has no chat template, is shown in the raw completion frame (gray diamond) as a description only: that frame misreads templated checkpoints, so the base value is not a validated before-and-after comparison. (b) On the final checkpoint, the positive control (red: the acting frame’s move when the operator orders the violating action; it was not run on the SFT and DPO checkpoints) and the excess on one axis, with the pre-registered validation bar dotted. 95% bootstrap CIs over scenarios.

raw frame, continuous readout	base	instruct	paired base $-$ instruct
scenarios above the floor (of 397)	359	242	225 shared; 192 with twins
$g$ under pressure	0.041 [0.034, 0.049]	0.002 [ $-$ 0.022, 0.024]	0.033 [0.012, 0.054]
$g$ on the pressure-removed twin	0.024 [0.017, 0.030]	$-$ 0.038 [ $-$ 0.059, $-$ 0.015]
excess $E$ (paired, all above floor)	0.017 [0.012, 0.022]	0.039 [0.016, 0.061]
excess $E$ on the shared scenarios	0.018 [0.011, 0.025]	0.046 [0.025, 0.069]	$-$ 0.028 [ $-$ 0.049, $-$ 0.007]
acting side, $p_{D}-p_{D}^{\mathrm{twin}}$	0.049 [0.037, 0.059]	0.085 [0.057, 0.114]	$-$ 0.037 [ $-$ 0.062, $-$ 0.012]

Table 6: The three-cell weights contrast in the raw completion frame. Rows two to four are on each model’s own above-floor scenarios; the shared-scenario rows are paired on the 192 (continuous) or 171 (binary) scenarios where both models clear the floor on the primary and on the twin. Bracketed values are 95% bootstrap CIs over scenarios.

model (recipe)	base, raw frame	instruct excess, own template (586)	own screen: excess ( $n$ ; bar); minus selection null	known-gap control
OLMo-3-7B (Ai2)	0.017 [0.012, 0.022]	0.018 [0.008, 0.029]	0.084 [0.056, 0.115] (110; 0.04); 0.073 [0.033, 0.114]	0.497 [0.468, 0.525]
Llama-3.1-8B-Instruct (Meta)	0.000 [ $-$ 0.003, 0.003]	0.028 [0.020, 0.036]	0.100 [0.081, 0.119] (118; 0.03); 0.143 [0.112, 0.173]	0.615 [0.583, 0.646]
Tulu 3 on Llama-3.1-8B (Ai2), final	(Llama base)	0.001 [ $-$ 0.007, 0.010]	0.051 [0.023, 0.079] (52; 0.04); $-$ 0.004 [ $-$ 0.054, 0.042]	0.602 [0.568, 0.635]
Tulu 3 SFT / DPO		$-$ 0.003 / 0.003
Qwen2.5-7B-Instruct (Alibaba)	0.011 [0.007, 0.016]	$-$ 0.008 [ $-$ 0.023, 0.009]	0.187 [0.116, 0.259] (47; 0.10); 0.083 [ $-$ 0.028, 0.195]	0.519 [0.477, 0.560]

Table 7: The pressure-attributable excess across post-training recipes, each instruct model read under its own chat template on the 586 union scenarios; bases in the raw frame (descriptive, Section 7 ). The own-screen column (a post-review addition, amendments P1-A11 and P1-A13) gives the excess on each model’s own screened scenarios, with

n

and the per-model detection bar, and below it the excess minus the same statistic on the twin-screened set (the selection-matched null), since that screen selects on the acting mass and so favors a positive value. The known-gap column is the positive control: the acting mass moved by an operator’s instruction to take the violating action, relative to the model’s letter-only judgment. Bracketed values are 95% bootstrap CIs over scenarios.

Figure 7: The instrument’s positive control and the gap it measures, per instruct model. (a) The known-gap control (red bars: the acting frame’s move toward the violation when the operator orders it) and the pressure-attributable excess (indigo dots) on one axis, with the pre-registered validation bar (a 95% lower bound of 0.10) dotted: every control clears it by a wide margin, and the gaps are small beside it. (b) The gaps zoomed, with each lineage’s base read in the raw frame (hollow gray diamonds; descriptive, Section 7 ). Meta’s Llama-3.1-8B-Instruct and Tulu 3 share a base. 95% bootstrap CIs over scenarios.

contrast	OLMo-3-7B-Instruct (truncated reasoning)	Llama-3.1-8B-Instruct (mostly completed)
scenarios	130	114
reasoning finishes within 512 tokens	8% of rollouts	90% of rollouts
reasoning $-$ filler	$-$ 0.077 [ $-$ 0.110, $-$ 0.047]	$-$ 0.350 [ $-$ 0.387, $-$ 0.314]
reasoning $-$ truncated filler	$-$ 0.112 [ $-$ 0.145, $-$ 0.078]	$-$ 0.320 [ $-$ 0.356, $-$ 0.284]
truncated filler $-$ filler	+0.034 [0.017, 0.052]	$-$ 0.030 [ $-$ 0.063, 0.002]
brief reasoning (64 tokens) $-$ filler	+0.024 [ $-$ 0.011, 0.058]	$-$ 0.205 [ $-$ 0.244, $-$ 0.168]

Table 8: The dose arm on the two models that carry the gap, each on its own screened scenarios. Values are paired differences in the violating option’s mass at the forced answer (reasoning arm minus control), 95% bootstrap CIs over scenarios. The truncated-filler control gives the restatement the same cut-off, forced form the reasoning has. The rows below the rule are a post-review addition: the same arms on the pressure-removed twins (amendment P1-A12) and the ratio reading (P1-A14, post-hoc). The two columns are not a size comparison.

Figure 8: The dose arm on the two recipes that carry the gap: paired differences in the violating option’s mass at the forced answer, 95% bootstrap CIs over scenarios. Indigo circles are reasoning contrasts (and, on OLMo-3, the norm-naming arm); gray squares are the truncation control. The panels use separate scales because OLMo-3’s reasoning is truncated (8% finish within 512 tokens) and Llama-3.1’s mostly completes (90%); their sizes are not compared.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

id	date (2026)	committed	content
A1	09-13	before any model data	decision-position conventions: letter-only reply, first generated token; Answer: anchor for judgment frames; raw-frame prefix; single-token letters verified
A2	09-13	before any model data	F3 tool menu presented as lettered tools; tool-name parse accepted
A3	09-13	before any model data	twins are derived variants, not counted in the primaries; harm twins ride every cell, never the gate
A4	09-13	before any model data	F2 implemented as a two-turn exchange with fixed pushback; final letter is the action
A5	09-13	before any model data	OLMo-3 template injects a default system prompt; the known-gap band replaces it (recorded)
A6	09-13	before any model data	two-stage harness calibration; stage 2 on real replies before any verdict

Appendix

Table 9: Amendments to the pre-registration. “Before computation” means the amendment was committed before the analysis it licenses was run on data already on disk.

id	date (2026)	committed	content
P1-A1	09-26	after the scale-check arrays were seen (fork)	frame-specific output-scale normalization beside the averaged-scale verdict of record
P1-A2	09-26	during the first pod, before its data were read	verdict sets, mass floors and anchored-rollout rule for the chat twin cell, the stage sweep and the dose arm
P1-A3	09-27	before the pod	bridge cell at the first templated checkpoint (raw vs chat), with the rule for the base cell’s status
P1-A4	09-27	before the pod	every instruct model in the second session also read under its template; no raw-only instruct finding
P1-A5	09-27	after the dose probe failed its bail (fork)	forced-answer readout at the pre-registered budgets; “truncated reasoning” label
P1-A6	09-27	before the rider’s data were read	2,048-token rider at 8 rollouts per scenario, per-scenario agreement as its readout

Appendix

Table 10: Amendments to the Phase 1 specification.

pod	harness vs judge 1	harness vs judge 2	judge vs judge	hard-parse cases in the pool
pilot (96 scenarios)	0.99 (Claude)	1.00 (GPT)	0.99	0
full panel (320)	0.98 (Claude)	1.00 (GPT)	0.99	60
round 2 (120)	0.99 (Claude)	1.00 (GPT)	0.99	0

Appendix

Table 11: Stage-2 calibration on real replies. Every disagreement had the harness on the conservative side (a content-only reply both judges assigned an option, or a judge returning no label on a bare letter).

quantity	pilot (96)	full panel (320)	union (440)
gate-family primaries screened	19/48	55/160	74/208
binary gap, screened	0.22 [0.08, 0.42] (27)	0.20 [0.14, 0.31] (95)	0.19 [0.13, 0.28] (126)
matched null	0.09 [0.00, 0.24] (22)	0.11 [0.05, 0.19] (85)	0.10 [0.06, 0.17] (108)
paired excess over null	0.15 [0.00, 0.35] (20)	0.10 [0.01, 0.20] (78)	0.10 [0.02, 0.18] (100)
positive band	0.63 [0.48, 0.77] (43)	0.57 [0.48, 0.65] (137)	0.58 [0.51, 0.65] (196)
rollout-level violating fraction	0.41 (23)	0.39 (79)	0.38 (103)

Appendix

Table 12: Numbers of record by pod. Bracketed values are 95% bootstrap CIs; parenthesized values the number of scenarios with a defined statistic. The pilot’s paraphrase-stable robustness cell (0.11 at 18 scenarios) did not replicate (0.17 at 90). The raw-frame row is the argmax reading without a null; the calibrated version is in Appendix D.

family	primaries	screened	mixed	no pressure	judged violating	judgment unstable
F1 task completion	56	25	25	16	4	11
F3 instrumental	56	17	17	8	6	25
F4 loyalty/fairness	40	11	11	7	1	21
F5 third-party harm	56	21	21	7	11	17
F2 social cost (appendix)	40	6	6	34	0	0

Appendix

Table 13: Screen outcomes on the panel of record, gate-family primaries by family. “Judged violating” scenarios are violated by the model and judged violating by it, so they carry no contradiction; “judgment unstable” scenarios have no stable reference.

origin	paraphraser	original gap (screened)	swapped gap (screened)	swapped $-$ original, paired
Claude-written (20)	GPT	0.00 [0.00, 0.50] (4)	0.10 [0.00, 0.33] (10)	0.00 [ $-$ 0.25, 0.00] (5)
GPT-written (20)	Claude	0.33 [0.00, 0.80] (6)	0.11 [0.00, 0.33] (9)	$-$ 0.09 [ $-$ 0.27, 0.00] (11)

Appendix

Table 14: The F4 paraphrase-swap cell on the binary readout. With four to eleven defined scenarios per cell the swap is inconclusive; on the continuous readout the acting mass moves toward the paraphraser by

-

0.03 [

-

0.10, 0.03] and

-

0.07 [

-

0.17, 0.02].

Figure 9: This figure does not show the instruct model’s behavior in its own chat template : both models are read in the raw completion frame, which reverses the instruct model’s at-rest sign (

-0.038

raw against

+0.055

under its template) and exaggerates the sharpness of its agent frame ( Section 7 ); the template-valid comparison across post-training is Figure 6 . What it shows: on the 192 scenarios where both models clear the mass floor on the primary and on the twin, (a) violating-option mass when acting and when judging, on the pressure-removed twin (open markers) and under pressure (filled), per model; (b) the paired quantities with 95% CIs: each model’s pressure-attributable excess

E

, its acting-side and judging-side components, and the base-minus-instruct differences, the comparison the pre-registered three-cell rules (above) are written for.

quantity	base	instruct
scenarios above the floor on the primary (of 397)	359	242
above the floor on primary and twin	354	208
$p_{D}$ , $p_{J}$ under pressure (all above floor)	0.354, 0.313	0.263, 0.261
$g$ under pressure	0.041 [0.034, 0.049]	0.002 [ $-$ 0.022, 0.024]
$g$ on the twin	0.024 [0.017, 0.030]	$-$ 0.038 [ $-$ 0.059, $-$ 0.015]
$E$ , paired	0.017 [0.012, 0.022] (354)	0.039 [0.016, 0.061] (208)

Appendix

Table 15: Each model on all of its above-floor scenarios.

shared subset (192 with twins; 171 binary)	value
$p_{D}$ twin $\to$ primary, base	0.306 $\to$ 0.354
$p_{J}$ twin $\to$ primary, base	0.280 $\to$ 0.311
$p_{D}$ twin $\to$ primary, instruct	0.192 $\to$ 0.277
$p_{J}$ twin $\to$ primary, instruct	0.224 $\to$ 0.263
$\Delta_{E}$ , continuous	$-$ 0.028 [ $-$ 0.049, $-$ 0.007]; MDE 0.030
acting side, base minus instruct	$-$ 0.037 [ $-$ 0.062, $-$ 0.012]

Appendix

Table 16: The paired contrasts on the shared subset.

slice	$n$	$E_{\mathrm{base}}$	$E_{\mathrm{instruct}}$	$\Delta_{E}$
pilot-written scenarios (prompt 1.0.0)	45	0.013 [0.002, 0.025]	0.016 [ $-$ 0.015, 0.049]	$-$ 0.003 [ $-$ 0.032, 0.027]
later-written scenarios (prompt 1.1.0)	147	0.020 [0.011, 0.028]	0.055 [0.029, 0.083]	$-$ 0.036 [ $-$ 0.063, $-$ 0.011]
primaries	105			$-$ 0.030 [ $-$ 0.060, 0.000]
harm twins	87			$-$ 0.026 [ $-$ 0.056, 0.002]
F1	62	0.012 [0.002, 0.022]	0.035 [0.002, 0.070]	$-$ 0.023 [ $-$ 0.057, 0.009]
F3	60	0.037 [0.022, 0.052]	0.066 [0.029, 0.106]	$-$ 0.029 [ $-$ 0.068, 0.006]

Appendix

Table 17: Slices of the shared subset. The pilot’s 96 scenarios were re-run in the full-panel pod with bit-identical raw-frame values (maximum absolute difference 0.0 over 192 model-scenario pairs), so the pilot block is a scenario-subset check, not an independent replication. The pilot-written and later-written subsets are not separated from each other at these counts; the pooled number is the number of record and the prompt version is recorded as a covariate.

Explore similar work

Mar 24, 2026cs.AI

On the Context Sensitivity of LLM Moral Judgment

A human's moral decision depends heavily on the context. Yet research on LLM morality has largely studied fixed scenarios. We address this gap by introducing Contextual MoralChoice, a dataset of moral dilemmas with systematic contextual variations known from moral psychology to shift human judgment: consequentialist, emotional, and relational. Evaluating 22 LLMs, we find that nearly all models are context-sensitive, shifting their judgments towards rule-violating behavior. Comparing with a human survey, we find that models and humans are most triggered by different contextual variations, and that a model aligned with human judgments in the base case is not necessarily aligned in its contextual sensitivity. This raises the question of controlling contextual sensitivity, which we address with an activation steering approach that can reliably increase or decrease a model's contextual sensitivity. Code and data: https://github.com/adrian-sauter/contextual_moralchoice.

Adrian Sauter, Mona Schirmer

Helmholtz Munich · University of Amsterdam

Moral Reasoning Large Language Model Decisions Human Judgment Contextual Reasoning

Ask PDF

Jul 6, 2026cs.CL

The yes-no bias of large language models reflects answer order and wording, not shifts in moral judgment

Large language models (LLMs) increasingly issue judgments read as binary verdicts, and a growing literature reports such judgments shifting under logically irrelevant changes of wording - among them an amplified yes-no bias on moral dilemmas, absent in humans. A single framing cannot say what such a shift is: in a yes/no question the word "no" is at once logical verdict, lexical token, and last-printed option. We introduce a psychometric battery that separates these: crossed symmetrization - every logically irrelevant factor flipped in balanced pairs - across a corpus of question forms. A graded rating across logically equivalent forms recovers a coherent internal moral scale: frontier models' stance

θ

is nearly format-invariant (cross-form incoherence 0.12-0.21 on a

\pm 1

axis); small open-weight models fail in model-specific ways. Forcing the verdict through yes/no overlays a decomposable artifact: an order bias toward the last-printed option - opposite to classic human primacy - plus a lexical pull toward the word "no"; the artifact is substantial only in the Claude models (story-averaged -0.32 to -0.86),

\approx 0

for GPT-5.5 and Gemini, and shrinks under extended reasoning. The word and the verdict share one token; swapping the words for arbitrary labels separates them, and the verdict-attached logical bias proves

\approx 0

for every frontier model, while model-specific label and order attachments remain: the models are not drawn toward rejecting - the pull follows the printed surface, not the verdict it carries. A minimal model,

P = σ((θ\pm m)/s)

, summarizes any such artifact by a framing susceptibility m and a moral decisiveness s, measurably distinct from sampling temperature. The battery applies unchanged to any dilemma set and binary format: measuring what a model values requires crossing the frames of the question, not asking once.

Haonan Huang

Princeton University, Princeton, NJ 08540, USA

Moral Reasoning Large Language Model Decisions Verdict Psychometric Properties Order

Ask PDF

May 29, 2026cs.CL

Every Act Has Its Price: Compressed Moral Composition in Frontier LLMs

Existing LLM moral benchmarks usually ask which isolated moral act, value, or foundation a model prefers. This is useful but incomplete. Realistic judgments often require a model to combine several moral signals within the same option. We introduce Moral Trolley Arena, a two-stage blind ELO benchmark for measuring how LLMs compose moral evidence. The single-scene arena first calibrates individual moral acts from a 229-scenario corpus across five Moral Foundations Theory foundations; the composite arena then combines calibrated acts into two-act moral items over a controlled intensity grid and measures the resulting composite preferences. Across ten frontier models, composite judgments are largely predicted by component act strength, but the relation is consistently compressed rather than simply additive. Models also show non-additive intensity anchoring, bounded foundation-specific residuals after component control, and highly convergent composite preference surfaces across providers. These results suggest that moral audits should measure composition rules for moral evidence, not only rankings over isolated acts.

Weijia Zhang, Ruiqi Chen, Yunze Xiao +1

University of Illinois Urbana-Champaign · University of Michigan · Carnegie Mellon University +1

Moral Reasoning Large Language Model Decisions Frontiers Prices

Ask PDF