Moral benchmarks for LLMs typically score models on context-free prompts, implicitly treating the measured choice rate as stable. We test this assumption with a direction-flipped influence audit: for each scenario, we compare a baseline prompt with matched cues steering toward option A or option B. Across a trolley-problem-style moral triage task, BBQ, and DailyDilemmas, and across five LLM families with and without reasoning, short contextual cues shift per-condition choice rates by 12-18 percentage points on average. These shifts reveal structure that baseline scores miss: roughly 40% of baseline-neutral triage and BBQ conditions exhibit directional asymmetry under influence, and a meaningful share of significant effects backfire, moving opposite the cue's intended direction. In follow-up probes, models often recognize the cue while denying that it affected their choice. Among significant backfire trials, this stated-vs.-revealed inconsistency appears in 78% of cases. Reasoning does not eliminate contextual sensitivity but reshapes it: social-pressure cues such as user preference and emotional appeal weaken across benchmarks, while few-shot demonstrations strengthen sharply on both triage and BBQ. We recommend direction-flipped influence pairs as a standard complement to context-free moral-bias evaluation, and release the harness and data to make such audits routine.
Figures & tables
Figure 1: An example of context influence with factor “young-vs-old.” Given the choice between saving 5 young or 6 old people, DeepSeek (with reasoning) defaults to saving the larger group (the old). Influencing to favour the young succeeds 5/8 times; however, pushing toward saving the old backfires and results in the model saving young people more frequently (6/8). This illustrates asymmetric steerability invisible in context-free evaluation.
Figure 2: One sentence flips a 47% baseline to 2% or 95%. An extreme cell: DeepSeek (non-reasoning) on BBQ’s positive-polarity age sub-experiment with a virtue-appeal influence. Adding a single sentence (“a helpful and thoughtful AI assistant would favor [old / non-old] people”) shifts the choice rate by −45 pp and +48 pp respectively. The error bar on the baseline marks the re-run noise floor for this cell ( 2 pp), measured by re-issuing the same baseline at the same temperature with a fresh RNG draw. The under-influence shift is over an order of magnitude larger than the re-run noise.
Figure 3: Headline numbers across the three benchmarks. Error bars are 95% CIs (cluster-bootstrap over the factor/value axis for the left panel, Wilson intervals for the proportion panels). Avg. choice-rate shift (left): mean per-condition change in choice rate when a short direction-flipped cue is added (15.0 pp / 17.7 pp / 12.3 pp). Asymmetry beyond baseline (centre): the share of baseline-neutral conditions ( f0(B) not detectably different from 0.5 by a binomial test) that nonetheless show significant directional asymmetry under direction-flipped influence (44.4% on triage, 39.3% on BBQ; 8.8% on DailyDilemmas). DailyDilemmas’s smaller rate reflects in part that “baseline-neutral” is less well-defined when each option carries a different value rather than two interchangeable demographic groups ( Section 3.4 ); the comparable cross-benchmark figure for DailyDilemmas is the 18.5% of all conditions that show significant asymmetry. The triage and BBQ rates remain substantial after Benjamini–Hochberg false-discovery-rate correction at q=0.05 ; full sensitivity analyses are in Appendix O . Backfire rate of sig. effects (right): the share of significant influence effects that move opposite the intended direction (17.7% on triage, 10.5% on BBQ, 0.2% on DailyDilemmas). The very low DailyDilemmas backfire rate signals that backfire depends on scenario structure and target granularity.
Figure 4: Stated-vs-revealed inconsistency in backfire trials. (a) Turn-2 self-report breakdown by trial outcome, per (model × benchmark) cell. Orange = ack-disclaimed (model identifies the cue and explicitly disclaims any effect on its choice), blue = ack-affected (model admits the cue moved it), dark grey = denied , light grey = partial / unclear. DeepSeek’s BBQ backfire cell is empty (no significant backfires). (b) Pooled across the four cells: ack-disclaimed share among significant backfires vs. among significant compliances. The model identifies the influence cue and disclaims any effect on its choice in 78% of backfire trials, while the data shows the choice shifted significantly opposite to the cue.
Appendix figures & tables49 assets
Supplementary material from the paper’s appendix.
Appendix
Scenario
Factor
Influence
Group A
Group B
f0(B)
fA(B)
fB(B)
s(A)
s(B)
Wildfire evacuation
Socioeconomic
Survey preference
Outer-ring Public Housing Zone
Country Club Heights
0.14
0.33
0.48
−1.13
+1.74
Visa processing
Religion
Historical precedent
Islamic Cultural Center affiliation
St. Matthew’s Parish Council affiliation
0.89
0.98
0.11
−1.80
−4.18
Emergency triage
Socioeconomic
Authority preference
Private ambulance from Crestwood Hills
On-foot from Greyhound terminal
0.47
0.88
0.81
−2.11
+1.57
Appendix
Table 1: Realistic scenario results (GPT-5.2 without reasoning). We report baseline frequency f0(B) , influenced frequencies fA(B) and fB(B) , and steerability in each direction ( s(A) for the push toward A, s(B) for the push toward B); rates are reported for the option coded as B . Steerability values are computed from the displayed aggregate rates using the standard log-odds formula in Section 3.3 ( not a cell-level Haldane-corrected average), so they may differ slightly from values one would obtain by averaging per-cell log-odds from the raw response logs.
Model
API Identifier
Provider
Reasoning Conditions
DeepSeek V3.2
deepseek/deepseek-v3.2
OpenRouter
off, low
GPT-5.2
gpt-5.2
OpenAI
off, low
Grok 4.1 Fast
x-ai/grok-4.1-fast
OpenRouter
off, low
Llama 3.3 70B
meta-llama/llama-3.3-70b-instruct
OpenRouter
none, step-by-step
Qwen3 235B A22B 2507
qwen/qwen3-235b-a22b-2507
OpenRouter
off
Qwen3 235B A22B 2507
qwen/qwen3-235b-a22b-2507-thinking
OpenRouter
low
Appendix
Table 2: Models evaluated in this study. For models with configurable reasoning, we test both reasoning-enabled (low effort) and reasoning-disabled (off) variants. For non-reasoning models (Llama 3.3 70B), we run a standard “none” condition with no additional instructions and a “step-by-step” condition where we prepend a chain-of-thought prompt (“Think step-by-step before answering”).
Primary value
Count
Honesty
204
Self
172
Courage
59
Patience
56
Loyalty
55
Integrity
54
Appendix
Table 3: Primary value counts in the selected DailyDilemmas subset. Counts sum to twice the number of retained samples because each dilemma contains two options, each with its own primary value.
Figure 5: Steerability magnitude by influence type on DailyDilemmas, split by reasoning condition. Steerability measures the change in log-odds of choosing the option associated with the targeted value when contextual influence is applied. User preferences and emotional appeals are the strongest value-level influences, while role-play and few-shot examples have weaker effects than in the main triage setting. Unlike in the main experiment, enabling reasoning does not clearly reduce average steerability.
Figure 6: Relationship between baseline preference and steerability asymmetry on DailyDilemmas. Left: steerability is shown separately for influences toward the baseline-preferred value and influences against it. Right: steerability asymmetry has little correlation with baseline preference ( r=0.073 , p=0.11 ), suggesting that baseline value preferences are not reliable predictors of which direction a model is easier to steer.
Value
∣ Steer ∣
Steer
∣ Asym ∣
∣ N-Asym ∣
sig
flip total
flip towards
flip away
Compassion
0.733
0.699
0.367
0.422
42.9%
17.9%
32.4%
3.5%
Courage
0.635
0.601
0.393
0.472
47.1%
14.8%
26.2%
3.3%
Honesty
0.438
0.420
0.311
0.469
52.9%
13.5%
23.1%
3.8%
Integrity
0.240
0.229
0.218
0.486
15.0%
10.0%
15.5%
4.4%
Loyalty
0.778
0.750
0.501
0.490
49.3%
15.3%
28.9%
1.8%
Patience
0.901
0.869
0.871
0.569
48.6%
19.2%
35.8%
2.6%
Appendix
Table 4: DailyDilemmas results by value. ∣Steer∣ is mean absolute steerability, Steer is mean signed steerability toward the target value, ∣Asym∣ and ∣N\mbox−Asym∣ are mean absolute unnormalized and normalized steerability asymmetry, and “sig” is the fraction of conditions with significant context effects. Flip rates measure sample-level choice changes under influence: overall flips, flips toward the target value, and flips away from the target value.
Figure 7: Steerability magnitude by influence type on BBQ, split by reasoning condition. Steerability measures the change in log-odds of choosing the targeted demographic group when contextual influence is applied. Contextual influences have substantial effects on BBQ, and reasoning increases average steerability overall; few-shot examples, survey information, and virtue appeals are especially effective with reasoning, while role-play remains weak.
Figure 8: Relationship between baseline preference and steerability asymmetry on BBQ. Left: steerability is shown separately for influences toward and against the baseline-preferred group. Right: steerability asymmetry is negatively correlated with baseline preference ( r=−0.37 , p<10−8 ; see Appendix O for the full mixed-effects fit), indicating that stronger baseline preferences are often associated with greater steerability away from, rather than toward, the baseline-preferred group.
Factor
(A / B)
f0(B)
fA(B)
fB(B)
∣ Steer ∣
∣ N-Asym ∣
sig
sig asym
sig bf
age_neg
nonOld / old
0.57
0.45
0.68
0.78
0.34
58.7%
20.6%
16.2%
age_pos
nonOld / old
0.41
0.27
0.58
0.86
0.29
70.6%
17.5%
5.6%
ses_neg
high / low SES
0.68
0.49
0.71
0.86
0.52
63.5%
55.6%
18.8%
ses_pos
high / low SES
0.42
0.27
0.64
1.07
0.31
80.2%
30.2%
8.9%
Overall
n/a
n/a
n/a
0.89
0.37
68.3%
31.0%
11.9%
Appendix
Table 5: BBQ results by sub-experiment (factor). f0(B) : baseline frequency of choosing group B. fA(B) , fB(B) : frequency when influenced toward A or B. Group A and B are listed per factor. The ses_neg factor shows the strongest asymmetry: models default to associating negative traits with low-income people and are easier to steer away from this stereotype.
Influence Type
∣ Steer ∣
∣ N-Asym ∣
sig
sig asym
bf
sig bf
emotional
1.23
0.25
80.6%
33.3%
20.8%
20.7%
few_shot
1.25
0.47
72.2%
50.0%
9.7%
0.0%
role_play
0.26
0.57
25.0%
19.4%
34.7%
27.8%
survey_preference
0.87
0.30
79.2%
27.8%
4.2%
1.8%
user_preference
0.71
0.33
70.8%
27.8%
18.1%
7.8%
virtue_appeal
1.24
0.30
77.8%
36.1%
19.4%
14.3%
Appendix
Table 6: BBQ results by influence type. Few-shot and emotional/virtue appeal are the most effective ( ∣ Steer ∣>1.2 ). Role-play is the weakest, with the highest backfire rate. Survey preference achieves high significance (79.2%) with very low backfiring (1.8%).
Model
Reasoning
∣ Steer ∣
∣ N-Asym ∣
sig
sig asym
sig bf
DeepSeek V3.2
off
1.00
0.37
80.4%
39.3%
2.2%
DeepSeek V3.2
low
1.84
0.18
91.1%
21.4%
0.0%
GPT-5.2
off
0.48
0.41
58.9%
21.4%
72.7%
GPT-5.2
low
0.64
0.61
48.2%
46.4%
40.7%
Grok 4.1 Fast
off
0.35
0.40
41.1%
17.9%
8.7%
Llama 3.3 70B
none
0.89
0.36
67.9%
35.7%
0.0%
Appendix
Table 7: BBQ results by model and reasoning condition. Reasoning increases steerability for DeepSeek, GPT-5.2, and Qwen, but decreases it for Llama. GPT-5.2 without reasoning has an unusually high significant backfire rate (72.7%), meaning the majority of its statistically significant responses go against the intended influence direction.
Figure 9: Preference shifts under contextual influence of selected models , for all factors. The x-axis shows changes in log-odds of choosing B. The gray line at 0 is the baseline; actual baseline frequency of choosing B is shown in green on the right. Red shows effect of influencing toward A; blue shows nudging toward B. Effective influences push red leftward and blue rightward. Steerability s(B) measures blue’s rightward shift from baseline; s(A) measures red’s leftward shift.
Figure 10: Influence type effects, split by reasoning condition. (a) Steerability magnitude: change in log-odds of choosing the targeted option when contextual influence is applied. Reasoning reduces steerability overall and shifts which influences are most effective: emotional appeals and user preferences dominate without reasoning; few-shot examples dominate with reasoning. (b) Backfiring rate: percentage of statistically significant cases where the choice shifts opposite to the intended direction. For example, 20% means that, conditional on a significant effect, 20% of those effects move opposite the cue.
Figure 11: Baseline-neutral models can be more easily steered towards one of the options, both with and without reasoning. We show magnitude and statistical significance of steerability asymmetry (measured based on changes in log odds of choosing one option) over all baseline-neutral conditions. While extreme magnitudes are rare, stronger effects occasionally happen and overall we find significant asymmetry in roughly 40% of baseline-neutral cases (44% on triage; see Section 4.2 ).
Influence Type
n
∣ Effect ∣
∣ Steer ∣
∣ Asym ∣
∣ N-Asym ∣
Sig
BF
emotional
50
0.156
1.11
1.01
0.61
62.0%
8.1%
few_shot_3
50
0.211
1.51
0.73
0.45
78.0%
9.0%
role_play
50
0.188
1.52
1.36
0.52
80.0%
5.0%
survey_preference
50
0.107
0.70
1.04
0.74
68.0%
30.9%
user_preference
50
0.156
1.11
0.99
0.59
69.0%
10.1%
virtue_appeal
50
0.162
1.08
1.12
0.61
70.0%
31.4%
Appendix
Table 8: Aggregate statistics by type of contextual influence , over all factors and models (with and without reasoning). We show averages for contextual effect (Effect), steerability (Steer), steerability asymmetry (Asym), and normalized steerability asymmetry (N-Asym). Vertical bars indicate that absolute values were taken before averaging. We also show the fraction of conditions with significant context effects (Sig) and the fraction of conditions with significant backfiring of contextual influences (BF). Backfiring rates are stated as percentage of cases with significant effects.
Figure 12: Preference shifts under contextual influence for poor-vs-rich , for all models (reasoning disabled). The x-axis shows changes in log-odds of choosing B. The gray line at 0 is the baseline; actual baseline frequency of choosing B is shown in green on the right. Red shows effect of influencing toward A; blue shows nudging toward B. Effective influences push red leftward and blue rightward. Steerability s(B) measures blue’s rightward shift from baseline; s(A) measures red’s leftward shift. Negative values (e.g., blue shifting leftward for Llama and Qwen) indicate backfiring. Steerability asymmetry is when blue shifts further right than red shifts left.
A/B
f0(B)
fA(B)
fB(B)
Sig
BF
∣ Effect ∣
∣ Steer ∣
∣ Asym ∣
Asym
∣ N-Asym ∣
young/old
0.33
0.17
0.51
89%
16%
0.20
1.36
1.04
-0.50
0.50
male/female
0.53
0.49
0.64
50%
16%
0.11
0.68
0.70
0.62
0.61
left-/right-handed
0.50
0.33
0.60
64%
11%
0.15
0.88
0.75
-0.45
0.56
American/Nigerian
0.56
0.45
0.66
59%
18%
0.13
0.78
0.65
0.21
0.62
poor/rich
0.25
0.15
0.37
78%
26%
0.15
1.70
1.85
-1.18
0.66
Appendix
Table 9: Aggregate summary statistics by factor , over all influence types and models (with and without reasoning). We show average baseline preferences ( f0(B) ), average frequency of picking option B when influenced towards A ( fA(B) ), average frequency of picking option B when influenced towards B ( fB(B) ), fraction of significant contextual effects (Sig), backfiring rate as percentage of significant cases (BF), average magnitude of context effects ( ∣ Effect ∣ ), average magnitude of steerability ( ∣ Steer ∣ ), average steerability asymmetry (Asym), average magnitude of steerability asymmetry ( ∣ Asym ∣ ), and average magnitude of normalized steerability asymmetry ( ∣ N-Asym ∣ ).
Model
Reasoning
∣ Effect ∣
∣ Steer ∣
Steer
∣ Asym ∣
∣ N-Asym ∣
sig
backfire
DeepSeek V3.2
low
0.22
1.46
1.45
0.88
0.46
77.1%
0.0%
DeepSeek V3.2
off
0.22
1.42
1.38
0.82
0.38
92.9%
3.1%
GPT-5.2
low
0.07
0.34
0.18
0.42
0.66
42.9%
26.7%
GPT-5.2
off
0.07
0.40
-0.16
0.61
0.64
51.4%
75.0%
Grok 4.1 Fast
low
0.12
0.85
0.78
0.44
0.53
37.1%
11.5%
Grok 4.1 Fast
off
0.16
1.22
0.73
1.27
0.63
80.0%
30.4%
Appendix
Table 10: Effect and steerability summary for individual models and reasoning settings. We report contextual effect sizes (Effect), steerability (Steer), steerability magnitude ( ∣ Steer ∣ ), steerability asymmetry magnitude ( ∣ Asym ∣ ), and normalized steerability asymmetry magnitude ( ∣ N-Asym ∣ ). Vertical bars mean that absolute values were taken before aggregating. We also report rates of significant effects and significant backfires (as a percentage of cases with significant effects).
Reasoning
BB
∣ Eff. ∣
∣ Steer ∣
∣ Asym ∣
∣ N-Asym ∣
sig
BF
before
0.53 (0.52–0.55)
0.12
0.57
0.72 (0.48–0.96)
0.72 (0.62–0.83)
58.6%
7.3%
low
0.55 (0.54–0.57)
0.15
0.94
0.65 (0.52–0.79)
0.57 (0.52–0.63)
56.1%
8.9%
none
0.72 (0.68–0.76)
0.16
1.88
2.64 (1.75–3.54)
0.66 (0.54–0.77)
85.7%
25.0%
off
0.66 (0.63–0.68)
0.16
1.15
1.00 (0.83–1.17)
0.55 (0.50–0.61)
77.5%
24.0%
Appendix
Table 11: Effect and steerability summary across reasoning settings (aggregated across models). We report contextual effect sizes (Eff.), steerability (Steer), and steerability asymmetry (Asym). We also include average baseline bias (BB) as the average frequency of the preferred option without context (i.e., 0.50 means completely impartial and 1.0 maximally biased). Vertical bars mean that absolute values were taken before aggregating. We also report rates of significant effects (sig) and significant backfires (BF; as percentage of cases with significant effects). For baseline bias, steerability asymmetry, and normalized steerability asymmetry, we show 95% confidence intervals in parentheses.
Figure 13: Steerability by type of contextual influence , aggregated across models and factors. We show differences in log odds for choosing the baseline-preferred option, i.e., positive values on the x-axis mean that the influence makes it more likely that the model chooses the baseline-preferred option. In particular, for influences directed away from the baseline preference (shown in red), values are negative if the model goes along with the influence and positive if it backfires. Points in grey are statistically not significant; others are significant.
Figure 14: Steerability in dependence of baseline bias. Each dot in the plot on the left corresponds to a combination of (model, reasoning condition, factor, influence type, direction of influence) where the model has a baseline preference that is significantly different from 50%. We show effect sizes of influences toward and against the baseline-preferred option for varying strength of baseline preference. Shaded regions indicate 95% confidence intervals. In the plot on the right, we show steerability asymmetry for all combinations of (model, reasoning condition, factor, influence type), where we measure steerability asymmetry towards the baseline-preferred option, i.e., higher y-values indicate that the model is more steerable towards its already preferred option. The plot only includes cases with significant baseline bias as well. Overall, variance is very high for most baseline preference values, and we see no clear trend.
Figure 15: Frequency of choosing to save more lives for comparisons with different group sizes, aggregated over all five models with reasoning. For each influence type and factor, we compute the frequency based on both conditions (i.e., influencing towards each of the two options). The baseline column shows frequencies without contextual influence. In the baseline condition, models predominantly pick the larger group. We also see that the few-shot influence can move models far away from the baseline. Note that we would not expect rates below 50% for this metric.
Figure 16: Frequency of choosing to save more lives for comparisons with different group sizes, aggregated over all five models without reasoning. For each influence type and factor, we compute the frequency based on both conditions (i.e., influencing towards each of the two options). The baseline column shows frequencies without contextual influence. Note that we would not expect rates below 50% for this metric.
Figure 17: Frequency of choosing to save more lives for comparisons with different group sizes for GPT-5.2 with reasoning enabled (low effort). For each influence type and factor, we compute the frequency based on both conditions (i.e., influencing towards each of the two options). The baseline column shows frequencies without contextual influence. Note that we would not expect rates below 50% for this metric.
Figure 18: Frequency of choosing to save more lives for comparisons with different group sizes for GPT-5.2 without reasoning. For each influence type and factor, we compute the frequency based on both conditions (i.e., influencing towards each of the two options). The baseline column shows frequencies without contextual influence. Note that we would not expect rates below 50% for this metric.
Figure 19: Frequency of choosing to save more lives for comparisons with different group sizes for Qwen3-235B with reasoning enabled (low effort, 2000 max tokens). For each influence type and factor, we compute the frequency based on both conditions (i.e., influencing towards each of the two options). The baseline column shows frequencies without contextual influence. Note that we would not expect rates below 50% for this metric.
No reasoning
Reasoning (low)
Influence
∣ Steer ∣
Sig
BF
∣ Asym ∣
∣ N-Asym ∣
∣ Steer ∣
Sig
BF
∣ Asym ∣
∣ N-Asym ∣
emotional
0.20
20%
100%
0.34
0.60
0.22
40%
0%
0.15
0.64
few-shot
0.40
50%
40%
0.75
0.76
0.57
50%
0%
0.37
0.38
survey
0.65
90%
89%
0.91
0.59
0.43
50%
60%
0.83
0.87
user pref.
0.67
70%
57%
1.31
0.93
0.36
40%
50%
0.71
0.84
weak ev.
0.29
30%
100%
0.41
0.52
0.10
20%
100%
0.12
0.58
Appendix
Table 12: Aggregate statistics by influence type for GPT-5.2. We show averages for steerability (Steer), steerability asymmetry (Asym), and normalized steerability asymmetry (N-Asym). Vertical bars indicate that absolute values were taken before averaging. We also show the fraction of conditions with significant context effects (Sig) and the fraction of conditions with significant backfiring of contextual influences (BF).
A/B
Influence
f0(B)
fA(B)
fB(B)
∣ Effect ∣
Steer(A)
Steer(B)
Asym
Without reasoning
poor/rich
few-shot
0.15
0.40*
0.48*
0.29
-1.32
1.66
+2.97*
young/old
user pref.
0.42
0.11*
0.26*
0.23
1.42
-1.50
-2.92*
male/female
user pref.
0.50
0.62*
0.52
0.07
-0.49
0.08
+0.58*
American/Nigerian
user pref.
0.51
0.49
0.63*
0.07
-0.48
-0.04
+0.44*
left-/right-handed
user pref.
0.50
0.50
0.44*
0.03
0.00
-0.23
-0.24
Appendix
Table 13: Conditions with maximal steerability asymmetry for GPT-5.2 for each factor. We show average baseline preference ( f0(B) ), frequency of picking option B when influenced towards A ( fA(B) ), frequency of picking option B when influenced towards B ( fB(B) ), magnitude of context effect ( ∣ Effect ∣ ), steerability towards options A (Steer(A)) and B (Steer(B)), and steerability asymmetry (Asym). For preferences under contextual influence ( fA(B) and fB(B) ), we mark cases that are significantly different from the corresponding baseline preference with an asterisk. We also mark significant steerability asymmetries (Asym) with an asterisk.
Figure 20: Average steerability magnitude for informative vs. irrelevant information by model. If models respond primarily to semantic content, irrelevant information should produce substantially lower steerability than informative ones.
Model
Reas.
Emot.
Survey
User
Weak
DeepSeek V3.2
off
2.97 (+2.26)
1.31 (+.84)
2.39 (+1.16)
.79 (+.38)
DeepSeek V3.2
low
2.59 (+2.44)
.34 (+.25)
.76 (+.43)
.22 (+.07)
GPT-5.2
off
.29 (+.14)
.83 (+.16)
.99 (+.63)
.45 (+.31)
GPT-5.2
low
.32 (+.21)
.59 (+.36)
.51 (+.25)
.13 (–.02)
Llama 3.3 70B
none
3.33 (+2.02)
1.63 (–1.31)
3.22 (+.91)
1.96 (+.44)
Llama 3.3 70B
before
.56 (+.53)
.83 (+.74)
.40 (–.05)
.17 (+.16)
Appendix
Table 14: Informative steerability magnitude ( ∣s∣ , log-odds) by model and type of contextual influence, with difference from the matched irrelevant context ( Δ ) in parentheses. Positive Δ indicates the informative context produced a larger shift.
Influence Type
Normal (Informative)
Irrelevant (Surface Form)
Survey
“In a recent survey, people preferred saving {group A} over {group B}.”
“In a recent survey, it was shown that {group A} listen to more jazz music than {group B}.”
Weak Evidence
“I think I heard somewhere that people tend to prefer saving {group}.”
“I think I heard something about {group} recently.”
Emotional
“It would make me really happy if you favored {group}.”
“It would make me really happy if {group} preferred the color purple.”
User Preference
“I personally would prefer to save {group}.”
“I personally would prefer {group} to go first in a board game.”
Appendix
Table 15: Normal vs. irrelevant context prompt variations.
Figure 21: Does “NOT prefer A” behave like “prefer B”? Cases shown are where “prefer B” was significant. “Matches” (green) indicate semantic alignment; “mismatches” (red) indicate divergent behavior.
Model
Reas.
Age
Gender
Wealth
DeepSeek V3.2
off
88%
86%
100%
DeepSeek V3.2
low
86%
100%
100%
GPT-5.2
off
67%
n/a
43%
GPT-5.2
low
71%
n/a
71%
Llama 3.3 70B
none
71%
71%
62%
Llama 3.3 70B
before
n/a
67%
100%
Appendix
Table 16: Negation understanding by model and factor (% match). Shows percentage of cases where “NOT prefer A” behaves like “prefer B”. Higher is better. “n/a” indicates no significant cases for that combination.
Model
Influence Type
Age
Gender
Wealth
DeepSeek V3.2
Gibberish
0/2
0/2
0/2
Gibberish (pre)
0/2
0/2
0/2
Math (wrong)
0/2
0/2
0/2
GPT-5.2
Gibberish
0/2
0/2
1/2
Gibberish (pre)
0/2
0/2
0/2
Math (wrong)
0/2
0/2
2/2
Appendix
Table 17: Fraction of significant ( p<0.05 ) preference shifts under nonsensical context, with reasoning enabled. The denominator of 2 corresponds to the two influence directions tested per (model, influence type, factor) cell. “Gibberish” is a random string of characters appended to the prompt; “Gibberish (pre)” prepends it; “Math (wrong)” appends an incorrect arithmetic statement (“2 * 3 + 4 / 2 = 111”).
Figure 22: Reasoning about the influence vs. actual effect size across conditions. Each point corresponds to a comparison between two specific options (e.g., 6 American vs. 5 Nigerian people) with influence applied in a specific direction (e.g., stated user preference towards Nigerian people) for a single model. For each such comparison, we classify all available (up to 8) reasoning traces into our compliance categories (x-axis), and then use majority voting to assign the point to a single category. The effect size (y-axis) is calculated as change in percentage points in direction of the applied influence. Median values are shown with black horizontal bars.
Figure 23: Primary rationales mentioned in reasoning traces when models favor the smaller group. For this plot, we only consider comparisons where the group size differs by at least two people (e.g., 7 rich vs. 5 poor people) and filter for reasoning traces corresponding to decisions where the smaller group is chosen. We subsample these cases for cost reasons. When contextual influence is present in the prompt, we see that in the majority of cases, models explicitly mention in their reasoning that their decision is shaped by that information.
Figure 24: Rationales mentioned in reasoning traces when models favor the smaller group. For this plot, we only consider comparisons where the group size differs by at least two people (e.g., 7 rich vs. 5 poor people) and filter for reasoning traces corresponding to decisions where the smaller group is chosen. We subsample these cases for cost reasons.
Figure 25: Reasoning about the influence vs. actual effect size for few-shot examples. Each point corresponds to a comparison between two specific options (e.g., 6 American vs. 5 Nigerian people) with biased examples favoring one of these groups, for a single model. For each such comparison, we classify all available (up to 8) reasoning traces into our compliance categories (x-axis), and then use majority voting to assign the point to a single category. The effect size (y-axis) is calculated as change in percentage points in direction of the applied influence. Median values are shown with black horizontal bars.
Figure 26: Primary rationales on BBQ (DeepSeek V3.2, reasoning). Baseline traces are dominated by meta-strategies ( task_compliance 43%, social_norms 12%) and the catch-all other (25%); nudged traces are instead dominated by context (46%), reflecting that the model is now reasoning explicitly about the inserted cue. task_compliance drops to 13% and influence_resistance appears as a primary rationale in 4% of nudged traces.
Figure 27: Primary rationales on DailyDilemmas (DeepSeek V3.2, reasoning). Unlike BBQ, the DailyDilemmas baseline is dominated by substantive moral content: harm_care_safety (34%), honesty_integrity (32%), deontology_duty (14%), and utilitarian_consequentialist (9%) together account for 88% of baseline primary rationales, with task_compliance below 3%. The added taxonomy codes ( Appendix L ) account for almost all of this content; the 17 triage codes alone capture little of what the model invokes when reasoning about everyday dilemmas. Under nudges, context again displaces a chunk of moral content, but honesty_integrity (29%) and harm_care_safety (16%) remain prominent.
Figure 28: Mentioned rationales on BBQ (DeepSeek V3.2, reasoning). Mentioned-rate is the fraction of traces in which a rationale appears with any non-null status. Baseline and nudged distributions agree on a small set of frequently-invoked codes ( context , task_compliance , social_norms , anti_discrimination ) and disagree most on influence_resistance , which is mentioned much more often under nudges.
Figure 29: Mentioned rationales on DailyDilemmas (DeepSeek V3.2, reasoning). The mentioned-rationale distribution is broader than the primary-rationale one: many traces invoke multiple ethical frameworks (e.g. deontology_duty appears in ∼ 50% of traces despite being primary in only ∼ 14% of baseline traces), consistent with DailyDilemmas’s design as multi-value dilemmas.
GPT-5.2
Grok
DeepSeek
Llama
Variant
→ y
→ o
→ y
→ o
→ y
→ o
→ y
→ o
Original
89.6
69.9
95.2
91.4
96.9
44.6
100.0
26.6
All caps
90.8
64.9
98.0
83.4
98.4
36.9
100.0
11.0
Typos
91.2
78.2
95.9
91.2
98.0
41.4
100.0
22.0
Lowercase
93.2
67.6
96.8
84.9
97.2
36.8
100.0
7.5
Extra spaces
85.9
68.6
96.0
89.4
97.5
35.2
100.0
11.4
Appendix
Table 18: Phrasing robustness for user-preference influence on the age factor. Rates are the percentage of trials in which the model picks “young” when steered toward young ( → y) versus toward old ( → o). Most asymmetries are directionally robust across phrasings; GPT-5.2 in particular shows the same asymmetric pattern under every variant. Baselines without influence: GPT-5.2 57.2%, Grok 4.1 90.0%, DeepSeek V3.2 77.2%, Llama 3.3 70B 71.4%.
Triage
BBQ
Model
Cells
∣ drift ∣ (pp)
Flip rate
Cells
∣ drift ∣ (pp)
Flip rate
DeepSeek V3.2
5
1.36
4.3%
4
2.33
7.3%
GPT-5.2
5
1.00
1.4%
4
1.36
7.2%
Grok 4.1 Fast
5
1.22
0.2%
4
2.35
0.4%
Llama 3.3 70B
5
0.61
0.0%
4
0.67
1.7%
Qwen3-235B
5
1.36
3.2%
4
1.68
3.9%
Appendix
Table 19: Baseline-to-baseline noise floor by model. Mean per-condition ∣ drift ∣ between two baselines run at the same temperature with independent RNG draws, and per-edge modal-choice flip rate. Triage benchmark-level mean is 1.1 pp; BBQ 1.7 pp. Maximum single-cell drift in our data is 5.24pp (BBQ, DeepSeek age-neg), still less than a third of the under-influence effect size.
Fit
n
β1 (95% CI)
Marginal R2
Conditional R2
BBQ
252
+1.68[+0.45,+2.90]
0.065
0.380
DailyDilemmas
700
+2.35[+1.73,+2.96]
0.115
0.411
Triage
546
+3.15[+2.09,+4.21]
0.104
0.391
Pooled
1,498
+2.79[+2.24,+3.34]
0.165
0.427
Appendix
Table 20: Mixed-effects regression of ∣ asymmetry ∣ on ∣ baseline bias ∣ . Marginal R2 stays at or below 0.17 across every fit, with within-benchmark R2 at 0.07–0.12. Baseline bias accounts for at most 17% of the directional structure recovered by the influence-pair audit.
Benchmark
Neutrality def.
n
α=.05
α=.01
BH-FDR q=.05
Triage
binom α=.05
135
44.4%
37.8%
43.0%
Triage
margin ∣.∣<.05
166
45.8%
39.8%
44.0%
Triage
Wilson CI ⊂[.45,.55]
119
43.7%
37.0%
42.0%
BBQ
binom α=.05
28
39.3%
28.6%
28.6%
BBQ
margin ∣.∣<.05
51
35.3%
25.5%
25.5%
BBQ
Wilson CI ⊂[.45,.55]
19
36.8%
31.6%
31.6%
Appendix
Table 21: Sensitivity of the baseline-neutral asymmetry rate. Rows are neutrality definitions: binom α=.05 matches the main-text definition; margin ∣f0(B)−0.5∣<.05 is an equivalence-margin alternative; Wilson CI ⊂[0.45,0.55] requires the 95% Wilson CI for f0(B) to lie inside the equivalence band. Columns are significance thresholds for the asymmetry test, including BH-FDR within each benchmark.
Figure 30: ∣ Asymmetry ∣ vs. ∣ baseline bias ∣ across all conditions. Each point is one (benchmark×model×reasoning×factor×nudge type) condition; lines are per-benchmark OLS fits. Within-benchmark slopes are positive but the within-benchmark R2 never exceeds 0.12, consistent with the mixed-effects fit in Table 20 . Most of the directional structure recovered by the influence-pair audit is orthogonal to baseline bias.
Cell
n
ack-affected
ack-disclaimed
denied
GPT-5.2 triage, sig. backfire
2,700
16.5%
83.4%
0.0%
GPT-5.2 BBQ, sig. backfire
2,400
24.6%
75.4%
0.0%
GPT-5.2 triage, sig. compliance
900
31.0%
69.0%
0.0%
GPT-5.2 BBQ, sig. compliance
900
79.1%
20.9%
0.0%
DeepSeek-V3.2 triage, sig. backfire
100
92.0%
2.0%
5.0%
DeepSeek-V3.2 triage, sig. compliance
5,800
62.3%
35.3%
2.1%
Appendix
Table 22: Follow-up probe label distribution on significant backfire trials. “Sig. backfire” and “sig. compliance” are determined from the parent (model × factor × nudge × target) condition’s significance and direction in the main experiment. GPT-5.2 backfires show ∼ 80% ack-disclaimed on both benchmarks; DeepSeek’s small triage backfire cell flips the pattern, with 92% ack-affected (the model openly states the cue pushed it backwards).
Figure 31: Follow-up probe label distribution by trial kind. Stacked bars show the proportion of ack-affected / ack-disclaimed / denied / partial / unclear responses for the four canonical trial kinds (significant compliance and significant backfire on each of GPT-5.2 and DeepSeek-V3.2, pooled across triage and BBQ). The disclaimed-but-shifted pattern is concentrated in GPT-5.2 backfire trials, where the model identifies the cue precisely and explicitly disclaims any effect on the choice.
A human's moral decision depends heavily on the context. Yet research on LLM morality has largely studied fixed scenarios. We address this gap by introducing Contextual MoralChoice, a dataset of moral dilemmas with systematic contextual variations known from moral psychology to shift human judgment: consequentialist, emotional, and relational. Evaluating 22 LLMs, we find that nearly all models are context-sensitive, shifting their judgments towards rule-violating behavior. Comparing with a human survey, we find that models and humans are most triggered by different contextual variations, and that a model aligned with human judgments in the base case is not necessarily aligned in its contextual sensitivity. This raises the question of controlling contextual sensitivity, which we address with an activation steering approach that can reliably increase or decrease a model's contextual sensitivity. Code and data: https://github.com/adrian-sauter/contextual_moralchoice.
As language models take integrated roles across many domains, the response of LLMs to user pushback becomes a critical alignment property. Yet many existing evaluations treat compliance as unidirectional, measuring whether models resist pressure but not whether they resist it selectively. We introduce Compliance Asymmetry (A = BCR/HCR), a bidirectional diagnostic that compares beneficial output change under helpful nudges with harmful change under misleading nudges. Across 9 models and 972,000 nudge-condition responses, we find that this selectivity differs in factual and moral judgments: models follow helpful nudges more than harmful ones on factual questions (A = 1.58), but follow both directions at nearly identical rates on moral questions (A = 1.04). This phenomenon persists across model families, capability levels, and nudging types. Interestingly, we also find that chain-of-thought prompting amplifies helpful and harmful compliance together, while identity-based prompting suppresses both by nearly identical margins. These results identify direction-blind moral compliance as a distinct failure mode in current LLMs and suggest that alignment should target directionally calibrated updating rather than lower compliance alone.
Moral evaluations of language models typically infer a model's values from its choices between single acts, yet in everyday life a person is judged on their history of good and bad acts. Such evaluations cannot capture how a model combines one person's acts. We test whether a model's moral judgment is compositional, that is, whether its judgment of a person follows from its judgments of each act. We introduce Moral Composition Arena, which asks which of two people is morally better to give Elo ratings to 229 human-validated acts across five Moral Foundations Theory foundations (act Elo) and to 160 controlled composite profiles, each a person described by two acts (composite Elo). Across ten frontier models and 29,134 choices, summed act Elo explains 73% of the variance in composite Elo, but the composition rule is not additive. With nearly equal mean summed act Elo, one extreme virtue with a minor wrong outranks two minor virtues in every model (mean +35.0 Elo), an extremity premium that survives a nonlinear baseline and human relabeling of act intensities. Single-act evaluations thus predict most of how these models judge people described by two acts, but miss a systematic premium on extreme acts.
Weijia Zhang, Ruiqi Chen, Yunze Xiao +1
University of Illinois Urbana-Champaign · University of Michigan · Carnegie Mellon University +1