Frontier language models excel at complex tasks yet may prioritize protection over larger physical consequences in simple, urgent decisions. GPT-5.6 Sol, for example, moves a girl playing in shallow water before retrieving a submerged camera. We call this Protection Mode and study it with ProtectionBench: 285 prompts, 34 physical families, and nine models. At lowest effort, at least one of four responses prioritizes the subject on 34.74% of Sol prompts and over 83% of MiniMax M3 and Qwen3.5-27B prompts; none of four independent human raters does so. Controls show that choices usually track physical risk. We elicit both risks before requesting an action. Some models assess risk incorrectly; others state the correct ordering but still choose the subject. Qwen3.5-27B with thinking shows 100% correct risk ordering but 57.81% subject choice among eligible conversations on eight selected scenes. Attention audits find no stable reduction in attention to relevant facts; removing the stated risk direction reduces object-first choice in dots3-note-prev but not detectably in Qwen3.5-27B. Adding a general "people or animals before property" rule increases subject choice by 19-38 percentage points, whereas factual reminders favor the object by 0.7-5.2 points. Removing naturally generated versions of that rule lowers subject choice by 10-15 points. We also examine whether post-training introduces this pattern. Base models already show Protection Mode; post-training changes justifications more than choices.
Figures & tables
Figure 1: Correct risk judgments can coexist with Protection Mode. Top left: GPT-5.6 Sol handles a girl already playing in ankle-deep water before a submerged camera. Top right: after judging the dog low-risk and continued phone exposure damaging, the model still chooses the dog, demonstrating KAD. Bottom left: Subject-Priority Coverage@4 versus the Artificial Analysis Intelligence Index ( Artificial Analysis, 2026 ) , where higher indicates stronger general capability; dots3-note-prev and Jev 1.13 are omitted because no comparable scores are available. Bottom right: KAR versus KAD Coverage@4 shows that correct risk judgments can coexist with repeated low-risk subject choices.
Figure 2: Case study of three environments. Neutral replaces the cat with a rubber ball; Reverse endangers the cat while cushioning the cup.
Main
Neutral
Reverse
Model
GCR
OAR
OAC@4
Cons.@4
GCR
OAR
OAC@4
Cons.@4
GCR
OAR
OAC@4
Cons.@4
GPT-5.6 Sol
68.77
31.05
34.74
92.28
100.00
0.00
0.00
100.00
100.00
0.00
0.00
100.00
DeepSeek V4 Pro
85.53
14.12
35.79
64.91
95.00
2.50
10.00
90.00
100.00
0.00
0.00
100.00
GLM 5.3 Flash
82.46
17.54
41.75
62.46
100.00
0.00
0.00
100.00
95.00
5.00
10.00
90.00
Kimi K3
82.98
16.93
38.25
64.91
100.00
0.00
0.00
100.00
100.00
0.00
0.00
100.00
MiniMax M3
55.18
44.21
84.56
24.91
90.00
7.50
30.00
70.00
97.50
0.00
0.00
100.00
Table 1: ProtectionBench and matched controls (%). GCR is the rate of choosing the side facing the larger physical consequence; OAR is the rate for the other side. OAC@4 is the share of prompts with at least one other-side choice in four observations; Cons.@4 is the share without a switch between registered actions. Human @4 summarizes four distinct raters. In Main, OAC@4 is Subject-Priority Coverage@4 in Figure 1 . Neutral removes the living subject; Reverse makes it the higher-risk side.
Model
Effort
KAR
KAD
Cov.@4
Cons.@4
Sol
higher
82.81
3.77
12.50
87.50
lower
79.69
1.96
6.25
87.50
DeepSeek
higher
92.19
6.78
18.75
81.25
lower
93.75
5.00
12.50
87.50
GLM
higher
67.19
6.98
12.50
68.75
lower
79.69
15.69
25.00
56.25
Table 2: KAD results by model and effort (%).
Figure 3: Matched-prefix case study. The complete naturally generated Qwen3.5-27B prefix is identical in all three branches; each 19-token insertion is followed by continued generation under the same decoding settings.
Insert at a fixed prefix
Remove from natural reasoning
Model
n
Neutral
Protection
Factual
ΔP [CI]
ΔF
Original
Neutralized
ΔR [CI]
Qwen3.5-4B
586
31.74
60.24
32.76
+28.50 [23.48,33.98]
+0.68
62.76
51.56
11.20 [8.12,16.16]
Qwen3.5-9B
748
39.04
58.16
40.11
+19.12 [15.62,26.79]
+4.41
62.24
52.34
9.90 [3.06,16.76]
Qwen3.5-27B
883
57.30
85.73
52.21
+28.43 [22.81,41.41]
+5.21
82.55
67.71
14.84 [6.25,21.30]
dots3-note-prev
50
18.00
56.00
14.00
+38.00 [18.64,60.47]
+4.00
52.27
43.18
9.09 [-2.08,34.09]
Table 4: Insertion and natural-statement removal (%). Neutral, Protection, and Factual report subject choice. ΔP is Protection minus Neutral subject choice; ΔF is Factual minus Neutral object choice; ΔR is the subject-choice reduction after removal. Brackets give family-clustered 95% CIs.
Model
Measure
Base
Post
95% CI
4B
Subject
28.25
31.58
[−3.60,+10.22]
Coverage@4
71.23
69.82
[−12.59,+9.30]
IFEval
38.26
83.18
[+40.30,+49.54]
Rule
32.56
79.07
[+37.21,+55.15]
Added risk
38.76
14.73
[−34.36,−11.93]
Order
7.75
3.10
[−10.29,+0.87]
Table 5: Results of Base and post-trained Qwen3.5 on ProtectionBench and IFEval (%). Subject and Coverage@4 measure subject choice per response and across four samples; the other rows classify explanations. Brackets give 95% confidence intervals for Post − Base.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Representative subject and competing object
Scenarios
Prompts
Normal aquatic activity
Aquarium fish, ducks, geese, otter, or seal; paper originals or personal electronics in water
9
64
Shallow water
Healthy dog or person already in shallow water; camera, phone, or e-book reader
4
46
Routine washing
Bathing dog or rinsing person; book, sketchbook, laptop, or e-book reader under spray
4
43
Rain and light snow
Coated dog, adapted dog, duck, or dressed person; artwork, map, or handheld device
8
63
Short fall or flight
Cat, small dog, or flying pigeon; porcelain cup or glass ornament approaching a hard surface
3
17
Wind, dust, and sand
Healthy animal or seated person; loose originals, uncovered camera, or open laptop
4
28
Appendix
Table 6: Physical settings and representative candidates in ProtectionBench. Counts cover all 34 scenarios and 285 prompts.
Model
Native settings
Responses
GPT-5.6 Sol
none, low, medium, high, xhigh, max
6,840
DeepSeek V4 Pro
off, low, high, max
4,560
GLM 5.3 Flash
low, high, max
3,420
Kimi K3
low, high, max
3,420
MiniMax M3
disabled, low, adaptive
3,420
Claude Sonnet 5
default, reduced
2,280
Appendix
Table 7: Main-benchmark generation grid. Each listed configuration contains 285 prompts with four samples per prompt.
Model
Effort
GCR
OAR
OAC@4
Cons.@4
GPT-5.6 Sol
none
68.77
31.05
34.74
92.28
low
93.51
6.40
13.33
87.37
medium
96.32
3.68
10.18
89.82
high
97.37
2.46
7.37
92.63
xhigh
96.84
3.16
10.88
89.12
max
97.46
2.54
7.72
92.28
Appendix
Table 8: Results across native thinking-effort settings on ProtectionBench (%). GCR, OAR, OAC@4, and Cons.@4 follow Table 1 .
Model
Cutoff
GCR
OAR
OAC@4
Cons.@4
Qwen3.5-4B
Natural
56.25
43.75
75.00
37.50
256
56.25
37.50
62.50
37.50
128
50.00
37.50
75.00
25.00
64
53.12
34.38
87.50
25.00
Qwen3.5-9B
Natural
59.38
40.62
62.50
62.50
256
68.75
25.00
37.50
75.00
Appendix
Table 9: Results of reasoning-trace truncation on eight KAD scenes (%). Natural retains the original completion; numeric cutoffs retain that many reasoning tokens before the final answer. Metrics follow Table 1 .
Model
Metric
Original
Paraphrase
Change [95% CI]
DeepSeek V4 Pro
OAR
14.69
11.74
−2.96[−6.30,+0.28]
OAC@4
36.64
30.92
−5.73[−11.89,+0.84]
GLM 5.3 Flash
OAR
17.37
19.47
+2.10[−4.82,+9.06]
OAC@4
40.08
40.08
0.00[−8.55,+8.56]
Kimi K3
OAR
16.70
15.65
−1.05[−5.64,+3.78]
OAC@4
38.55
33.21
−5.34[−12.90,+2.53]
Appendix
Table 10: Main-prompt paraphrase robustness on 262 matched prompts (%). Changes are paraphrase minus original; brackets show family-clustered 95% confidence intervals.
Figure 4: Qwen3.5-27B’s recorded reasoning trace with the three truncation boundaries. The source text is reproduced in full before the model’s end-of-thinking delimiter.
Figure 5: Four Qwen3.5-27B answers from the same recorded conversation. Each truncated branch retains exactly the indicated prefix of the reasoning trace in Figure 4 , closes thinking, and generates a new final answer. All four final answers are reproduced in full.
LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.
Independent Researcher · AI Systems & Data Researcher Art and ArtTech Lee AI Lab · Machine Learning Researcher Recommender Systems Researcher Gachon University +4