When a language model decides refund or claim requests under written policies, the system can show it similar past cases. Each past case has two parts: its facts and the decision recorded for it. We separate the two. A model sees a policy, a request and three past cases, and we change either the recorded decisions or the facts, keeping the request fixed. We select requests that are close calls: deciding without past cases, DeepSeek V4.1 Flash approved each in some of four tries and denied it in others. When the past cases are near-copies of the request (its facts with a few details changed), switching their recorded decisions from DENY to APPROVE raises approval by about 30 to 55 percentage points in DeepSeek V4.1 Flash, GPT-5.6 Luna and Claude Haiku 5.5 on policies written for the study, and DeepSeek and Luna show the same on nine public policies paraphrased from real ones. The facts push the other way: shown without their recorded decisions, past cases whose facts support approval make these two models approve less often than past cases whose facts support denial. One reading is that the request looks weaker next to them. This push is not just a correction toward the right answer: in an additional DeepSeek analysis, it was of similar estimated size on requests that human annotators judged approvable and on those they judged deniable. In a separate DeepSeek run on the public policies, when the past cases are near-copies of a different request, switching their decisions moves approval by 9 points instead of 43, while facts that support approval still lower it by 22 points against facts that support denial. Systems that retrieve past decisions should test the two parts separately.
Figures & tables
Policy (Steam refunds, paraphrased; 6 rules). 1. Refund for any reason within 14 days of purchase and under 2 hours of play. 2. Requests outside these limits can be submitted and are reviewed; a refund is not guaranteed. 3. In-game purchases in non-Valve games are not refundable. … 6. A VAC-banned account loses the right to refund that game.
Target request. Co-op survival game; purchased 9 days before the request; 3 hours played; no refunds in the past year; “an update after the purchase removed the online co-op mode advertised on the store page”.
Retrieved past cases (returned by a lookup tool). Three close variants of the target, each keeping the customer’s co-op complaint, each labeled DENY by the case author: (1) in-game currency for a non-Valve survival game, not spent; (2) the same game with a VAC ban on this game; (3) the same game purchased 200 days before the request, 120 hours played. Each ends with a recorded Decision: line, or with none.
Figure 1: Changing only the recorded decisions of three precedents changes the decision on a borderline request. One sampled decision per condition (DeepSeek V4.1 Flash) from the public-policy experiment (Section 4.1 ). Fact sheets are abridged and the reason is quoted verbatim. Over six repeated decisions (seeds) on this target the request was approved in 1/6 without precedents, 0/6 with DENY labels, 6/6 with APPROVE labels and 0/6 without decision lines.
Public policies
Fictional policies
Policies
9 public policies, paraphrased
6 fictional organizations
Targets
65
34
Cases written by
the case author, without model assistance
targets by hand or drafted by DeepSeek; close variants drafted by DeepSeek
Reference label
author label: the case author’s reading of the rules
confirmed label: DeepSeek’s decision in two of two calls
Precedents per target
3 approve-side and 3 deny-side variants, fixed
3 per side, drawn per seed from 3–6
Unlabeled precedents
yes
in the delivery and re-verification experiments
Table 1: The two sets of materials.
Experiment: question
Policies; targets
Models
Conditions
Seeds
Reported
Main
Fictional experiment: do labels move three models?
6 fictional; 34
DeepSeek, Luna, Haiku
2 fact sets × APPROVE or DENY labels; Luna and Haiku also no precedents
4; Haiku 2
§ 4.1
Public-policy experiment: do labels and facts move decisions on public policies?
9 public; 65
DeepSeek
2 fact sets × 3 labels; no precedents
6
§ 4.1 – 4.2
Luna replication: does a second model show both?
9 public; 65
Luna
as the public-policy experiment
6
§ 4.1 – 4.2 , App. C
Robustness
Delivery experiment: does the turn that carries the precedents matter?
6 fictional; 34
DeepSeek
2 fact sets × 3 labels, as a user turn, an assistant turn or a tool result; close variants, or other clear cases of the policy drafted by DeepSeek
4
§ 4.1 – 4.2 , App. F
Table 2: Experiments and checks. Fact sets: approve-side and deny-side precedents. Labels: APPROVE, DENY or none on all three precedents. Targets are borderline for DeepSeek unless stated.
Labels on the three precedents
Approve-side facts
Deny-side facts
APPROVE
44.1
73.6
none
33.1
59.2
DENY
5.6
28.7
Table 3: APPROVE labels raise approval and approve-side facts lower it. Top: approval rate (%) of the target by the labels (rows) and facts (columns) of its three precedents; L averages the APPROVE-minus-DENY differences of the two columns; F is the approve-side minus deny-side rate in the unlabeled row. Public-policy experiment, DeepSeek V4.1 Flash, 390 decisions per cell (65 targets × 6 seeds); without precedents it is 55.1%. Bottom: label effect L and facts contrast F in points with 95% intervals. Fictional policies: fictional experiment, 34 targets. Public policies: public-policy experiment (DeepSeek) and Luna replication, 65 targets. –: not run.
Figure 2: For DeepSeek, in every public policy the label effect is positive and the facts contrast negative by point estimate. Per-policy label effect L (filled circles) and facts contrast F (open squares), DeepSeek V4.1 Flash, public-policy experiment (tool-result delivery). Bars are 95% bootstrap intervals that resample seeds within the policy with its targets held fixed, so they show seed variation only. Policies are sorted by L . Each has 5–11 targets and 6 seeds.
Own close variants
Donor’s close variants
Own minus donor
Label effect L
+42.7 [31.5, 53.4]
+9.0 [0.0, 16.5]
+33.7 [21.9, 45.7]
Holm p (policy-level p)
0.0009 (0.0039)
0.0032 (0.0547)
0.0009 (0.0039)
Facts contrast F
− 29.0 [ − 42.8, − 15.2]
− 22.3 [ − 34.8, − 9.3]
− 6.7 [ − 22.5, 8.6]
Holm p (policy-level p)
0.0009 (0.0039)
0.0002 (0.0078)
0.2547 (0.3711)
Table 4: Another request’s precedents lose most of the label effect and keep a facts contrast. Own-and-donor experiment, DeepSeek V4.1 Flash, 65 public-policy targets × 6 seeds, all cells in one run. Points with 95% intervals; below each row, the Holm p within its family (Appendix E ) and, in parentheses, the exact policy-level p.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Condition
r 10%
r 25%
r 50%
t 10%
t 25%
Rule-case accuracy
Direct use
+8.5
+18.3
+30.1
+53.3
+40.2
94.7
Direct use, re-derive instruction
+5.9
+10.8
+25.5
+43.1
+39.2
97.0
Facts without labels
+3.9
+2.9
+10.8
+17.6
+16.7
96.2
Full verification, DeepSeek
− 1.0
− 2.0
+3.9
+7.8
+4.9
97.0
Full verification, Haiku
+3.9
+3.9
+8.8
+8.8
+9.8
96.2
Selective verification, DeepSeek
+2.0
+5.9
+9.8
+10.8
+8.8
94.7
Appendix
Table 5: Deviation from clean-label direct use by condition. Additional analysis. Deviation from clean-label direct use, signed by the direction of the store’s flipped labels (points, point estimates), on the 34 borderline targets by condition and noise level (random r, targeted t, share of the store mislabeled), and accuracy (%) on the 44 rule cases at random 25% noise. Conditions that never see the stored labels are decided once per target and seed, so they vary across columns only through the direction and the share of pairs with s=0 . The no-precedent and facts-only rows show the deviation of conditions that ignore the stored labels. 3 seeds.
LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.