Decision models map stated rules and observed states to discrete answers. Aggregate accuracy leaves unresolved which rule and information changes guide those answers. We connect controlled input changes to transfer, joint reliability, and directed information use through executable arithmetic rules. Semantic rule separation and crossed rule and state tests make transfer and joint success separately measurable. We compare equivalent wordings, calibrate joint correctness through marginal-preserving reassociation, and alter tables to define alternative answer targets. Adaptation gains increase with nearest-source answer agreement; fully conflicting quartets yield small pooled gains and different gain signs across source splits. Equivalent rewrites have opposing task-level effects, linking transfer gains to rule presentation. Joint-score departures depend on which rule and state associations the reference preserves. A decision service changes answers more across equivalent wordings than across identical-text repeats and usually follows the answer targets implied by altered table values or mappings.
Figures & tables
Figure 1: Overview of the three behavioral diagnostics: rule transfer and wording, joint reliability, and external information use. Scores and interventions illustrate the design.
Figure 2
Development rules
Method
Natural
Diag. item
QAcc
Controlled Base
38.67
33.81
0.52
Rank-4 readout
39.11
35.11
0.17
LoRA
58.03
53.13
13.19
Jev
60.44
55.34
22.05
Table 1: LoRA retains higher natural accuracy on development and reserved rules. Scores are percentages. Development averages two rule splits; reserved evaluation uses batch 16, equal weights for reserved blocks, and MLP readouts selected by source-validation NLL without reserved-rule fitting or any checkpoint selection.
Nearest-source agreement groups
Group
Base
LoRA
Gain
QAcc
Natural accuracy
Low
37.06
48.53
+11.48
—
Middle
38.31
57.71
+19.40
—
High
39.06
67.13
+28.07
—
Diagnostic item accuracy and QAcc
Table 2: Transfer depends on source-answer agreement and conflict. Accuracy and LoRA QAcc are percentages; gains are pp. Base QAcc is zero throughout. Agreement groups rank pairs within task/ K ; load counts conflicting answers per intact quartet. Pooled scores weight source splits equally. Fully conflicting cohorts contain 39 and 32 quartet memberships, covering eight and seven task/ K strata. Source splits reuse target questions with potentially different cohort memberships; these finite-cohort scores describe adaptation under source–target conflict.
Figure 2: Equivalent wording changes task-level adaptation gains in opposite directions. Paired changes in the LoRA–Base advantage use batch 8 and 95% shared-state intervals. Zero denotes an unchanged advantage on the same semantically equivalent decision questions.
Figure 3: Jev changes answers more across equivalent wordings than across identical-text repeats. (a) Disagreement rates average items within splits. (b) The paired excess remains positive after interrupted quartets are excluded. Each wording repeats twice; brackets and whiskers give the 95% quartet-resampling intervals.
Method
QAcc
Ref. mean
95% reference range
Base
0.52
1.09
[0.00, 2.60]
Rank-4 readout
0.17
1.05
[0.17, 2.09]
LoRA
13.19
12.21
[10.07, 14.41]
Jev
22.05
21.29
[19.79, 22.92]
Maximum selection only
LoRA
13.19
7.60
[4.17, 10.42]
Table 3: Pooled LoRA and Jev QAcc lie within the marginal-preserving reference ranges, while maximum-selection scores exceed their ranges. Scores are percentages. Central 95% ranges describe 1,000 stratified reassociations that preserve prediction marginals; they summarize the reference distribution rather than confidence intervals for performance on newly sampled rules.
Figure 4: Complete value tables benefit Jev more than the local predictors on these paired inputs. Lines connect missing-value (D3) and complete-value (D2) scores. Endpoint labels use a shared 0–100% accuracy scale.
Jev intervention
Overall T/N (%)
Initially correct T/B (%)
Formula rows
169/220 (76.82)
169/191 (88.48)
Record columns
173/190 (91.05)
173/177 (97.74)
Target values
243/320 (75.94)
243/282 (86.17)
Table 4: Jev usually switches to the altered-table target. Each cell gives transitions over its denominator, followed by the percentage. Overall rates use all conflicts; conditional rates use the initially correct conflicts.
Appendix figures & tables36 assets
Supplementary material from the paper’s appendix.
Appendix
State (x,y,z)
r0
Answer
r1
Answer
(0,−8,0)
0
no / A
8
yes / B
(0,6,−10)
6
yes / B
−16
no / A
Appendix
Table 5: Changing either the rule or the state reverses the binary answer in this evaluated quartet. Exact values determine the gold answers; zero has the answer no.
Same labels after semantic option remapping. Fitting augmentation.
Source natural validation
640
80
Checkpoint, architecture, feature, and regularization selection where those choices are in the protocol.
Source diagnostic validation
320
160
Joint-response reporting and earlier operational qualification. Not the checkpoint-selection objective.
Development natural
640
80
New-rule natural evaluation.
Appendix
Table 6: Training, selection, and evaluation use distinct data roles in each replication cell. Reversed renderings augment training without adding complete states; audit data remain excluded from all model-selection decisions.
Model repository
Frozen revision / response version
Qwen/Qwen3.5-2B-Base
b1485b2fa6dfa1287294f269f5fb618e03d52d7c
Qwen/Qwen3.5-2B
15852e8c16360a2fea060d615a32b45270f8a8fc
Mapika/decider-2b
533964dae8be954c5b5e19fa4948e48408094c1e
startlux-models/StartLux-Decision-2B
3e2e456409a7fe44be69eee231566f7830872ea2
TypeSafe Jev (request alias: jev-latest )
jev-1.13.0 (successful responses)
Appendix
Table 7: Fixed checkpoint revisions and the recorded service version identify the evaluated systems. Jev’s recorded version identifies the service response, while its internal checkpoint and parameter count remain undisclosed.
System
Source rules
Development rules
Natural
Natural
Diagnostic item
QAcc
Qwen Base, native BF16
39.58
34.90
33.07
0.00
Qwen Instruct
37.63
35.03
31.51
3.13
Decider v11
43.10
39.32
38.80
3.13
StartLux 2B, FP32 research path
38.28
38.54
41.67
6.25
Controlled Base
38.15
35.68
33.33
0.00
Appendix
Table 8: Higher item accuracy coexists with lower all-four correctness across the tested systems. Scores are macro-averaged percentages across the tested tasks. Each system retains its interface and training history.
Figure 5: LoRA gains recur across rule splits and state seeds, while joint correctness varies across cells. Left panels retain all development systems; right panels compare source-selected MLPs with LoRA on reserved rules. Numbers give accuracy (%). Natural accuracy uses a shared 0–100 color scale, while QAcc uses a shared 0–30 scale across all development and reserved panels for the corresponding metric, keeping cell comparisons on the same scale.
Base
Rank-4 readout
LoRA
Jev
Split
State seed
Natural
QAcc
Natural
QAcc
Natural
QAcc
Natural
QAcc
1
101
41.15
0.00
38.54
1.04
60.16
16.67
57.42
21.88
1
202
39.58
0.00
37.37
0.00
61.46
13.54
57.55
19.79
1
303
36.72
0.00
40.23
0.00
65.36
13.54
60.03
17.71
2
101
36.46
0.00
37.76
0.00
52.08
7.29
62.11
23.96
2
202
40.10
0.00
39.19
0.00
53.91
18.75
63.02
28.13
Appendix
Table 9: LoRA gains recur across development cells, while the relative ranking of LoRA and Jev changes across rule splits. Scores are percentages, with equal weight assigned to each task family within every evaluated cell.
Split
State seed
Readout natural
Readout QAcc
LoRA natural
LoRA QAcc
Gap (pp)
1
101
40.22
0.00
52.48
7.48
+12.26
1
202
42.69
1.92
53.83
7.37
+11.14
1
303
42.29
0.00
55.49
4.27
+13.20
2
101
42.29
2.03
61.98
5.77
+19.69
2
202
42.53
0.96
57.75
13.57
+15.22
2
303
44.95
3.31
60.96
18.91
+16.01
Appendix
Table 10: Every fixed LoRA model exceeds its selected readout on reserved-rule natural accuracy. Scores are percentages, and gaps are percentage points with equal weights across all of the reserved evaluation rule blocks.
Figure 6: Individual correctness and all-four correctness reveal different task profiles. Jev leads in binary QAcc; LoRA leads in rating and routing and ties Jev in selection. Both panels use the same development questions, task/split weighting, and 0–100% scale; labels give every individual and joint score to one decimal place.
Model
Task
Item accuracy
QAcc
Base
Binary
48.96
2.08
Base
Rating
26.39
0.00
Base
Routing
27.43
0.00
Base
Selection
32.47
0.00
Rank-4 readout
Binary
50.00
0.00
Rank-4 readout
Rating
29.69
0.00
Appendix
Table 11: Task-level item and joint scores distinguish binary strength from performance on ratings and record selection. Percentages give equal weight to option counts, state seeds, and rule splits within each evaluated task.
Figure 7: Partial correctness occupies much more probability mass than all-four success for most systems. Segment widths retain the macro-weighted percentage of quartets with zero to four correct answers; dark blue marks all-four success. Every model and seed is retained, and labels annotate segments with macro weights of at least 10%.
Item accuracy
QAcc
Model
Missing values
Complete values
Missing values
Complete values
Base
35.72
40.19
0.00
2.08
Rank-4 readout
35.81
39.28
0.00
3.13
LoRA
49.48
52.95
7.47
11.98
Jev
54.43
90.63
20.31
75.69
Appendix
Table 12: Complete values improve joint correctness most for Jev on the paired development inputs. Scores are percentages under the same task, option-count, state-seed, and rule-split macro weighting used in Figure 4 .
Figure 8: Answer binding depends on context and instructions even when the correct semantic answer is supplied. Direct instructions raise local-model accuracy, while Jev performs best with minimal context and legacy instructions. Numbers give accuracy (%) using a single 0–100 sequential color scale shared across all systems, context lengths, and instruction conditions.
Figure 9: LoRA exceeds the prospectively fixed 5 pp margin over the selected readout in every reserved block. Bars begin at zero and labels give the natural-accuracy difference (pp); the dashed line marks the decision margin.
Figure 10: Probe gains over shuffled targets are stronger on the source audit than on development inputs at many LoRA locations. Numbers give real minus shuffled-target accuracy (pp) for value and answer targets at every tested location. The diverging scale is centered at zero and shared across every evaluated layer and model panel.
Figure 11: Source-selected patches produce little change in counterfactual target success relative to the native baseline. Numbers retain absolute success (%); shading shows the within-column change from the native row (pp) on a shared, zero-centered scale. Audit and development retain all of the positive and negative controls.
Task
Model
QAcc
Ref. mean
Ref. 95% range
Obs. minus ref.
binary decision
Base
2.08
4.29
[0.00, 10.42]
-2.20
binary decision
Jev
66.67
66.55
[62.50, 70.83]
0.11
binary decision
LoRA
22.92
27.90
[20.83, 35.42]
-4.98
minimum routing
Base
0.00
0.07
[0.00, 0.69]
-0.07
minimum routing
Jev
3.47
3.00
[0.69, 5.56]
0.47
minimum routing
LoRA
9.03
5.90
[3.45, 9.03]
3.13
Appendix
Table 13: Joint-score departures depend on task and on the associations retained by reassociation. Task-specific QAcc is compared with 1,000-draw reference ranges; pooled rule-pair and state-vector controls use 2,000 draws. Scores are percentages. The reported ranges describe the reference distribution for each set of preserved associations.
Figure 12: Validation trajectories approach the frozen readout at selected early steps, but independent audit gaps remain. Curves show saved LoRA–readout gaps (pp); dotted vertical lines mark selected steps and the horizontal zero line denotes equality. Below-panel labels give audit natural-accuracy / QAcc gaps at those selected steps.
Model
Intervention
N
B
T
T/N (%)
T/B (%)
Base
Formula-row mismatch
220
69
8
3.64
11.59
Base
Record-column mismatch
190
66
5
2.63
7.58
Base
Target-value counterfactual
320
116
12
3.75
10.34
LoRA
Formula-row mismatch
220
90
17
7.73
18.89
LoRA
Record-column mismatch
190
84
21
11.05
25.00
LoRA
Target-value counterfactual
320
147
19
5.94
12.93
Appendix
Table 14: Directed answer transitions distinguish following a supplied target from an undirected answer change. Counts preserve the applicable conflict set: T/N uses all conflicts and T/B conditions on original correctness.
Metric
Models
Original
Rewrite 1
Rewrite 2
Change 1 [95% CI]
Change 2 [95% CI]
Natural
All six
19.30
16.66
19.21
-2.64 [-4.80, -0.62]
-0.09 [-1.76, 1.62]
Natural
Split 1
16.10
15.15
16.87
-0.95 [-3.10, 1.10]
0.77 [-0.87, 2.51]
Natural
Split 2
22.50
18.18
21.55
-4.33 [-6.64, -2.20]
-0.95 [-2.64, 0.80]
QAcc
All six
9.99
5.84
9.44
-4.15 [-5.65, -2.80]
-0.55 [-1.44, 0.30]
QAcc
Split 1
6.37
4.88
6.16
-1.50 [-3.13, 0.11]
-0.21 [-1.28, 0.89]
QAcc
Split 2
13.60
6.80
12.71
-6.80 [-9.44, -4.31]
-0.89 [-2.17, 0.39]
Appendix
Table 15: Functional rewrites reduce the pooled adaptation advantage; verbal rewrites produce little pooled natural-accuracy change. All entries compare LoRA with Base at batch 8. Changes are paired differences (pp); 95% intervals condition on the fixed rule allocations and model checkpoints under shared, paired input resampling.
Figure 13: LoRA remains above Base across equivalent wordings, with a larger reduction under functional notation. Points connect evaluations of fixed systems at batch 8; labels give accuracy (%). Natural accuracy and QAcc use separately labeled zero-based scales. LoRA averages six instances, and 9B is a distinct instruction-tuned system.
Task
Model
Original
Rewrite 1
Rewrite 2
Change 1 [95% CI]
Change 2 [95% CI]
Binary
Base
69.95
50.96
58.17
-18.99 [-24.77, -12.74]
-11.78 [-17.07, -6.25]
Binary
LoRA
77.72
73.60
78.04
-4.13 [-5.77, -2.48]
0.32 [-0.48, 1.16]
Routing
Base
27.64
27.48
29.25
-0.16 [-3.77, 3.37]
1.60 [-0.88, 4.09]
Routing
LoRA
44.87
38.29
41.57
-6.58 [-7.71, -5.46]
-3.30 [-4.22, -2.34]
Rating
Base
22.84
23.48
24.20
0.64 [-0.88, 2.09]
1.36 [-0.24, 2.96]
Rating
LoRA
54.91
48.33
53.15
-6.58 [-7.76, -5.46]
-1.76 [-2.60, -0.95]
Appendix
Table 16: Binary-task increases offset adaptation-gain reductions in rating, routing, and selection. Natural accuracy is reported in percentages and paired changes in pp, at batch 8 with complete-state resampling intervals.
Same-text padding changes
Wording disagreement at fixed padding, FP32
Model
BF16
FP32
Functional (%, 95% CI)
Verbal (%, 95% CI)
Base
26/960
0/960
44.27 [34.38, 54.69]
31.25 [23.44, 39.06]
LoRA, split 1
11/960
0/960
23.44 [16.67, 30.21]
12.50 [8.33, 16.67]
LoRA, split 2
8/960
1/960
25.00 [19.27, 31.25]
17.71 [12.50, 22.92]
Appendix
Table 17: Wording sensitivity persists with FP32 inference and fixed padding on the controlled subset. Wording-flip rates compare each rewrite with the original on natural questions; same-text padding comparisons retain the wording and measure changes in the selected answer under a different padding condition on identical question inputs.
Mapping
Split
Conflict n
Mass (%)
Conditional weighted
Conditional subset
Contribution conflict
Contribution other
native
1
1907
43.73
-2.43
-3.11
-1.06
16.93
native
2
1722
38.90
12.62
11.37
4.91
17.26
permuted
1
1595
35.96
1.91
1.70
0.69
15.18
permuted
2
1454
32.97
14.74
13.70
4.86
17.31
Appendix
Table 18: Nearest-source nonconflicts account for most of the full-set adaptation gain. Conditional gains and weighted contributions are in pp. Weighted estimates retain full-set conflict mass; subset estimates recompute the task/ K means on retained questions before averaging scores equally across strata within each task family.
Mapping
Split
Shared conflict n
LoRA–Base (pp)
native
1
918
-0.77
native
2
918
10.08
permuted
1
727
7.11
permuted
2
727
10.12
Appendix
Table 19: The common conflict intersection permits both source splits to be compared on identical questions. All ten task/ K strata remain represented; percentages use the subset-normalized task and option-count macro.
Natural questions
Split
Agreement
Questions
Base
LoRA
Gain
95% gain CI
Conflict (%)
1
Low
1600
37.03
44.29
+7.26
[4.29, 10.31]
52.50
1
Middle
1280
39.26
54.82
+15.56
[12.48, 18.55]
41.67
1
High
1280
38.15
65.10
+26.95
[23.96, 29.86]
34.83
2
Low
1600
37.08
52.78
+15.69
[12.93, 18.33]
46.82
2
Middle
1280
37.37
60.61
+23.24
[20.42, 26.24]
39.84
Appendix
Table 20: Low-agreement pairs retain positive pooled transfer gains but lower joint correctness than high-agreement pairs. Accuracy and QAcc are percentages; gains and their intervals are percentage points. Diagnostic Base and LoRA columns give item accuracy, and QAcc gives LoRA all-four correctness. Base QAcc is zero in every group. Pooled counts retain both source-reference memberships for questions shared across the two source pools.
Split
Source–target answers
Questions
Weight (%)
Base
LoRA
Readout
Gain (pp)
1
Conflict
1907
43.73
34.49
32.07
34.02
-2.43
1
Agree
2253
56.27
40.83
70.93
47.73
+30.09
2
Conflict
1722
38.90
31.10
43.72
34.50
+12.62
2
Agree
2438
61.10
42.49
70.74
48.83
+28.25
Appendix
Table 21: Conflict-conditioned absolute accuracy exposes different transfer outcomes across the two source splits. Scores are percentages; gains compare LoRA with Base under the original weights conditioned on each subset. The readout column reports the MLP selected separately using the validation inputs of each source-rule split.
Split
Load
Quartets
Task/ K
Weight (%)
Base
LoRA
Gain (pp)
QAcc
1
0
37
7
13.14
32.93
47.29
+14.36
4.88
1
1
19
4
11.22
46.43
58.33
+11.90
14.29
1
2
129
10
50.32
39.17
48.89
+9.71
7.15
1
3
30
4
11.75
24.09
41.74
+17.65
4.55
1
4
39
8
13.57
20.47
14.37
-6.10
0.00
2
0
46
7
17.74
36.90
59.69
+22.79
23.09
Appendix
Table 22: All-four conflict yields little pooled accuracy gain and low joint correctness. Load counts conflicting members per complete quartet. Base and LoRA columns give diagnostic item accuracy; QAcc gives LoRA all-four correctness. Base QAcc is zero throughout. Weight is original diagnostic mass, and Task/ K counts represented strata, listed by source split in pooled rows. Pooled counts reuse the same quartets under two source references.
Actual start gap
Within pairs
Disagree (%)
Across pairs
Disagree (%)
<2 s
523
9.37
1991
35.26
2–4 s
324
9.88
1363
35.95
4–8 s
105
6.67
452
34.51
8–16 s
0
N/A
2
100.00
≥16 s
8
12.50
32
37.50
Appendix
Table 23: Cross-wording disagreement exceeds same-text disagreement in every elapsed-time stratum with both comparisons. The 8–16 s stratum has no same-text pairs; the counts identify the available paired coverage.
Priority policy
Exception policy
Finite-state transitions
System
Natural
QAcc
Natural
QAcc
Natural
QAcc
Base
60.00
0.00
31.87
0.00
25.31
0.00
Arithmetic LoRA
51.88
0.00
43.12
2.50
21.88
0.00
Qwen3.5-9B
78.12
22.50
76.25
2.50
32.81
0.00
Always deny
58.13
0.00
78.13
0.00
Not applicable
Appendix
Table 24: The tested systems show different competence across Boolean policies and finite-state transitions. All systems have zero transition-task QAcc. Scores are descriptive finite-set percentages; the arithmetic LoRA is reused without new-task training and the always-deny baseline applies to both evaluated Boolean policy rule families.
Task
Model
Strata
Size range
Singletons
Weight (%)
Constant
Weight (%)
Binary
Base
24
8–8
0
0.00
0
0.00
Binary
Jev
24
8–8
0
0.00
12
50.00
Binary
LoRA
24
8–8
0
0.00
5
20.83
Routing
Base
262
1–6
102
17.71
157
47.05
Routing
Jev
262
1–6
102
17.71
130
30.73
Routing
LoRA
262
1–6
102
17.71
129
29.34
Appendix
Table 25: Small and constant-prediction strata constrain the variation available to reassociation. Counts pool disjoint strata across six cells; weights average cell-level task macro weights. Constant counts include singletons.
Split/state
K
LoRA QAcc
LoRA dˉ [ q.025,q.975 ]
Jev QAcc
Jev dˉ [ q.025,q.975 ]
1/101
2
50.00
+21.48 [+0.00, +37.50]
25.00
+2.75 [-12.50, +25.00]
1/101
4
0.00
+0.00 [+0.00, +0.00]
0.00
+0.00 [+0.00, +0.00]
1/101
8
0.00
+0.00 [+0.00, +0.00]
12.50
+9.09 [+0.00, +12.50]
1/202
2
25.00
+11.99 [-12.50, +25.00]
12.50
+8.77 [+0.00, +12.50]
1/202
4
0.00
-2.69 [-12.50, +0.00]
12.50
+10.86 [+0.00, +12.50]
1/202
8
0.00
+0.00 [+0.00, +0.00]
12.50
+10.90 [+0.00, +12.50]
Appendix
Table 26: Maximum-selection departures vary across cells and option counts. QAcc is in percentages and mean differences dˉ in pp; brackets give quantiles of db , with eight complete diagnostic quartets in every cell/ K stratum.
Intervention
Micro
Task/ K macro
Equal cell
T/N
T/B
T/N
T/B
T/N
T/B
Formula rows
76.82
88.48
78.81
86.47
79.18
86.58
Record columns
91.05
97.74
91.10
97.46
91.08
97.41
Target values
75.94
86.17
77.34
84.62
77.34
84.35
Appendix
Table 27: Jev's directed transitions persist under micro, task-balanced, and equal-cell weighting. All weighting schemes reuse the same paired requests and archived integer transition counts, changing their aggregation rather than the observed answers or the conflict questions entering the archived counts of paired answer transitions.
Figure 14: The fixed native chat template lowers 9B natural accuracy under all three wordings. (a) Lines connect raw and native scores on paired inputs. (b) Paired native–raw differences are negative, with 95% intervals from resampling the 13 reserved blocks. The score axis in (a) is restricted to 29–49% for resolving paired changes.
Figure 15: The original native instruction asks for the minimum-scoring record and specifies the tie-break. The evaluated native request string is retained exactly.
Figure 16: The complete-value request supplies scores for every rule while retaining the decision task. The displayed strings preserve the evaluated request, including its instruction, records, formula labels, and numeric values.
Evaluation
Controlled factor
Evaluation setting
CheckList / contrast sets
Local input changes
Behavioral and contrastive tests
LogicBench
Logical inference patterns
Three kinds of logic
RuleArena
Rules and tool assistance
Real-world regulations
CFQ / COGS / Ordered CommonGen
Combinations and concept order
Compositional generalization
Mind Your Format / symbol binding
Templates and option symbols
Answer-interface variation
Cacheable decision models
Joint versus independent encoding
Rule-following decision scores
Appendix
Table 28: Controlled input changes provide different kinds of evidence across evaluation designs. Our oracle-checked setting crosses rules and states and intervenes on equivalent wording and supplied-value mappings.
Certo is a small non-generative decision model (Qwen3-4B): it scores candidate actions from their text and returns a probability, instead of generating an answer. The accurate design reads the state, the rules, and each candidate together (a joint scorer), so cost grows with the menu. Independent encoding lets each candidate be encoded once and reused across states (about 5x cheaper at 77 candidates), but separates state from candidate. We ask how much rule-sensitivity survives that move, and whether it can be trained back. Four experiments on Certo: (1) the tested conversion to cacheable scoring loses rule-sensitivity (recall@1 1.00 -> 0.24) while the joint scorer holds 1.00, and a shortlist+rerank rescue fails; (2) targeted counterfactual supervision restores strong performance on held-out synthetic rule tasks (paraphrase, counterfactual, composition; reproducible across seeds), though we do not isolate whether predictions depend on the supplied rule; (3) on real rules the added benefit is not established -- after fixing a truncation confound, the joint scorer wins significantly on the short tier (0.861 vs 0.500) and directionally on the hard tier (0.655 vs 0.483, n=29); (4) a matched cross-domain real-prose mixture did not help and reduced contract accuracy (-9.3, -16.2 points). A cacheable encoder can be made rule-sensitive on its training distribution, but transfer to unseen-source real rules is not established; the joint scorer keeps an edge at the cost of caching.
Typed decision models return structured results, but output-type correctness alone does not ensure that decisions follow explicit option definitions. Each option pairs a name with a definition that defines its intended meaning; the name, however, can provide a competing semantic cue. We study this conflict in Jev and two open-weight models by changing only the name-definition mapping, leaving the question, state, and the names and definition texts themselves unchanged. We measure decision flips at the level of the selected definition, rather than the returned name. On 1200 decision tasks with task-specific definitions, decision-flip rates are up to 70.4 pp higher with yes/no names than with the 0/1 control. This gap holds across all 4 binary decision rules. With yes/no names, reassignment also lowers their mean AUC from 93.8% to a below-chance 23.2%. In the binary evaluations, random strings used as option names yield mean flip rates close to those of neutral controls across all three models, with comparable balanced accuracy before reassignment. Together, these results support option-name polarity as a contributor to decision instability beyond reassignment alone. The type-error rate remains 0% throughout, showing that type-correct outputs can still fail to follow explicit option definitions.
Yu Sun, Junhao Xu, Jiajia Shi +1
Fudan University · University of Science and Technology of China
An ordinary-looking background detail can turn a correct model decision into a confident mistake. We demonstrate this fragility in four decision systems, including Jev, across seven datasets covering knowledge, reasoning, and tool routing. Within 64 accepted target evaluations per decision, we uncover short context additions that redirect 61.4%-73.2% of each system's initially correct decisions toward a wrong option fixed in advance. The additions supply background or procedural information rather than explicit answer-selection instructions, leaving the original question and choices intact. We construct them through probability-guided context optimization, which uses shifts in the option distribution to refine surrounding text under naturalness and answer-preservation constraints. Redirection affects initially confident decisions, often produces high-confidence wrong choices, and transfers across models. In blinded human evaluation, 91.6% of 250 sampled successful contexts are judged natural, answer-preserving, and free of decisive answer-changing evidence by a majority of three independent annotators. These findings expose a weakness in current decision models: context that looks entirely compatible with an input can redirect the choices that agents, routers, and evaluators rely on.