Retaining Repair Hypotheses for Cost-Aware Execution
Authors: Jae Hun Hwang, Hong-In Won
Organizations: Manufacturing AI Research Center, Korea Institute of Industrial Technology, Incheon, Republic of Korea · Department of Artificial Intelligence Convergence, Hanyang University, Republic of Korea · HYU-KITECH Joint Department, Graduate School, Hanyang University, Seoul, Republic of Korea
Several corrections of an executable model can fit the same observations while predicting different execution outcomes. We study a repair-to-execution interface that retains compatible finite guard repairs, conditions their weights on diagnostic and ordinary-execution observations, and uses their predictions to choose routes, priced tests, or a backup. On 24 synthetic two-stage devices, an equal mixture of a language-model-generated repair and a uniform belief over compatible repairs (the AI-weighted belief) lowers no-test cost by 2.14 percent relative to committing to the generated repair. Access charges can erase the benefit of diagnosis on isolated tasks, while repeated work can recover it under the AI-weighted belief. With normal-execution learning in every repeated-work arm, workload-valued testing costs 3.33 percent less than same-budget random selection under the AI-weighted belief. Weighting changes the comparison: uniform normal learning costs 540.75 less than AI-weighted normal learning on this cohort, with a device-paired interval excluding zero. Testing recovers part of that gap, but uniform normal learning still has a lower aggregate cost than AI-weighted workload-valued selection. Under uniform beliefs, information-gain selection has the lowest total among the compared selectors, while workload-valued testing costs more than normal learning. These results support retaining alternatives to a generated repair and evaluating diagnostic selection together with repair weighting and learning from execution.
Figures & tables
Figure 1: Compatible repairs turn model ambiguity into route, test and backup choices. (a) Illustrative synthetic device. (b) Repeated-work decision loop: tests and ordinary execution update the retained belief. Access-charged stage-B tests include reset (0.25), stage-A setup and query.
STRIPS preconditions and effects of a planning domain; grounded or lifted repairs
Consistency with partially observed traces or with positive and negative plans; repair sets are diagnoses, and a modeler may judge alternative repairs iteratively [ 2 ]
Recovered model or cardinality-minimal or optimal repair; model-quality measures, plan validity, and repair benchmarks. Execution-time costs are not the evaluated quantity
Replay data and an optimistic goal-reachability constraint [ 6 ] ; program-search values reflect predictive accuracy on stored transitions [ 7 ] ; acting supplies further data
Program that fits data and supports planning; GIF-MCTS compares with an earlier WorldCoder version [ 7 ]
Hypothesis revision with experiments: LLM-SMC-S [ 8 ]
Weighted natural-language rule particles revised by a language model
Bayesian posterior over particles; experiments chosen by information gain
Posterior prediction and informative experiments over language hypotheses; not costed route execution
This study
Finite executable guard repairs, as truth-table masks or syntax-counted disjunctive normal form (DNF) programs
Compatible repairs weighted by uniform, MDL-weighted, or AI-weighted mass and conditioned on every observation; predictions enter route, test, and backup choices; normal execution also updates the belief
Total task cost with priced tests, reset and setup access, and backup, on a fixed synthetic cohort with matched-belief controls
Table 1: Methodological comparison drawn from the cited sources; the last column gives the objective each study evaluated.
Figure 2: Repair-to-execution mechanism for one recorded task (device 5 of Figure 4, menu 7). Candidate h predicts query outcome y; bars show weights before and after acceptance (navy marks the AI repair). Observed acceptance conditions the belief and changes expected route costs; stars mark the chosen alternatives before (grey, Backup) and after the observation (green, (5,11)); × marks a candidate eliminated by the accepted observation. Panel (d) gives realised abstract task cost (backup cost 8), including the cheaper uniform no-test alternative. The query uses direct, unit-cost stage-B access. Routes are context pairs (qA,qB) , indexed 0–15. Stage A retains 20 candidates throughout this local stage-B test.
Decision rule
Cost
First-route success
Tests
AI point repair
9227.25
681
0
AI-weighted, no test
9029.75
718
0
AI-weighted, test cost 0.25
8322.5
899
1815
AI-weighted, test cost 1
9018.75
745
139
Uniform, no test
9263.75
737
0
Uniform, test cost 0.25
8377.75
909
2308
Table 2: Small-grammar results over 1,536 menu decisions.
Diagnostic condition
Comparator
Total difference
Mean difference, 95% interval
No test
Uniform
-234.00
-9.75 [-26.43, 4.54]
No test
MDL-weighted belief
-267.50
-11.15 [-41.31, 7.88]
Test cost 0.25
Uniform
-55.25
-2.30 [-6.81, 3.51]
Test cost 0.25
MDL-weighted belief
-24.25
-1.01 [-2.84, 0.00]
Test cost 1
Uniform
-112.75
-4.70 [-14.78, 5.44]
Test cost 1
MDL-weighted belief
-115.00
-4.79 [-16.20, 2.40]
Table 3: AI-mixture cost minus the indicated belief, aggregated over the same 24 devices. Negative totals favor AI. Intervals summarize the mean device difference using 20,000 paired resamples; see Section 4 for their interpretation.
Figure 3: Diagnostic cost differences from the matching no-test policy. Panel (a) aggregates 1,536 separately initialized menu decisions on 24 devices; panel (b) shows cumulative differences over 64 sequential jobs, with normal-execution updates and at most four access-charged tests per device (reset and setup included). Panel (a) lines join three tested cost levels. Negative values indicate lower total cost. Curves are observed cohort aggregates, without uncertainty intervals. Colors in (b) follow (a); the initial positive difference includes diagnostic spending before the first job.
Decision rule
Cost
First-route success
Tests
AI-weighted, normal learning
8500.25
762
0
AI-weighted, workload-valued selection
8069.25
885
96
MDL-weighted belief, normal learning
8502.5
761
0
MDL-weighted belief, workload-valued selection
8070.5
885
96
Uniform, normal learning
7959.5
886
0
Uniform, workload-valued selection
7977.75
913
96
Table 4: Repeated work, 24 devices, 64 jobs each.
Figure 4: Device-paired total-cost differences for the fixed cohort. Each dot is one device’s total over 64 menus: separately initialized decisions in (a,b), direct tests at cost 0.25 in (b), and sequential work with access-charged tests in (c). Negative values favor the first policy. The dashed line is the sample mean. The three panels show mixture versus point repair, cheap tests versus the matching no-test mixture, and workload-valued selection versus the matching AI-weighted normal-learning baseline. Each device index identifies the same device in all three panels.
Decision rule
Cost
First-route success
AI-selected history, point repair
3106.25
42
AI-selected history, MDL repair
3081.625
49
AI-selected history, full syntax-counted belief
3054.625
51
Exact syntactic-EIG history, full syntax-counted belief
3061.125
44
Immediate backup
3072
0
Table 5: Larger grammar, 384 menu decisions, execution cost only.
Belief and selector
Total cost
Diagnostic cost
First-route success
AI-weighted, normal learning
8500.25
0
762
AI-weighted, workload-valued selection
8069.25
224.5
885
AI-weighted, information gain
8216.25
218
859
AI-weighted, random (mean)
8346.97
228.09
834.13
Uniform, normal learning
7959.5
0
886
Uniform, workload-valued selection
7977.75
236.75
913
Table 6: Matched-budget selection on the 24-device, 64-job cohort.
Figure 5: Absolute total cost of all eight belief–selector combinations on the fixed 24-device cohort. Circles show deterministic policies; squares and whiskers show random-selector means and ranges across 32 seeds. The dashed reference is uniform normal learning. All marks include test, setup, reset, execution, and fallback costs.