Governance Records as Supervision: Verifier-Selected Self-Training for Structured Workflow Repair

Jesus Salas

cs.AIAug 18, 2026

Governance Records as Supervision: Verifier-Selected Self-Training for Structured Workflow Repair

Authors: Jesus Salas

Organizations: Microsoft

Abstract

Machine-verifiable workflows produce governance records linking a task contract, model attempt, verifier decision, accepted output, and target origin. We test whether verifier-admitted outputs can supervise a bounded model by consolidating occasional or expensive capability into reliable one-shot execution. On fresh, structure-disjoint PlanBench replanning cases, Qwen3-14B thinking produced 24 plans admitted by independently authored VAL. They trained the same checkpoint for non-thinking execution, without oracle targets or a stronger teacher. VAL acceptance rose from 1/80 to 57/80. A prospective replication held targets, model revision, recipe, and evaluation corpus fixed across eight LoRA seeds and three inference realizations per seed. Every seed produced a clear lift: adapters reached 45/80 to 70/80 against 1/80 for every matched base report; the exact seed-level sign-flip test gave p=0.0078125. Target-selection performance was less stable. An initial matched seed gave 102/160 accepted plans after VAL selection versus 69/160 after blinded model self-selection. Across eight prospective seeds, the contrast was seed-dependent, included one clear reverse seed, and did not replicate (p=0.3672). VAL also had a positive descriptive aggregate over schema-only selection but failed its preregistered seed-level reliability gate (p=0.0703). The verifier remains the admission authority; no reliable downstream capability advantage of semantic selection is established. A complementary Phi arm supports stronger-teacher distillation. Earlier synthetic studies bound teachability, cumulative learning, transfer, and stopping. The evidence supports robust consolidation of one fixed, machine-checkable capability, not arbitrary planning, enterprise validity, or unrestricted self-improvement.

Figures & tables

Path	Target producer	Training recipient	Target provenance	Frozen reading
Qwen	Qwen3-14B thinking	Same checkpoint, non-thinking	No oracle; no stronger teacher	Primary lift replicated
Phi	Phi-4-reasoning	Separate base Phi-4	No oracle; stronger teacher	Full-band distillation

Table 1: The two external paths share the governance-selected pipeline but not the same causal classification.

Figure 1: Fresh external outcomes on the shared 80-case PlanBench evaluation. The Qwen path is teacher-free same-checkpoint self-distillation. The Phi path uses a separate stronger reasoning teacher. Exact counts are printed above the bars.

Figure 2: Prospective replication of the primary teacher-free Self-24 intervention. Filled points are per-seed means and open points are three frozen inference realizations. The base result is 1/80 in every matched report. The training seed, not the repeated case, is the inferential unit.

Figure 3: Selector outcomes across eight prospective training seeds. The left panel is a descriptive pooled ladder. The right panel shows each seed’s mean paired net across three inference realizations; both sets of seed-level contrasts cross zero and neither passes its preregistered seed-level reliability test. The original base result on this 160-case corpus was 1/160, measured at one training seed and one inference realization; it is excluded from the common eight-seed ladder.

Figure 4: Verified performance versus mean request latency on the external evaluation. Arrows connect the expensive target-producing regime to the cheap trained specialist. The source-versus-specialist semantic comparisons are descriptive compression diagnostics, not preregistered superiority tests.

Construction and partition	Base	First adapter	Second adapter	Later adapter
Generator A, initial	10/48	21/48	n/a	n/a
Generator A, later	9/48	21/48	33/48	35/48
Generator B, Qwen	12/80	44/80	62/80	n/a
Generator C, Qwen	57/128	128/128	126/128	n/a
Generator B, Phi self-training	10/80	77/80	n/a	n/a

Table 2: Fresh synthetic one-response repair after training only on model- generated, verifier-selected targets. The two Generator A rows use different case-disjoint evaluation partitions. The later Generator A step does not clear its frozen materiality gate.

Figure 5: Qwen self-sourced one-response learning across three synthetic constructions. Generator C’s high base rate and first-round saturation limit its evidence about iteration.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Harvest measure	Observed	Frozen gate
Distinct VAL-admitted targets	24/32	at least 16
First-attempt successes	13/32	reported
Successes within eight attempts	24/32	reported
Beyond-first-attempt headroom	11	at least 8
Completed HTTP requests	118/118	at least 98%
Completion tokens	419,201	at most 2,500,000

Appendix

Table 3: Formal Qwen target acquisition on the fresh harvest partition.

Qwen arm	VAL accepted	Schema-valid	Mean latency	Completion tokens
Base, non-thinking	1/80	71/80	0.624 s	1,729
Base, thinking	30/80	77/80	65.964 s	254,960
Self-24, non-thinking	57/80	80/80	1.186 s	2,965

Appendix

Table 4: Fresh Qwen evaluation on the external 80-case partition. Token counts are totals across each arm.

Selector	VAL-valid targets	Train time	Final loss
VAL	24/24	64.1 s	0.3545
Qwen self-selection	20/24	64.2 s	0.3905
Schema-only frozen order	15/24	64.0 s	0.5077

Appendix

Table 5: Matched target sets and training outcomes. Only target selection changes across arms.

Evaluation arm	VAL accepted	Schema-valid	Mean latency
Base, non-thinking	1/160	129/160	1.395 s
Schema-selected adapter	55/160	152/160	1.410 s
Model-self-selected adapter	69/160	159/160	1.385 s
VAL-selected adapter	102/160	160/160	1.202 s

Appendix

Table 6: Fresh outcomes for the matched target-selection ablation.

Training seed	Adapter reports	Base reports	Mean net	Reading
2026113002	59, 61, 59	1, 1, 1	$+58.67$	clear lift
2026113003	57, 59, 57	1, 1, 1	$+56.67$	clear lift
2026113004	45, 47, 45	1, 1, 1	$+44.67$	clear lift
2026113005	51, 52, 51	1, 1, 1	$+50.33$	clear lift
2026113006	59, 58, 57	1, 1, 1	$+57.00$	clear lift
2026113007	65, 65, 66	1, 1, 1	$+64.33$	clear lift

Appendix

Table 7: Accepted plans out of 80 for the eight prospective Self-24 training seeds. Each report column contains the three frozen inference realizations.

Seed	Schema	Model self	VAL	VAL $-$ self	VAL $-$ schema
2026120305	102	123	190	$+67$	$+88$
2026120306	164	120	141	$+21$	$-23$
2026120307	129	248	288	$+40$	$+159$
2026120308	147	151	118	$-33$	$-29$
2026120309	110	197	226	$+29$	$+116$
2026120310	146	215	149	$-66$	$+3$

Appendix

Table 8: Prospective common-seed selector results. Totals are descriptive; exact tests use the eight training-seed effects rather than pooled cases.

Phi arm	VAL accepted	Schema-valid	Mean latency	Completion tokens
Base Phi-4	2/80	35/80	1.027 s	3,362
Phi-4-reasoning	30/80	41/80	150.283 s	544,408
Distilled Phi-4	51/80	80/80	0.981 s	2,531

Appendix

Table 9: Fresh Phi evaluation on the same external 80-case partition. Token counts are totals across each arm.

Stage	Model	Training-target source	Fresh evaluation	Evidentiary role
Oracle precursor	Phi-4, Qwen3-14B	Procedural schedule, verifier-confirmed	144 rows per model	Teachability and signal attribution
Generator A	Qwen3-14B	Same-model accepted samples	48-row one-response and 32-row search pools	Bounded loop, cumulative learning, stopping
Generator B	Qwen3-14B	Same-model accepted samples	80-row one-response and 32-row search pools	Native construction replication and transfer boundary
Generator C	Qwen3-14B	Same-model accepted samples	128-row one-response and 32-row search pools	Construction-separated first-round replication
Generator B	Phi-4	Same-model accepted samples	80-row one-response and 32-row search pools	Cross-model method replication

Appendix

Table 10: Synthetic evidence map. All evaluation partitions are case-disjoint from their training targets. Generator separation is by implementation, not by independent authorship.

Training state	Inference feedback	Pass	Transport	Compound
Base	Localized	31/144	128/144	5/48
Localized-trained	Localized	139/144	144/144	45/48
Base	Verdict-only	35/144	135/144	9/48
Localized-trained	Verdict-only	115/144	144/144	33/48
Verdict-only-trained	Localized	133/144	144/144	41/48
Verdict-only-trained	Verdict-only	136/144	144/144	44/48

Appendix

Table 11: Qwen training and inference feedback cross. The primary causal comparison fixes localized inference and changes only the training trace.

Search condition	Accepted	Pass@1	Later	Calls	Transport
Fixed generic	12/32	8/32	4	678	678/678
Fixed localized	16/32	10/32	6	591	590/591
Adaptive localized	16/32	11/32	5	539	539/539

Appendix

Table 12: Target-hidden search attribution. “Later” counts targets first accepted after at least one verifier rejection.

Construction and model	Earlier arm	Later arm	Paired reading
A, Qwen initial	Base 10/48	Self-16 21/48	14 gains, 3 losses, $p=0.012726$
A, Qwen Round 2	Self-16 25/48	Default30 35/48	Net $+10$ , $p=0.021271$
A, Qwen Round 3	Default30 33/48	Round3 35/48	3 gains, 1 loss, $p=0.625$ ; stop
B, Qwen Round 1	Base 12/80	B-Round1 44/80	Net $+32$ , $p=4.07\times 10^{-9}$
B, Qwen Round 2	B-Round1 44/80	B-Round2 62/80	Net $+18$ , $p=4.01\times 10^{-5}$
C, Qwen final	Base 57/128	C-Round2 126/128	69 gains, 0 losses, $p=3.388\times 10^{-21}$

Appendix

Table 13: Fresh one-response transitions from verifier-selected self-training. The Generator A rows use separate case-disjoint partitions. Generator C Round 1 reaches 128/128; its final Round 2 arm reaches 126/128 and does not improve on the saturated first-round measurement.

Figure 6: First-attempt and 32-attempt success on matched pools. Panels use different case sets and are not pooled. Training expands fixed-budget reachability through Default30 on Generator A and Round 1 on Generator B; Generator C saturates the frozen frontier after Round 1.

Figure 7: Zero-shot transfer from Generator A adapters to Generator B. Default30 passes the aggregate gate but crosses the frozen family-regression guard. Round3 is a predeclared secondary trajectory arm and cannot replace the primary treatment after results are opened.

Self-Training for Language Models

Self-training for language models uses their own generations, feedback, and interaction traces as supervision to improve capabilities with less labeled data.

See all 83 papers

Jun 18, 2026

Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning

Test-time reasoning is increasingly used as a serving-time control knob, but extra reasoning is not uniformly valuable: it can repair failed attempts, waste compute on already-correct answers, or introduce harmful answer changes. We study this as a deployment allocation problem rather than a new-verifier problem. We introduce \sevra, Selective Verification for Reasoning Allocation, a serving-layer controller that decides whether to preserve a frozen solver's initial answer or invoke active verification. Using a frozen Qwen3-4B solver, we log intervention outcomes and train recoverability-aware gates from serving-visible attempt state. On \mathfive, selective verification reaches 76.3% accuracy, compared with 75.5% for always verifying, while reducing post-generation tokens by 26.8% and harmful flips from 2.2% to 1.0%. However, an 8,192-token initial solve reaches 76.0% accuracy with 28% fewer total model tokens, showing that selective recovery is useful but not the best tested cost frontier. In frozen transfer to \gsm, the selective policy verifies only 3.0% of examples, improves accuracy from 93.4% to 94.5%, and reduces verification tokens by 91.2% relative to always verifying; again, a longer initial solve matches its accuracy with fewer realized tokens. On CommonsenseQA, always-on verification hurts, while Self-Consistency@5 improves accuracy at about five times the realized token cost. The resulting deployment rule is: tune the initial budget first, then use selective recovery when explicit checks, bounded retries, auditability, or regression-risk control matter.

Sajib Acharjee Dip, Dawei Zhou, Liqing Zhang

Department of Computer Science, Virginia Tech · Fralin Biomedical Research Institute, Virginia Tech · FBRI Cancer Research Center, Washington, DC

LLM Inference Efficiency Cost-Aware Inference Reasoning Verification

Ask PDF

Jul 30, 2026

SVR: Self-Verifying Refinement via Joint Verdict-Confidence Reinforcement Learning for Adaptive Test-Time Compute

Scaling test-time computation can improve language-model reasoning, but uniform budgets waste computation on easy inputs, while verifier-guided refinement relies on external feedback. We introduce Self-Verifying Refinement (SVR), an oracle-free multi-turn reinforcement learning framework that learns to use self-verification as a compute-control policy. At each turn, the model produces a solution together with a discrete correctness verdict and a confidence score; it retains the current answer only when the verdict is Correct and confidence exceeds a threshold, and otherwise continues refinement using its own self-verification. Ground-truth correctness is used only to construct training rewards and is never exposed to the policy through refinement prompts or required at inference. SVR is trained with GRPO on fixed-horizon trajectories using rewards that promote solution correctness, calibration-aware self-verification, and stop-ready correct states; adaptive stopping is activated only at inference. On seven mathematical reasoning benchmarks with Qwen3.5-2B, SVR achieves a macro-average accuracy of 0.563 with only 2.99 inference turns on average. In the evaluated complete-system comparison, it exceeds standard GRPO, strong multi-turn baselines, and a fixed-budget oracle-guided score-feedback reference while requiring substantially fewer turns than fixed ten-turn inference. These results demonstrate that learned self-verification can serve as an effective internal control signal for answer retention and adaptive test-time compute allocation.

Hongyu Chen, Liang Lin, Guangrun Wang

1Sun Yat-sen University · 2Guangdong Key Laboratory of Big Data Analysis and Processing · 3X-Era AI Lab

Language Model Self-Verification RL for Language Model Reasoning Language Model Self-Assessment Inference-Time Adaptation of Language Models

Ask PDF

Sep 17, 2026

How Do Agent Harnesses Create Value? Planning Information and Release Control in Stateful LLM Agents

Agent harnesses supply planning guidance, organize execution, and check completion. We study how these components affect success, erroneous acceptance, and cost in two Retail experiments and an Airline pilot in

τ^2

-bench. The primary comparison pairs prewritten task-specific plans (Fixed) with shuffled policy text matched in word count (Sham), isolating the contribution of guidance content. Across 265 matched cells, Fixed improves oracle-verified success by 7.17 percentage points (90% task-clustered bootstrap interval, 1.15--13.36 points), with gains concentrated in higher-complexity tasks. A read-only terminal verifier rejects 61% of Retail oracle-invalid episodes while withholding 17% of correct ones, at less than one cent of additional cost per episode. Which component matters more depends on the loss assigned to erroneous acceptance: at low liability the planning gain dominates; at high liability the verifier's avoided false passes dominate---and a standalone verifier captures nearly all the false-pass benefit of the full planning-plus-verification stack at a fraction of its cost.

Yukun Zhang, Kemu Xu, Yishen Chen

The Chinese University of Hong Kong, Shenzhen, China · University of Edinburgh, Edinburgh, United Kingdom · The Chinese University of Hong Kong, Hong Kong, China

Agent Harness Optimization LLM Agent Verification LLM Agent Harnesses LLM Agent Planning

Ask PDF

Abstract

Self-Training for Language Models

Explore similar work

Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning

SVR: Self-Verifying Refinement via Joint Verdict-Confidence Reinforcement Learning for Adaptive Test-Time Compute

How Do Agent Harnesses Create Value? Planning Information and Release Control in Stateful LLM Agents