A Verifier Can Leak the Answer: Diagnosability Before Optimization in Closed-Loop Agent Debugging
Organizations: Blossom AI Blossom AI Labs San Francisco, CA, USA · Tokyo, Japan
Abstract
Agent developers increasingly compare prompts, tools, policies, and diagnosis algorithms through simulator-grounded verifiers. A verifier can nevertheless make a solver comparison vacuous: if its probes or predicates encode the target identity, an exact optimizer may appear effective without resolving any genuine ambiguity. We report such a failure in an aggregate-trace debugger for a closed-loop decision agent. Exact minimum hitting set (MHS) and a propagation-aware greedy method returned identical supports in 12/12 development cases and the same planted-fault recovery in 9/12. A subsequent audit found that exact-anchor predicates produced the planted pair in 9/9 cases. After removing those anchors, overall planted-pair recovery was 8/9; hard-probe singleton pairs nevertheless matched the planted pair in 9/9, and no case retained a nonempty residual conflict family after propagation (0/9). The optimizer was correct, but the verifier had already disclosed the answer. We replace solver-first evaluation with a support-gated verification contract. A clean reference map must first show repeated component exposure; a matched reference/current gate must then establish comparable runtime evidence; only afterward may an independently calibrated signal rule return a detection. In a preregistered heldout comprising 1,440 cases and 21,600 partition rows, 55/72 regime-component units passed the reference gate, 54/55 passed the runtime gate, and stable false admission was 0/20 represented components with a one-sided exact 95% upper bound of 0.1391. Within admitted units, affected clean traffic predicted detection better than nominal fault-cell fraction. The main lesson is structural: verify evidence eligibility and non-revelation before optimizing the component selector. Otherwise a stronger solver can merely certify a stronger verifier artifact.
Figures & tables
| Audit | Finding | Consequence |
|---|---|---|
| Exact MHS vs weighted-greedy support equality | 12/12 | no observed decision advantage |
| Exact planted recovery, exact MHS | 9/12 | good recovery, not superiority |
| Exact planted recovery, weighted greedy | 9/12 | matched exact MHS |
| Exact-anchor singleton pairs match planted pair | 9/9 | anchor layer reveals the target |
| Recovery after removing exact anchors | 8/9 | deleting one shortcut is insufficient |
| Hard-probe singleton pairs match planted pair | 9/9 | probe-scope interaction still reveals target |
| Layer | Endpoint | Formal result | Interpretation |
|---|---|---|---|
| execution | protocol, hashes, schedule, validator | 56/56 checks; 1,440 cases | EXECUTED |
| evidence | reference admission | 55/72 units; 20/24 components | unsupported units abstain |
| evidence | runtime admission | 54/55 reference-admitted units | one additional runtime abstention |
| safety | component stable false admission | 0/20; upper 0.1391 | SAFETY CONFIRMED under 0.20 rule |
| structure | traffic versus cell fraction | 0.1264 [0.0593, 0.1918] nats/row | traffic is better within admitted units |
| structure | aggregate traffic-target power | monotone counts in both families | structural hypothesis PASS |
Fault Diagnosis
Fault diagnosis identifies, localizes, and explains failures in engineered and computational systems using sensor data, logs, models, and domain knowledge.
See all 146 papers