Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation
Organizations: Manipal University Jaipur
Abstract
LLM-based verifiers are increasingly used to evaluate agent executions, but their reliability is typically measured under a single representation of the trajectory. In deployed agent systems, the same execution may be exposed through composed tools, delegation boundaries, or compacted histories, changing what execution evidence is directly observable to the verifier. It therefore remains unclear whether correct verifier judgments persist across such abstraction boundaries. We introduce Verification Transport, a reliability test that measures judgment stability under abstraction-induced changes in execution observability. We construct matched native, abstracted, and restored views of the same frozen executions while holding the task, realized trajectory, outcome, ground truth, evaluation criterion, and verifier fixed. Across four agent-evaluation benchmarks, seven verifier backbones, four verification protocols, and three abstraction boundaries, we find substantial transport failure, reaching about 31% under macro abstraction and delegation. Verification protocols exhibit distinct failure modes: direct verification more often produces incorrect verdicts, whereas evidence-grounded and deliberative protocols tend to abstain under limited observability. This suggests that lower transport failure rates do not necessarily imply greater reliability. Restoring the corresponding execution provenance recovers 85.5% of transport failures, while recovery is lower when provenance is lossy or unavailable. These results show that native-view accuracy alone is insufficient to characterize verifier reliability when execution evidence crosses abstraction boundaries.
Figures & tables
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Condition | View | Verifier-visible representation |
| Native | Benchmark’s original trajectory representation. | |
| Macro abstraction | Multiple internal tool operations and observations are exposed through a higher-level macro boundary. | |
| Macro restoration | The exact internal operations and observations abstracted by are reintroduced. | |
| Delegation abstraction | An existing execution block is exposed through a delegation boundary while its internal trajectory is not directly visible. | |
| Delegation restoration | The exact delegated execution from the original trajectory is reintroduced. | |
| History compaction | Earlier execution evidence is deterministically compacted while recent execution evidence remains visible. |
| Macro | Delegation | Compaction | ||||
| Protocol | WV | NV | WV | NV | WV | NV |
| V-LLM | 15.8 | 0.7 | 16.5 | 0.8 | 13.2 | 0.9 |
| V-EVIDENCE | 10.6 | 24.0 | 10.4 | 23.6 | 10.9 | 7.8 |
| V-AGENTIC | 17.5 | 7.5 | 16.5 | 7.0 | 13.2 | 5.0 |
| V-DELIBERATIVE | 9.4 | 39.9 | 9.2 | 41.0 | 11.6 | 19.1 |
| Overall | 13.4 | 17.7 | 13.2 | 17.7 | 12.2 | 8.0 |
| Protocol | Cov. | Abs. Cov. | FCR | FRR | PRR | BRR | HRR |
| V-LLM | 98.4 | 98.7 | 41.3 | 6.6 | N/A | N/A | N/A |
| V-EVIDENCE | 97.0 | 80.4 | 29.4 | 4.7 | N/A | N/A | N/A |
| V-AGENTIC | 97.1 | 93.3 | 26.2 | 13.1 | 37.4 | N/A | N/A |
| V-DELIBERATIVE | 94.6 | 67.3 | 24.8 | 5.4 | N/A | 8.7 | 28.5 |
LLM Agent Verification
LLM agent verification checks whether agents’ actions, evidence, policies, and completion claims satisfy explicit requirements.
See all 170 papers