cs.AINov 9, 2026

Veritas: A Verifier Generation Harness for Agent Tasks

Authors: Daoan Zhang, Qinbo Bai, Dehai Min, Xingyu Mu, Ziyi Chen, Jianing Yin

Organizations: ByteDance Inc., USA · University of Illinois Chicago · New York University

Abstract

The effectiveness of a self-improving system relies heavily on the quality of its verifier, which judges the outcome of each improvement round. However, the verifiers of most tasks are written by hand, which is expensive and does not scale to tasks that come without one. We propose Veritas, a harness that generates verifiers automatically. Unlike general-purpose agent harnesses, Veritas adds three mechanisms for verifier generation: an evidence header stating what the verifier will observe, a library of domain rules that tasks leave unstated, and authoring principles, partly refined by a self-improvement loop. We evaluate on six benchmarks: four scored by execution, whose official verifiers we first certify, and two that compare pairs of responses. With the model fixed, the verifiers Veritas generates agree better with the official verdicts than those of seven other harnesses on all four execution-scored benchmarks. On the two pairwise benchmarks they are at or near the best. They also agree better than those of the vendor's own harness for two of three other model families. Its mechanisms also improve 9 of 12 harness-benchmark pairs scored by execution across three harnesses.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents

    Oct 19, 2026Jianing Yin, Ziyi Chen, Hao Wang +4LLM Agent VerificationAI Agent Benchmarks

  2. Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification

    Aug 27, 2026Jinghan Xu, Yikai Zhang, Aili Chen +3Agent Harness OptimizationLLM Agent Harnesses

  3. VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks

    Oct 1, 2026Caiqi Zhang, Rujun Han, Zifeng Wang +4LLM Agent VerificationLong-Horizon LLM Agents