Valid Stopping in Adaptive Generator-Verifier Loops

Mahmoud Hegazy; Michael I. Jordan; Aymeric Dieuleveut

stat.MLOct 5, 2026

Valid Stopping in Adaptive Generator-Verifier Loops

Authors: Mahmoud Hegazy, Michael I. Jordan, Aymeric Dieuleveut

Organizations: CMAP, École polytechnique, IP Paris, France · Inria Paris, France · EECS, UC Berkeley, USA

Abstract

Numerous agentic workflows are based on a generator-verifier loop: a generator proposes candidates, a cheap verifier scores them, and the workflow terminates when a proposal is verified as good enough. The verifier typically proxies a more costly ground-truth oracle, and as the generator searches adaptively against it, false acceptances may accumulate. Proposals can pass the proxy but fail under the costlier ground-truth check. We study when to stop these loops while controlling the false discovery rate of the accepted proposals. Our construction introduces tools of independent interest in distribution-free statistical testing and conformal risk control, including analysis of $e$ -values constructed through index betting and a novel conformal risk control procedure for non-monotone losses. We validate the approach in synthetic settings and on a protein-design benchmark.

Figures & tables

Figure 1 : Type-I and Type-II error rates across reference sizes

M

for

L_{t}

E_{t}^{\mathrm{idx}}

, and

\widehat{E}_{t}^{\mathrm{idx}}

. Solid curves with open markers correspond to

\alpha=0.05

and dashed curves with filled markers to

\alpha=0.01

Figure 2 : Normalized log-growth

\mathbb{E}_{Q}[\log E]/\KL(Q\|P)

of the oracle index e-value

E_{t}^{\mathrm{idx}}

as a function of the fractional-moment exponent

\delta

, for reference sizes

M\in\{500,1000,5000\}

. Each point averages

2\times 10^{5}

repetitions.

R\leftarrow\left\{\lambda\in\Lambda\setminus C:\frac{1}{n+1}\sum_{i=1}^{n+1}L_{i}(C\cup\{\lambda\})\leq\alpha\right\}

Algorithm 1 Greedy Selector

Figure 3 : Mean selected parameter on the clipped-Gaussian grid, with 95% Monte Carlo intervals. The dotted line marks the highest fixed safe parameter

\lambda^{\star}=\max_{\preceq}\{\lambda_{k}:\mathbb{E}[\ell_{i}(\lambda_{k})]\leq\alpha\}

. The three panels show the three clipped-Gaussian noise levels in the sweep.

Figure 4 : Selection time of robust-core CRC against the grid size. Each panel corresponds to one noise level

\sigma

of the clipped-Gaussian losses and shows the median selection time in milliseconds against the grid size

|\Lambda|

, at level

\alpha=0.1

, with one curve per calibration size

n

. Curves show the median over 100 independent calibration datasets of each dataset’s median time over five repetitions, and shaded bands show pointwise 95% bootstrap intervals for this median.

Figure 5 : Synthetic Gaussian-ring validation. Panels A and B show the same sampled 50-round trajectory with no stopping rule applied, so they do not depend on

\alpha

. Panel A shows the oracle field

Q_{g}\coloneqq B+G_{g}

, which coincides with the initial verifier field

V_{g,\xi,1}

, and Panel B shows the final verifier field

V_{g,\xi,T}

. The filled contours give the value of the field at each point of

\mathbb{R}^{2}

, with brighter colors for larger values. The yellow star marks

\mu_{g}

, and the red cross marks

\mu_{g+\xi}

. Panel C reports the mean stopped FDP against the target level

\alpha

, with 95% Monte Carlo intervals over 20 independent repetitions.

Figure 6 : Motif scaffolding examples. Each column shows a MotifBench-derived task after scaffolding. The fixed motif is magenta and the designed scaffold is grey.

Figure 7 : One round of the protein feedback loop. Four exploit candidates continue from the stored midpoint state associated with the previous winner, while four explore candidates start from the original motif specification. The cheap ProteinMPNN–ESMFold pipeline scores the resulting proposals, and the midpoint state associated with the highest-scoring proposal seeds round $t{+}1$ .

Figure 8 : Cheap-verifier trends across rounds. At each round, we retain the design with the highest cheap-verifier score seen in each trajectory. Solid circles denote RFdiffusion and dashed squares denote Proteina. Panel A reports cumulative-best score. Panel B reports ESMFold motif RMSD on the left axis and mean pLDDT on the right axis.

Figure 9 : Stopped risk and utility on held-out trajectories. The robust-core calibrated online

e

-BH rule is evaluated over target levels

\alpha\in\{0.05,0.10,0.15,0.20\}

and target acceptance counts

\kappa\in\{1,5,10\}

. Panels A and B report stopped FDP for RFdiffusion and Proteina. Panels C and D report returned designs, with counts capped at

\kappa

, for the same two generators.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Task	Motif residues	Motif length	Left range	Right range
01_1LDB	A186–206	21	0–100	0–100
02_1ITU	A124–147	24	0–100	0–100
03_2CGA	A184–194	11	0–100	0–100
04_5WN9	A170–189	20	10–40	10–40
05_5ZE9	A229–243	15	0–100	0–100
06_6E6R	A23–35	13	0–65	0–65

Appendix

Table 1: MotifBench source tasks used to construct the 300 targets, each fixing one pair of left and right scaffold lengths.

Explore similar work

Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agents
Jul 20, 2026Yitao Wu, Si Shen, Rui Yang +2AI Agent Reliability LLM Self-Correction
saofund.ai, Shenzhen, China · Shenzhen Xingqing Zhiti Technology Co., Ltd., Shenzhen, China · School of Information Technology and Engineering, Lanzhou University, Lanzhou, China
Adaptive Generate-Rank-Verify: Inference-Time Search with Costly Verification
May 17, 2026Shaddin Dughmi, Mahdi Haghifam, Yusuf Hakan KalayciAdaptive Inference Inference-Time Search
University of Southern California · Data Science Institute at the University of Chicago · Northwestern University +2
Adaptive auditing of AI systems with anytime-valid guarantees
May 7, 2026Siyu Zhou, Patrick Vossler, Venkatesh Sivaraman +2Algorithmic Auditing Anytime-Valid Inference
University of California, San Francisco · Stanford University

Jul 20, 2026, cs.AIJ/K move · Enter open · S save

Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agents

Yitao Wu, Si Shen, Rui Yang, Hong Peng+1

saofund.ai, Shenzhen, China · Shenzhen Xingqing Zhiti Technology Co., Ltd., Shenzhen, China · School of Information Technology and Engineering, Lanzhou University, Lanzhou, China

AI Agent Reliability LLM Self-Correction LLM Answer Verification LLM Agent Verification LLM Agent Reliability

Open paper AskPDF arXiv

Verify-repair loops are a standard means for large language model (LLM) agents to correct faulty plans in code generation, mathematical reasoning, and tool use. When both the verifier and the repairer are noisy, repair can damage already-correct plans, and reported acceptance keeps rising while true validity falls, so existing methods lack a principled basis for deciding when repair should stop. We propose VRR-Stop, a robust stopping framework for noisy verify-repair-repeat (VRR) loops. A four-parameter noise model separates verifier false acceptance and false rejection from the repair and damage behavior of the repairer. Belief filtering turns repeated verification votes into an estimate of committed validity, and the loop commits or repairs according to the sign of the true marginal gain, which requires only sign identifiability rather than accurate recovery of all parameters. When verifier discrimination approaches zero, calibration itself fails and estimation error can flip the stopping sign, so we pair VRR-Stop with VRR-Guard, an estimation-free fallback that replaces the incumbent candidate only under a sufficient verification margin. On a GSM8K stress setting, VRR-Stop improves final true validity by 60.6 percentage points over fixed five-round repair at an average cost of 0.72 repair rounds. Across settings, stopping reliability is governed jointly by verifier discrimination and the decision margin rather than by the absolute size of estimation error.

Abstract

Explore similar work

Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agents

Adaptive Generate-Rank-Verify: Inference-Time Search with Costly Verification

Adaptive auditing of AI systems with anytime-valid guarantees