AI agents increasingly act as analysts whose decision can often be cast as hypothesis tests. Classical tests control the Type I error rate, the probability of rejecting a true null hypothesis, for fixed procedures. However, AI analysts introduce internal randomness and can evolve continuously over time. As AI analysts evolve, the risk of approving versions with Type I error rates above the target can accumulate, and adaptive use of evaluation feedback can lead to overfitting. We introduce continual validation, which evaluates each proposed AI analyst version before deployment using simulated data under the null hypothesis, with scenarios that can be randomized and expanded over time. We prove that, regardless of how updates are chosen or how many are proposed, the lifetime probability of ever deploying a version whose Type I error rate exceeds the allowed level remains bounded. This guarantees that every subsequent decision controls the Type I error rate at the target level across these scenarios. At the same time, updates with sufficiently low Type I error rates pass validation with high probability, allowing the analyst to continuously improve its performance without sacrificing rigor. In two examples involving evolving AI analysts for bioequivalence studies and product A/B tests, Type I error rates reached 0.098 and 0.183 without continual validation, compared with at most 0.045 and 0.044 with it, while retaining 82% and 96% of the power from optimally selected valid updates.
Figures & tables
Figure 1: Continual validation and the two examples. ( A ) A human developer or the AI analyst itself proposes updates. The validator freezes the updated version and runs it alongside a reference analysis on datasets newly simulated under the null hypothesis. The update is deployed only when the upper confidence bound on its Type I error rate is at most the validation level α′ across all simulated scenarios. The bound for update 2 exceeds α′ in one scenario so it is rejected and v1 stays in use. The proposer receives this result as feedback. Over any number of updates the chance of ever deploying a version above α′ remains at most the lifetime risk δ . In every real study where the data follow a simulated null scenario a true H0 is rejected with probability at most α=α′+δ(1−α′) . The examples use α=0.05 and δ=0.005 which gives α′=0.0452 . ( B ) The two examples and the experiment setup. In Example 1 the agent decides whether a test product and a reference product are bioequivalent under the standard average bioequivalence framework. The true ratio is the ratio of their geometric mean exposures for a measure such as the peak concentration. The null hypothesis states that this ratio lies outside the regulatory acceptance limits of 0.80 and 1.25. The data come from replicate crossover studies where subjects receive the formulations in successive periods and the reference product is given at least twice. The agent claims bioequivalence and rejects H0 when the computed confidence interval lies entirely within these limits. The agent can also defer a study to a human. In Example 2 users of a shopping application are randomized to the current version or to a new feature over 14 days. The true change represents the difference in the share of users who make a purchase. The agent uses the statistics engine of the GrowthBook platform and makes launch decisions according to the rule specified by each version. A false launch constitutes a Type I error. In the experiment Claude Sonnet 5.5 proposes one update per round. The update either changes a setting or introduces a new decision rule. Each method is run 20 times for 10 rounds. Example 1 also tests the open-weight models Qwen 3.8 and DeepSeek-R1 as developers under three of the methods.
Figure 2: Continual validation and four comparison methods evaluated across two evolving AI analysts. Each method was tested in 20 independent runs of 10 rounds from the same initial version. In each round, Claude Sonnet 5.5 acted as the developer and proposed one update based on feedback from earlier rounds. ( A to D ) Bioequivalence analyst. ( E to H ) Product experiment analyst. ( A and E ) Updates proposed under continual validation in the first ten runs, with one row per run. Circles denote setting changes, squares denote new decision rules, and crosses denote malformed updates. Filled markers indicate deployed updates. Colors indicate the true Type I error rate of the proposed version, showing rates within α′ (grey), above α′ (pink), and above α (dark red). ( B and F ) Largest true Type I error rate of the deployed version over the full validation set after each round. Lines show the median over runs and bands extend to the worst run for no validation (vermilion), the static benchmark (orange), the improvement test (bluish green), Learn then Test (sky blue), and continual validation (blue). The dashed line marks the validation level α′=0.0452 , the dotted line marks the target level α=0.05 , and the shaded region lies above α . In F , the axis ends at 0.2 and an arrow indicates the largest rate in the worst run of the improvement test. ( C and G ) Share of runs that ever deployed a version above α , showing Wilson 95 percent intervals and the raw count out of 20 runs. ( D and H ) Mean true statistical power of the deployed version after each round. The dashed line represents an oracle that deploys only valid updates that strictly increase power.
Figure 3: Continual validation with random scenarios in the bioequivalence analyst. ( A ) Largest true Type I error rate of the deployed version at sample sizes of 30, 42, and 60, outside the original validation grid of 24, 36, and 48. Filled circles indicate deployed versions, while open circles indicate rejected updates. Orange circles highlight four rejected updates that would increase Type I error outside the original grid while leaving the analysis unchanged on the grid. The dashed and dotted lines mark the validation level α′=0.0452 and the target level α=0.05 , respectively. The shaded region indicates rates above α . ( B ) Mean true power of the deployed version across 20 runs with Claude Sonnet 5.5 proposing updates. The dashed line shows the oracle in Fig. 2 , and the percentages indicate power relative to the oracle after ten rounds.
Figure A1: Why common checks fail and what validation costs. ( A ) Combining individually valid tests can inflate Type I error. Two bioequivalence tests each control Type I error near 5 percent, but using the second whenever the first fails exceeds this level. Dots show nine designs, each evaluated using 400,000 simulated studies. ( B ) Repeated validation at a fixed confidence level accumulates risk. The curves show the probability of ever deploying a version above α′ when every proposed version lies just above this threshold. Fixed 95 percent and 99.5 percent checks accumulate risk, whereas continual validation keeps it below the lifetime level δ . ( C ) Simulated datasets per scenario required for an update to pass validation as the update number increases. Without reference comparison, a version with Type I error rate α0 passes with probability 0.9 (gray). With paired comparison, a version that adds no rejections passes with certainty (solid blue), while one that adds rejections at rate (α′−α0)/2 passes with probability 0.9 (dashed blue). Panels B and C use α=0.05 , δ=0.005 , δ0=0.001 , and α0=0.040 , giving α′=0.0452 .
Figure A2: Updates written in advance. The same ten updates, in the same order, judged by every method in ( A ) Example 1 and ( B ) Example 2. A filled circle marks an update the method deployed and an open circle one it did not. The color gives the true Type I error rate of the version the update would create, within α′ or too close to α′ to resolve (blue), above α′ (orange), or also above α (vermilion).
Figure A3: Random scenarios in Example 1. ( A ) Updates written in advance. The same ten updates, in the same order, judged by continual validation on the fixed scenarios and with random scenarios, in the first of three runs, and the other runs made the same decisions. Bold labels mark the four updates that relax the analysis only at sample sizes outside the grid. A filled circle marks a deployed update and an open circle a refused one. The color gives the largest true Type I error rate of the version the update would create, on the validation scenarios or at the sample sizes outside the grid, within α′ (blue), above α′ (orange) or above α (vermilion). ( B ) Largest true Type I error rate of the deployed version at 30, 42 and 60 volunteers, the sample sizes of the off-grid scenarios outside the grid, after each round of these updates. ( C ) Mean true power of the deployed version over 20 runs with Claude Sonnet 5.5 as developer, on the fixed scenarios (the runs of Fig. 2 D ) and with random scenarios, and of the oracle (dashed).
A major bottleneck in characterizing the failure modes of generative AI systems is the cost and time of annotation and evaluation. Consequently, adaptive testing paradigms have gained popularity, where one opportunistically decides which cases and how many to annotate based on past results. While this framework is highly practical, its extreme flexibility makes it difficult to draw statistically rigorous conclusions, as it violates classical assumptions: the number of observations is typically limited (often 10 to 50 cases) and decisions regarding sampling and stopping are made in the midst of data collection rather than based a pre-specified rule. To characterize what statistical inferences can be drawn from highly adaptive audits, we introduce a hypothesis testing framework from two "dueling" perspectives: (i) the model's null that asserts there is no failure mode with performance below a target threshold versus (ii) the auditor's null that asserts they have a sampling strategy that will uncover a failure mode. Leveraging Safe Anytime-Valid Inference (SAVI), we formalize the auditor as conducting "testing by betting," which translates into simultaneous e-processes for testing the dueling null hypotheses. Furthermore, if the auditor is sufficiently powerful, we prove that these two hypotheses are asymptotically inverses of each other, in that passage of such an audit certifies the AI system as being globally robust. Empirically, we demonstrate that our proposed testing procedures maintain anytime-valid Type-I error control, outperform pre-specified testing methods, and can reach statistically rigorous conclusions sometimes with as few as 20 observations.
Siyu Zhou, Patrick Vossler, Venkatesh Sivaraman +2
University of California, San Francisco · Stanford University
We adapt split conformal prediction and adaptive conformal inference (ACI) to continuous AI agent evaluation, providing distribution-free coverage guarantees for forecasted quality scores. Conformal intervals achieve calibration error below 0.02 across all nominal levels at the 24h horizon, while ACI correctly widens intervals by 35% following agent releases then reconverges. We further develop compositional uncertainty bounds for multi-agent pipelines (validated via simulation across inter-stage correlations rho in [-0.5, 0.9]), a conformal abstention rule for pairwise rankings with controlled false-ranking rate, and FDR-corrected abstention for leaderboard-scale multiple testing. Evaluating 50 agents via 18 real-time signals collected hourly, we show that per-agent conditional coverage is well-concentrated around the nominal level (mean 80.4%, 90% of agents within [72%, 90%]), and that cross-source sentiment divergence predicts ranking instability (r=0.64, p<0.01). A circularity-controlled validation confirms the framework captures signal beyond benchmarks (rho_s=0.52, p<0.01, n=35). Code and data are released under CC BY 4.0.
Yuxuan Gao, Megan Wang, Yi Ling Yu
OpenMesh · University of Pennsylvania · Columbia Univer
Classification accuracy and true-positive and false-positive rates alone do not determine whether restarting an AI agent improves success. Using an exact alarm-to-action identity for an independent clean restart with zero monitoring overhead, we show analytically that a monitor with fixed 90% accuracy can swing from a five-percentage-point loss to a 35-point gain in success rate, depending solely on the replacement attempt's residual-budget success probability. We introduce two techniques for evaluating monitors and progress signals in decision terms. First, a decision-sufficiency criterion, a binary-action specialization of Blackwell's comparison of experiments, identifies which progress representations or context compressions preserve the information needed to choose optimally between the specified actions, independent of trace reconstruction fidelity. Second, a finite-sample, model-free decision audit uses simultaneous binomial intervals over existing agent traces, without additional model calls, to certify a preference for "restart" or "continue" under stated sampling assumptions, or return "abstain" when evidence is insufficient. In a study of 194 externally evaluated model executions on 19 selected SymPy issues from SWE-bench Verified, with a retrospective audit of 130 of these traces, we find a 2.29-point apparent restart gain traced to a single surviving failed attempt, a 42.29-point loss on a second model, no positive replication of the initial gain on prospective task cohorts, and zero of 103 live comparisons meeting the specified certification bar. These results motivate evaluating agent monitors through recovery opportunity, decision value, and statistical support, rather than classification performance alone.