Evidence-Gated Research: Statistically Controlled Model Adoption in Adaptive Search

Yifan Guo

cs.LGOct 1, 2026

Evidence-Gated Research: Statistically Controlled Model Adoption in Adaptive Search

Authors: Yifan Guo

Organizations: Department of Applied Mathematics and Statistics Stony Brook University

Abstract

Adaptive model search is path dependent: once a challenger is adopted, it becomes the reference from which later candidates are generated. A statistically unsupported replacement can therefore alter hypotheses that have not yet been proposed. We introduce Evidence-Gated Research (EGR), a statistical adoption layer for moving-incumbent search. EGR freezes each challenger before decision evidence is revealed, builds anytime-valid evidence across a predeclared set of environments, routes evidence predictably toward unresolved components, composes a persistent candidate e-value, and passes that e-value to an online controller. Under explicit conditional-validity and predictability conditions, the resulting procedure controls false discovery rate for the declared all-environment adoption target even though earlier adoptions change later challengers. In a 5,000-trajectory closed-loop benchmark, development-only e-LOND attains persistent FDR 0.621, whereas no persistent false-adoption path is observed for the audited EGR variants in that finite run. In matched replay over 600 challenger--incumbent pairs, Stagewise EGR preserves fixed-anytime alternative crossing decisions while using 56.1% less decision evidence at the representative threshold. A three-environment public-data study and a 40,000-sample controlled neural benchmark reproduce the evidence-efficiency pattern. These results identify model replacement as a distinct statistical control point in adaptive model development.

Figures & tables

Figure 1: EGR inserts a statistically controlled adoption gate between adaptive proposal generation and the next search state. The challenger is frozen before decision evidence is opened; component e-processes are updated under predictable routing, combined through a persistent target, and passed to a compatible online controller. Only adoption changes the incumbent.

Object	aLTT	e-LOND / e-GAI	EGR (ours)
Native target	candidate / hyperparameter risk	online e-value stream	persistent challenger–incumbent adoption
Within-candidate acquisition	sequential	assumes e-value supplied	predictable multi-env. acquisition
Changing incumbent	extra specification	extra specification	explicitly modeled
Across-candidate control	family selection	online FDR	compatible online controller

Table 1: Statistical objects addressed by related frameworks. Our construction is shown in the final column.

Condition	Required?	Role
Candidate frozen before evidence	Yes	prevents candidate-time leakage
Joint-filtration validity	Yes	links null to selected updates
Predictable allocation / stopping	Yes	preserves e-process validity
Cross-environment independence	No	minimum composition is pointwise
Raw-unit dependence	Not automatic	must be handled in evidence unit

Table 2: Operational validity contract. The guarantee is local to the declared target and evidence unit.

Figure 2: Target mismatch and stream multiplicity. Development-only procedures can appear powerful while missing the declared persistent target; repeated reuse of an anytime-valid candidate test accumulates false adoptions, whereas EGR with online correction remains controlled in the tested regimes.

Metric	Dev mean	Dev e-LOND	Two-env mean	Fixed IUT + e-LOND	Stagewise EGR	EGR-ALLOC
Persistent FDR	.6240	.6210	.0340	.0000	.0000	.0000
Pers.-alt. power	.907	.461	.867	.188	.196	.251
Final audit utility	.493	.489	.705	.641	.642	.657
Evidence / candidate	240.0	221.6	240.0	240.0	145.5	153.7

Table 3: Headline closed-loop results. Methods are columns so EGR variants appear last. Lower is better for FDR/evidence; higher is better for power/audit utility.

Benchmark	Metric	Fixed IUT + e-LOND	Stagewise EGR	EGR-ALLOC
Digits 3-env.	FDR	0	0	0
( $N=800$ )	Power	.3137	.3221	.3349
	Evidence	1200.0	388.8	392.6
Controlled 40k	FDR	0	0	0
( $N=500$ )	Power	.2149	.2380	.2666
	Evidence	1200.0	444.6	440.9

Table 4: Three-environment audited comparisons. EGR columns are last; Monte Carlo uncertainty is reported in Appendix C .

Figure 3: Persistent-alternative power versus decision-evidence cost in the three-environment studies. Error bars show 95% Monte Carlo intervals. The EGR variants achieve substantially lower evidence use; exact numerical summaries are in Table 4 .

Figure 4: Matched frozen-candidate replay over 600 challenger–incumbent pairs. Stagewise EGR has the same alternative crossing event as fixed-anytime IUT while placing evidence sequentially; EGR-ALLOC can recycle saved early-stage budget. At $q=20$ , Stagewise uses 56.1% less evidence at identical power.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Metric	Fixed IUT	Stagewise EGR	EGR-ALLOC
RBF-SVC	FDR	.0000	.0000	.0000
	Power	.522	.554	.583
	Evidence	1200.0	567.5	584.3
Neural MLP	FDR	.0000	.0005	.0000
	Power	.319	.355	.374
	Evidence	1600.0	782.3	781.0

Appendix

Table 5: Original public-data adaptive searches. EGR methods are the final columns.

Policy	Metric	Fixed IUT	Stagewise EGR	EGR-ALLOC
Local	Power	.3192	.3224	.3309
	Evidence	1200.0	389.3	390.7
Wide-local	Power	.5466	.5519	.5705
	Evidence	1200.0	379.0	379.4
Global	Power	.6993	.6877	.6855
	Evidence	1200.0	379.1	378.4

Appendix

Table 6: E31 proposal-policy robustness. Every displayed audited method has zero observed FDR in 400 trajectories per policy; EGR methods are the final columns.

Metric	Fixed endpoint	Fixed anytime	Stagewise EGR	EGR-ALLOC
Pers.-alt. power	.6082	.6477	.6477	.6699
Persistent-null crossing	.000013	.000355	.000355	.000395
Mismatch-null crossing	.000000	.001667	.001667	.001905
Evidence	1600.0	1600.0	702.4	715.7

Appendix

Table 7: Frozen-pair replay at

q=20

. Reported null rates are candidate-level crossing frequencies, not online-stream FDR.

Variants $L$	Frozen	Leak+reuse	Leak+restart
1	.063 [.049,.081]	.063 [.049,.081]	.054 [.041,.070]
4	.060 [.047,.077]	.223 [.196,.254]	.042 [.031,.057]
16	.062 [.048,.079]	.988 [.929,1.051]	.059 [.046,.076]
64	.052 [.040,.068]	3.802 [3.685,3.922]	.063 [.049,.081]

Appendix

Table 8: E33 persistent-null crossing at

q=20

over 100,000 trials per

L

(percent; Wilson 95% interval in brackets).

Explore similar work

Search Shapes Conclusions: Auditing Evidence Selection Bias in Deep Research Agents
Sep 30, 2026Shuyao Xiao, Shengling Wang, Xuan Chen +8Deep Research Adaptive Sampling
Beijing Normal University · Ke Holdings
PEAR: Progressive Evidence-Based AutoResearch for Industrial Search Systems
Sep 28, 2026Yifan Wang, Shipeng Zhu, Fei Xiong +6Research Automation Agentic Optimization
Global E-Commerce Agentic Search Team
EviSearch: Trustworthy Extraction and Synthesis of Clinical Trial Evidence with Agents that Improve with Use
Mar 23, 2026Naman Ahuja, Abhijit Chakraborty, Muhammad Ali Khan +4Systematic Review Clinical Trials
Arizona State University · Mayo Clinic

Sep 30, 2026, cs.AIJ/K move · Enter open · S save

Search Shapes Conclusions: Auditing Evidence Selection Bias in Deep Research Agents

Shuyao Xiao, Shengling Wang, Xuan Chen, Ke Chao+7

Beijing Normal University · Ke Holdings

Deep Research Adaptive Sampling Shapes Constraint Language

Open paper AskPDF arXiv

Deep Research agents synthesize evidence into cited reports, yet a well-cited report can still reach a misleading conclusion. Citation correctness checks whether cited sources support individual claims. It does not show whether adaptive search exposed a representative view of all documents made available for evaluation, which we call the candidate pool. Early findings redirect later queries, document choices, and stopping, so the documents an agent reads form a selective sample. Existing evaluations rarely account for this selection. We formulate the problem as adaptive evidence sampling and introduce Causal Evidence Selection Correction (CESS). CESS predicts each candidate document's evidence direction and corrects the candidate-pool average using the logged probabilities of selecting each document and reaching each search round. Shrinkage stabilizes short searches, while intervals replace point estimates when some documents cannot be sampled. We also prove that estimating the average evidence direction of a common pool differs from measuring how a change in search policy alters the evidence read. The latter requires intervention. On questions from the MS2 systematic-review benchmark, CESS reduces mean absolute error against the candidate-pool average by

9.2\%

and reduces the estimate's change under opposing document rankings by

39.4\%

relative to averaging the evidence scores of documents read. Across trajectories from a public Open Deep Research agent, the corresponding reductions reach

60.1\%

and

87.2\%

. A further 4,800 trajectories under paired interventions confirm that correcting a pool estimate and measuring a policy effect are different tasks. CESS therefore audits whether the evidence direction underlying a report reflects the documents available for evaluation, while a separate intervention analysis measures the effect of search decisions.

Abstract

Explore similar work

Search Shapes Conclusions: Auditing Evidence Selection Bias in Deep Research Agents

PEAR: Progressive Evidence-Based AutoResearch for Industrial Search Systems

EviSearch: Trustworthy Extraction and Synthesis of Clinical Trial Evidence with Agents that Improve with Use