Triage-PiEvo: Evidence Admission for Principle-Evolving Scientific Agents

Peize Li; Yingming Pu; Xingguo Xu; Yiming Cui; Yuansheng Gao; Tao Lin

cs.LGOct 10, 2026

Triage-PiEvo: Evidence Admission for Principle-Evolving Scientific Agents

Authors: Peize Li, Yingming Pu, Xingguo Xu, Yiming Cui, Yuansheng Gao, Tao Lin

Organizations: Westlake University · Kuaishou Technology · Zhejiang University · Dalian University of Technology ·

Abstract

Large language model (LLM)-based AI Scientists advance scientific discovery, yet they suffer from inefficiencies due to noisy observations. Existing principle-evolving agents act on every anomaly, leaving its source unresolved: a new mechanism or an evaluation artifact, so invalid revisions bias evolution. To address this, we formulate revision as evidence admission: anomalies steer search immediately, but only independently verified evidence may rewrite persistent principles, and contradicted principles are retracted. We realize this in TRIAGE-PIEVO , which pairs an anomaly-responsive search portfolio with a staged verification gate that alone controls what enters principle state. Across four scientific tasks with three backbones, TRIAGE-PIEVO (1) attains the highest average solution quality with +7.5%~11.8% over the strongest baseline per backbone, matching the strongest baseline's full-budget quality with one-third of the evaluation budget on Qwen3-235B; and additional stress tests and cross-harness deployment demonstrate that (2) in constructed stress tests, it admits none of the corrupted, hidden-source, or exploit evidence that ungated variants admit, while missing no planted mechanism, and (3) plugged unchanged into three external research harnesses, it cuts false evidence admissions by over 60% and raises verified scores by 31%~43%.

Figures & tables

Figure 1: Source-aware triage. Latent source

Z_{t}

mediates the verified retain-or-revise decision.

Figure 2: Overview of Triage-PiEvo . Left: an outcome-blind AFE portfolio cycles candidate proposals across exploitation, frontier expansion, and mechanism-discriminating probes. Middle: Triage treats an anomalous residual as source-ambiguous and routes it to quarantine, search history, or verification rather than attributing it to a source at runtime. Right: principles advance Proposed $\to$ Provisional $\to$ Accepted only with independent held-out support; contradictory evidence retracts them.

Figure 3: AFE portfolio schematic. Three complementary roles on a shared design manifold: exploitation (

\uparrow\hat{\mu}

), frontier expansion (

\uparrow N,D

), and mechanism probes (

\uparrow\hat{\sigma}

Qwen3-235B (No Thinking)
Model / Method	MBO	NHO	SPO	TMC	Average
Vanilla MAS	83.35 $\pm$ 29.95	52.83 $\pm$ 20.36	37.29 $\pm$ 0.75	56.34 $\pm$ 3.69	57.45
ReAct	75.48 $\pm$ 4.94 $\downarrow$ 0.9x	61.95 $\pm$ 17.08 $\uparrow$ 1.2x	37.49 $\pm$ 0.46	72.80 $\pm$ 4.48 $\uparrow$ 1.3x	61.93 $\uparrow$ 1.1x
AI Scientist v1	73.37 $\pm$ 39.82 $\downarrow$ 0.9x	41.50 $\pm$ 3.12 $\downarrow$ 0.8x	37.04 $\pm$ 0.61	63.96 $\pm$ 8.93 $\uparrow$ 1.1x	53.97 $\downarrow$ 0.9x
AI Scientist v2	84.40 $\pm$ 20.07	52.34 $\pm$ 15.56	41.40 $\pm$ 1.15 $\uparrow$ 1.1x	63.52 $\pm$ 2.86 $\uparrow$ 1.1x	60.42 $\uparrow$ 1.1x
AI Researcher	105.70 $\pm$ 21.92 $\uparrow$ 1.3x	37.21 $\pm$ 5.68 $\downarrow$ 0.7x	37.39 $\pm$ 0.85	78.80 $\pm$ 3.76 $\uparrow$ 1.4x	64.78 $\uparrow$ 1.1x

Table 1: Solution Quality (SQ, %)

\uparrow

across four discovery tasks and three backbone LLMs. Entries report mean

\pm

std over three seeds; Bold indicates the best result within each backbone.

Figure 4: Best-so-far trajectories across four benchmarks with Qwen3-235B backbone. Across all scientific domains, Triage-PiEvo accumulates high-quality candidates and sustains the highest solution quality.

Qwen3-235B (No Thinking)
Model / Method	MBO		NHO		SPO		TMC		Average
	APD $\uparrow$	AUOC $\uparrow$	APD $\uparrow$	AUOC $\uparrow$	APD $\uparrow$	AUOC $\uparrow$	APD $\uparrow$	AUOC $\uparrow$	Avg APD $\uparrow$	Avg AUOC $\uparrow$
Vanilla MAS	53.58 $\pm$ 15.25	70.09 $\pm$ 17.75	32.23 $\pm$ 3.72	49.33 $\pm$ 17.82	52.03 $\pm$ 1.05	36.34 $\pm$ 0.38	51.89 $\pm$ 3.85	52.67 $\pm$ 3.49	47.43	52.11
ReAct	59.43 $\pm$ 8.60	70.58 $\pm$ 7.95	33.31 $\pm$ 1.86	47.46 $\pm$ 7.04	56.13 $\pm$ 4.49	35.74 $\pm$ 0.76	72.93 $\pm$ 0.33	66.95 $\pm$ 4.67	55.45 $\uparrow$ 1.2x	55.18 $\uparrow$ 1.1x
AI Scientist v1	52.93 $\pm$ 15.22	56.16 $\pm$ 26.05	32.98 $\pm$ 2.32	38.84 $\pm$ 2.06	42.55 $\pm$ 5.61	35.62 $\pm$ 1.46	64.43 $\pm$ 7.02	57.50 $\pm$ 6.15	48.22	47.03 $\downarrow$ 0.9x
AI Scientist v2	60.26 $\pm$ 10.57	76.03 $\pm$ 7.92	28.39 $\pm$ 3.49	39.51 $\pm$ 3.29	48.18 $\pm$ 1.41	37.03 $\pm$ 0.22	55.66 $\pm$ 2.28	54.75 $\pm$ 6.66	48.12	51.83

Table 2: Average Pairwise Distance (APD,

\uparrow

) and Area Under the Optimisation Curve (AUOC,

\uparrow

) across four tasks and three backbone LLMs. Higher APD indicates broader candidate coverage, while higher AUOC indicates faster accumulation of high-quality candidates.

Figure 5: Cost-efficiency trade-offs across autonomous research harnesses. (a) Token expenditure versus reported Task Score and externally Verified Score. Triage substantially narrows the verification gap (shaded region) with minimal token overhead. (b) Wall-clock execution runtime versus optimization performance across ResearchHarness, AI-Researcher, and Claude Code. Full numerical results are reported in Table 6 (Appendix D ).

Variants	Avg SQ $\uparrow$	Avg AUOC $\uparrow$
Triage-PiEvo	65.98	60.45
w/o Gate	62.73 ( $\Delta{=}{-}3.25$ )	58.23 ( $\Delta{=}{-}2.22$ )
Triage-only	60.60 ( $\Delta{=}{-}5.38$ )	55.08 ( $\Delta{=}{-}5.37$ )
PiEvo ( Pu et al., 2026 )	64.78 ( $\Delta{=}{-}1.20$ )	58.15 ( $\Delta{=}{-}2.30$ )

Table 3: Module ablations across four tasks. SQ and AUOC are averaged across tasks and seeds.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Constant	Role	Value
$\tau_{\mathrm{anom}}$	Residual threshold to open diagnostic state	0.85
$\epsilon_{\mathrm{rep}}$	Same-channel agreement tolerance	0.12
$\epsilon_{\mathrm{ind}}$	Cross-channel agreement tolerance	0.12
$\tau_{\mathrm{gen}}$	Held-out gain required to authorize a revision	0.16
$k$	Independent held-out records required for acceptance	2

Appendix

Table 4: Evidence-admission gate hyperparameters. All values are fixed across all experiments.

Figure 6: Gate sensitivity analysis. (a) Missed-mechanism rate as a joint function of the held-out gain threshold $\tau_{\mathrm{gen}}$ and held-out probe attenuation. The default operating point (circle) lies in the zero-error region, and the boundary moves monotonically with effective held-out gain. (b) Verification cost and missed-mechanism rate as a function of the anomaly threshold $\tau_{\mathrm{anom}}$ ; the default (dashed) lies on the flat zero-miss plateau before coverage collapses at more restrictive thresholds.

Figure 7: Performance progression and cost-effectiveness trade-off. (a) Solution Quality (SQ, %) over budget

T\in[1,24]

. (b) Cost-effectiveness trade-off: Solution Quality versus relative compute and token cost, defining the empirical efficiency frontier.

Figure 8: Execution runtime and discovery throughput across methods. (a) Mean wall-clock runtime per 24-step discovery run across all eight methods under Qwen3-235B. Triage-PiEvo incurs only

+13.6\%

overhead over standard PiEvo due to conditional anomaly-triggered verification, while remaining substantially faster than tree-search or multi-agent debate frameworks (AI Scientist v2, AI Researcher, PiFlow). (b) Scientific exploitation throughput: Exploitation Speed (AUOC, %) versus mean runtime (seconds). Triage-PiEvo achieves the peak discovery throughput, reaching the highest AUOC (

71.73\%

) at competitive execution latency.

Method	Official Score $\uparrow$	Target Attainment $\uparrow$	Verified Score $\uparrow$	Valid Runs (%) $\uparrow$	Critical Failure (%) $\downarrow$	Cost / Task ( $)$ \downarrow$
Claude Code	0.348 $\pm$ 0.032	0.312	0.298	78.5%	14.8%	0.84
Codex CLI	0.315 $\pm$ 0.041	0.274	0.261	69.2%	22.4%	1.12
AI-Researcher	0.332 $\pm$ 0.036	0.295	0.284	74.1%	18.6%	0.68
ResearchHarness	0.286 $\pm$ 0.045	0.241	0.230	65.8%	26.2%	0.75
Triage-PiEvo (Ours)	0.418 $\pm$ 0.028	0.386	0.382	86.5%	5.2%	0.92

Appendix

Table 5: Performance on AutoResearchEval Optimization Tasks. Comparison of Triage-PiEvo against autonomous scientific and coding agent harnesses across benchmark tasks. Official Score entries show empirical mean

\pm

SD over independent seeds; the other columns report means. Bold indicates the best result.

Harness	Variant	Task Score $\uparrow$	Verified Score $\uparrow$	False Admission (%) $\downarrow$	Revision Accuracy (%) $\uparrow$	Token Cost $\downarrow$	Runtime (s) $\downarrow$
ResearchHarness	Base	0.286	0.230	38.4%	44.2%	14.2k	48.5
	+ Self-Review	0.301	0.248	35.2%	48.5%	18.6k	62.1
	+ Triage (Ours)	0.345	0.328	14.2%	68.4%	21.4k	71.3
AI-Researcher	Base	0.332	0.284	32.6%	51.8%	16.5k	56.2
	+ Self-Review	0.348	0.305	29.8%	56.2%	22.1k	74.8
	+ Triage (Ours)	0.389	0.371	11.8%	73.6%	25.8k	86.5

Appendix

Table 6: Triage as a Plug-in across Autonomous Research Harnesses. Incorporating Triage enhances task score and verified reliability while reducing false evidence admissions across the three host architectures.

Panel A: Unreliable Evidence Admission Rate (%) $\downarrow$ (Evaluator Corruptions & Artifacts)
Injected Evaluation Regime	Cells / Trials	Model Performance
		PiEvo	w/o Gate	Triage-PiEvo (Ours)
Transient Noise & Dropout ( $z_{\mathrm{noise}},z_{\mathrm{exec}}$ )	24 cells	100.0%	100.0%	0.0%
Channel-Specific Faults ( $z_{\mathrm{fault}}$ )	24 cells	100.0%	100.0%	0.0%
Hidden Surrogate & Drift Artifacts ( $z_{\mathrm{surr}},z_{\mathrm{ood}},z_{\mathrm{drift}}$ )	48 cells	100.0%	100.0%	0.0%
Subtotal: Overall False Admission Rate	96 cells	100.0%	100.0%	0.0%

Appendix

Table 7: Controlled stress tests under evaluator corruptions and planted mechanisms. Panel A : Across 96 corrupted evaluator and hidden artifact cells, ungated baselines uniformly admit 100% of non-generalizing anomalies into persistent state, whereas Triage-PiEvo achieves 0.0% false admission. Panel B : Across 144 independent planted mechanism trials, Triage-PiEvo successfully recovers 100.0% (144/144) of true mechanisms without confirmation masking.

Method	Exploit Admissions $\downarrow$	Exploit Filter	Apparent Proxy	Held-out True	Held-out Gain
	(Count / Run)	Rate (%) $\uparrow$	Score $\uparrow$	Score $\uparrow$	$\Delta_{\mathrm{heldout}}$
Vanilla MAS	4.67 $\pm$ 1.15	0.0%	0.812 $\pm$ 0.045	0.412 $\pm$ 0.061	$-0.023$ $\pm$ 0.078
ReAct	4.00 $\pm$ 0.82	0.0%	0.785 $\pm$ 0.038	0.428 $\pm$ 0.055	$-0.007$ $\pm$ 0.073
PiEvo ( Pu et al., 2026 )	5.33 $\pm$ 0.58	0.0%	0.894 $\pm$ 0.021	0.435 $\pm$ 0.048	Baseline
Triage-PiEvo w/o Gate	6.00 $\pm$ 0.00	0.0%	0.932 $\pm$ 0.015	0.408 $\pm$ 0.052	$-0.027$ $\pm$ 0.071
Triage-PiEvo (Ours)	0.00 $\pm$ 0.00	100.0%	0.748 $\pm$ 0.029	0.644 $\pm$ 0.035	+0.209 $\pm$ 0.054

Appendix

Table 8: Robustness against proxy evaluator exploitation. In environments with exploitable surrogate simulator shortcuts, ungated baseline agents repeatedly incorporate deceptive artifacts (

5.33\sim 6.00

admissions per run), reporting inflated apparent proxy scores while collapsing on genuine held-out benchmarks. Triage-PiEvo filters 100% of exploit candidates, raising the true held-out score by

\mathbf{+0.209\pm 0.054}

Variant	False Revision (%) $\downarrow$	True Mechanism Recall (%) $\uparrow$	Verified Score $\uparrow$	Retraction Acc. (%) $\uparrow$	Verification Cost ( $)$ \downarrow$
Full Triage (Ours)	4.8%	92.4%	0.382	93.5%	0.24
$k=1$ (Permissive Gate)	22.4%	89.2%	0.362	78.5%	0.14
$k=3$ (Strict Gate)	6.5%	71.8%	0.370	90.8%	0.38
w/o Held-out Verification	34.5%	83.5%	0.315	61.2%	0.18
w/o Channel Reliability	19.2%	81.2%	0.352	76.3%	0.22
w/o Backtracking	16.5%	84.0%	0.338	51.8%	0.21

Appendix

Table 9: Ablation of Evidence Admission Mechanisms. Systematic evaluation of each architectural component in Triage . The full framework achieves the optimal balance between high mechanism recall, high verification accuracy, and low false revision risk.

AI Agents for Scientific Discovery

AI agents for scientific discovery use language models, tools, and experiments to generate, test, and refine scientific knowledge.

See all 288 papers

Collaborative Principle Evolution via Evidence Transfer for Scientific Discovery
Sep 28, 2026Yingming Pu, Hongyu Chen, Tao LinMulti-Agent Collaboration Multi-Agent Systems
Zhejiang University, Hangzhou, Zhejiang, China · Westlake University, Hangzhou, Zhejiang, China
ExpEvolve: Structuring and Reusing Experience for Open-Ended Discovery
Oct 21, 2026Yongqiang Chen, Chenxi Liu, Deyu Zou +4AI Agents for Scientific Discovery Large Language Model-Guided Scientific Discovery
MBZUAI · Carnegie Mellon University · TMLR Group, Hong Kong Baptist University +2
CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad
Mar 15, 2026Yongqiang Chen, Chenxi Liu, Zhenhao Chen +3AI Agents for Scientific Discovery Large Language Model-Guided Scientific Discovery
MBZUAI · Carnegie Mellon University · TMLR Group, Hong Kong Baptist University +1

Sep 28, 2026J/K move · Enter open · S save

Collaborative Principle Evolution via Evidence Transfer for Scientific Discovery

Yingming Pu, Hongyu Chen, Tao Lin

Zhejiang University, Hangzhou, Zhejiang, China · Westlake University, Hangzhou, Zhejiang, China

Multi-Agent Collaboration Multi-Agent Systems Large Language Model-Guided Scientific Discovery Autonomous Research Agents

Open paper AskPDF arXiv

Large Language Model (LLM)-based agents promise to automate scientific discovery, yet exploring the vast hypothesis space remains costly. Existing principle-evolution methods accelerate this loop, but operate sequentially, which caps exploration breadth and wastes wall-clock time on challenging problems. To address this, we formulate collaborative scientific discovery as evidence transfer between parallel principle-evolution branches. We present COEVOLVE, which realizes this transfer through a coordination core over parallel branches. By integrating value-of-information-gated routing and context-discounted likelihood injection, COEVOLVE enables branches to collaborate through shared measurements while keeping their principle posteriors separate. Across six scientific-discovery tasks under a matched evaluation budget, COEVOLVE attains a mean solution quality of 66.5% versus 57.0% for single-branch principle evolution, with a 1.80x mean wall-clock speedup on the GPT-5.6-Terra backbone; on five auto-research tasks delegated to an autonomous research harness, it is the only arm whose mean stays above the published SOTA anchor on every task. These results establish when evidence sharing accelerates parallel discovery and when transfer safeguards are necessary to limit negative or inert transfers

Abstract

AI Agents for Scientific Discovery

Explore similar work

Collaborative Principle Evolution via Evidence Transfer for Scientific Discovery

ExpEvolve: Structuring and Reusing Experience for Open-Ended Discovery

CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad