ScienceFlow: A long-horizon agent for ML research, scientific discovery and beyond

Mingming Zhao; Jiqian Dong; Kangping Xu; Zadid Hasan; Chengrui Fan; Shan Jiang; Shuai Mao; Yating Ling

cs.AIAug 14, 2026

ScienceFlow: A long-horizon agent for ML research, scientific discovery and beyond

Authors: Mingming Zhao, Jiqian Dong, Kangping Xu, Zadid Hasan, Chengrui Fan, Shan Jiang, Shuai Mao, Yating Ling,

Organizations: Noah’s Ark Lab, Huawei

Abstract

Enabling LLM agents to sustain productive, stable, and goal-aligned research over extended horizons is a central challenge for autonomous machine learning and scientific discovery, as progress hinges on continuously managing evolving state, exploration decisions, and computational resources. Pioneering autoresearch agents, despite great success, still lack mechanisms for continuity, recovery from dead ends, and value-driven compute allocation, which inherently undermines overall search efficiency, wastes computational resources, and lowers the chance of ultimate success. To bridge this gap, we introduce ScienceFlow, an end-to-end autoresearch agent framework that organizes long-horizon research work into research segments grounded in executable workspaces. It represents research progress as recoverable executable states, enabling efficient exploration, revision, and execution. Transitions between research segments are governed by Executable-State Transition through Re-Anchoring (ESTRA), which selects either the live state or an archived state as the next anchor and determines whether to continue or redirect the research trajectory. An evidence-aware execution controller allocates resources to physical jobs based on resource availability, remaining budget, and validated progress. We evaluate ScienceFlow on tasks spanning machine learning, scientific modeling, and mathematical optimization. Results on diverse long-horizon benchmarks demonstrate its ability to sustain effective research processes, highlighted by a SOTA 70.22 percent Any-Medal score on the full MLE-bench within a 24-hour budget, outperforming prior reported results by 4.92 percentage points. The efficacy of ScienceFlow further demonstrates that efficient state management, adaptive exploration, and objective-aligned execution are critical for scaling autonomous research beyond short-horizon interactions.

Figures & tables

Panel of Figure 1 — Figure 1: Performance overview. (a) End-to-end performance on the full MLE-bench, where ScienceFlow achieves $70.22\pm 1.18\%$ Any-Medal. (b) Capability analysis across Search, Storage, Compute, Model Adaptation, and Time dimensions under full and constrained-reference settings. Each dimension is evaluated independently.

Figure 2: ScienceFlow system architecture. Research workers operate over recoverable executable states and adapt long-horizon trajectories through ESTRA-governed transitions between research segments, while an evidence-aware execution controller coordinates physical resource allocation and runtime execution.

Agent	LLM(s)	Lite (%)	Medium (%)	High (%)	All (%)
Other reported systems
Famou-Agent	Gemini-2.5-Pro	75.76 $\pm$ 1.52	57.89 $\pm$ 1.52	40.00 $\pm$ 0.00	59.56 $\pm$ 0.89
CAIR MARS+	Gemini-3-Pro-Preview	78.79 $\pm$ 1.52	60.53 $\pm$ 1.52	44.44 $\pm$ 2.22	62.67 $\pm$ 0.77
AIBuildAI	Claude-Opus-4.6	77.27 $\pm$ 0.00	61.40 $\pm$ 0.88	46.67 $\pm$ 0.00	63.11 $\pm$ 0.44
Famou-Agent 2.0	Gemini-3-Pro-Preview	80.30 $\pm$ 1.52	64.04 $\pm$ 2.32	42.22 $\pm$ 2.22	64.44 $\pm$ 1.18
Iris	Claude-Opus-4.6	80.30 $\pm$ 1.50	64.00 $\pm$ 0.90	44.40 $\pm$ 2.20	64.90 $\pm$ 0.40

Table 1: Representative results on the full 75-task MLE-bench. Any-Medal rates (%) are reported as mean

\pm

SEM when three runs are available. Best results in each column are shown in bold .

Figure 3: A 24-hour ScienceFlow trajectory on the Statoil Iceberg Classifier Challenge , showing validation and retrospective test progress together with checkpointed workspace states and resource-control events. Retrospective test scores are shown only for analysis and are never exposed to the research agent.

Figure 4: State-matched counterfactual replay on the Tabular Playground Series May 2022 task. Seven historical ESTRA decision points are replayed from their captured executable states under matched budgets. Each point compares four alternatives formed by current or archived anchors and extend or redirect directions. Bars show the mean change in retrospective private-test AUROC relative to the source state across three replay seeds, with error bars denoting sample standard deviation. Red arrows mark the actions selected in the historical run.

Figure 5: MLE-bench Lite mechanism ablation over a 24-hour window. Curves show the cumulative Any-Medal rate averaged over three seeds, and shaded bands show the sample standard deviation.

Mechanism	Sub-metric	Value	Interpretation
ESTRA	ESTRA decisions	561 total	469 continue; 92 switch.
	Post-ESTRA completions	71 completed points	Post-ESTRA outcomes.
	Comparable improvements	19 / 70 (27.1%)	Selective gains.
Workspace state	Checkpoint reuse	436 / 437 (99.8%)	Near-complete reuse.
	Snapshot object reuse	177,234 / 221,925 (79.9%)	Snapshot dedup.
	Storage footprint	2204.6 GiB $\to$ 189.2 GiB	Storage-covered sources.

Table 2: Operational telemetry for ScienceFlow mechanisms on 54 of 75 MLE-bench tasks (72.0% coverage), including ESTRA activity, workspace-state reuse, storage efficiency, and resource control.

Panel of Figure 6 — Figure 6: Backbone sensitivity and efficiency on the Tabular Playground Series May 2022 task under the same two-worker, 24-hour protocol, showing cumulative input-token consumption and endpoint performance-cost trade-offs over three independent runs. “DS” is an abbreviation for “DeepSeek” in the figure legends.

Task	Method	Model	Performance
Circle Packing $\sum_{i}r_{i}$ ( $\uparrow$ )	AlphaEvolve ( 46 )	Gemini 2.0 Pro + Flash	2.6358627564
	ShinkaEvolve ( 36 )	Claude Sonnet 4 + GPT-4.1 family	2.6359828390
	ThetaEvolve ( 70 )	DeepSeek-R1-0528-Qwen3-8B	2.6359830774
	MLEvolve ( 17 )	Gemini-3.1-Pro-Preview	2.6359830395
	OpenEvolve ( 58 )	DeepSeek-V4-Flash-Preview	2.6344194866
	ScienceFlow (Ours)	DeepSeek-V4-Flash-Preview	2.6359830849

Table 3: Best-score comparison on three mathematical optimization tasks. Arrows indicate the optimization direction. Bold entries denote the best compared results, including ties at the reported precision; underlined entries denote the second-best results. Published baselines retain their reported protocols, while OpenEvolve is rerun locally under the configuration described in the text. Uncertainty-inequality results compare only Hermite–Gaussian constructions.

Rank	Team	Model	Score $\downarrow$	Submitted (UTC+8)
1	TGMA	–	337.646	2026-07-01 13:50
2	AC_TUWien	–	337.700	2026-07-01 01:05
3	ScienceFlow (Ours)	DS-V4-Pro-Preview / DS-V4-Preview + GLM-5.1	393.229	2026-07-01 09:33
4	fcmaes	–	436.015	2026-06-30 14:01
5	Team HRI	–	526.078	2026-06-30 21:17
6	SINTEF	–	587.050	2026-06-30 20:27

Table 4: Public leaderboard for the KTTSP-hard track. Scores report total mission elapsed time in days (lower is better), and timestamps identify each team’s best submission in UTC+8. Boldface marks the winning score and the ScienceFlow entry. In the model column, DS-V4-Preview denotes DeepSeek-V4-Flash-Preview, and DS-V4-Pro-Preview denotes DeepSeek-V4-Pro-Preview. Team names link to their public ESA Optimise profiles. Leaderboard snapshot accessed on July 6, 2026.

Figure 7: Persistent-memory growth and validation progress for worker W01 during the ten-day KTTSP-hard campaign. The right axis uses

\log_{10}(\mathrm{score}/2000)

coordinates with raw mission-day tick labels; lower is better. Invalid evaluation outputs are excluded.

Panel of Figure 8 — Figure 8: Route and timing refinement within the W01 KTTSP-hard trajectory. Figures (a) and (b) visualize two evaluator-valid candidates that visit the same target set. Target color progresses from blue to yellow with scheduled visit order, and thin links connect consecutive targets.

Task	Metric	Random	OpenCode	Pi	Codex	Claude Code	ScienceFlow (Ours)
Standalone scientific design tasks
TFBind8	BKM	0.7533	0.9764	0.9730	0.9718	0.9648	0.9769
Superconductor	NDCG	0.2800	0.4929	0.8671	0.8188	0.8132	0.8583
UTR MRL	NE	-0.0006	0.6523	0.7533	0.7424	0.7367	0.7605
TFBind10 Pho4	NE	0.0000	0.2467	0.2473	0.2024	0.3197	0.2543
GFP	NE	0.0001	0.2600	0.2692	0.2586	0.1977	0.2732

Table 5: Task-level results under a two-hour budget with DeepSeek-V4-Flash-Preview. BKM, NE, and NDCG denote best-

K

mean, normalized enrichment, and global NDCG. Best and second-best agent scores are bolded and underlined .

Agent	Standalone Design (6)	Rat Clinical Pathology (6)	Group-balanced Score $\uparrow$	7-group Avg. Rank $\downarrow$	Task Wins
Pi	52.77	43.12	51.39	2.79	1/12
Claude Code	51.65	49.64	51.37	3.69	3/12
Codex	51.49	43.79	50.39	3.64	1/12
OpenCode	43.84	46.44	44.21	3.33	2/12
ScienceFlow (Ours)	54.16	55.91	54.41	1.55	5/12

Table 6: Group-balanced results. Scores map Random to 0 and the attainable optimum to 100. The DrugMatrix endpoints form one group, weighted equally with each standalone task. Average rank uses the same seven groups. Best and second-best values are bolded and underlined ; lower rank is better.

System	Distinct candidates	Pool coverage
ScienceFlow	66	16.9%
Pi	38	9.7%
OpenCode	32	8.2%
Codex	24	6.2%
Claude Code	32	8.2%

Table 7: Candidate coverage in the Phosphorus case study. Repeated conditions across batches are counted once; coverage is relative to the 390-condition candidate pool.

Panel of Figure 9 — Figure 9: Cross-task progress under matched query budgets. The DrugMatrix endpoints form one group, and ScienceFlow’s worker submissions are merged in chronological order.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Contents	Update scope
$P_{\mathrm{worker}}$	Generic research-worker and workspace-interaction policy	Worker lifecycle
$P_{\mathrm{runtime}}$	Runtime behavior, safety, and tool-use policy	Worker lifecycle
$P_{\mathrm{tools}}$	Tool names, descriptions, argument schemas, and call policy	Worker lifecycle
$P_{\mathrm{task}}$	Objective, evaluator contract, budget envelope, task adapter, and initial analysis guidance	Task run
$P_{\mathrm{anchor}}(a_{n})$	Workspace view, folded or unfolded memory, validation evidence, and resource status for the selected anchor	Research segment
$P_{\mathrm{dir}}(d_{n})$	Extend-or-redirect instruction selected by ESTRA	Research segment

Appendix

Table A1: Model-facing context components in assembly order. The first four components form the cache-friendly prefix; subsequent components vary with the recoverable state or current research segment.

Component	Contents
Current state	Active workspace summary, latest result card, and current validation evidence
Archived anchors	Recoverable states with workspace summaries and associated validation evidence
Memory view	Folded research history, recent result cards, and indexed evidence available through Unfold
Peer evidence	Optional validated scores and compact method summaries from other workers
Resource envelope $B_{t}$	Remaining wall-clock budget and current execution constraints

Appendix

Table A2: Evidence provided to ESTRA at a research-segment boundary. Optional components are included only when corresponding records are available.

Agent	LLM(s) used	Lite (%)	Medium (%)	High (%)	All (%)
ScienceFlow (Ours) 5	DeepSeek-V4-Flash-Preview	80.30 $\pm$ 1.52	74.56 $\pm$ 0.88	44.44 $\pm$ 2.22	70.22 $\pm$ 1.18
Public study
Iris ( 22 ) 4	Claude-Opus-4.6	80.30 $\pm$ 1.50	64.00 $\pm$ 0.90	44.40 $\pm$ 2.20	64.90 $\pm$ 0.40
MLEvolve ( 17 ) 6	Gemini-3.1-Pro-Preview	80.30 $\pm$ 1.50	64.00 $\pm$ 0.90	46.70 $\pm$ 0.00	65.30 $\pm$ 0.80
Official leaderboard
Famou-Agent 2.0 ( 38 )	Gemini-3-Pro-Preview	80.30 $\pm$ 1.52	64.04 $\pm$ 2.32	42.22 $\pm$ 2.22	64.44 $\pm$ 1.18

Appendix

Table A3: Full-set MLE-bench Any-Medal results. Values are percentages reported as mean

\pm

SEM over three runs. Baseline results retain their published reporting protocols. Systems are grouped by source, and the best result in each column is bolded.

Competition Name	Year	Score	Cost($)	GPUs	Any-Medal	First Medal Time (h)
detecting-insults-in-social-commentary	2012	0.9588	0.0618	1	$\checkmark$	2.54
the-icml-2013-whale-challenge-right-whale-redux	2013	0.9486	1.4995	1	$\checkmark$	0.12
mlsp-2013-birds	2013	0.9099	0.1483	1	$\checkmark$	1.95
random-acts-of-pizza	2015	0.7713	1.0641	1	$\checkmark$	0.35
denoising-dirty-documents	2015	0.0116	0.0141	1	$\checkmark$	1.08
text-normalization-challenge-russian-language	2017	0.9791	0.3177	1	$\checkmark$	1.35

Appendix

Table A4: Per-task results on the 22-task MLE-bench Lite split. Field definitions follow Section B.1 .

Competition Name	Year	Score	Cost($)	GPUs	Any-Medal	First Medal Time (h)
AI4Code	2022	0.7911	1.6478	1	-	-
alaska2-image-steganalysis	2020	0.8992	1.6478	2	$\checkmark$	22.27
billion-word-imputation	2014	4.4668	0.1794	2	$\checkmark$	7.87
cassava-leaf-disease-classification	2021	0.8995	0.3477	2	$\checkmark$	11.77
cdiscount-image-classification-challenge	2017	0.7248	0.4307	2	$\checkmark$	23.04
chaii-hindi-and-tamil-question-answering	2021	0.7458	0.0056	1	$\checkmark$	6.11

Appendix

Table A5: Per-task results on the 38-task MLE-bench Medium split. Field definitions follow Section B.1 .

Competition Name	Year	Score	Cost($)	GPUs	Any-Medal	First Medal Time (h)
3d-object-detection-for-autonomous-vehicles	2019	0.0597	0.1874	1	$\checkmark$	1.73
bms-molecular-translation	2021	4.8469	1.0800	1	-	-
google-research-identify-contrails-reduce-global-warming	2023	0.4474	4.1043	1	-	-
hms-harmful-brain-activity-classification	2024	0.8100	4.1043	1	-	-
iwildcam-2019-fgvc6	2019	0.2107	0.1442	1	$\checkmark$	0.68
nfl-player-contact-detection	2023	0.6585	4.1043	1	$\checkmark$	1.63

Appendix

Table A6: Per-task results on the 15-task MLE-bench High split. Field definitions follow Section B.1 .

Time (UTC)	Worker	Stage	Duration	Search event
Jun 21 16:12	W00	S60	521.9054	Early peer best
Jun 21 16:20	W01	S25	1746.8935	Structural ESTRA redirect
Jun 25 18:04	W01	S78	1321.8824	Fine inclination bins
Jun 25 19:12	W01	S79	1321.8824	Operator-level ESTRA redirect
Jun 25 23:01	W01	S80	573.0305	Phase-greedy narrow bins
Jun 26 00:00	W01	S85	525.9948	Near peer best

Appendix

Table A7: Validated checkpoints and ESTRA decisions in the KTTSP-hard peer-guided overtaking case. Mission duration is measured in days; lower is better.

Task group	Visible	Evaluation	$N/K/Q$	Trusted evaluator	Source / terms
TFBind8 BBO	32,768 sequences	65,536 sequences	32/5/20	Normalized PBM E-score lookup; BKM	PBM/UniPROBE ( 4 ; 31 ) ; source-specific academic-use terms
TFBind10 Pho4 BBO	2,087,323 rows (524,300 seq.)	$4^{10}$ sequences	128/16/20	Four-replicate affinity posterior; NE	BET-seq ( 37 ; 21 ) ; CC BY 4.0
Superconductor ranking	16,795 records (12,179 groups)	2,985 groups	32/5/20	Median critical temperature by composition; NDCG	UCI Superconductivity Data ( 27 ; 26 ) ; CC BY 4.0
UTR MRL ranking	76,877 50-mers	5,043 50-mers	128/16/10	Two-replicate mean ribosome load; NE	GEO GSE114002/mRNABench ( 55 ; 54 ) ; terms unknown
GFP ranking	41,372 proteins	10,343 proteins	128/16/20	Protein-level median log brightness; NE	Sarkisyan Figshare ( 8 ; 56 ) ; CC BY 4.0
Hopper Controller ranking	1,920 policies	1,280 policies	32/5/10	Mean of 500 frozen Hopper-v5 rollouts; NDCG	Design-Bench policy assets (terms not separately stated); Gymnasium (MIT), MuJoCo (Apache-2.0) ( 66 ; 64 )

Appendix

Table A8: SciModelingBench task metadata. Visible scale describes the data given to the agent; evaluation scale describes the exact reference domain or label-hidden pool. Values in parentheses are canonical candidates represented by row-level observations. DrugMatrix contains six endpoint tasks from one study collection. BKM, NE, and NDCG denote best-

K

mean , normalized enrichment , and global NDCG . Data terms come from the original source, while the Hopper row separately identifies simulator software licenses.

Long-Horizon LLM Agents

Long-horizon large language model agents plan, use tools, and preserve task state across extended, multi-step interactions.

See all 446 papers

Toward Autonomous Long-Horizon Engineering for ML Research
Apr 14, 2026Guoxin Chen, Jie Chen, Lei Chen +7Long-Horizon LLM Agents Multi-Agent LLM Systems
AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation
May 27, 2026Shanghua Gao, Ada Fang, Marinka ZitnikProtein Fitness Prediction Multi-Agent Collaboration
Harvard University
PrimeScientist: Strategic Allocation of Research Effort in Autonomous Research
Sep 15, 2026Xinle Yu, Fan Bai, Kaiser Sun +5AI for Science Autonomous Research Agents
UC San Diego · Johns Hopkins University

Apr 14, 2026J/K move · Enter open · S save

Toward Autonomous Long-Horizon Engineering for ML Research

Guoxin Chen, Jie Chen, Lei Chen, Jiale Zhao+6

Long-Horizon LLM Agents Multi-Agent LLM Systems Memory-Augmented Agents

Open paper AskPDF arXiv

Agentic systems increasingly automate pieces of AI research. Yet turning underspecified research objectives into runnable, experimentally validated ML systems remains a central bottleneck. We study this operational setting as long-horizon ML research engineering: converting a research specification into a runnable ML system through repeated implementation, experimentation, and refinement. The central challenge is to sustain cumulative project progress across heterogeneous stages under delayed, confounded feedback. We introduce AiScientist, a multi-agent system built around thin control over thick state: a lightweight hierarchical research team coordinates through a File-as-Bus workspace that preserves decision-relevant artifacts across roles and invocations. On PaperBench, AiScientist improves over the strongest matched baselines by 9.92 and 11.15 points with Gemini-3-Flash and GLM-5, respectively. On MLE-Bench Lite, it reaches 81.82 Any Medal% under both backbones, improving over the strongest matched baselines by 4.55 and 16.67 points, and exceeding a Codex/GPT-5.5 xhigh frontier harness reference by 13.64 Any Medal points. Ablations and process analyses show that durable project state is central to later-round refinement: removing File-as-Bus lowers PaperBench score by 6.41 points and MLE-Bench Lite Any Medal% by 31.82 points. These results suggest that long-horizon AI research is not only a problem of stronger local reasoning, but a systems problem of maintaining cumulative, inspectable project progress.

Abstract

Long-Horizon LLM Agents

Explore similar work

Toward Autonomous Long-Horizon Engineering for ML Research

AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentation

PrimeScientist: Strategic Allocation of Research Effort in Autonomous Research