Enabling LLM agents to sustain productive, stable, and goal-aligned research over extended horizons is a central challenge for autonomous machine learning and scientific discovery, as progress hinges on continuously managing evolving state, exploration decisions, and computational resources. Pioneering autoresearch agents, despite great success, still lack mechanisms for continuity, recovery from dead ends, and value-driven compute allocation, which inherently undermines overall search efficiency, wastes computational resources, and lowers the chance of ultimate success. To bridge this gap, we introduce ScienceFlow, an end-to-end autoresearch agent framework that organizes long-horizon research work into research segments grounded in executable workspaces. It represents research progress as recoverable executable states, enabling efficient exploration, revision, and execution. Transitions between research segments are governed by Executable-State Transition through Re-Anchoring (ESTRA), which selects either the live state or an archived state as the next anchor and determines whether to continue or redirect the research trajectory. An evidence-aware execution controller allocates resources to physical jobs based on resource availability, remaining budget, and validated progress. We evaluate ScienceFlow on tasks spanning machine learning, scientific modeling, and mathematical optimization. Results on diverse long-horizon benchmarks demonstrate its ability to sustain effective research processes, highlighted by a SOTA 70.22 percent Any-Medal score on the full MLE-bench within a 24-hour budget, outperforming prior reported results by 4.92 percentage points. The efficacy of ScienceFlow further demonstrates that efficient state management, adaptive exploration, and objective-aligned execution are critical for scaling autonomous research beyond short-horizon interactions.
Figures & tables
Figure 1: Performance overview. (a) End-to-end performance on the full MLE-bench, where ScienceFlow achieves 70.22±1.18% Any-Medal. (b) Capability analysis across Search, Storage, Compute, Model Adaptation, and Time dimensions under full and constrained-reference settings. Each dimension is evaluated independently.
Figure 2: ScienceFlow system architecture. Research workers operate over recoverable executable states and adapt long-horizon trajectories through ESTRA-governed transitions between research segments, while an evidence-aware execution controller coordinates physical resource allocation and runtime execution.
Agent
LLM(s)
Lite (%)
Medium (%)
High (%)
All (%)
Other reported systems
Famou-Agent
Gemini-2.5-Pro
75.76 ± 1.52
57.89 ± 1.52
40.00 ± 0.00
59.56 ± 0.89
CAIR MARS+
Gemini-3-Pro-Preview
78.79 ± 1.52
60.53 ± 1.52
44.44 ± 2.22
62.67 ± 0.77
AIBuildAI
Claude-Opus-4.6
77.27 ± 0.00
61.40 ± 0.88
46.67 ± 0.00
63.11 ± 0.44
Famou-Agent 2.0
Gemini-3-Pro-Preview
80.30 ± 1.52
64.04 ± 2.32
42.22 ± 2.22
64.44 ± 1.18
Iris
Claude-Opus-4.6
80.30 ± 1.50
64.00 ± 0.90
44.40 ± 2.20
64.90 ± 0.40
Table 1: Representative results on the full 75-task MLE-bench. Any-Medal rates (%) are reported as mean ± SEM when three runs are available. Best results in each column are shown in bold .
Figure 3: A 24-hour ScienceFlow trajectory on the Statoil Iceberg Classifier Challenge , showing validation and retrospective test progress together with checkpointed workspace states and resource-control events. Retrospective test scores are shown only for analysis and are never exposed to the research agent.
Figure 4: State-matched counterfactual replay on the Tabular Playground Series May 2022 task. Seven historical ESTRA decision points are replayed from their captured executable states under matched budgets. Each point compares four alternatives formed by current or archived anchors and extend or redirect directions. Bars show the mean change in retrospective private-test AUROC relative to the source state across three replay seeds, with error bars denoting sample standard deviation. Red arrows mark the actions selected in the historical run.
Figure 5: MLE-bench Lite mechanism ablation over a 24-hour window. Curves show the cumulative Any-Medal rate averaged over three seeds, and shaded bands show the sample standard deviation.
Mechanism
Sub-metric
Value
Interpretation
ESTRA
ESTRA decisions
561 total
469 continue; 92 switch.
Post-ESTRA completions
71 completed points
Post-ESTRA outcomes.
Comparable improvements
19 / 70 (27.1%)
Selective gains.
Workspace state
Checkpoint reuse
436 / 437 (99.8%)
Near-complete reuse.
Snapshot object reuse
177,234 / 221,925 (79.9%)
Snapshot dedup.
Storage footprint
2204.6 GiB → 189.2 GiB
Storage-covered sources.
Table 2: Operational telemetry for ScienceFlow mechanisms on 54 of 75 MLE-bench tasks (72.0% coverage), including ESTRA activity, workspace-state reuse, storage efficiency, and resource control.
Figure 6: Backbone sensitivity and efficiency on the Tabular Playground Series May 2022 task under the same two-worker, 24-hour protocol, showing cumulative input-token consumption and endpoint performance-cost trade-offs over three independent runs. “DS” is an abbreviation for “DeepSeek” in the figure legends.
Task
Method
Model
Performance
Circle Packing ∑iri ( ↑ )
AlphaEvolve ( 46 )
Gemini 2.0 Pro + Flash
2.6358627564
ShinkaEvolve ( 36 )
Claude Sonnet 4 + GPT-4.1 family
2.6359828390
ThetaEvolve ( 70 )
DeepSeek-R1-0528-Qwen3-8B
2.6359830774
MLEvolve ( 17 )
Gemini-3.1-Pro-Preview
2.6359830395
OpenEvolve ( 58 )
DeepSeek-V4-Flash-Preview
2.6344194866
ScienceFlow (Ours)
DeepSeek-V4-Flash-Preview
2.6359830849
Table 3: Best-score comparison on three mathematical optimization tasks. Arrows indicate the optimization direction. Bold entries denote the best compared results, including ties at the reported precision; underlined entries denote the second-best results. Published baselines retain their reported protocols, while OpenEvolve is rerun locally under the configuration described in the text. Uncertainty-inequality results compare only Hermite–Gaussian constructions.
Rank
Team
Model
Score ↓
Submitted (UTC+8)
1
TGMA
–
337.646
2026-07-01 13:50
2
AC_TUWien
–
337.700
2026-07-01 01:05
3
ScienceFlow (Ours)
DS-V4-Pro-Preview / DS-V4-Preview + GLM-5.1
393.229
2026-07-01 09:33
4
fcmaes
–
436.015
2026-06-30 14:01
5
Team HRI
–
526.078
2026-06-30 21:17
6
SINTEF
–
587.050
2026-06-30 20:27
Table 4: Public leaderboard for the KTTSP-hard track. Scores report total mission elapsed time in days (lower is better), and timestamps identify each team’s best submission in UTC+8. Boldface marks the winning score and the ScienceFlow entry. In the model column, DS-V4-Preview denotes DeepSeek-V4-Flash-Preview, and DS-V4-Pro-Preview denotes DeepSeek-V4-Pro-Preview. Team names link to their public ESA Optimise profiles. Leaderboard snapshot accessed on July 6, 2026.
Figure 7: Persistent-memory growth and validation progress for worker W01 during the ten-day KTTSP-hard campaign. The right axis uses log10(score/2000) coordinates with raw mission-day tick labels; lower is better. Invalid evaluation outputs are excluded.
Figure 8: Route and timing refinement within the W01 KTTSP-hard trajectory. Figures (a) and (b) visualize two evaluator-valid candidates that visit the same target set. Target color progresses from blue to yellow with scheduled visit order, and thin links connect consecutive targets.
Task
Metric
Random
OpenCode
Pi
Codex
Claude Code
ScienceFlow (Ours)
Standalone scientific design tasks
TFBind8
BKM
0.7533
0.9764
0.9730
0.9718
0.9648
0.9769
Superconductor
NDCG
0.2800
0.4929
0.8671
0.8188
0.8132
0.8583
UTR MRL
NE
-0.0006
0.6523
0.7533
0.7424
0.7367
0.7605
TFBind10 Pho4
NE
0.0000
0.2467
0.2473
0.2024
0.3197
0.2543
GFP
NE
0.0001
0.2600
0.2692
0.2586
0.1977
0.2732
Table 5: Task-level results under a two-hour budget with DeepSeek-V4-Flash-Preview. BKM, NE, and NDCG denote best- K mean, normalized enrichment, and global NDCG. Best and second-best agent scores are bolded and underlined .
Agent
Standalone Design (6)
Rat Clinical Pathology (6)
Group-balanced Score ↑
7-group Avg. Rank ↓
Task Wins
Pi
52.77
43.12
51.39
2.79
1/12
Claude Code
51.65
49.64
51.37
3.69
3/12
Codex
51.49
43.79
50.39
3.64
1/12
OpenCode
43.84
46.44
44.21
3.33
2/12
ScienceFlow (Ours)
54.16
55.91
54.41
1.55
5/12
Table 6: Group-balanced results. Scores map Random to 0 and the attainable optimum to 100. The DrugMatrix endpoints form one group, weighted equally with each standalone task. Average rank uses the same seven groups. Best and second-best values are bolded and underlined ; lower rank is better.
System
Distinct candidates
Pool coverage
ScienceFlow
66
16.9%
Pi
38
9.7%
OpenCode
32
8.2%
Codex
24
6.2%
Claude Code
32
8.2%
Table 7: Candidate coverage in the Phosphorus case study. Repeated conditions across batches are counted once; coverage is relative to the 390-condition candidate pool.
Figure 9: Cross-task progress under matched query budgets. The DrugMatrix endpoints form one group, and ScienceFlow’s worker submissions are merged in chronological order.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Component
Contents
Update scope
Pworker
Generic research-worker and workspace-interaction policy
Worker lifecycle
Pruntime
Runtime behavior, safety, and tool-use policy
Worker lifecycle
Ptools
Tool names, descriptions, argument schemas, and call policy
Workspace view, folded or unfolded memory, validation evidence, and resource status for the selected anchor
Research segment
Pdir(dn)
Extend-or-redirect instruction selected by ESTRA
Research segment
Appendix
Table A1: Model-facing context components in assembly order. The first four components form the cache-friendly prefix; subsequent components vary with the recoverable state or current research segment.
Component
Contents
Current state
Active workspace summary, latest result card, and current validation evidence
Archived anchors
Recoverable states with workspace summaries and associated validation evidence
Memory view
Folded research history, recent result cards, and indexed evidence available through Unfold
Peer evidence
Optional validated scores and compact method summaries from other workers
Resource envelope Bt
Remaining wall-clock budget and current execution constraints
Appendix
Table A2: Evidence provided to ESTRA at a research-segment boundary. Optional components are included only when corresponding records are available.
Agent
LLM(s) used
Lite (%)
Medium (%)
High (%)
All (%)
ScienceFlow (Ours) 5
DeepSeek-V4-Flash-Preview
80.30 ± 1.52
74.56 ± 0.88
44.44 ± 2.22
70.22 ± 1.18
Public study
Iris ( 22 ) 4
Claude-Opus-4.6
80.30 ± 1.50
64.00 ± 0.90
44.40 ± 2.20
64.90 ± 0.40
MLEvolve ( 17 ) 6
Gemini-3.1-Pro-Preview
80.30 ± 1.50
64.00 ± 0.90
46.70 ± 0.00
65.30 ± 0.80
Official leaderboard
Famou-Agent 2.0 ( 38 )
Gemini-3-Pro-Preview
80.30 ± 1.52
64.04 ± 2.32
42.22 ± 2.22
64.44 ± 1.18
Appendix
Table A3: Full-set MLE-bench Any-Medal results. Values are percentages reported as mean ± SEM over three runs. Baseline results retain their published reporting protocols. Systems are grouped by source, and the best result in each column is bolded.
Competition Name
Year
Score
Cost($)
GPUs
Any-Medal
First Medal Time (h)
detecting-insults-in-social-commentary
2012
0.9588
0.0618
1
✓
2.54
the-icml-2013-whale-challenge-right-whale-redux
2013
0.9486
1.4995
1
✓
0.12
mlsp-2013-birds
2013
0.9099
0.1483
1
✓
1.95
random-acts-of-pizza
2015
0.7713
1.0641
1
✓
0.35
denoising-dirty-documents
2015
0.0116
0.0141
1
✓
1.08
text-normalization-challenge-russian-language
2017
0.9791
0.3177
1
✓
1.35
Appendix
Table A4: Per-task results on the 22-task MLE-bench Lite split. Field definitions follow Section B.1 .
Competition Name
Year
Score
Cost($)
GPUs
Any-Medal
First Medal Time (h)
AI4Code
2022
0.7911
1.6478
1
-
-
alaska2-image-steganalysis
2020
0.8992
1.6478
2
✓
22.27
billion-word-imputation
2014
4.4668
0.1794
2
✓
7.87
cassava-leaf-disease-classification
2021
0.8995
0.3477
2
✓
11.77
cdiscount-image-classification-challenge
2017
0.7248
0.4307
2
✓
23.04
chaii-hindi-and-tamil-question-answering
2021
0.7458
0.0056
1
✓
6.11
Appendix
Table A5: Per-task results on the 38-task MLE-bench Medium split. Field definitions follow Section B.1 .
Table A6: Per-task results on the 15-task MLE-bench High split. Field definitions follow Section B.1 .
Time (UTC)
Worker
Stage
Duration
Search event
Jun 21 16:12
W00
S60
521.9054
Early peer best
Jun 21 16:20
W01
S25
1746.8935
Structural ESTRA redirect
Jun 25 18:04
W01
S78
1321.8824
Fine inclination bins
Jun 25 19:12
W01
S79
1321.8824
Operator-level ESTRA redirect
Jun 25 23:01
W01
S80
573.0305
Phase-greedy narrow bins
Jun 26 00:00
W01
S85
525.9948
Near peer best
Appendix
Table A7: Validated checkpoints and ESTRA decisions in the KTTSP-hard peer-guided overtaking case. Mission duration is measured in days; lower is better.
Table A8: SciModelingBench task metadata. Visible scale describes the data given to the agent; evaluation scale describes the exact reference domain or label-hidden pool. Values in parentheses are canonical candidates represented by row-level observations. DrugMatrix contains six endpoint tasks from one study collection. BKM, NE, and NDCG denote best- K mean , normalized enrichment , and global NDCG . Data terms come from the original source, while the Hopper row separately identifies simulator software licenses.
Agentic systems increasingly automate pieces of AI research. Yet turning underspecified research objectives into runnable, experimentally validated ML systems remains a central bottleneck. We study this operational setting as long-horizon ML research engineering: converting a research specification into a runnable ML system through repeated implementation, experimentation, and refinement. The central challenge is to sustain cumulative project progress across heterogeneous stages under delayed, confounded feedback. We introduce AiScientist, a multi-agent system built around thin control over thick state: a lightweight hierarchical research team coordinates through a File-as-Bus workspace that preserves decision-relevant artifacts across roles and invocations. On PaperBench, AiScientist improves over the strongest matched baselines by 9.92 and 11.15 points with Gemini-3-Flash and GLM-5, respectively. On MLE-Bench Lite, it reaches 81.82 Any Medal% under both backbones, improving over the strongest matched baselines by 4.55 and 16.67 points, and exceeding a Codex/GPT-5.5 xhigh frontier harness reference by 13.64 Any Medal points. Ablations and process analyses show that durable project state is central to later-round refinement: removing File-as-Bus lowers PaperBench score by 6.41 points and MLE-Bench Lite Any Medal% by 31.82 points. These results suggest that long-horizon AI research is not only a problem of stronger local reasoning, but a systems problem of maintaining cumulative, inspectable project progress.
Scientific research proceeds through iterative cycles of hypothesis generation, experiment design, execution, and revision. AI agents can automate parts of this process, but existing approaches typically follow a single research trajectory or coordinate through a central planner with fixed objectives. As a result, they struggle to sustain parallel exploration, adapt as experimental evidence changes, or preserve knowledge of failed directions over long-running experiments. We introduce AutoScientists, a decentralized team of AI agents for long-running computational scientific experimentation. Agents interpret a shared experimental state, self-organize into teams around promising hypotheses, critique proposals before using experimental compute, and share successes and failures to reduce redundant exploration. Under matched experimental budgets, AutoScientists improves over prior AI agents across biomedical machine learning, language-model training optimization, and protein fitness prediction. On BioML-Bench, spanning biomedical imaging, protein engineering, single-cell omics, and drug discovery, AutoScientists achieves a mean leaderboard percentile of 74.4% across 24 tasks, improving over the strongest AI agent by +8.33%. On GPT training optimization, AutoScientists reaches a target validation bits-per-byte 1.9x faster than Autoresearch and continues discovering improvements from a starting champion where the single-agent approach finds none (7 vs. 0 accepted improvements). On ProteinGym fitness prediction, AutoScientists discovers a method for ACE2-Spike binding that improves over the current state-of-the-art model by +12.5% in Spearman correlation. Applied without modification across all 217 ProteinGym assays, the same method improves over the prior state of the art by +6.5% (Spearman correlation).
Autonomous research agents aim to automate scientific workflows, from proposing ideas to conducting experiments and analyzing results. Yet current AI and research agents can propose more directions than available resources allow them to pursue. Moreover, each attempt could consume substantial resources, requiring agents to reconsider how to invest in subsequent research. Thus, deciding how to invest research effort strategically should be a defining capability of autonomous research agents. Accordingly, we introduce PrimeScientist, which jointly determines research direction and resource investment across successive research attempts. Specifically, we formulate this challenge of strategic research effort allocation as a sequential decision problem where remaining resources should explicitly guide the research policy. We first introduce an executable plan tree that preserves competing plans and their outcomes across attempts. Building on this representation, we propose an adaptive MCTS-based allocation policy that balances exploration and exploitation using experimental feedback and remaining resources. Comprehensive evaluations across AI research, systems and code optimization, and machine learning engineering show that strategic allocation improves research quality and sample efficiency together. Across 12 AI research tasks, PrimeScientist improves average reward by 10.3% with 50.6% fewer research attempts than AutoResearch under the same resource budget. We believe making research effort allocation an explicit optimization target establishes effective resource use as a core research capability for autonomous agents to drive scientific breakthroughs at scale.