Modern coding agents are fundamentally reshaping the role of human input for computational research: they write code, launch in silico experiments, analyze their results, and autonomously operate with ever-increasing time horizons. Despite these developments, fully autonomous research agents suffer from a limited ideation radius and myopic hill-climbing search dynamics. To bridge this gap, we introduce The Agentic Scientist, an end-to-end distributed pipeline for human researchers to conduct computational research alongside modern coding agents. Building on its predecessor, our system decomposes research into task drafting, idea generation, planning, sandboxed experimentation, result aggregation, baseline and ablation construction, post-hoc analysis, and report writing. Importantly, each step in the pipeline is powered by the next generation of coding agents and leverages agentic self-verification. The overall research process is driven by a meta-controller that manages parallel research agents and enforces rubric verification at each stage. This design enables the same orchestration layer to be reused across heterogeneous domains with different execution environments, agentic coding harnesses, and evaluation metrics. Our system can be used across a spectrum of autonomy: either as a co-scientist tool or to perform end-to-end scientific investigation. Within the co-scientist setting, we leveraged a development version of our system to submit a single research paper, which got accepted for publication at ALIFE 2026. We further apply The Agentic Scientist fully autonomously to open problem-solving tasks in machine learning, and demonstrate the system's capability of scientific discovery across different domains of computational research. Our ablations suggest that reliable scientific autonomy depends not only on base model capabilities, but also on the cross-agent orchestration layer.
Figures & tables
Figure 1: The Agentic Scientist Four Stage Workflow.
Figure 2: Task drafting workflow.
Figure 3: Investigation workflow.
Figure 4: Evidence-to-paper pipeline.
Figure 5: autoillustrate pipeline , generated by autoillustrate itself. (A) Six phases turn a document and an instruction into an editable draw.io figure: LLM sub-agents decide content, while deterministic Python owns geometry and the generative image model only restyles. (B) In Phase 2, panels are rendered from a combination of automatically-generated templates whose coordinates are determined by LLM-generated Python code. (C) In Phases 5–6, the enhanced candidate with the best audit results is selected; if none is acceptable, the deterministic render is used as a fallback.
Figure 6: Camera-ready ALIFE 2026 paper after the single submitted manuscript was accepted. The full manuscript and additional details can be found in Appendix D .
Figure 7: Creation process of the ALIFE paper: human researchers defined the initial direction, baseline, and rubrics; supported the experiment domain transition; verified results, references, and claims before submission; and incorporated reviewer feedback into the camera-ready paper.
Revision stage
Inserted
Deleted
Net change
Agentic Scientist → submitted
664 (17.5%)
405 (10.7%)
+259 ( +6.8 %)
Submitted → camera-ready
1,329 (32.8%)
630 (15.6%)
+699 ( +17.3 %)
Table 2: Revision statistics across manuscript stages. Counts are word-equivalents computed directly from the latexdiff LaTeX sources. Percentages are relative to the pre-revision manuscript length.
Task
Starting Point
Research Objective
Learned optimization
Baseline : ELO’s expert-supervised MLP optimizer, meta-trained on (Fashion-)MNIST ( Huang et al., 2026b ) ; Evaluation : CIFAR-10 and SVHN.
Improve transfer to unseen datasets and network architectures under a fixed meta-training budget. Search iteration budget : 2 H100s, 150 min.
World models
Baseline : Action-free Pong model, with a trained discrete tokenizer and masked-token dynamics; Evaluation : Error reduction against TinyWorlds
Reduce held-out free-running rollout error across prediction horizons relative to Majmudar (2026) . Search iteration budget : 4 H100s, 180 min.
Improve reconstruction at fixed sparsity while preserving model behavior and feature utilization. Search iteration budget : 2 H100s, 60 min.
Tabular distribution shift
Baseline : TabICLv2’s ( Qu et al., 2026b ) stage-one synthetic-prior pretraining from scratch recipe; Evaluation : 4 held-out real datasets vs. reference.
Improve in-context balanced accuracy on held-out real table data & across context sizes. Search iteration budget : 4 H100s, 150 min.
Hyperparameter optimization
Baseline : Adaptive incumbent-centred local search; Evaluation : BBOB ( Hansen et al., 2021 ) , HPO-B ( Arango et al., 2021 ) , YAHPO ( Pfisterer et al., 2022 ) .
Improve sample-efficient optimization over continuous, mixed, and hierarchical search spaces. Search iteration budget : 8 CPU cores, 30 min.
Table 3: Research tasks, starting implementations, and objectives used in the aggregate comparison.
Figure 8: Pre-revision reviewer scores by (a) model release date and (b) harness, averaged across five tasks; (c) within-task percentile ranks of final performance and reviewer scores, colored by task. The least-squares fit uses a pointwise 95% band from 10,000 bootstrap resamples of the 14 complete harness–model profiles across all five tasks, conditional on the observed tasks and fixed ranks.
Figure 9: Search scores (top), rubric pass rates (middle), and final performance versus reviewer score (bottom) across five tasks. Bottom panels show within-task performance percentiles against original 1–10 reviewer scores; annotated least-squares slopes give reviewer-score points per 10 percentile points. Final performance is positively associated with reviewer scores on four of the five tasks.
Figure 10: (Left) Search trajectories for Doc-to-LoRA scientist runs given 40 idea iteration budget with Codex + GPT-5.6-Sol, Claude + Opus-5, and Pi + DeepSeek-V4-Pro on the multi-NIAH task. Both Codex and Claude runs improve steadily on the task. (Right) Exact-match accuracy is shown as a heatmap grouped by the number of test sample needles. Red crosses indicate invalid iterations.
Figure 11: Doc-to-LoRA composition mechanism ablations figure in the generated report ( Section C.1 ). (a) Hypernetwork’s head redesign, auxiliary objective, and unconditional merging stay at or below the 51.7% exact-match accuracy, while query-conditioned composition reaches 98.0% . (b) Starting from the query-blind (unconditional) merge, query-dependent merging provides a significant improvement; relevance-based text placement and delimiter-aligned chunking close the remaining gap.
Figure 12: Search trajectories for 30 idea iterations with GPT-5.6-Sol + Codex, Opus-5 + Claude, DeepSeek V4 Flash + Pi, and Kimi K3 + OpenCode. Dots are individual Ant search-validation evaluations and step curves are the cumulative best.
Figure 13: Post-search evaluation across eight Brax-v1 legacy-spring environments. Curves show the mean best-so-far return over 16 shared seeds; shaded regions denote one standard error. The grey Ant panel marks the search-validation reference environment, evaluated here with new seeds and the reduced transfer-policy interface; the other seven environments were unseen during agentic search.
Figure 14: Post-search test accuracy of the best program of each GRPO run. Each evaluation selects a run’s best program using the fixed validation set, evaluates it on the held-out test set across five seeds, and reports the mean and standard deviation.
Figure 15: Search dynamics of The Agentic Scientist on RL fine-tuning across three seeds per worker configuration. Left: per-evaluation scores against iteration index and wall-clock duration. Right: the combined 1- and 2-worker search tree per task, with examples of cross-worker inspirations relayed through shared memory. Small amounts of parallelism improve wall clock time and effectively transfer knowledge across workers. Further scaling of the parallelism does not show improvements.
Figure 16: Cross-agent comparison under the same orchestration scaffold across three seeds per agent. Left to Right : score evolution during the search, test performance after reevaluation, and cumulative API cost per evolution cycle. Crosses mark evaluations whose program modified the evaluation protocol (identified by manual audit); they are excluded from all running-best curves.
Figure 17: Effect of shared knowledge on 2-worker GRPO search across three seeds per configuration. Crosses mark evaluations from the no-shared-knowledge runs whose program modified the evaluation protocol (identified by manual audit); they are excluded from all running-best curves.
Figure 18: autoillustrate applied to the Darwin Gödel Machine paper ( Zhang et al., 2026a ) : the editable diagram (left) and its corresponding enhanced figure (right).
Method
Variant
Faithfulness
Conciseness
Readability
Aesthetic
Overall
Editable
source
autoillustrate
Claude
32.0
37.2
47.8
83.4
37.5
✓
+ no critic
25.2
37.5
50.0
82.7
31.8
✓
+ low effort
24.5
37.3
47.8
82.5
29.8
✓
Codex
33.6
37.5
34.4
56.3
28.4
✓
PaperBanana
Gemini
18.5
55.3
42.0
71.6
25.0
✗
Table 4: Mean scores on PaperBananaBench, using GPT-6-Astra as VLM judge.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 19: First pages of the five highlighted task manuscripts, selected by highest AI reviewer score and final task performance. The generated papers are watermarked by The Agentic Scientist .
Environment
CMA-ES
GPT-5.6-Sol + Codex
Opus-5 + Claude Code (CMA-Anchored LearnedES)
DeepSeek V4 Flash + Pi
Ant
3328.12±28.41
3053.53±39.02
3509.26±59.09
3373.30±48.44
HalfCheetah
3704.73±19.53
3435.84±95.04
3448.89±111.84
3768.87±138.32
Hopper
1464.10±36.34
923.09±44.29
860.66±39.68
996.09±21.72
Humanoid
4368.72±105.59
4488.64±165.19
4682.69±187.04
3400.08±187.88
Reacher
−17.03±0.94
−3.32±0.30
−9.23±0.42
−21.23±1.89
Walker2d
1699.83±43.04
2094.39±53.41
1769.62±43.21
1821.07±41.05
Appendix
Table 5: Mean terminal best-so-far return ± one standard error over 16 shared seeds after 2,000 generations. Ant is the search-validation reference environment and uses new transfer seeds and the reduced transfer-policy interface; the other seven environments were unseen during agentic search. Bold denotes the highest mean within an environment.
Rubric strictness
Key ED constraints
Observed effect
Loose
No explicit wording to prohibit backpropagation
Best validation accuracy 0.9694 but failed to implement ED
Intermediate
Explicit wording to prohibit backprop
Best validation accuracy 0.976 . Implemented ED correctly
Strict
No backprop No W⊤ backward transport Explicit local update with one global error sign Detailed polarity invariance
Table 6: Rubric ablation design for MNIST Error Diffusion runs. The experiment varied the strictness and wording of the method-validity rubric while keeping the MNIST task, fixed split, search budget, and high-level exclusions constant.
Figure 20: autoillustrate applied to the agent draft of the Error Diffusion ALIFE paper, before human edits: the editable diagram (a) and its corresponding enhanced figure (b) . The figure in the submitted and camera-ready versions of the ALIFE paper was instead generated with Nano Banana Pro as a non-editable raster image; the human authors kept its correct right panel and manually redrew the left and middle panels to correct their inaccuracies. The prompt given to autoillustrate was: “Overview of the dual-stream Error Diffusion framework. Left: The excitatory/inhibitory architecture maintains separate streams ( p , n ) with four non-negative weight matrices per layer, enforcing Dale’s principle structurally. Center: The Error Diffusion update broadcasts output error directly to hidden layers without weight transport or random feedback matrices. Right: The shared architecture is applied to classification (with domain-specific sigmoid widths, mean-centered error, and asymmetric initialization) and reinforcement learning (with ReLU activations and PPO integration).”
Figure 21: Examples of figures generated by autoillustrate on PaperBananaBench ( Zhu et al., 2026a ) .
Hyperparameter
Value
Codebase version
autoillustrate v0.2.0
Coding agent
Claude Code ; Codex for the Codex variant
Agent model
claude-opus-4-8 ; gpt-5.6-sol for the Codex variant
Effort level
high
Panel inspects (Phase 3)
3: one inspect + up to two repair passes per panel
Enhancement attempts
4: Phase 5 enhance + 3 verify + 2 repair passes
Appendix
Table 7: Hyperparameters for the PaperBananaBench evaluation.
Method
Variant
Faithfulness
Conciseness
Readability
Aesthetic
Overall
Editable
source
autoillustrate
Claude
43.7
45.7
45.0
75.2
50.0
✓
+ no critic
42.8
41.6
45.2
75.3
46.6
✓
+ low effort
37.0
43.2
46.2
74.8
42.1
✓
Codex
42.3
39.9
31.0
49.8
34.9
✓
PaperBanana
Gemini
36.0
57.0
48.1
74.7
46.7
✗
Appendix
Table 8: Mean scores on PaperBananaBench, using Gemini-3.1-Pro as VLM judge.
Figure 22: Example PaperBanana (Claude Opus 4.8 + GPT Image 2) diagram on which the two VLM judges disagree on Faithfulness. Gemini-3.1-Pro rationale: “Accurately captures the structural differences and integrates specific text details like delta/relative trajectories ( ar/d ) and frame-averaged encoders. It strictly aligns with the caption’s visual simplification. No veto errors.” GPT-6-Astra rationale: “Correctly depicts the proposed method’s encoder/backbone distinction and action frames. However, in the original Diffusion Policy panel, the visual-feature connection terminates above the clean-action label rather than entering the U-Net. The separate, unlabeled arrow entering the U-Net is disconnected from those features. This omits the essential visual-conditioning connection and constitutes a logical-flow veto, not merely simplification.”
AutoIllustrate Diagrams
Cause of Failure
Edge attached to the wrong source. The equivariance loss LEqv is fed the downsampled latent directly from Downs , bypassing the decoder Dψ , whereas the method defines the loss on the decoded output Dψ(zdown) .
Space-inefficient layout. Disconnected annotations float above the main pipeline, yielding a non-rectangular bounding box with large dead zones that waste vertical space under standard publication formatting.
Insufficient visual abstraction. Raw equations are transcribed into the figure rather than distilled, most visibly the full argmax objective placed in a box at the top of Stage 3, and modules are labelled with bare symbols ( fθ , Wϕ ) instead of their function, reducing the diagram to a box-ified restatement of the text.
Appendix
Table 9: Example failure cases of AutoIllustrate.
Harness
Version
Assigned models
Codex
0.149.0
GPT-6 Astra; GPT-5.6-Sol; GPT-5.5; GPT-5.4
Claude Code
2.1.240
Fable 5.1; Opus 5; Opus 4.7; Opus 4.6
Pi
0.84.2
GPT-5.6-Sol; Opus 5; DeepSeek V4 Flash
OpenCode
1.18.21
GPT-5.6-Sol; Opus 5; DeepSeek V4 Flash
Appendix
Table 10: Harness and model configurations for the aggregate sweep. All runs use high reasoning effort.
Stage
Setting
Value
Agent search
Idea budget
40 ideation cycles
Model / harness
gpt-5.6-sol / Codex; claude-opus-5 / Claude; deepseek-v4-pro / Pi
Hardware
One H100 GPU with 16 CPU cores
Training time limit
3 hours; candidates above 3 hours are invalid
Dataset
Training datasets
NIAH samples range from 32 to 256 tokens
Evaluation datasets
NIAH samples range from 32 tokens to 64K tokens
Appendix
Table 11: Doc-to-LoRA multi-needle NIAH agent-search and evaluation protocol.
Table 12: MetaEvo training, search-validation, and transfer protocol. BBOB tasks meta-train neural weights; Ant subsequently supplies the score used by the agentic program search.
Setting
MNIST
CIFAR-10
MuJoCo / Brax
Craftax
Model / Harness
Sonnet 4.6 / Claude Code
Opus 4.6 / Claude Code
Composer 2.5 Fast / Cursor
GPT-5.5 / Codex
Agent search budget
20 ideation cycles
60 ideation cycles
60 ideation cycles
30 ideation cycles
Evaluator resources
1 GPU, 4 CPUs
1 GPU, 8 CPUs
1 GPU, 4 CPUs
1 GPU, 4 CPUs
Dataset / environment
MNIST: 55k train / 5k validation / 10k test
CIFAR-10: 45k train / 5k validation / 10k test
Ant, HalfCheetah, Humanoid
Craftax-Symbolic-v1
Final training budget
250 epochs
250 epochs
400M requested steps per seed
1B requested steps per seed
Appendix
Table 13: Error-Diffusion task and evaluation parameters.
Stage
Setting
Value
Agent search
Idea budget
50 ideation cycles
Model / harness
claude-opus-4-7 / Claude
Program evaluations
1 per idea
Pruning
Every 20 cycles
Hardware
Two H100 GPUs with 16 CPU cores
Training time limit
2.5 hours
Appendix
Table 14: GRPO agent-search and evaluation protocol. A fixed Qwen2.5-Math-1.5B model is trained on the MATH dataset with GRPO; final validation accuracy (mean reward) supplies the score used by the agentic parallel investigation phase, and the best program of each run is re-scored on a held-out test set.
AI coding agents are increasingly used for scientific work, but their end-to-end autonomous research capability remains difficult to verify. We present ResearchClawBench, a benchmark for evaluating autonomous scientific research across 40 tasks from 10 scientific domains. Each task is grounded in a real published paper, provides related literature and raw data, and hides the target paper during evaluation. Expert-curated multimodal rubrics decompose the target scientific artifacts into weighted criteria, enabling evaluation of target-paper-level re-discovery while leaving room for new discovery. We evaluate seven autonomous research (auto-research) agents under a unified protocol and seventeen native LLMs through the lightweight ResearchHarness. Current systems remain far from reliable re-discovery: the strongest autonomous agent, Claude Code, averages 21.5, and the strongest ResearchHarness LLM, Claude-Opus-4.7, averages 20.7, with an LLM frontier mean of only 26.5. Error analysis shows that failures concentrate in experimental protocol mismatch, evidence mismatch, and missing scientific core. ResearchClawBench provides a reproducible evaluation frontier for measuring progress toward autonomous scientific research.
A major challenge for niche scientific and technical domains in leveraging coding agents is the lack of access to up-to-date, domain- specific knowledge. Foundational models often demonstrate limited reasoning capabilities in specialized fields and cannot inherently incorporate knowledge that evolves through ongoing research and experimentation. Materials scientists exploring novel compounds, communication engineers designing and evaluating new protocols, and bioengineering researchers conducting iterative experiments all face this limitation. These experts typically lack the resources to fine-tune large models or continuously embed new findings, creating a barrier to adopting AI-driven coding agents. To address this, we introduce a framework that gives coding agents instanta- neous access to research repositories and technical documentation, enabling real-time, context-aware operation. Our open-source im- plementation allows users to upload documents via doc-search.dev and includes zed-fork, which enforces domain-specific rules and workflows. Together, these tools accelerate the integration of coding agents into specialized scientific and technical workflows
Automating scientific discovery requires more than generating papers from ideas. Real research is iterative: hypotheses are challenged from multiple perspectives, experiments fail and inform the next attempt, and lessons accumulate across cycles. Existing autonomous research systems often model this process as a linear pipeline: they rely on single-agent reasoning, stop when execution fails, and do not carry experience across runs. We present AutoResearchClaw, a multi-agent autonomous research pipeline built on five mechanisms: structured multi-agent debate for hypothesis generation and result analysis, a self-healing executor with a Pivot/Refine decision loop that transforms failures into information, verifiable result reporting that prevents fabricated numbers and hallucinated citations, human-in-the-loop collaboration with seven intervention modes spanning full autonomy to step-by-step oversight, and cross-run evolution that converts past mistakes into future safeguards. On ARC-Bench, a 25-topic experiment-stage benchmark, AutoResearchClaw outperforms AI Scientist v2 by 54.7%. A human-in-the-loop ablation across seven intervention modes reveals that precise, targeted collaboration at high-leverage decision points consistently outperforms both full autonomy and exhaustive step-by-step oversight. We position AutoResearchClaw as a research amplifier that augments rather than replaces human scientific judgment. Code is available at https://github.com/aiming-lab/AutoResearchClaw.