cs.AIOct 13, 2026
SaveThe Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery
Organizations: Sakana AI
Abstract
Modern coding agents are fundamentally reshaping the role of human input for computational research: they write code, launch in silico experiments, analyze their results, and autonomously operate with ever-increasing time horizons. Despite these developments, fully autonomous research agents suffer from a limited ideation radius and myopic hill-climbing search dynamics. To bridge this gap, we introduce The Agentic Scientist, an end-to-end distributed pipeline for human researchers to conduct computational research alongside modern coding agents. Building on its predecessor, our system decomposes research into task drafting, idea generation, planning, sandboxed experimentation, result aggregation, baseline and ablation construction, post-hoc analysis, and report writing. Importantly, each step in the pipeline is powered by the next generation of coding agents and leverages agentic self-verification. The overall research process is driven by a meta-controller that manages parallel research agents and enforces rubric verification at each stage. This design enables the same orchestration layer to be reused across heterogeneous domains with different execution environments, agentic coding harnesses, and evaluation metrics. Our system can be used across a spectrum of autonomy: either as a co-scientist tool or to perform end-to-end scientific investigation. Within the co-scientist setting, we leveraged a development version of our system to submit a single research paper, which got accepted for publication at ALIFE 2026. We further apply The Agentic Scientist fully autonomously to open problem-solving tasks in machine learning, and demonstrate the system's capability of scientific discovery across different domains of computational research. Our ablations suggest that reliable scientific autonomy depends not only on base model capabilities, but also on the cross-agent orchestration layer.
Figures & tables
Figure 1: The Agentic Scientist Four Stage Workflow.
Figure 2: Task drafting workflow.
Figure 3: Investigation workflow.
Figure 4: Evidence-to-paper pipeline.
Figure 5: autoillustrate pipeline , generated by autoillustrate itself. (A) Six phases turn a document and an instruction into an editable draw.io figure: LLM sub-agents decide content, while deterministic Python owns geometry and the generative image model only restyles. (B) In Phase 2, panels are rendered from a combination of automatically-generated templates whose coordinates are determined by LLM-generated Python code. (C) In Phases 5–6, the enhanced candidate with the best audit results is selected; if none is acceptable, the deterministic render is used as a fallback.
Figure 6: Camera-ready ALIFE 2026 paper after the single submitted manuscript was accepted. The full manuscript and additional details can be found in Appendix D .
Figure 7: Creation process of the ALIFE paper: human researchers defined the initial direction, baseline, and rubrics; supported the experiment domain transition; verified results, references, and claims before submission; and incorporated reviewer feedback into the camera-ready paper.
| Revision stage | Inserted | Deleted | Net change | |
| Agentic Scientist submitted | 664 (17.5%) | 405 (10.7%) | ( %) | |
| Submitted camera-ready | 1,329 (32.8%) | 630 (15.6%) | ( %) |
Table 2: Revision statistics across manuscript stages. Counts are word-equivalents computed directly from the latexdiff LaTeX sources. Percentages are relative to the pre-revision manuscript length.
| Task | Starting Point | Research Objective |
| Learned optimization | Baseline : ELO’s expert-supervised MLP optimizer, meta-trained on (Fashion-)MNIST ( Huang et al., 2026b ) ; Evaluation : CIFAR-10 and SVHN. | Improve transfer to unseen datasets and network architectures under a fixed meta-training budget. Search iteration budget : 2 H100s, 150 min. |
| World models | Baseline : Action-free Pong model, with a trained discrete tokenizer and masked-token dynamics; Evaluation : Error reduction against TinyWorlds | Reduce held-out free-running rollout error across prediction horizons relative to Majmudar (2026) . Search iteration budget : 4 H100s, 180 min. |
| Sparse autoencoders (SAE) | Baseline : TopK SAE activation-trained on Pythia- 160M-deduped & Gemma-2-2B-it; Evaluation : Task-specific SAEBench ( Karvonen et al., 2025 ) . | Improve reconstruction at fixed sparsity while preserving model behavior and feature utilization. Search iteration budget : 2 H100s, 60 min. |
| Tabular distribution shift | Baseline : TabICLv2’s ( Qu et al., 2026b ) stage-one synthetic-prior pretraining from scratch recipe; Evaluation : 4 held-out real datasets vs. reference. | Improve in-context balanced accuracy on held-out real table data & across context sizes. Search iteration budget : 4 H100s, 150 min. |
| Hyperparameter optimization | Baseline : Adaptive incumbent-centred local search; Evaluation : BBOB ( Hansen et al., 2021 ) , HPO-B ( Arango et al., 2021 ) , YAHPO ( Pfisterer et al., 2022 ) . | Improve sample-efficient optimization over continuous, mixed, and hierarchical search spaces. Search iteration budget : 8 CPU cores, 30 min. |
Table 3: Research tasks, starting implementations, and objectives used in the aggregate comparison.
Figure 8: Pre-revision reviewer scores by (a) model release date and (b) harness, averaged across five tasks; (c) within-task percentile ranks of final performance and reviewer scores, colored by task. The least-squares fit uses a pointwise 95% band from 10,000 bootstrap resamples of the 14 complete harness–model profiles across all five tasks, conditional on the observed tasks and fixed ranks.
Figure 9: Search scores (top), rubric pass rates (middle), and final performance versus reviewer score (bottom) across five tasks. Bottom panels show within-task performance percentiles against original 1–10 reviewer scores; annotated least-squares slopes give reviewer-score points per 10 percentile points. Final performance is positively associated with reviewer scores on four of the five tasks.
Figure 10: (Left) Search trajectories for Doc-to-LoRA scientist runs given 40 idea iteration budget with Codex + GPT-5.6-Sol, Claude + Opus-5, and Pi + DeepSeek-V4-Pro on the multi-NIAH task. Both Codex and Claude runs improve steadily on the task. (Right) Exact-match accuracy is shown as a heatmap grouped by the number of test sample needles. Red crosses indicate invalid iterations.
Figure 11: Doc-to-LoRA composition mechanism ablations figure in the generated report ( Section C.1 ). (a) Hypernetwork’s head redesign, auxiliary objective, and unconditional merging stay at or below the exact-match accuracy, while query-conditioned composition reaches . (b) Starting from the query-blind (unconditional) merge, query-dependent merging provides a significant improvement; relevance-based text placement and delimiter-aligned chunking close the remaining gap.
Figure 12: Search trajectories for 30 idea iterations with GPT-5.6-Sol + Codex, Opus-5 + Claude, DeepSeek V4 Flash + Pi, and Kimi K3 + OpenCode. Dots are individual Ant search-validation evaluations and step curves are the cumulative best.
Figure 13: Post-search evaluation across eight Brax-v1 legacy-spring environments. Curves show the mean best-so-far return over 16 shared seeds; shaded regions denote one standard error. The grey Ant panel marks the search-validation reference environment, evaluated here with new seeds and the reduced transfer-policy interface; the other seven environments were unseen during agentic search.
Figure 14: Post-search test accuracy of the best program of each GRPO run. Each evaluation selects a run’s best program using the fixed validation set, evaluates it on the held-out test set across five seeds, and reports the mean and standard deviation.
Figure 15: Search dynamics of The Agentic Scientist on RL fine-tuning across three seeds per worker configuration. Left: per-evaluation scores against iteration index and wall-clock duration. Right: the combined 1- and 2-worker search tree per task, with examples of cross-worker inspirations relayed through shared memory. Small amounts of parallelism improve wall clock time and effectively transfer knowledge across workers. Further scaling of the parallelism does not show improvements.
Figure 16: Cross-agent comparison under the same orchestration scaffold across three seeds per agent. Left to Right : score evolution during the search, test performance after reevaluation, and cumulative API cost per evolution cycle. Crosses mark evaluations whose program modified the evaluation protocol (identified by manual audit); they are excluded from all running-best curves.
Figure 17: Effect of shared knowledge on 2-worker GRPO search across three seeds per configuration. Crosses mark evaluations from the no-shared-knowledge runs whose program modified the evaluation protocol (identified by manual audit); they are excluded from all running-best curves.
Figure 18: autoillustrate applied to the Darwin Gödel Machine paper ( Zhang et al., 2026a ) : the editable diagram (left) and its corresponding enhanced figure (right).
| Method | Variant | Faithfulness | Conciseness | Readability | Aesthetic | Overall | Editable |
| source | |||||||
| autoillustrate | Claude | 32.0 | 37.2 | 47.8 | 83.4 | 37.5 | ✓ |
| + no critic | 25.2 | 37.5 | 50.0 | 82.7 | 31.8 | ✓ | |
| + low effort | 24.5 | 37.3 | 47.8 | 82.5 | 29.8 | ✓ | |
| Codex | 33.6 | 37.5 | 34.4 | 56.3 | 28.4 | ✓ | |
| PaperBanana | Gemini | 18.5 | 55.3 | 42.0 | 71.6 | 25.0 | ✗ |
Table 4: Mean scores on PaperBananaBench, using GPT-6-Astra as VLM judge.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 19: First pages of the five highlighted task manuscripts, selected by highest AI reviewer score and final task performance. The generated papers are watermarked by The Agentic Scientist .
| Environment | CMA-ES | GPT-5.6-Sol + Codex | Opus-5 + Claude Code (CMA-Anchored LearnedES) | DeepSeek V4 Flash + Pi |
| Ant | ||||
| HalfCheetah | ||||
| Hopper | ||||
| Humanoid | ||||
| Reacher | ||||
| Walker2d |
Table 5: Mean terminal best-so-far return one standard error over 16 shared seeds after 2,000 generations. Ant is the search-validation reference environment and uses new transfer seeds and the reduced transfer-policy interface; the other seven environments were unseen during agentic search. Bold denotes the highest mean within an environment.
| Rubric strictness | Key ED constraints | Observed effect |
| Loose | No explicit wording to prohibit backpropagation | Best validation accuracy but failed to implement ED |
| Intermediate | Explicit wording to prohibit backprop | Best validation accuracy . Implemented ED correctly |
| Strict | No backprop No backward transport Explicit local update with one global error sign Detailed polarity invariance | Constrained search toward global/mean error broadcast ED; best validation accuracy |
Table 6: Rubric ablation design for MNIST Error Diffusion runs. The experiment varied the strictness and wording of the method-validity rubric while keeping the MNIST task, fixed split, search budget, and high-level exclusions constant.
Figure 20: autoillustrate applied to the agent draft of the Error Diffusion ALIFE paper, before human edits: the editable diagram (a) and its corresponding enhanced figure (b) . The figure in the submitted and camera-ready versions of the ALIFE paper was instead generated with Nano Banana Pro as a non-editable raster image; the human authors kept its correct right panel and manually redrew the left and middle panels to correct their inaccuracies. The prompt given to autoillustrate was: “Overview of the dual-stream Error Diffusion framework. Left: The excitatory/inhibitory architecture maintains separate streams ( , ) with four non-negative weight matrices per layer, enforcing Dale’s principle structurally. Center: The Error Diffusion update broadcasts output error directly to hidden layers without weight transport or random feedback matrices. Right: The shared architecture is applied to classification (with domain-specific sigmoid widths, mean-centered error, and asymmetric initialization) and reinforcement learning (with ReLU activations and PPO integration).”
Figure 21: Examples of figures generated by autoillustrate on PaperBananaBench ( Zhu et al., 2026a ) .
| Hyperparameter | Value |
| Codebase version | autoillustrate v0.2.0 |
| Coding agent | Claude Code ; Codex for the Codex variant |
| Agent model | claude-opus-4-8 ; gpt-5.6-sol for the Codex variant |
| Effort level | high |
| Panel inspects (Phase 3) | 3: one inspect up to two repair passes per panel |
| Enhancement attempts | 4: Phase 5 enhance 3 verify 2 repair passes |
Table 7: Hyperparameters for the PaperBananaBench evaluation.
| Method | Variant | Faithfulness | Conciseness | Readability | Aesthetic | Overall | Editable |
| source | |||||||
| autoillustrate | Claude | 43.7 | 45.7 | 45.0 | 75.2 | 50.0 | ✓ |
| + no critic | 42.8 | 41.6 | 45.2 | 75.3 | 46.6 | ✓ | |
| + low effort | 37.0 | 43.2 | 46.2 | 74.8 | 42.1 | ✓ | |
| Codex | 42.3 | 39.9 | 31.0 | 49.8 | 34.9 | ✓ | |
| PaperBanana | Gemini | 36.0 | 57.0 | 48.1 | 74.7 | 46.7 | ✗ |
Table 8: Mean scores on PaperBananaBench, using Gemini-3.1-Pro as VLM judge.
Figure 22: Example PaperBanana (Claude Opus 4.8 + GPT Image 2) diagram on which the two VLM judges disagree on Faithfulness. Gemini-3.1-Pro rationale: “Accurately captures the structural differences and integrates specific text details like delta/relative trajectories ( ) and frame-averaged encoders. It strictly aligns with the caption’s visual simplification. No veto errors.” GPT-6-Astra rationale: “Correctly depicts the proposed method’s encoder/backbone distinction and action frames. However, in the original Diffusion Policy panel, the visual-feature connection terminates above the clean-action label rather than entering the U-Net. The separate, unlabeled arrow entering the U-Net is disconnected from those features. This omits the essential visual-conditioning connection and constitutes a logical-flow veto, not merely simplification.”
| AutoIllustrate Diagrams | Cause of Failure |
| Edge attached to the wrong source. The equivariance loss is fed the downsampled latent directly from , bypassing the decoder , whereas the method defines the loss on the decoded output . | |
| Space-inefficient layout. Disconnected annotations float above the main pipeline, yielding a non-rectangular bounding box with large dead zones that waste vertical space under standard publication formatting. | |
| Insufficient visual abstraction. Raw equations are transcribed into the figure rather than distilled, most visibly the full argmax objective placed in a box at the top of Stage 3, and modules are labelled with bare symbols ( , ) instead of their function, reducing the diagram to a box-ified restatement of the text. |
Table 9: Example failure cases of AutoIllustrate.
| Harness | Version | Assigned models |
| Codex | 0.149.0 | GPT-6 Astra; GPT-5.6-Sol; GPT-5.5; GPT-5.4 |
| Claude Code | 2.1.240 | Fable 5.1; Opus 5; Opus 4.7; Opus 4.6 |
| Pi | 0.84.2 | GPT-5.6-Sol; Opus 5; DeepSeek V4 Flash |
| OpenCode | 1.18.21 | GPT-5.6-Sol; Opus 5; DeepSeek V4 Flash |
Table 10: Harness and model configurations for the aggregate sweep. All runs use high reasoning effort.
| Stage | Setting | Value |
| Agent search | Idea budget | 40 ideation cycles |
| Model / harness | gpt-5.6-sol / Codex; claude-opus-5 / Claude; deepseek-v4-pro / Pi | |
| Hardware | One H100 GPU with 16 CPU cores | |
| Training time limit | 3 hours; candidates above 3 hours are invalid | |
| Dataset | Training datasets | NIAH samples range from 32 to 256 tokens |
| Evaluation datasets | NIAH samples range from 32 tokens to 64K tokens |
Table 11: Doc-to-LoRA multi-needle NIAH agent-search and evaluation protocol.
| Stage | Setting | Value |
| Agent search | Idea iterations | 30 per agent |
| Models / agents | gpt-5.6-sol / Codex; claude-opus-5 / Claude; deepseek-v4-flash / Pi; kimi-k3 / OpenCode | |
| Valid evaluations | 28 Codex; 30 Claude; 28 DeepSeek + Pi; 30 Kimi + OpenCode | |
| Outer meta-evolution | Optimizer | Full CMA-ES |
| Population / generations | 256 / 1,500 | |
| Selected candidates’ initial std. | 0.10 (Codex, Claude, Kimi); 0.01 (DeepSeek + Pi) |
Table 12: MetaEvo training, search-validation, and transfer protocol. BBOB tasks meta-train neural weights; Ant subsequently supplies the score used by the agentic program search.
| Setting | MNIST | CIFAR-10 | MuJoCo / Brax | Craftax |
| Model / Harness | Sonnet 4.6 / Claude Code | Opus 4.6 / Claude Code | Composer 2.5 Fast / Cursor | GPT-5.5 / Codex |
| Agent search budget | 20 ideation cycles | 60 ideation cycles | 60 ideation cycles | 30 ideation cycles |
| Evaluator resources | 1 GPU, 4 CPUs | 1 GPU, 8 CPUs | 1 GPU, 4 CPUs | 1 GPU, 4 CPUs |
| Dataset / environment | MNIST: 55k train / 5k validation / 10k test | CIFAR-10: 45k train / 5k validation / 10k test | Ant, HalfCheetah, Humanoid | Craftax-Symbolic-v1 |
| Final training budget | 250 epochs | 250 epochs | 400M requested steps per seed | 1B requested steps per seed |
Table 13: Error-Diffusion task and evaluation parameters.
| Stage | Setting | Value |
| Agent search | Idea budget | 50 ideation cycles |
| Model / harness | claude-opus-4-7 / Claude | |
| Program evaluations | 1 per idea | |
| Pruning | Every 20 cycles | |
| Hardware | Two H100 GPUs with 16 CPU cores | |
| Training time limit | 2.5 hours |
Table 14: GRPO agent-search and evaluation protocol. A fixed Qwen2.5-Math-1.5B model is trained on the MATH dataset with GRPO; final validation accuracy (mean reward) supplies the score used by the agentic parallel investigation phase, and the best program of each run is re-scored on a held-out test set.