What happens when you teach an LLM-based agent the scientific method? Motivation: Scientific discovery emerges from cycles of hypothesis, implementation, empirical testing, and feedback. Can this process be automated? We approach automated algorithm design through the lens of the scientific method, where an LLM-based agent goes through each step of the process in an ordered, iterative fashion. Results: We present The Little Scientist, a framework in which a "Scientist agent" works inside an evaluation environment that benchmarks its code and returns structured per-instance diagnostics. When the Scientist plateaus at a local optimum, a "Kuhn agent" injects a paradigm-shifting conjecture paired with a cross-disciplinary inspiration, forcing exploration of a different region of the LLM's latent space. We demonstrate the framework on two problems that require fundamentally different modes of discovery. For protein fitness prediction, the Scientist discovered Delta V, an ensemble calibration strategy that ranks first on the ProteinGym DMS Substitutions Zero-Shot leaderboard across all five official evaluation metrics, exceeding the #2 model (VenusREM) by +0.033 mean Spearman correlation across 217 DMS assays. For DNA motif discovery, the Scientist wrote an algorithm from scratch--DALE (Dual-seed Algorithm for Latent Enumeration)--that outperforms STREME (the default in the MEME Suite) across 132 ENCODE transcription factors (mean AUROC 0.842 vs. 0.803, Wilcoxon p < 10^{-6}) while running 11x faster. This demonstrates that the framework can produce genuinely novel algorithms, not just optimize existing components. Together, these results show that an LLM agent stepping through the scientific method can discover both new algorithms and new ensemble strategies that outperform prior solutions. The entire research program consumed 704M tokens on a single virtual machine with no GPUs
Figures & tables
Figure 1: Overview of The Little Scientist framework. The Scientist agent works inside an evaluation environment that benchmarks its code and returns structured diagnostics. When the Scientist plateaus, the Kuhn agent injects a paradigm-shifting conjecture paired with a cross-disciplinary inspiration, forcing exploration of a different region of the LLM’s latent space.
Phase
Runs
Tokens
Avg tokens/run
Days
Initial search (V1)
1,588
97.6M
61K
13
Kuhn paradigm agent
63
33.5M
531K
3
full-framework search (V2)
500
97.9M
196K
10
Total
2,151
229.0M
106K
26 active
Table 1: Measured compute usage for the motif discovery case study. All figures from OpenClaw cron_run_logs .
Figure 2: Score progression for both case studies. (A) Motif discovery: the infrastructure overhaul gap (gray band) and Kuhn paradigm shift (annotation) separate the stalled Phase 1 from the full-framework Phase 2. Dashed lines show STREME and MEME baseline accuracy. (B) Protein fitness prediction: the discontinuous jump on Aug 4 occurred when the human investigator provided pre-computed model predictions as input data, enabling the Scientist to adopt an ensemble approach. Dashed lines show the top ProteinGym leaderboard entries.
Figure 5: Genomic negatives benchmark (53 TFs, Markov-1 scoring). DALE outperforms all three comparison tools.
Figure 6: Per-TF comparison: DALE vs. STREME (53 TFs, genomic negatives). Points above the diagonal indicate TFs where DALE outperforms STREME.
Figure 7: Runtime per TF across tools (132-TF shuffled benchmark). DALE processes all 132 TFs in under 45 seconds.
Test
Shuffled (132 TFs)
Genomic (53 TFs)
Wilcoxon p
4.1×10−7
6.5×10−4
Bootstrap 95% CI
[0.023, 0.046]
[0.010, 0.046]
McNemar χ2
13.1 ( p=2.9×10−4 )
6.4 ( p=1.2×10−2 )
Cliff’s δ
+0.31
+0.38
Cohen’s d (paired)
+0.48
+0.42
Wins / Losses
88 / 44
35 / 16
Table 5: Statistical significance of DALE vs. STREME.
Metric
DALE
STREME
Any JASPAR match
112 (94%)
104 (87%)
Significant ( e<0.05 )
93 (78%)
88 (74%)
Strong match ( e<0.001 )
55 (46%)
51 (43%)
Median best e-value
1.6×10−3
1.1×10−3
Top-5 moderate ( q<0.05 )
100%
100%
Top-1 strict ( q<0.001 , ≥ 70% overlap)
50%
62%
Table 6: TOMTOM JASPAR match summary ( n=119 TFs with motifs tested).
Figure 8: ProteinGym score progression over the 8-day search. The discontinuous jump on Aug 4 occurred when the human investigator provided pre-computed predictions from five state-of-the-art models as input data, enabling the Scientist to adopt an ensemble approach. Dashed lines show the top 3 ProteinGym leaderboard entries.
Component
Runs
Tokens
Model
Scientist agent
287
377.3M
GLM-4.7
Kuhn agent
104
95.7M
GLM-4.7
Infrastructure
38
2.0M
—
Total
429
475.0M
8 calendar days
Table 7: Measured compute usage for the ProteinGym case study. All figures from OpenClaw cron_run_logs .
Rank
Model
Input Modalities
Spearman
AUC
MCC
NDCG
Top- k
1
Delta V (agent-discovered)
Ensemble of 5 models
0.551
0.801
0.430
0.800
0.254
2
VenusRAR ( Tan and others, 2026 )
Multi-modal ensemble + CoT
0.551 ∗
—
—
—
—
3
VenusREM
Structure & MSA
0.518
0.783
0.404
0.770
0.244
4
ProSST (K=2048)
Sequence & Structure
0.507
0.777
0.398
0.757
0.236
5
ProSST (K=4096)
Sequence & Structure
0.498
0.773
0.385
0.774
0.232
6
S3F-MSA
Structure & MSA
0.496
0.771
0.387
0.792
0.243
Table 8: ProteinGym DMS Substitutions Zero-Shot leaderboard (top entries). All competitor scores from the official ProteinGym evaluation pipeline (v1.3, August 2026). Delta V ranks #1 on all five official metrics. “Avg. Spearman” is the official category-mean aggregation. Delta V’s flat mean across all 217 assays is 0.569. ∗ VenusRAR reports matching Spearman on the same benchmark; AUC/MCC/NDCG/Top- k were not reported.
Figure 9: ProteinGym DMS Substitutions Zero-Shot leaderboard (top entries). Delta V exceeds all 96 scored models on the leaderboard, ranking #1 on all five official metrics.
Category
#UniProt
Delta V
VenusRAR-Rank
Best Other
Margin
Activity
37
0.549
0.539
VenusREM (0.495)
+0.054
Expression
11
0.554
0.497
VenusREM (0.533)
+0.057
Organismal Fitness
62
0.523
0.558
VenusREM (0.459)
− 0.035
Stability
66
0.642
0.658
ProSST K=2048 (0.653)
− 0.016
Binding
10
0.486
0.497
ProSST K=4096 (0.472)
− 0.011
Official mean
186
0.551
0.551
VenusREM (0.518)
+0.033
Table 9: Per-category Spearman scores: Delta V vs. VenusRAR-Rank vs. best competitor. Scores from official ProteinGym pipeline. VenusRAR-Rank scores from Tan and others (2026) , Table 2. Delta V wins 2 of 5 categories head-to-head; VenusRAR-Rank wins 3. The approaches capture different signal — Delta V is strongest on Expression (+0.057), VenusRAR-Rank on Stability (+0.016).
Figure 10: Search efficiency comparison between Scientist and Kuhn agents on the ProteinGym zero-shot task. Top row: cumulative improvements over time and magnitude of each improvement. Middle row: rolling acceptance rate and best score trajectory. Bottom row: per-achievement cost and summary statistics.
Configuration
Smoke peak
Full eval
Runs to peak
Tokens
Baseline (synthetic eval, loose prompt, no diagnostics)
Table 10: Development timeline: effect of progressive infrastructure additions on search outcomes. Each row adds one component to the previous configuration. Smoke-test AUROC is the 5-TF proxy metric used during search; full-benchmark AUROC is the 132-TF final metric. Rows are sequential additions, not independent variables — see §7.2 for limitations.
Large Language Models (LLMs) have been used for novel discovery of algorithms, theorems, drugs, and other tasks through the use of harnesses that prompt an LLM to iteratively optimize an objective. In this work, we study the relationship between the population of previous iterates and eventual discovery success. We generalize past work on harness design to develop a suite of 12 harnesses called 'Modular' and characterize their performance across 5 diverse discovery tasks, finding that discovery success is brittle and sensitive to harness design. We uncover mode collapse, characterized by a dramatic drop in the diversity of iterates, as a common failure mode. We find that popular state-of-the-art harnesses and diversity-inducing harness interventions, which aim to prolong this collapse, yield inconsistent gains. Our results instead uncover that the performance of early discoveries is predictive of eventual success. We therefore propose a universally applicable intervention that performs an initial stage of parallel exploration in order to initialize subsequent iterative optimization. Our method provides consistent gains across many harnesses and target applications, confirming the importance of initialization in LLM-driven discovery.
Mansi Sakarvadia, Marco Ciccone, Colin Raffel
University of Chicago · Vector Institute · University of Toronto
Frameworks that use large language models for scientific discovery typically rely on a fixed, human-designed algorithm that decides what the model sees at each step, leaving the model only the role of proposer. The model knows nothing of the search beyond what it is shown. As models grow more capable, a question arises: does a search strategy chosen by a human before the run scale better than promoting the model from proposer to planner and letting it own the search? The Bitter Lesson suggests that choosing the strategy in advance is the kind of hand-designed structure that general methods eventually outscale. We introduce AgentDiscover, in which a coding agent plans the search using its context as working memory, runs experiments, and records every attempt in a database of ideas, candidates, and their relations. This database serves as the agent's long-term memory and is structured so that the selection rules of classical algorithms such as MAP-Elites and Monte Carlo tree search each reduce to a single query, which the agent is free to use, combine, or replace. A server maintains the database and steers the agent after every submission, keeping it on course over long runs. In our experiments, AgentDiscover is more cost-efficient than existing frameworks, reaching better scores at lower cost. On tasks in kernel engineering, biology, algorithm design, and mathematics, AgentDiscover outperforms prior discovery frameworks. Its programs would have placed first among human competitors in seven past AtCoder heuristic contests, and on eleven mathematical and systems optimization tasks it matches or exceeds every baseline that uses the same model. Our code is available at https://github.com/mhdfb/AgentDiscover.
The application of Large Language Models (LLMs) in accelerating scientific discovery has garnered increasing attention, with a key focus on constructing research agents endowed with innovative capability, i.e., the ability to autonomously generate novel and significant research ideas. Existing approaches predominantly rely on sophisticated prompt engineering and lack a systematic training paradigm. To address this, we propose DeepInnovator, a training framework designed to trigger the innovative capability of LLMs. Our approach comprises two core components. (1) Standing on the shoulders of giants''. We construct an automated data extraction pipeline to extract and organize structured research knowledge from a vast corpus of unlabeled scientific literature. (2) Conjectures and refutations''. We introduce a ``Next Idea Prediction'' training paradigm, which models the generation of research ideas as an iterative process of continuously predicting, evaluating, and refining plausible and novel next idea. Both automatic and expert evaluations demonstrate that our DeepInnovator-14B significantly outperforms untrained baselines, achieving win rates of 80.53%-93.81%, and attains performance comparable to that of current leading LLMs. This work provides a scalable training pathway toward building research agents with genuine, originative innovative capability, and will open-source the dataset to foster community advancement. Source code and data are available at: https://github.com/HKUDS/DeepInnovator.
Tianyu Fan, Fengji Zhang, Yuxiang Zheng +5
♣The University of Hong Kong · ♠City University of Hong Kong · ♢Shanghai Jiao Tong University +1