The Little Scientist: LLM Agent-Driven Discovery via the Scientific Method

Travis Smith

q-bio.QMAug 16, 2026

The Little Scientist: LLM Agent-Driven Discovery via the Scientific Method

Authors: Travis Smith

Organizations: Independent Researcher, S˜ao Miguel, Azores, Portugal

Abstract

What happens when you teach an LLM-based agent the scientific method? Motivation: Scientific discovery emerges from cycles of hypothesis, implementation, empirical testing, and feedback. Can this process be automated? We approach automated algorithm design through the lens of the scientific method, where an LLM-based agent goes through each step of the process in an ordered, iterative fashion. Results: We present The Little Scientist, a framework in which a "Scientist agent" works inside an evaluation environment that benchmarks its code and returns structured per-instance diagnostics. When the Scientist plateaus at a local optimum, a "Kuhn agent" injects a paradigm-shifting conjecture paired with a cross-disciplinary inspiration, forcing exploration of a different region of the LLM's latent space. We demonstrate the framework on two problems that require fundamentally different modes of discovery. For protein fitness prediction, the Scientist discovered Delta V, an ensemble calibration strategy that ranks first on the ProteinGym DMS Substitutions Zero-Shot leaderboard across all five official evaluation metrics, exceeding the #2 model (VenusREM) by +0.033 mean Spearman correlation across 217 DMS assays. For DNA motif discovery, the Scientist wrote an algorithm from scratch--DALE (Dual-seed Algorithm for Latent Enumeration)--that outperforms STREME (the default in the MEME Suite) across 132 ENCODE transcription factors (mean AUROC 0.842 vs. 0.803, Wilcoxon p < 10^{-6}) while running 11x faster. This demonstrates that the framework can produce genuinely novel algorithms, not just optimize existing components. Together, these results show that an LLM agent stepping through the scientific method can discover both new algorithms and new ensemble strategies that outperform prior solutions. The entire research program consumed 704M tokens on a single virtual machine with no GPUs

Figures & tables

Figure 1: Overview of The Little Scientist framework. The Scientist agent works inside an evaluation environment that benchmarks its code and returns structured diagnostics. When the Scientist plateaus, the Kuhn agent injects a paradigm-shifting conjecture paired with a cross-disciplinary inspiration, forcing exploration of a different region of the LLM’s latent space.

Phase	Runs	Tokens	Avg tokens/run	Days
Initial search (V1)	1,588	97.6M	61K	13
Kuhn paradigm agent	63	33.5M	531K	3
full-framework search (V2)	500	97.9M	196K	10
Total	2,151	229.0M	106K	26 active

Table 1: Measured compute usage for the motif discovery case study. All figures from OpenClaw cron_run_logs .

Figure 2: Score progression for both case studies. (A) Motif discovery: the infrastructure overhaul gap (gray band) and Kuhn paradigm shift (annotation) separate the stalled Phase 1 from the full-framework Phase 2. Dashed lines show STREME and MEME baseline accuracy. (B) Protein fitness prediction: the discontinuous jump on Aug 4 occurred when the human investigator provided pre-computed model predictions as input data, enabling the Scientist to adopt an ensemble approach. Dashed lines show the top ProteinGym leaderboard entries.

Tool	Version	Parameters
DALE	1.0	Default (widths 6–17)
STREME	5.5.5	--dna --minw 6 --maxw 17
MEME	5.5.5	-mod zoops -minw 6 -maxw 17 -revcomp
proto-DALE	—	Earlier prototype, same width range

Table 2: Tools compared in this study.

Tool	Mean AUROC	Time/TF	Speedup vs. STREME
DALE	0.842	0.30 s	11 $\times$
proto-DALE	0.843	1.56 s	2.1 $\times$
STREME 5.5.5	0.803	3.26 s	1.0 $\times$

Table 3: Benchmark results on 132 ENCODE K562 ChIP-seq datasets (shuffled negatives).

Figure 3: AUROC distribution across 132 ENCODE K562 ChIP-seq datasets (shuffled negatives). DALE consistently outperforms STREME. Dashed line indicates random performance.

Figure 4: Per-TF comparison: DALE vs. STREME (132 TFs, shuffled negatives). Points above the diagonal indicate TFs where DALE outperforms STREME.

Tool	Mean AUROC	Time/TF
DALE	0.891	0.3 s
MEME 5.5.5	0.881	$\sim$ 90 s
proto-DALE	0.874	1.4 s
STREME 5.5.5	0.863	4.4 s

Table 4: Genomic negatives benchmark (53 TFs, Markov-1 scoring).

Figure 5: Genomic negatives benchmark (53 TFs, Markov-1 scoring). DALE outperforms all three comparison tools.

Figure 6: Per-TF comparison: DALE vs. STREME (53 TFs, genomic negatives). Points above the diagonal indicate TFs where DALE outperforms STREME.

Figure 7: Runtime per TF across tools (132-TF shuffled benchmark). DALE processes all 132 TFs in under 45 seconds.

Test	Shuffled (132 TFs)	Genomic (53 TFs)
Wilcoxon $p$	$4.1\times 10^{-7}$	$6.5\times 10^{-4}$
Bootstrap 95% CI	[0.023, 0.046]	[0.010, 0.046]
McNemar $\chi^{2}$	13.1 ( $p=2.9\times 10^{-4}$ )	6.4 ( $p=1.2\times 10^{-2}$ )
Cliff’s $\delta$	$+0.31$	$+0.38$
Cohen’s $d$ (paired)	$+0.48$	$+0.42$
Wins / Losses	88 / 44	35 / 16

Table 5: Statistical significance of DALE vs. STREME.

Metric	DALE	STREME
Any JASPAR match	112 (94%)	104 (87%)
Significant ( $e<0.05$ )	93 (78%)	88 (74%)
Strong match ( $e<0.001$ )	55 (46%)	51 (43%)
Median best e-value	$1.6\times 10^{-3}$	$1.1\times 10^{-3}$
Top-5 moderate ( $q<0.05$ )	100%	100%
Top-1 strict ( $q<0.001$ , $\geq$ 70% overlap)	50%	62%

Table 6: TOMTOM JASPAR match summary (

n=119

TFs with motifs tested).

Figure 8: ProteinGym score progression over the 8-day search. The discontinuous jump on Aug 4 occurred when the human investigator provided pre-computed predictions from five state-of-the-art models as input data, enabling the Scientist to adopt an ensemble approach. Dashed lines show the top 3 ProteinGym leaderboard entries.

Component	Runs	Tokens	Model
Scientist agent	287	377.3M	GLM-4.7
Kuhn agent	104	95.7M	GLM-4.7
Infrastructure	38	2.0M	—
Total	429	475.0M	8 calendar days

Table 7: Measured compute usage for the ProteinGym case study. All figures from OpenClaw cron_run_logs .

Rank	Model	Input Modalities	Spearman	AUC	MCC	NDCG	Top- k
1	Delta V (agent-discovered)	Ensemble of 5 models	0.551	0.801	0.430	0.800	0.254
2	VenusRAR ( Tan and others, 2026 )	Multi-modal ensemble + CoT	0.551 ∗	—	—	—	—
3	VenusREM	Structure & MSA	0.518	0.783	0.404	0.770	0.244
4	ProSST (K=2048)	Sequence & Structure	0.507	0.777	0.398	0.757	0.236
5	ProSST (K=4096)	Sequence & Structure	0.498	0.773	0.385	0.774	0.232
6	S3F-MSA	Structure & MSA	0.496	0.771	0.387	0.792	0.243

Table 8: ProteinGym DMS Substitutions Zero-Shot leaderboard (top entries). All competitor scores from the official ProteinGym evaluation pipeline (v1.3, August 2026). Delta V ranks #1 on all five official metrics. “Avg. Spearman” is the official category-mean aggregation. Delta V’s flat mean across all 217 assays is 0.569. ∗ VenusRAR reports matching Spearman on the same benchmark; AUC/MCC/NDCG/Top- k were not reported.

Figure 9: ProteinGym DMS Substitutions Zero-Shot leaderboard (top entries). Delta V exceeds all 96 scored models on the leaderboard, ranking #1 on all five official metrics.

Category	#UniProt	Delta V	VenusRAR-Rank	Best Other	Margin
Activity	37	0.549	0.539	VenusREM (0.495)	+0.054
Expression	11	0.554	0.497	VenusREM (0.533)	+0.057
Organismal Fitness	62	0.523	0.558	VenusREM (0.459)	$-$ 0.035
Stability	66	0.642	0.658	ProSST K=2048 (0.653)	$-$ 0.016
Binding	10	0.486	0.497	ProSST K=4096 (0.472)	$-$ 0.011
Official mean	186	0.551	0.551	VenusREM (0.518)	+0.033

Table 9: Per-category Spearman scores: Delta V vs. VenusRAR-Rank vs. best competitor. Scores from official ProteinGym pipeline. VenusRAR-Rank scores from Tan and others (2026) , Table 2. Delta V wins 2 of 5 categories head-to-head; VenusRAR-Rank wins 3. The approaches capture different signal — Delta V is strongest on Expression (+0.057), VenusRAR-Rank on Stability (+0.016).

Figure 10: Search efficiency comparison between Scientist and Kuhn agents on the ProteinGym zero-shot task. Top row: cumulative improvements over time and magnitude of each improvement. Middle row: rolling acceptance rate and best score trajectory. Bottom row: per-achievement cost and summary statistics.

Configuration	Smoke peak	Full eval	Runs to peak	Tokens
Baseline (synthetic eval, loose prompt, no diagnostics)	0.664	—	1,588	97.6M
+ Real benchmark data (15 ChIP-seq TFs, AUROC)	0.565	—	11	—
+ Causal scaffold prompt (7-field structured plan)	0.709	—	11	—
+ Per-instance diagnostics restored	0.794	—	23	—
+ Kuhn paradigm agent	0.794	—	63	33.5M
+ All components (full framework)	0.846	0.842	500	97.9M

Table 10: Development timeline: effect of progressive infrastructure additions on search outcomes. Each row adds one component to the previous configuration. Smoke-test AUROC is the 5-TF proxy metric used during search; full-benchmark AUROC is the 132-TF final metric. Rows are sequential additions, not independent variables — see §7.2 for limitations.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Metric	DALE	STREME
Top-1 strict ( $q<0.001$ , $\geq$ 70% overlap)	23 (50.0%)	31 (62.0%)
Top-1 moderate ( $q<0.05$ )	40 (87.0%)	45 (90.0%)
Top-5 strict	29 (63.0%)	37 (74.0%)
Top-5 moderate ( $q<0.05$ )	46 (100%)	50 (100%)
No match at all	6	5

Appendix

Table 11: TOMTOM JASPAR match rates (

n=53

TFs).

Explore similar work

Initialization Improves LLM-Driven Discovery
Sep 30, 2026Mansi Sakarvadia, Marco Ciccone, Colin RaffelLarge Language Model-Guided Optimization Large Language Model-Guided Scientific Discovery
University of Chicago · Vector Institute · University of Toronto
AgentDiscover: Autonomous Discovery with Minimal Search Scaffolding
Oct 4, 2026Mahdi Farahbakhsh, Ilan Sela, Fatemeh Doudi +4AI Agents for Scientific Discovery Automated Algorithm Discovery
Texas A&M University
DeepInnovator: Triggering the Innovative Capabilities of LLMs
Feb 21, 2026Tianyu Fan, Fengji Zhang, Yuxiang Zheng +5Research Idea Generation LLM Agent Training
♣The University of Hong Kong · ♠City University of Hong Kong · ♢Shanghai Jiao Tong University +1

Sep 30, 2026J/K move · Enter open · S save

Initialization Improves LLM-Driven Discovery

Mansi Sakarvadia, Marco Ciccone, Colin Raffel

University of Chicago · Vector Institute · University of Toronto

Large Language Model-Guided Optimization Large Language Model-Guided Scientific Discovery

Open paper AskPDF arXiv

Large Language Models (LLMs) have been used for novel discovery of algorithms, theorems, drugs, and other tasks through the use of harnesses that prompt an LLM to iteratively optimize an objective. In this work, we study the relationship between the population of previous iterates and eventual discovery success. We generalize past work on harness design to develop a suite of 12 harnesses called 'Modular' and characterize their performance across 5 diverse discovery tasks, finding that discovery success is brittle and sensitive to harness design. We uncover mode collapse, characterized by a dramatic drop in the diversity of iterates, as a common failure mode. We find that popular state-of-the-art harnesses and diversity-inducing harness interventions, which aim to prolong this collapse, yield inconsistent gains. Our results instead uncover that the performance of early discoveries is predictive of eventual success. We therefore propose a universally applicable intervention that performs an initial stage of parallel exploration in order to initialize subsequent iterative optimization. Our method provides consistent gains across many harnesses and target applications, confirming the importance of initialization in LLM-driven discovery.

Abstract

Explore similar work

Initialization Improves LLM-Driven Discovery

AgentDiscover: Autonomous Discovery with Minimal Search Scaffolding

DeepInnovator: Triggering the Innovative Capabilities of LLMs