ExpEvolve: Structuring and Reusing Experience for Open-Ended Discovery

Yongqiang Chen; Chenxi Liu; Deyu Zou; Zhenhao Chen; Tongliang Liu; Bo Han; Kun Zhang

cs.LGOct 21, 2026

ExpEvolve: Structuring and Reusing Experience for Open-Ended Discovery

Authors: Yongqiang Chen, Chenxi Liu, Deyu Zou, Zhenhao Chen, Tongliang Liu, Bo Han, Kun Zhang

Organizations: MBZUAI · Carnegie Mellon University · TMLR Group, Hong Kong Baptist University · The Chinese University of Hong Kong ·

Abstract

Evolutionary coding agents such as AlphaEvolve are one of the notable successes in using Large Language Models (LLMs) to build AI Scientists. These agents tackle open-ended scientific problems by iteratively improving and evolving programs, leveraging the prior knowledge and reasoning capabilities of LLMs. Despite their success, mutations over programs are mostly low-level upon surface heuristics, without a persistent unit of control. In contrast, science compounds through ideas. In this work, we propose ExpEvolve, which injects idea-level extraction and control into evolution: it discovers and leverages the high-level factors from programs that explain scoring differences among different programs. We characterize theoretical conditions on when discovery with idea factors improves sample efficiency. We also build a practical framework to show that the extracted high-level idea factors are helpful for guiding exploitation from experience and exploration when discovery comes to a plateau. Across 11 open-ended mathematical tasks, ExpEvolve significantly improves the final performance and sample efficiency up to 5.0x. The improvements also transfer to 4 other LLMs and to 6 machine learning tasks from MLS-Bench. Analysis shows that ExpEvolve can discover idea-level factors to provide better exploration and exploitation for evolutionary discovery.

Figures & tables

Figure 1: Comparison between experience reuse at program-level and idea-level. All programs solve the same grid task. Left: Experience reuse at program-level is prone to low-level edits. Program-level evolution derives new programs from a high-scoring parent, such as changing the heuristic weight or bounding the search loop. Both edited programs score lower. Right: Experience reuse at idea-level compares structurally different programs through a shared procedure-level factor, lp = refine_lp(lp) . Programs that contain the factor score higher than the program that omits it. This contrast singles out the factor, which is then inserted into new programs of different structures.

Panel of Figure 2 — Figure 2: Evolutionary discovery as a POMDP. (A) In each iteration, an evolutionary coding agent builds the proposal context from its memory, e.g., parent and inspiration programs or summarized insights, and generates candidate programs with an LLM $G$ . The evaluator scores every program, whose outcome depends on the hidden mechanism $\theta_{\mathrm{sci}}$ , and the scored programs enter the archive and update the memory. (B) The corresponding temporal graphical model. The memory $m_{t}$ guides the next program $p_{t}$ , and the score $y_{t}$ depends on both $p_{t}$ and the static but unknown mechanism $\theta_{\mathrm{sci}}$ , which in turn updates $m_{t+1}$ .

Figure 3: Illustration of idea factor discovery.

Figure 4: Illustration of two-level exploration-exploitation with discovered idea factors.

Figure 5: Illustration of idea-guided exploration at plateaus.

	20 generations $\times$ 10 programs				10 generations $\times$ 20 programs				ShinkaEvolve
Task	ExpEvolve		SCS		ExpEvolve		SCS		ShinkaEvolve
	Mean	Best	Mean	Best	Mean	Best	Mean	Best	Mean	Best
Hadamard $\uparrow$	0.9211 $\pm$ 0.000	0.9211	0.6641 $\pm$ 0.169	0.8596	0.9259 $\pm$ 0.008	0.9357	0.8349 $\pm$ 0.101	0.9211	0.6467 $\pm$ 0.139	0.8066
Pack $\uparrow$	2.63542 $\pm$ 0.001	2.63598	2.61697 $\pm$ 0.028	2.63572	2.63499 $\pm$ 0.002	2.63598	2.62305 $\pm$ 0.011	2.63104	2.58201 $\pm$ 0.037	2.61408
Linprog $\uparrow$	2.63542 $\pm$ 0.001	2.63598	2.63542 $\pm$ 0.001	2.63598	2.63542 $\pm$ 0.001	2.63598	2.55161 $\pm$ 0.006	2.55608	2.51434 $\pm$ 0.026	2.54142
SumDiff $\uparrow$	1.11609 $\pm$ 0.006	1.11937	1.11087 $\pm$ 0.001	1.11143	1.12224 $\pm$ 0.001	1.12281	1.11608 $\pm$ 0.005	1.12006	1.10954 $\pm$ 0.000	1.10954

Table 1: Final performance at the budget of

200

evaluated programs. Mean reports the seed mean

\pm

seed standard deviation over

3

seeds, and Best reports the best seed.

Figure 6: Best-so-far scores along the budget of

200

evaluated programs.

Figure 7: (a) Best-so-far scores of SCS and ExpEvolve with other LLMs. (b) Best-so-far scores of SCS , SCS+meta, and ExpEvolve .

Figure 8: Quality of discovered idea factors. (a) Pairwise accuracy of the discovered idea factors. (b) The idea factor with the largest weight. (c) Illustration of usage of factors in sumdiff task.

Panel of Figure 9 — Figure 9: Results on MLS-Bench. (a) Mean final scores. (b,c) Trajectories on two example tasks.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 10 — Figure 10: The iterative scientific discovery loop. Left: Conceptual flow of the agent. The agent maintains a scratchpad memory ( $m$ ), proposes a program ( $p$ ), and observes the outcome ( $y$ ) which is constrained by the unknown world state ( $\theta_{\mathrm{sci}}$ ). The outcome feeds back into the memory for the next step. Right: The diagram illustrates how the AI Scientist probes the unknown world state $\theta_{\mathrm{sci}}$ . By proposing a candidate program $p_{t}$ , the agent triggers an experiment yielding outcome $y_{t}$ . This observation provides evidence about $\theta_{\mathrm{sci}}$ , which is integrated into the agent’s scratchpad memory $m_{t+1}$ . Over time steps $t,t+1,\dots$ , this recurrent process allows the agent to navigate the performance landscape and converge towards optimal programs despite the static but unknown nature of $\theta_{\mathrm{sci}}$ .

Symbol	Meaning
$\mathcal{P}$	Program / pipeline / model space (candidate designs)
$K$	Number of candidate programs, $K:=\|\mathcal{P}\|$ (finite in Theorem 3.1 )
$p\in\mathcal{P}$	A program to evaluate (action)
$\mathcal{X}$	Design-variable space (encoding of programs)
$x_{p}\in\mathcal{X}$	Encoding of program $p$ (e.g., design variables $X=x_{p}$ )
$z$ , $Z(p)$	An idea and the idea representation of program $p$ (Definition 3.2 )

Appendix

Table 2: Notation used in the formulation and theorems.

Hyperparameter	$N=20$ , $B=10$	$N=10$ , $B=20$
Fraction of random exploration $\rho$	$0.6$	$0.4$
Discovery interval $n_{\mathrm{disc}}$ (generations)	$2$	$1$
Plateau window $\kappa$ (generations)	$4$	$2$
Warm-up generations	$2$	$1$
Action rotation $N_{\mathrm{rot}}$ (generations)	$6$	$3$
Shared by both configurations

Appendix

Table 3: Hyperparameters of ExpEvolve in the main experiments. The budget of

200

evaluations is run with two configurations of generations

N

and batch size

B

. Both configurations are run on all

11

tasks, each with

3

seeds.

Task	Direction	Previous best	AlphaEvolve
First autocorrelation inequality	$\downarrow$	$1.5098$	$1.5053$
Second autocorrelation inequality	$\uparrow$	$0.88922$	$0.8962$
Uncertainty inequality	$\downarrow$	$0.3523$	$0.3521$
Erdős minimum overlap	$\downarrow$	$0.380927$	$0.380924$
Sums and differences of sets	$\uparrow$	$1.14465$	$1.1584$
Max–min distance ratio ( $16$ points)	$\downarrow$	$12.890$	$12.88927$

Appendix

Table 4: Tasks, optimization directions, and reference values. The previous best values and the values of AlphaEvolve are collected by Gideoni et al. (2026) . For the uncertainty inequality, all methods in Gideoni et al. (2026) reach

0.3482

under their reformulated parameterization, which we use in our experiments. For the Hadamard matrix, the previous best is the normalized score of the best-known construction.

Task	Method	Config.	Seed 1	Seed 2	Seed 3	Mean
Hadamard $\uparrow$	ExpEvolve	$10\times 20$	0.9211	0.9357	0.9211	0.9259 $\pm$ 0.0084
	ExpEvolve	$20\times 10$	0.9211	0.9211	0.9211	0.9211 $\pm$ 0.0000
	SCS	$10\times 20$	0.8596	0.9211	0.7240	0.8349 $\pm$ 0.1008
	SCS	$20\times 10$	0.8596	0.5665	0.5662	0.6641 $\pm$ 0.1693
	ShinkaEvolve	–	0.5603	0.5733	0.8066	0.6467 $\pm$ 0.1386
Pack $\uparrow$	ExpEvolve	$10\times 20$	2.63304	2.63598	2.63595	2.63499 $\pm$ 0.00169

Appendix

Table 5: Final best score of each seed at the budget of

200

, and the mean

\pm

standard deviation over seeds.

Task	Method	Config.	Seed 1	Seed 2	Seed 3	Mean
Kissing $\uparrow$	ExpEvolve	$10\times 20$	582	582	582	582 $\pm$ 0
	ExpEvolve	$20\times 10$	414	582	582	526 $\pm$ 97
	SCS	$10\times 20$	582	582	582	582 $\pm$ 0
	SCS	$20\times 10$	406	438	394	413 $\pm$ 23
	ShinkaEvolve	–	414	406	566	462 $\pm$ 90
Second autocorr. $\uparrow$	ExpEvolve	$10\times 20$	0.8825	0.8825	0.8825	0.8825 $\pm$ 0.0000

Appendix

Table 6: Final best score of each seed (continued) at the budget of

200

evaluated programs (

199

for ShinkaEvolve ), and the mean

\pm

standard deviation over seeds.

Task	Method	Config.	$40$	$80$	$120$	$160$	$200$
Hadamard $\uparrow$	ExpEvolve	$10\times 20$	0.8801 $\pm$ 0.0355	0.9211 $\pm$ 0.0000	0.9259 $\pm$ 0.0084	0.9259 $\pm$ 0.0084	0.9259 $\pm$ 0.0084
	ExpEvolve	$20\times 10$	0.9006 $\pm$ 0.0355	0.9006 $\pm$ 0.0355	0.9211 $\pm$ 0.0000	0.9211 $\pm$ 0.0000	0.9211 $\pm$ 0.0000
	SCS	$10\times 20$	0.5047 $\pm$ 0.0113	0.6446 $\pm$ 0.1866	0.6494 $\pm$ 0.1828	0.7144 $\pm$ 0.1502	0.8349 $\pm$ 0.1008
	SCS	$20\times 10$	0.6506 $\pm$ 0.1821	0.6506 $\pm$ 0.1821	0.6556 $\pm$ 0.1771	0.6641 $\pm$ 0.1693	0.6641 $\pm$ 0.1693
	ShinkaEvolve	–	0.4909 $\pm$ 0.0171	0.5093 $\pm$ 0.0093	0.5297 $\pm$ 0.0258	0.5588 $\pm$ 0.0131	0.6467 $\pm$ 0.1386
Pack $\uparrow$	ExpEvolve	$10\times 20$	2.62275 $\pm$ 0.00820	2.62721 $\pm$ 0.00567	2.63201 $\pm$ 0.00090	2.63499 $\pm$ 0.00169	2.63499 $\pm$ 0.00169

Appendix

Table 7: Best-so-far scores along the budget, as the mean

\pm

standard deviation over

3

seeds after

40

80

120

160

, and

200

evaluated programs.

Task	Method	Config.	$40$	$80$	$120$	$160$	$200$
Kissing $\uparrow$	ExpEvolve	$10\times 20$	578 $\pm$ 6	582 $\pm$ 0	582 $\pm$ 0	582 $\pm$ 0	582 $\pm$ 0
	ExpEvolve	$20\times 10$	348 $\pm$ 0	453 $\pm$ 113	523 $\pm$ 103	523 $\pm$ 103	526 $\pm$ 97
	SCS	$10\times 20$	564 $\pm$ 3	582 $\pm$ 0	582 $\pm$ 0	582 $\pm$ 0	582 $\pm$ 0
	SCS	$20\times 10$	394 $\pm$ 38	394 $\pm$ 38	402 $\pm$ 34	402 $\pm$ 33	413 $\pm$ 23
	ShinkaEvolve	–	426 $\pm$ 68	459 $\pm$ 92	459 $\pm$ 92	461 $\pm$ 91	462 $\pm$ 90
Second autocorr. $\uparrow$	ExpEvolve	$10\times 20$	0.8825 $\pm$ 0.0000	0.8825 $\pm$ 0.0000	0.8825 $\pm$ 0.0000	0.8825 $\pm$ 0.0000	0.8825 $\pm$ 0.0000

Appendix

Table 8: Best-so-far scores along the budget (continued), as the mean

\pm

standard deviation over

3

seeds after

40

80

120

160

, and

200

evaluated programs. For ShinkaEvolve , the last column reports

199

programs.

Task	Config.	Final of SCS	$n_{\texttt{SCS}}$	$n_{\texttt{ExpEvolve}}$	$n_{\texttt{SCS}}/n_{\texttt{ExpEvolve}}$
Hadamard	$10\times 20$	$0.8349$	$186$	$10$	$18.6$
Hadamard	$20\times 10$	$0.6641$	$131$	$5$	$26.2$
SumDiff	$10\times 20$	$1.11608$	$180$	$81$	$2.2$
SumDiff	$20\times 10$	$1.11087$	$181$	$97$	$1.9$
Heilbronn	$10\times 20$	$0.0336$	$199$	$52$	$3.8$
Heilbronn	$20\times 10$	$0.0313$	$169$	$27$	$6.3$

Appendix

Table 9: Numbers of evaluated programs that ExpEvolve and SCS need to reach the final best score of SCS , averaged over

3

seeds.

AI Agents for Scientific Discovery

AI agents for scientific discovery use language models, tools, and experiments to generate, test, and refine scientific knowledge.

See all 287 papers

CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad
Mar 15, 2026Yongqiang Chen, Chenxi Liu, Zhenhao Chen +3AI Agents for Scientific Discovery Large Language Model-Guided Scientific Discovery
MBZUAI · Carnegie Mellon University · TMLR Group, Hong Kong Baptist University +1
EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery
Mar 9, 2026Yougang Lyu, Xi Zhang, Yuyue Zhao +11Multi-Agent LLM Systems AI Agents for Scientific Discovery
Huawei Technologies Co., Ltd. · Vrije Universiteit Amsterdam
SpecEvo: Speculative Evolution with Large Language Models for Cost-Efficient Scientific Discovery
Oct 20, 2026Ha Minh Hieu, Tongyao Zhu, Do Xuan Long +1Large Language Model-Guided Scientific Discovery LLMs for Evolutionary Optimization
Hanoi University of Science and Technology · National University of Singapore

Mar 15, 2026J/K move · Enter open · S save

CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad

Yongqiang Chen, Chenxi Liu, Zhenhao Chen, Tongliang Liu+2

MBZUAI · Carnegie Mellon University · TMLR Group, Hong Kong Baptist University+1

AI Agents for Scientific Discovery Large Language Model-Guided Scientific Discovery Open-Ended Evolution Large Language Model-Guided Optimization

Open paper AskPDF arXiv

Evolve-based agent such as AlphaEvolve is one of the notable successes in using Large Language Models (LLMs) to build AI Scientists. These agents tackle open-ended scientific problems by iteratively improving and evolving programs, leveraging the prior knowledge and reasoning capabilities of LLMs. Despite the success, existing evolve-based agents lack targeted guidance for evolution and effective mechanisms for organizing and utilizing knowledge acquired from past evolutionary experience. Consequently, they suffer from decreasing evolution efficiency and exhibit oscillatory behavior when approaching known performance boundaries. To mitigate the gap, we develop CausalEvolve, equipped with a causal scratchpad that leverages LLMs to identify and reason about guiding factors for evolution. At the beginning, CausalEvolve first identifies outcome-level factors that offer complementary inspirations in improving the target objective. During the evolution, CausalEvolve also inspects surprise patterns during the evolution and abductive reasoning to hypothesize new factors, which in turn offer novel directions. Through comprehensive experiments, we show that CausalEvolve effectively improves the evolutionary efficiency and discovers better solutions in 4 challenging open-ended scientific tasks.

Abstract

AI Agents for Scientific Discovery

Explore similar work

CausalEvolve: Towards Open-Ended Discovery with Causal Scratchpad

EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery

SpecEvo: Speculative Evolution with Large Language Models for Cost-Efficient Scientific Discovery