Component-Aware Feedback for Self-Evolving Programs

Ethan Lin; Jinming Nian; Yi Fang

cs.AISep 29, 2026

Component-Aware Feedback for Self-Evolving Programs

Authors: Ethan Lin, Jinming Nian, Yi Fang

Organizations: Santa Clara University

Abstract

LLM-guided evolutionary search can discover complex programs, but existing methods mostly only save candidate programs and fitness scores while discarding which component edits produced which fitness metric changes. Existing methods force the mutator LLM to infer the effect of prior edits from cluttered histories, making program search slow and unstable. This is especially true for locally servable LLMs to evolve multi-component systems. We introduce component-aware feedback, which compares each evaluated program with its parent, identifies the components that changed, and logs them with the associated metric differences into an attribution memory that later mutations read. The memory keeps each change in two reference frames, local against the parent it came from and global against the seed program, which shows both the immediate effect of a change and the cumulative progress made since the seed. We study this on LLM reranking, a multi-objective optimization problem where a multi-stage pipeline must balance quality against serving cost. Across twelve \textsc{Bright} datasets, our method reaches the strongest baseline's final quality after a median of one third of the search budget and ends 7.2% higher in held-out nDCG@10, and under a cost-aware objective it finds pipelines that are on average more accurate while using 11% fewer tokens per query, showing component-aware feedback to be a promising direction for more efficient self-evolving systems.

Figures & tables

Figure 1: Local and global component-aware feedback. A standard LLM-guided search (top left) rebuilds its context from the database at every iteration and keeps nothing between iterations. We add an observe step (1), which diffs a child against its parent, and an attribution memory, which records each edit and is read by the context step (top right). Local and global entries (bottom) differ only in their reference point. A local entry (2) compares a child with its parent, and a global entry (3) compares it with the seed.

	BM25		seed program		AdaEvolve			Ours
dataset	train	test	train	test	train	test	best	train	test	best	$\Delta$ %
StackExchange
bio.	0.184	0.167	$0.372$ $\pm$ .006	0.362	$0.572$ $\pm$ .027	$0.581$ $\pm$ .044	0.619	$\mathbf{0.602}$ $\pm$ .005	$\mathbf{0.613}$ $\pm$ .021	0.633	$+5.6$
earth.	0.238	0.363	$0.404$ $\pm$ .012	0.509	$0.492$ $\pm$ .016	$0.559$ $\pm$ .014	0.575	$\mathbf{0.529}$ $\pm$ .020	$\mathbf{0.573}$ $\pm$ .035	0.606	$+2.4$
econ.	0.180	0.118	$0.299$ $\pm$ .009	0.201	$0.407$ $\pm$ .024	$0.253$ $\pm$ .030	0.278	$\mathbf{0.412}$ $\pm$ .018	$\mathbf{0.274}$ $\pm$ .023	0.299	$+8.1$
psy.	0.089	0.228	$0.307$ $\pm$ .006	0.501	$0.434$ $\pm$ .023	$\mathbf{0.540}$ $\pm$ .035	0.563	$\mathbf{0.448}$ $\pm$ .008	$0.533$ $\pm$ .066	0.604	$-1.3$

Table 1: Ranking performance on Bright . nDCG@10 on the train and held-out test splits, split 60/40 per dataset. Best is the best single run on test.

\Delta

% is our relative gain over AdaEvolve on test. Mean

\pm

std over 3 runs.

StackExchange
	AdaEvolve				Ours
dataset	nDCG@10	$\Delta$	tok/q	cheaper	nDCG@10	$\Delta$	tok/q	cheaper
bio.	$0.542$ $\pm$ .012	-0.039	10,118	6.2x	$\mathbf{0.558}$ $\pm$ .010	-0.055	12,071	15.1x
earth.	$0.521$ $\pm$ .021	-0.038	27,539	2.2x	$\mathbf{0.540}$ $\pm$ .030	-0.033	14,930	2.6x
econ.	$\mathbf{0.225}$ $\pm$ .007	-0.029	14,453	5.2x	$0.217$ $\pm$ .044	-0.056	7,055	5.2x
psy.	$0.441$ $\pm$ .031	-0.099	10,422	5.6x	$\mathbf{0.488}$ $\pm$ .019	-0.045	25,736	2.8x

Table 2: Cost-aware search on Bright . Test nDCG@10 and LLM tokens per query of the final program. Both searches optimize the cost-aware objective rather than nDCG alone.

\Delta

is the change in test nDCG@10 against Table 1 . Cheaper is the quality-only program’s tokens per query divided by this program’s. Mean

\pm

std over 3 runs.

Figure 2: Convergence on Bright . Best-so-far nDCG@10 on the train split, one panel per dataset. Solid lines are the mean over 3 runs, faint lines the individual runs. The marker gives the iteration at which our mean first reaches AdaEvolve’s final score.

Figure 3: Convergence under the cost-aware objective. Best-so-far objective value on the train split, one panel per dataset. The seed program scores 1. Solid lines are the mean over 3 runs, faint lines the individual runs. The marker gives the iteration at which our mean first reaches AdaEvolve’s final value.

		optimized on that dataset		optimized on biology		retained
dataset	seed	AdaEvolve	Ours	AdaEvolve	Ours	AdaEvolve	Ours
earth.	$0.509$	$0.559$ $\pm$ .014	$\mathbf{0.573}$ $\pm$ .035	$0.517$ $\pm$ .029	$\mathbf{0.548}$ $\pm$ .037	$92\%$	$\mathbf{96\%}$
rob.	$0.239$	$0.254$ $\pm$ .015	$\mathbf{0.275}$ $\pm$ .005	$0.220$ $\pm$ .024	$\mathbf{0.284}$ $\pm$ .028	$87\%$	$\mathbf{103\%}$
leet.	$0.190$	$0.207$ $\pm$ .014	$\mathbf{0.247}$ $\pm$ .017	$0.126$ $\pm$ .056	$\mathbf{0.177}$ $\pm$ .022	$61\%$	$\mathbf{71\%}$
theoQ.	$0.124$	$0.286$ $\pm$ .048	$\mathbf{0.348}$ $\pm$ .054	$0.222$ $\pm$ .104	$\mathbf{0.311}$ $\pm$ .108	$78\%$	$\mathbf{89\%}$
mean	$0.266$	$0.327$	$\mathbf{0.361}$	$0.271$	$\mathbf{0.330}$	$79\%$	$\mathbf{90\%}$

Table 3: Cross-domain generalization on Bright . Programs optimized on biology scored on other datasets’ held-out splits (nDCG@10). Seed and optimized on that dataset are the unevolved program and each method’s search run on that dataset respectively, all from Table 1 . Retained is the transferred program’s score as a percentage of the same method’s score when optimized on that dataset. Mean

\pm

std over 3 runs.

Figure 4: Mathematical optimization. Best-so-far objective against the fraction of the search budget spent, one panel per benchmark, higher is better. Solid lines are the mean over 3 runs, faint lines the individual runs.

	bio.		leet.		theoQ.		mean
method	train	test	train	test	train	test	train	test
prompt evolution	$0.408$ $\pm$ .012	$0.373$ $\pm$ .011	$0.376$ $\pm$ .007	$0.221$ $\pm$ .019	$0.146$ $\pm$ .001	$0.140$ $\pm$ .004	$0.310$	$0.245$
AdaEvolve	$0.572$ $\pm$ .027	$0.581$ $\pm$ .044	$0.360$ $\pm$ .016	$0.207$ $\pm$ .014	$0.314$ $\pm$ .060	$0.286$ $\pm$ .048	$0.416$	$0.358$
Ours	$0.602$ $\pm$ .005	$0.613$ $\pm$ .021	$0.381$ $\pm$ .031	$\mathbf{0.247}$ $\pm$ .017	$\mathbf{0.381}$ $\pm$ .037	$\mathbf{0.348}$ $\pm$ .054	$\mathbf{0.454}$	$\mathbf{0.403}$
$-$ attribution	$\mathbf{0.609}$ $\pm$ .036	$\mathbf{0.635}$ $\pm$ .060	$\mathbf{0.392}$ $\pm$ .023	$0.208$ $\pm$ .004	$0.343$ $\pm$ .073	$0.302$ $\pm$ .030	$0.448$	$0.382$

Table 4: Component ablation on Bright . nDCG@10 on one dataset per Bright section. Prompt evolution evolves the reranking prompt, and the other rows evolve the program. Mean

\pm

std over 3 runs.

Explore similar work

Aug 12, 2026cs.AI

$\varepsilon$ -MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution

LLM-based program evolution systems such as FunSearch and AlphaEvolve have shown strong ability to discover novel algorithms, but typically optimize each task in isolation, discarding search experience after completion. We introduce

\varepsilon

-MemEvo, a framework for cross-task knowledge transfer in LLM program evolution.

\varepsilon

-MemEvo stores prior experience as task-agnostic tactic memories: compact natural-language summaries of successful algorithmic strategies rather than raw code, enabling transfer across tasks with different APIs and evaluators. To avoid negative transfer from semantically mismatched memories,

\varepsilon

-MemEvo uses an adaptive injection gate that decides whether retrieved memories should be injected, and at what intensity. We evaluate

\varepsilon

-MemEvo on 8 diverse optimization benchmarks spanning mathematical optimization and systems engineering, using a content-level Leave-One-Out protocol that excludes target-task memory entries. On the primary GPT-5 backbone,

\varepsilon

-MemEvo improves AUCC over AdaEvolve on all 8 tasks, with a mean relative gain of +8.7%, and improves early-stage convergence by +9.4% on average. Ablations show that naive memory injection can fail catastrophically, while adaptive gating remains safe across all five ablation tasks. The data-updated posterior is interpretable in observed states: it favors skip during improving search and shifts from skip to hint across early and late plateaus. These gains incur less than 1% computational overhead.

Aofan Liu, Shiyuan Song, Yiyan Qi

Evolutionary Search Cross-Task Knowledge Transfer Peak Memory Agentic Memory Memes

Ask PDF

Aug 11, 2026cs.AI

EvoMem: Memory-Augmented Evolution for Code Optimization

Successful mutation strategies in evolutionary code search may contain reusable knowledge that is useful beyond a single run, and in some cases may transfer across related tasks and domains. However, existing LLM-driven evolutionary frameworks largely discard such knowledge, repeatedly rediscovering similar ideas and limiting opportunities for cross-run and cross-task learning. We introduce EvoMem, a persistent memory architecture for LLM-based evolutionary program search that captures and reuses candidate mutation knowledge. EvoMem converts successful mutation events into structured, task-aware advice for future runs. It operates in two phases: after each run, it extracts and stores promising ideas with provenance, and during subsequent evolution, it retrieves a small set of relevant instructions based on the current task and program context to guide mutation. Across geometric optimization, multi-hop question answering, GPU kernel optimization, and related benchmarks, our experiments show positive average improvements in target metrics or search speed for most evaluated settings, while also revealing variability across tasks. Overall, EvoMem provides evidence that persistent memory can reduce some redundant exploration and improve the reuse and adaptation of successful strategies in LLM-driven evolutionary search.

Viktor Volkov, Valentin Khrulkov, Andrey V. Galichin +8

AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia · Lomonosov Moscow State University, Moscow, Russia

Evolutionary Search Memory-Augmented Framework Evolutionary Optimization Methods Code Optimization

Ask PDF

May 14, 2026cs.AI

SMCEvolve: Principled Scientific Discovery via Sequential Monte Carlo Evolution

LLM-driven program evolution has emerged as a powerful tool for automated scientific discovery, yet existing frameworks offer no principled guide for designing their individual components and provide no guarantee that the search converges. We introduce SMCEvolve, which recasts program search as sampling from a reward-tilted target distribution and approximates it with a Sequential Monte Carlo (SMC) sampler. From this view, three core mechanisms emerge as principled components: adaptive parent resampling, mixture of mutation with acceptance, and automatic convergence control. We further provide a finite-sample complexity analysis that bounds the LLM-call budget required to reach a target approximation error. Across math, algorithm efficiency, symbolic regression, and end-to-end ML research benchmarks, SMCEvolve surpasses state-of-the-art evolving systems while using fewer LLM calls under self-determined termination. The code is available at https://github.com/kongwanbianjinyu/SMCEvolve.

Jiachen Jiang, Huminhao Zhu, Zhihui Zhu

Department of Computer Science and Engineering The Ohio State University

Evolutionary Search Alphaevolve Symbolic Regression Model Discovery Mle-Bench Lite Monte Carlo+1

Ask PDF

Abstract

Explore similar work

ε\varepsilonε-MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution

EvoMem: Memory-Augmented Evolution for Code Optimization

SMCEvolve: Principled Scientific Discovery via Sequential Monte Carlo Evolution

$\varepsilon$ -MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution