Evolutionary Ensemble Search: Council-Guided Program Evolution with Persistent Memory

Juan P. Madrigal-Cianci; Eshan Chordia

cs.NESep 11, 2026

Evolutionary Ensemble Search: Council-Guided Program Evolution with Persistent Memory

Authors: Juan P. Madrigal-Cianci, Eshan Chordia

Organizations: Impulse AI

Abstract

Evolutionary Ensemble Search (EES) constructs machine-learning procedures through expert-guided program evolution. A role-specialized council turns task evidence and experimental results into structured search directions. An orchestrator allocates these directions to execution specialists and an evolutionary engine. The engine selects measured parents, diagnoses their errors, and produces descendants through code mutation, structured pipeline edits, and crossover. Each child must execute and acquire its own validation evidence. Population archives retain useful alternatives, while compatible predictions compete in a validation-gated ensemble stage. Search adapts through parent-relative operator credit, session memory, and problem-indexed lessons retrieved across runs. We specify these mechanisms, distinguish their execution profiles, and define the contracts required to compare candidates as their computations change. A public MLE-bench Lite development ledger records medal-threshold artifacts on 19 of 22 tasks (86.36%), with best outcomes of 11 gold, five silver, and three bronze. The procedures span text, images, tables, audio, scientific geometry, and deterministic transformations. The campaign includes grade feedback between runs, external-source routes, and mixed confirmation procedures; its aggregate is an achieved development result, not a blind autonomous-agent success rate. The report contributes a concrete architecture for cumulative executable search and a versioned account of its cross-modal development outcomes.

Figures & tables

Profile	Search representation	Selection and feedback
Multi-island	Generated training code in general, tree-ensemble, and neural populations.	Within-island tournament selection, code mutation and crossover, and Pareto consolidation.
Nested	Structured pipelines and optional code candidates.	Diagnostic interventions, fitness–offspring parent sampling, operator credit, archive reuse, and cross-run lessons.
Benchmark	Registered modality operators and bounded configuration variants.	Comparable-validation frontier, hypothesis probes, prediction alignment, and terminal benchmark grading.

Table 1: EES execution profiles. The nested profile supplies the detailed adaptive search policy in Sections 4.2 – 4.4 .

Panel of Figure 1 — Figure 1: Council-guided executable search. The inner loop selects measured parents, constructs edits, executes children, and updates the population. Numerical operator feedback and distilled run records influence subsequent proposals. Predictive ensembling occurs only after alignment and validation checks; the external benchmark grade is outside this loop.

Expert	Contribution to the search state
Discovery Driver	Next experiment, urgency, executable proposal, and a low-cost opportunity.
Methodology Advisor	Problem formulation, methodological fit, confidence, and checkpoint advice.
Memory Keeper	Established findings, attempted approaches, failed routes, and unresolved questions.
ML Strategist	Model family, feature intervention, validation design, and convergence assessment.
Code Reviewer	Implementation defects, statistical errors, leakage risks, and corrective actions.

Table 2: Role-specialized council outputs. The first three roles are always active; the final two occupy rotating slots.

Operation	What changes	What the child must establish
Configuration mutation	Seed, solver, model family, regularization, or training allocation.	Performance of the changed configuration under comparable validation.
Pipeline mutation	Feature expression, encoding, interaction, preprocessing block, or inference rule.	An executable composition with valid intermediate and output semantics.
Diagnostic code mutation	A code block targeted using residuals, generalization gaps, or component diagnostics.	A functioning edited program and its own predictions, cost, and score.
Program crossover	Compatible components from multiple parents, or a new program conditioned on them.	Validity and measured behavior of the combined procedure; parent scores are not combined as child fitness.
Ensemble mutation	Member addition, pruning, or a change in predictive topology.	Aligned predictions and a measured comparison for adoption or reuse.

Table 3: Variation in EES. Examples specify admissible operations, not reconstructed trajectories or measured improvements in the Lite campaign. Supported operations depend on the execution profile.

Record set	Gold	Silver	Bronze	No medal	Any medal
Best recorded	11	5	3	3	19/22
C1	10	4	5	3	19/22
C2	10	4	5	3	19/22
C3	9	6	4	3	19/22

Table 4: Recorded campaign outcomes. The best-artifact column and each confirmation column are different record sets. Counts are recomputed from the task-level entries, not additional benchmark experiments.

Task	Recorded procedure	Best result
NOMAD	Crystal-geometry descriptors and multi-output ExtraTrees regression.	0.05373; gold
Dirty Documents	CPU background estimation and paired-image reconstruction with per-pixel serialization.	0.01919; silver
Dogs vs. Cats	Image fine-tuning and combination candidates; the best artifact is distinct from confirmation members.	0.00597; gold
Spooky Author	Gutenberg-based provenance features with a TF–IDF fallback.	0.12422; gold
Pizza	External target lookup combined with a learned text fallback.	1.00000; gold
Normalization	Training-derived token lookup and deterministic transformation rules for English and Russian.	Bronze in both

Table 5: Representative procedures from the public solution notes [ 13 ] . The differences concern executable computation, not only estimator hyperparameters.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Entry point	Role
Text	TF–IDF portfolio	Word and character features with validation-ranked solvers.
Text	Embedding	Dense representation and linear prediction.
Text	Provenance	Corpus matching with a learned fallback.
Tabular	Portfolio	Encoded features, model-family variants, and seeds.
Image	Embedding	Pretrained visual representations with validation models.
Image	Generic media	Resource-bounded image feature baseline.

Appendix

Table 6: Callable entry points in the benchmark profile.

Task	Best	C1	C2	C3	U	Flag
Aerial Cactus $\uparrow$	1.00000 G	1.00000 G	1.00000 G	1.00000 G	3
APTOS 2019 $\uparrow$	0.92020 S	0.91942 B	0.91942 B	0.91930 B	2
Dirty Documents $\downarrow$	0.01919 S	0.01926 S	0.01928 S	0.01919 S	3
Insults $\uparrow$	0.91118 G	0.90164 G	0.90219 G	0.90284 G	3
Dog Breed $\downarrow$	0.02439 B	0.02439 B	0.02439 B	0.02439 B	1	E
Dogs vs. Cats $\downarrow$	0.00597 G	0.00920 G	0.00870 G	0.00974 G	3

Appendix

Table 7: Best recorded scores and all three confirmation scores, with their medal classes. Arrows indicate whether higher or lower is better. U is the number of distinct submission hashes within C1–C3. E marks an explicitly documented external-source or historical-lookup route; an empty flag is not a clean-data certification.

Automated Algorithm Discovery

Automated algorithm discovery develops methods that generate and test executable algorithms for optimization, science, software, and machine learning.

See all 170 papers

EvoMem: Memory-Augmented Evolution for Code Optimization
Aug 11, 2026Viktor Volkov, Valentin Khrulkov, Andrey V. Galichin +8Memory-Augmented Language Models Evolutionary Optimization
AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia · Lomonosov Moscow State University, Moscow, Russia
Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery
May 21, 2026Halil Alperen Gozeten, Xuechen Zhang, Emrullah Ildiz +3Evolutionary Optimization Evolutionary Multitask Optimization
University of Michigan - Ann Arbor · University of California San Diego
SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution
Apr 27, 2026Sichun Luo, Yi Huang, Haochen Luo +7Automated Algorithm Discovery Evolutionary Optimization
1The University of Hong Kong · 2JIUTIAN Research, China Mobile · 3City University of Hong Kong

Aug 11, 2026J/K move · Enter open · S save

EvoMem: Memory-Augmented Evolution for Code Optimization

Viktor Volkov, Valentin Khrulkov, Andrey V. Galichin, Danil Sivtsov+7

AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia · Lomonosov Moscow State University, Moscow, Russia

Memory-Augmented Language Models Evolutionary Optimization Large Language Model-Guided Optimization LLMs for Evolutionary Optimization

Open paper AskPDF arXiv

Successful mutation strategies in evolutionary code search may contain reusable knowledge that is useful beyond a single run, and in some cases may transfer across related tasks and domains. However, existing LLM-driven evolutionary frameworks largely discard such knowledge, repeatedly rediscovering similar ideas and limiting opportunities for cross-run and cross-task learning. We introduce EvoMem, a persistent memory architecture for LLM-based evolutionary program search that captures and reuses candidate mutation knowledge. EvoMem converts successful mutation events into structured, task-aware advice for future runs. It operates in two phases: after each run, it extracts and stores promising ideas with provenance, and during subsequent evolution, it retrieves a small set of relevant instructions based on the current task and program context to guide mutation. Across geometric optimization, multi-hop question answering, GPU kernel optimization, and related benchmarks, our experiments show positive average improvements in target metrics or search speed for most evaluated settings, while also revealing variability across tasks. Overall, EvoMem provides evidence that persistent memory can reduce some redundant exploration and improve the reuse and adaptation of successful strategies in LLM-driven evolutionary search.

Abstract

Automated Algorithm Discovery

Explore similar work

EvoMem: Memory-Augmented Evolution for Code Optimization

Evolutionary Multi-Task Optimization for LLM-Guided Program Discovery

SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution