SEDIMA: Cross-Run Hierarchical Insight Memory for Evolutionary Search Agents

Amirhossein Abaskohi; Mahdi Mostajabdaveh; Zirui Zhou

cs.NEOct 1, 2026

SEDIMA: Cross-Run Hierarchical Insight Memory for Evolutionary Search Agents

Authors: Amirhossein Abaskohi, Mahdi Mostajabdaveh, Zirui Zhou

Organizations: University of British Columbia · Huawei Technologies Canada

Abstract

Large language model (LLM)-driven evolutionary search is a powerful paradigm for automated program and algorithm discovery, yet existing systems are largely memoryless: each run explores from scratch, so agents repeatedly rediscover the same improvements and re-encounter the same dead ends. We introduce SEDIMA, a persistent hierarchical insight memory for evolutionary search agents. SEDIMA distills raw traces into natural-language insights, clusters them by semantic similarity using attention-weighted centroids, and retrieves relevant guidance to condition future mutations, accumulating transferable knowledge across runs and problems rather than within a single trajectory. As a drop-in module that leaves the search operators unmodified, SEDIMA improves average final performance by 5.5% on AlgoTune and 6.6% on ALE-Bench LITE under a fixed budget of 100 evaluated candidates. Under OpenEvolve, SEDIMA requires 32.3% fewer iterations on average to reach baseline-best performance across the five evaluated backbones.

Figures & tables

Figure 1: Overview of Sedima . A persistent three-level memory (right) augments an evolutionary search loop (left, dashed): evaluated children are distilled into hierarchical insights on save , and relevant insights are injected into the mutation prompt on retrieval .

	OpenEvolve		ShinkaEvolve
Model	Base	+ Sedima	Base	+ Sedima
GPT-5.4	1.48	1.56	1.53	1.60
DeepSeek V4 Pro	1.75	1.85	1.80	1.89
Gemini 3 Pro	1.83	1.91	1.87	1.94
Qwen 3.7 Max	1.69	1.80	1.74	1.84
Qwen 3 Coder	1.44	1.55	1.49	1.59

Table 1: AlgoTune, harmonic mean speedup.

	OpenEvolve		ShinkaEvolve
Model	Base	+ Sedima	Base	+ Sedima
GPT-5.4	1613.1	1709.9	1660.0	1743.7
DeepSeek V4 Pro	1007.2	1087.8	1055.1	1123.6
Gemini 3 Pro	1182.8	1259.6	1215.3	1281.8
Qwen 3.7 Max	1186.0	1275.0	1225.2	1304.6
Qwen 3 Coder	482.5	535.6	510.0	555.9

Table 2: ALE-Bench LITE, average performance.

Figure 2: Mean AlgoTune speedup over 100 OpenEvolve iterations with GPT-5.4, with and without Sedima , averaged over three seeds.

Memory at deployment	AlgoTune	ALE-Bench
Cold (empty)	1.67	1113.7
Warm, same benchmark	1.75	1187.8
Warm, other benchmark	1.70	1146.5

Table 3: Effect of memory provenance at deployment.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Short	Long	Overall
OpenEvolve	1166.8	1021.8	1094.3
+ Sedima	1261.7	1085.5	1173.6
ShinkaEvolve	1209.4	1056.8	1133.1
+ Sedima	1292.6	1111.2	1201.9

Appendix

Table 4: ALE-Bench LITE by contest format, averaged over models.

Operation	Memory content	Function
Level 1 write	Task: capacitated facility location. Score: $2.36\times$ . Backend: CBC. Structure: dense binary assignment variables, facility-opening variables, exactly-one customer assignments, and capacity constraints. Diagnosis: solver execution remains the dominant cost.	Preserves the candidate score, implementation details, evaluator evidence, and provenance of the higher-level insight.
Level 2 write	Boolean assignment models with exactly-one and capacity constraints may be solved more efficiently using a Boolean-oriented constraint solver. Native cardinality constraints should be exposed directly instead of being represented only as generic linear equalities.	Abstracts the facility-location trace into a reusable principle applicable to other discrete optimization problems.
Level 3 write	Cluster: Boolean assignment models and solver selection. Related entries include job assignment, capacitated scheduling, routing, and resource-activation problems.	Groups experiences by computational structure rather than benchmark identity or evolutionary lineage.
Retrieval query	The current program uses CBC for a dense Boolean facility-location model. Runtime is dominated by the solver, and each customer participates in an exactly-one assignment constraint.	Matches the stored cluster through shared concepts: Boolean decisions, assignment structure, capacities, and solver-bound execution.
Retrieved guidance	Replace the generic MILP backend with CP-SAT, use native AddExactlyOne constraints, and convert floating-point coefficients to bounded integers.	Provides a concrete mutation direction that can be directly translated into code.
Resulting candidate	CBC is replaced with CpModel and CpSolver ; generic assignment equalities are replaced with AddExactlyOne ; numerical coefficients are scaled to integers; and a bounded solve time is introduced.	Produces the $3.99\times$ candidate, a further $1.69\times$ improvement over the $2.36\times$ intermediate candidate.

Appendix

Table 5: Example snapshot of the information written to and retrieved from Sedima . Level 1 preserves task-specific evidence, while Levels 2 and 3 support transfer across problems.

Retrieved information	Resulting code modification
Boolean-dominated optimization models may benefit from CP-SAT.	Replace pywraplp.Solver and CBC with CpModel and CpSolver .
Exactly-one assignment structure should be exposed natively.	Replace generic sum-equals-one constraints with AddExactlyOne .
CP-SAT requires bounded integer coefficients.	Scale costs, demands, and capacities by a shared precision factor and convert them to integers.
Difficult solver runs should have bounded execution time.	Set max_time_in_seconds to $30$ .

Appendix

Table 6: Connection between the retrieved memory and the resulting code modifications.

Variant	AlgoTune-20	ALE-Bench LITE
Sedima (full)	1.69	1173.6
Level 2 only	1.65	1138.9
Mean centroid	1.67	1157.4
Random retrieval	1.62	1102.6
No memory	1.61	1094.3

Appendix

Table 7: Component ablation under OpenEvolve, averaged over three seeds and five mutation models. AlgoTune results use the first 20 tasks, while ALE-Bench uses the full LITE set.

Embedding model	AlgoTune-20	ALE-Bench LITE
Qwen3-Embedding-4B 2025	1.69	1173.6
Qwen3-Embedding-8B 2025	1.68	1168.4
GTE-Qwen2-7B-Instruct 2023	1.67	1159.7
BGE-M3 2024	1.65	1147.2
Qwen3-Embedding-0.6B 2025	1.64	1136.8

Appendix

Table 8: Embedding-model ablation under OpenEvolve, averaged over three seeds and five mutation models. AlgoTune results use the first 20 tasks, while ALE-Bench uses the full LITE set.

Task	GPT-5.4	DeepSeek V4 Pro	Gemini 3 Pro	Qwen 3.7 Max	Qwen 3 Coder
aes_gcm_encryption	1.12	1.06	1.18	1.05	1.02
affine_transform_2d	1.00	1.00	1.00	1.00	1.00
aircraft_wing_design	1.08	1.12	2.13	1.21	1.14
articulation_points	2.37	4.95	2.39	3.87	2.77
base64_encoding	1.06	1.00	1.10	1.00	1.00
battery_scheduling	8.54	14.85	31.72	15.62	12.94

Appendix

Table 9: Per-task speedup on AlgoTune using OpenEvolve+ Sedima (part 1 of 2).

Task	GPT-5.4	DeepSeek V4 Pro	Gemini 3 Pro	Qwen 3.7 Max	Qwen 3 Coder
matrix_multiplication	1.03	1.02	1.03	1.02	1.01
matrix_sqrt	1.02	1.02	1.02	1.02	1.01
max_clique_cpsat	4.67	12.24	13.52	10.57	6.01
max_common_subgraph	4.92	4.19	12.34	4.38	2.49
max_flow_min_cost	2.68	6.90	4.03	5.99	6.21
max_independent_set_cpsat	4.54	3.18	3.31	2.87	1.74

Appendix

Table 10: Per-task speedup on AlgoTune using OpenEvolve+ Sedima (part 2 of 2).

Task	GPT-5.4	DeepSeek V4 Pro	Gemini 3 Pro	Qwen 3.7 Max	Qwen 3 Coder
aes_gcm_encryption	1.13	1.07	1.19	1.06	1.02
affine_transform_2d	1.00	1.00	1.00	1.00	1.00
aircraft_wing_design	1.09	1.13	2.16	1.22	1.16
articulation_points	2.66	5.17	2.53	4.04	2.91
base64_encoding	1.07	1.00	1.11	1.00	1.00
battery_scheduling	9.85	16.16	34.92	17.21	14.44

Appendix

Table 11: Per-task speedup on AlgoTune using ShinkaEvolve+ Sedima (part 1 of 2).

Task	GPT-5.4	DeepSeek V4 Pro	Gemini 3 Pro	Qwen 3.7 Max	Qwen 3 Coder
matrix_multiplication	1.04	1.02	1.03	1.02	1.01
matrix_sqrt	1.02	1.02	1.02	1.02	1.01
max_clique_cpsat	5.37	13.14	13.90	11.77	6.93
max_common_subgraph	5.64	4.52	13.20	5.00	2.67
max_flow_min_cost	3.00	7.47	4.45	6.83	7.32
max_independent_set_cpsat	5.17	3.41	3.61	3.21	1.90

Appendix

Table 12: Per-task speedup on AlgoTune using ShinkaEvolve+ Sedima (part 2 of 2).

Problem	Format	GPT-5.4	DeepSeek V4 Pro	Gemini 3 Pro	Qwen 3.7 Max	Qwen 3 Coder
ahc008	Long	1206.4	660.1	830.8	777.5	258.5
ahc011	Long	1767.2	1177.8	1287.5	1388.6	584.1
ahc015	Short	1508.7	870.8	1154.4	1061.0	411.7
ahc016	Long	1306.9	743.3	1055.4	847.9	295.2
ahc024	Short	1900.3	1313.3	1358.2	1491.4	715.2
ahc025	Long	1495.7	885.2	1166.4	1032.3	369.6

Appendix

Table 13: Per-task performance on ALE-Bench LITE using OpenEvolve+ Sedima .

Problem	Format	GPT-5.4	DeepSeek V4 Pro	Gemini 3 Pro	Qwen 3.7 Max	Qwen 3 Coder
ahc008	Long	1254.7	683.3	851.7	793.5	277.9
ahc011	Long	1801.8	1219.1	1306.8	1445.6	609.8
ahc015	Short	1537.5	915.2	1169.1	1095.7	434.3
ahc016	Long	1359.2	769.4	1071.1	865.4	314.3
ahc024	Short	1936.7	1353.2	1389.2	1525.0	739.7
ahc025	Long	1525.1	934.7	1183.9	1053.5	385.8

Appendix

Table 14: Per-task performance on ALE-Bench LITE using ShinkaEvolve+ Sedima .

Memory-Augmented Agents

Memory-augmented agents use stored experience and knowledge to act coherently across long tasks, changing environments, and repeated interactions.

See all 340 papers

SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution
Apr 27, 2026Sichun Luo, Yi Huang, Haochen Luo +7Automated Algorithm Discovery Evolutionary Optimization
1The University of Hong Kong · 2JIUTIAN Research, China Mobile · 3City University of Hong Kong
EvoMem: Memory-Augmented Evolution for Code Optimization
Aug 11, 2026Viktor Volkov, Valentin Khrulkov, Andrey V. Galichin +8Memory-Augmented Language Models Evolutionary Optimization
AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia · Lomonosov Moscow State University, Moscow, Russia
$\varepsilon$ -MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution
Aug 12, 2026Aofan Liu, Shiyuan Song, Yiyan QiEvolutionary Optimization Large Language Model-Guided Optimization

Apr 27, 2026J/K move · Enter open · S save

SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution

Sichun Luo, Yi Huang, Haochen Luo, Fengyuan Liu+6

1The University of Hong Kong · 2JIUTIAN Research, China Mobile · 3City University of Hong Kong

Automated Algorithm Discovery Evolutionary Optimization Large Language Model-Guided Optimization LLM-Based Program Synthesis LLMs for Evolutionary Optimization

Open paper AskPDF arXiv

Large Language Model (LLM)-guided evolutionary search is increasingly used for automated algorithm discovery, yet most current methods track search progress primarily through executable programs and scalar fitness. Even when natural-language reasoning is used through heuristic descriptions or reflection, it typically remains transient mutation context or unstructured memory, rather than organized as persistent population-level state over strategic directions. As a result, evolutionary search can struggle to distinguish syntactically different implementations of the same idea, preserve lower-fitness but strategically promising directions, or detect when an entire family of strategies has saturated. We introduce \model, a modular strategy-space layer that turns language-level strategic reasoning into first-class population-level evolutionary state in LLM-driven program search. \model represents each candidate program with an explicit natural-language strategy, clusters the archive by strategy semantics, retrieves behaviorally complementary inspirations, and periodically navigates the strategy landscape to avoid saturated directions. Without modifying the underlying evolutionary algorithms, \model improves existing evolutionary backbones across algorithm discovery, systems optimization, and agent-scaffold design tasks in most settings. Across four systems benchmarks, \model achieves a 20.6% average relative improvement, with the best single run on Prism scoring 3

\times

higher. These results suggest that persistent strategy representations provide a practical mechanism for improving the effectiveness and cost-efficiency of LLM-guided evolutionary search, pointing toward compound AI systems whose search capabilities benefit from the structured accumulation and reuse of algorithmic strategies.

Abstract

Memory-Augmented Agents

Explore similar work

SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolution

EvoMem: Memory-Augmented Evolution for Code Optimization

ε\varepsilonε-MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution

$\varepsilon$ -MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution