What Makes a Redundant Representation Remember? Lineage Isolation, Not Masking

Jia Huang; Yangjun Ou

cs.NESep 4, 2026

What Makes a Redundant Representation Remember? Lineage Isolation, Not Masking

Authors: Jia Huang, Yangjun Ou

Organizations: Guanghua School of Management,Peking University · School of Mathematical Sciences,Peking University

Abstract

Memory-based evolutionary algorithms for dynamic optimization often carry a redundant second copy of the genotype and expose only one copy to the objective, on the assumption that the shielded copy accumulates information about past optima. We show this assumption is false as usually implemented, and identify the structural property that actually determines whether the shielded copy retains information. We formalize such methods as a gated dual-copy representation with two independent design axes: a gating rule deciding which copy is evaluated, and an inheritance rule deciding whether the two copies mix across generations. A ablation shows retained information is governed almost entirely by the inheritance rule (21.4 vs. 1.3 bits) and is nearly invariant to the gating rule. Per-locus independent inheritance reshuffles cross-locus structure every generation, so shielding preserves the variance of the hidden copy while destroying the pattern that constitutes a memory. Under isolated inheritance the memory effect is real: against a single-copy baseline matched for representation budget, the method gains +0.010 AUC when optima recur periodically and loses 0.078 when they drift unidirectionally---a 0.089 separation under otherwise identical settings, which excludes explanations based on added capacity. We show the readout rate is also the corruption rate, predicting and confirming an interior optimum replicated across two implementations. We report one negative result with a mechanism: dual-copy representations lower the mutational error threshold, because gated expression is a selector rather than a joint decoder and therefore provides no coding gain. Finally, we document a benchmarking hazard: on dynamic benchmarks the choice of recombination operator alone shifted our baseline by 0.062 AUC, six times the effect size under study.

Figures & tables

Gating	Inheritance	AUC	Bits	Div. loci
Hard	mixing	0.8165	$+1.28$	0.046
Hard	isolated	0.8628	$\mathbf{+21.38}$	0.402
Leaky	mixing	0.8123	$+0.83$	0.040
Leaky	isolated	0.8523	$\mathbf{+14.76}$	0.325

Table 1:

2\times 2

ablation over the two design axes.

T{=}40

\mu_{g}{=}0.02

, 6 seeds. Single-copy baseline AUC

=0.8715

. Retention differs by

16\times

across inheritance rules and barely at all across gating rules.

Figure 1: Retained information across the

2\times 2

ablation, grouped by inheritance rule. The gating rule—the mechanism credited in prior work—has almost no effect.

$T$	Periodic	Unidirectional
40	$\mathbf{+0.0104}$	$\mathbf{-0.0784}$
60	$-0.0082$	—
80	$-0.0269$	$-0.0926$
120	$-0.0599$	—
160	$-0.0756$	—
320	$-0.0942$	—

Table 2: AUC gain over the representation-matched single-copy baseline. Hard gate, isolated ,

\mu_{g}{=}0.05

, 8 seeds. The two regimes are identical in every setting except whether past optima recur.

Figure 2: Gain versus environment period under both regimes. The separation at

T{=}40

is the memory contribution; it cannot arise from added capacity, which is identical in the two regimes.

$\mu_{g}$	0.005	0.02	0.05	0.10	0.20
Hard gate (AUC $\uparrow$ )	0.834	0.863	0.883	0.870	0.848
Leaky gate (gens $\downarrow$ )	22.5	17.2	14.3	19.7	30.6

Table 3: Gate-flip rate sweep, two implementations differing in gating rule, metric, and inheritance details. Both peak at

\mu_{g}\approx 0.05

Figure 3: Interior optimum in the gate-flip rate, replicated across two implementations. Right axis is inverted so that “up” is better on both.

Figure 4: Static environment. If redundancy provided error correction the dual-copy arms would tolerate higher

\mu

. They do not: the threshold moves the wrong way, and further so the leakier the gate.

Baseline	1-pt $N$	1-pt $2N$	Unif. $N$	Unif. $2N$
AUC	0.796	0.810	0.847	0.872

Table 4: Baseline sensitivity to the recombination operator (

T{=}40

). The shift is six times the effect size under study.

Explore similar work

Mar 28, 2026

Persistent Memory Through Triple-Loop Consolidation Under Stochastic Unit Turnover

Dissipative cognitive architectures maintain computation through continuous energy expenditure, where units that exhaust their energy are stochastically replaced with fresh random state. This creates a fundamental challenge: how can persistent, context-specific memory survive when all learnable state is periodically destroyed? Existing memory mechanisms -- including elastic weight consolidation, synaptic intelligence, and surprise-driven gating -- rely on gradient computation and are inapplicable to systems that do not perform it. We introduce Deep Memory (DM), a backpropagation-free persistent memory mechanism operating through a triple-loop consolidation cycle: (1) recording of expert-specific content centroids, (2) seeding of replaced units with stored representations, and (3) stabilization through continuous re-entry. Discrete expert routing via Mixture-of-Experts (MoE) gating is required, in the regimes tested, to prevent the centroid convergence that would render stored memories identical. We derive a Foster-Lyapunov drift bound for the full triple loop, showing that seeding rescales the turnover noise floor. Across

1{,}007

simulation runs over thirteen blocks: (i) removing stable context-expert binding removes specialization (

\mathrm{MI}=1.10

vs.

0.001

;

n=91

); (ii) DM achieves

R=0.984

vs.

0.385

without memory (

n=16

); (iii) continuous seeding reconstructs representations after interference (

R_\mathrm{recon}=0.978

; one-shot fails;

n=30

); (iv) the mechanism operates within a characterized

(K,p)

envelope (

n=350

); (v) recording

\times

seeding is the minimal critical dyad (

n=40

); (vi) associative and reservoir baselines (Hopfield, ESN) are compared under matched turnover (

n=370

). DM is thus a falsifiable, bounded mechanism for persistent memory in backpropagation-free cognitive systems, with functional parallels to hippocampal consolidation.

Jianwei Lou

RailMind Systems, Neuss, Germany

Computational Cognitive Modeling Memory Consolidation

Ask PDF

Jun 10, 2026

Mathematical perspective on genetic algorithms with optimization guided operators

Recent work in ML applies genetic algorithms at inference time to iteratively improve solutions to optimization problems. The basic mutation and recombination operators involved are qualitatively different from those studied classically. Mutations are no longer random; an ML algorithm mutates a solution with the goal of improving an objective. Similarly, recombination is not based on random collages of parent solutions. Instead, it is an ML optimization-based operator whose goal is to synthesize improved solutions from its inputs. Thus, these mutation and recombination operators are more likely to improve the objective, but their computational cost is much higher. We introduce a general model of genetic algorithms and formulating optimization in this model as a query-complexity problem, using the language of reinforcement learning. We then study specialized models. We show that some optimization problems require generation, mutation, and recombination to be solved. We then obtain qualitatively tight algorithms for a family of problems within this framework that captures the nontrivial role of diversity in the solution pool, a key feature of practical ML genetic algorithms.

Anna Brandenberger, Ilan Doron-Arad, Elchanan Mossel

Department of Mathematics, MIT

Query Complexity Genetic Algorithms

Ask PDF

Aug 11, 2026

EvoMem: Memory-Augmented Evolution for Code Optimization

Successful mutation strategies in evolutionary code search may contain reusable knowledge that is useful beyond a single run, and in some cases may transfer across related tasks and domains. However, existing LLM-driven evolutionary frameworks largely discard such knowledge, repeatedly rediscovering similar ideas and limiting opportunities for cross-run and cross-task learning. We introduce EvoMem, a persistent memory architecture for LLM-based evolutionary program search that captures and reuses candidate mutation knowledge. EvoMem converts successful mutation events into structured, task-aware advice for future runs. It operates in two phases: after each run, it extracts and stores promising ideas with provenance, and during subsequent evolution, it retrieves a small set of relevant instructions based on the current task and program context to guide mutation. Across geometric optimization, multi-hop question answering, GPU kernel optimization, and related benchmarks, our experiments show positive average improvements in target metrics or search speed for most evaluated settings, while also revealing variability across tasks. Overall, EvoMem provides evidence that persistent memory can reduce some redundant exploration and improve the reuse and adaptation of successful strategies in LLM-driven evolutionary search.

Viktor Volkov, Valentin Khrulkov, Andrey V. Galichin +8

AXXX, Moscow, Russia · Applied AI Institute, Moscow, Russia · Lomonosov Moscow State University, Moscow, Russia

Memory-Augmented Language Models Evolutionary Optimization Large Language Model-Guided Optimization

Ask PDF