Mitigating Catastrophic Forgetting with Physics-Indexed Specialist Memory: the Evolutionary Neuroplastic Agent
Organizations: Facultad de Ciencias Médicas, Universidad Nacional del Litoral, Santa Fe, Argentina
Abstract
Continual reinforcement learning requires adaptation to changing conditions while preserving performance under previously encountered dynamics. It was investigated whether specialists generated by evolutionary search can provide a reusable memory when combined with environment detection and failure-driven search. The Evolutionary Neuroplastic Agent (ENA) evolves small neural networks without gradients, archives selected policies by detected dynamic regime, and retrieves them through Bayesian policy reuse. The architecture was evaluated in non-stationary CartPole and HalfCheetah environments. In the 30-replicate CartPole ablation, the complete architecture achieved a mean success rate of 83.2% across the five test environments. Among variants that learned the harder CartPole environments, the complete architecture showed the most consistent retention. A selection ablation also showed that the retrieval rule affected performance. In the evaluated sequences, ENA adapted more slowly than PPO on CartPole and achieved lower returns than SAC on HalfCheetah. The evidence concerns the complete architecture and does not isolate gradient-free optimization as the cause of retention or establish superiority over other continual-learning methods.
Figures & tables
| Method | Memory | Online detection | Recurrence | Gradient-free | Probing |
|---|---|---|---|---|---|
| EWC [ 3 ] | regularized shared weights | – | – | – | – |
| CLEAR [ 4 ] | replay into shared weights | – | – | – | – |
| OWL [ 5 ] | shared extractor (EWC), one head per task | – | bandit at test | – | – |
| RL-CD [ 6 ] | model and policy per context | ✓ | ✓ | tabular | – |
| MBCD [ 7 ] | model and policy per context | ✓ | ✓ | – | – |
| SWOKS [ 8 ] | policy per context | ✓ | ✓ | – | – |
| Variant | Plasticity | HoF | Detection |
|---|---|---|---|
| ENA-NoSkill | – | – | – |
| ENA-Plasticity | ✓ | – | – |
| ENA-HoF | – | ✓ | – |
| ENA-HoF+EnvDetection | – | ✓ | ✓ |
| ENA-Plasticity+HoF | ✓ | ✓ | – |
| ENA-Combined | ✓ | ✓ | ✓ |
| RQ | Evidence | Section |
|---|---|---|
| RQ1 | skill ablation (reliability and retention); forgetting of single-policy learners | V-A , V-E , V-G |
| RQ1a | ablation: Hall of Fame with and without detection; selection ablation | V-A , V-D |
| RQ1b | ablation: candidate steps with and without the archive | V-A |
| RQ1c | recognition per phase (both domains); detection benchmark (HalfCheetah) | A-C , V-F |
| RQ2 | ablation; compute study; controller study | V-A , V-B , V-C |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Continual RL
Continual reinforcement learning studies how agents adapt to changing tasks while retaining skills learned earlier.
See all 58 papers