math.OCAug 24, 2026

Dimensionless Controls of Plasticity Under Alternating Tasks: From Evolutionary Biology to Continual Learning

Authors: Owen Skriloff

Organizations: University of Chicago

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Plasticity under changing environments is central to both evolutionary biology and continual learning. Motivated by recent work on genotype--phenotype maps, we study a minimal deep-learning analogue where a network is trained alternately on two Boolean label sets, and ask which biological controls of plasticity survive the translation to gradient descent. Reinterpreting four proposed biological factors as quantities of training dynamics, we find the system reduces to two dimensionless controls: the task disagreement rr, the fraction of disagreeing labels, and the reach ηTηT, the product of learning rate and switching period. We derive two bounds on plasticity: rr alone fixes an extremal geometric floor on the utopia distance, while rr and ηTηT jointly bound forgetting. Across 9,720 trajectories, an ANOVA confirms that rr, ηη, and TT dominate, while the effect of neutral-set size (emphasized in the biological setting) is negligible. The optimal reach itself follows an approximate inverse power law ηT∗∝r−1.18ηT^{*}\propto r^{-1.18}, yielding a heuristic that sets the optimal reach ηT∗ηT^* from the task disagreement alone. The analogy that survives is therefore dynamical rather than geometric, and our setting enables a view of plasticity through the lens of other driven systems in physics and engineering.

Figures & tables

Figure 1: Example trajectory of alternating training on two functions. The left panel shows the accuracy proxies a1(t)a_{1}(t) and a2(t)a_{2}(t) over iterations. The right panel shows the corresponding phase plane trajectory and utopia point over iterations.
Figure 2: Empirical demonstration of the bounds of Theorems 1 (left) and 2 (right) across the 9,720 trajectories. For Theorem 1 , we use the uniform estimate M=100M=100 .
Factorlog⁡Δa1\log\Delta a_{1}log⁡dU\log d_{U}
K~\tilde{K} bin1×10−41{\times}10^{-4}3×10−4 ∗∗∗3{\times}10^{-4\,***}
NSS ( ν\nu ) class3×10−4 ∗∗∗3{\times}10^{-4\,***}1×10−4 ∗∗∗1{\times}10^{-4\,***}
Hamming ( rr ) class0.4966∗∗∗0.4966^{***}0.9422∗∗∗0.9422^{***}
η\eta0.2178∗∗∗0.2178^{***}0.0019∗∗∗0.0019^{***}
TT0.1348∗∗∗0.1348^{***}0.0030∗∗∗0.0030^{***}
Residual0.15030.15030.05250.0525
Table 1: Type II ANOVA effect sizes for the two log-transformed plasticity outcomes. Models use complexity bin, neutral-set-size class, Hamming-distance class, learning rate, and switching period as categorical predictors. Reported values are ω2\omega^{2} effect sizes. Significance levels are denoted by ∗∗∗ for p<0.001p<0.001 . Note that while K~\tilde{K} and NSS yield statistically significant effects, their effect sizes are negligible compared to the other factors.
Figure 3: Heatmap of dUd_{U} values across (T,η)(T,\eta) space with the most plastic trajectories (lowest dUd_{U} values) occurring along a diagonal (constant ηT\eta T ) (left). Task disagreement rr level curves of the utopia distance dUd_{U} versus nominal reach ηT\eta T (center) show that as rr grows the optimum rises and shifts to smaller ηT\eta T while curves become more convex. The optimal reach ηT∗\eta T^{*} versus rr over 37 task pairs (right) shows a power-law fit ηT∗∝r−1.18\eta T^{*}\propto r^{-1.18} (95% CI [−1.34,−1.01][-1.34,-1.01] , R2=0.86R^{2}=0.86 ).
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

SettingValue
Hidden layers10
Layer width40
ActivationTanh
Initializationi.i.d. N(0,1/width)\mathcal{N}(0,1/\text{width})
Loss functionBinary cross-entropy (BCE)
OptimizerSGD (full batch)
Appendix
Table 2: Architecture and training hyperparameters.
#K~\tilde{K}P(y1)\mathbb{P}(y_{1})P(y2)\mathbb{P}(y_{2})ν\nudHd_{H}
156.01.80×10−51.80\times 10^{-5}6.00×10−66.00\times 10^{-6}2.40×10−52.40\times 10^{-5}3
256.01.00×10−51.00\times 10^{-5}1.20×10−51.20\times 10^{-5}2.20×10−52.20\times 10^{-5}64
356.01.80×10−51.80\times 10^{-5}6.00×10−66.00\times 10^{-6}2.40×10−52.40\times 10^{-5}125
456.02.20×10−52.20\times 10^{-5}4.60×10−54.60\times 10^{-5}6.80×10−56.80\times 10^{-5}3
556.03.00×10−53.00\times 10^{-5}3.40×10−53.40\times 10^{-5}6.40×10−56.40\times 10^{-5}65
656.04.60×10−54.60\times 10^{-5}2.20×10−52.20\times 10^{-5}6.80×10−56.80\times 10^{-5}125
Appendix
Table 3: Attributes of the 27 label pairs (y1,y2)(y_{1},y_{2}) used in the computational experiments.
 e−ℓe^{-\ell}reciprocalaccuracyBrier
e−ℓe^{-\ell}110.9980.9980.8230.8230.9410.941
reciprocal0.9980.998110.8300.8300.9430.943
accuracy0.8230.8230.8300.830110.9390.939
Brier0.9410.9410.9430.9430.9390.93911
Appendix
Table 4: Spearman rank correlations between the four performance proxies, computed per run ( n=7,212n=7{,}212 ; all p<10−3p<10^{-3} ).
Figure 4: Monotonicity audit of Theorem 1 . The y2y_{2} phase is strictly decreasing in 97.5%97.5\% of phases, and the weaker endpoint condition holds in 98.8%98.8\% . The median normalized positive variation is 00 , and the distribution is sharply concentrated at 00 with a heterogeneous right tail.
StudyVariantParamsMean dUd_{U}
Optimizerfull-batch SGD15,12115{,}1210.5650.565
 minibatch-1615,12115{,}1210.5390.539
 minibatch-3215,12115{,}1210.5480.548
 momentum 0.90.915,12115{,}1210.4870.487
 Adam15,12115{,}1210.4560.456
Architectureshallow ( 3×403\times 40 )3,6413{,}6410.6260.626
Appendix
Table 5: Mean steady-state dUd_{U} (exp proxy) across optimizer and architecture variants, each with its own reach sweep over 99 task pairs and 55 seeds.
Figure 5: dUd_{U} versus nominal reach ηT\eta T for the three architectures; the optimum location and depth are preserved across a 16×16\times parameter range.
More in this topic

Continual Learning

Continual Learning studies how models acquire knowledge from changing data streams while retaining or selectively revising what they learned before.

See all 305 papers

Explore similar work

CardsList

  1. Non-Equilibrium Stochastic Dynamics as a Unified Framework for Insight and Repetitive Learning: A Kramers Escape Approach to Continual Learning

    Apr 5, 2026Gunn KimContinual LearningStability-Plasticity Dilemma

    Department of Physics, Sejong University, Seoul 05006, Republic of Korea

  2. Predicting Plasticity in Deep Continual Learning: A Theoretical Perspective

    May 9, 2026Jiuqi Wang, Jayanth Srinivasa, Claire Chen +3Deep Learning OptimizationLoss of Plasticity

    University of Virginia · Cisco Research · California Institute of Technology +1

  3. A Bellman Optimality Equation for Plasticity

    Sep 9, 2026Jeremy Lucas, Doina PrecupStability-Plasticity DilemmaReinforcement Learning

    Mila – Quebec Artificial Intelligence Institute Montreal, Quebec, Canada