cs.LGNov 15, 2026

Does Learning Order Explain Forgetting? The Lie Bracket as an Early-Warning Signal for Catastrophic Forgetting Without Old Data

Authors: Ephraim Kofi Ayi Kwapong

Organizations: Responsible Artificial Intelligence Lab (RAIL), Kwame Nkrumah University of Science and Technology (KNUST), Ghana

Abstract

A network trained on task A and then task B ends at different weights than one trained on B and then A. To leading order this gap is set by the Lie bracket of the two tasks' gradient fields, and Dherin (2023) conjectured that a non-vanishing bracket may be linked to catastrophic forgetting. I test this conjecture empirically. Across Split MNIST, Split Fashion-MNIST and Split CIFAR-10, the forgetting of A is almost exactly the difference in A's loss between the two training orders (Spearman rho = 0.96-1.00). I then ask whether the bracket can predict that forgetting early in training on B and without any of A's data, using only a small summary stored when A finishes: its gradient and its top 20 Hessian eigenpairs. When the old task is well learned and forgetting is slow (task-incremental), this stored bracket ranks forgetting with rho = 0.70-0.81, against 0.33-0.58 for the penalty of Elastic Weight Consolidation, and matches or exceeds re-evaluating A on its own data. When forgetting is fast (domain-incremental), the stored summary goes stale and the EWC penalty is as good or better. The bracket fails outright when the old task is only half learned; I trace this to a residual gradient on A an order of magnitude larger, which makes first-order (slope) damage matter. The bracket is therefore a useful early-warning signal for the quiet kind of forgetting, under a clear condition.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. When Forgetting is not Catastrophic: On the Mechanics of Spurious Forgetting

    Oct 6, 2026Vedant Palit, Florent Draye, Nicolas Zucchet +2Memorization in Language ModelsLLM Fine-Tuning

  2. Erased, but Not Gone: Output Forgetting Is Not True Forgetting

    Jun 23, 2026Teresa Pui Yee Yong, Win Kent Ong, Chee Seng ChanMachine UnlearningMachine Unlearning Evaluation

  3. From Order to Distribution: An Exact Operator Framework for Forgetting in Continual Learning

    Apr 15, 2026Zonghuan Xu, Xingjun MaContinual LearningCatastrophic Forgetting