cs.LGNov 3, 2026

Curriculum unlearning in humans and neural networks

Authors: Jirko Rubruck, Alice Zhang, Andrew Saxe, Christopher Summerfield

Organizations: Department of Experimental Psychology, University of Oxford, Oxford, UK · Gatsby Unit & Sainsbury Wellcome Centre, University College London, London, UK

Abstract

During learning, humans often benefit from curricula that teach simple rules before exceptions. However, we lack a computational explanation for this phenomenon. Here, using a canonical behavioural task and modelling with neural networks, we develop and test a theory for why simple-to-complex curricula succeed. We show that during learning, both humans and neural networks show simplicity biases, a tendency to learn simple rules that are underfit to the data. In neural networks this bias is accompanied by compression of neural representations. Simple-to-complex curricula help both types of learners overcome this compression, developing representations (and policy) with appropriate dimensionality for the task. We test and confirm a further prediction from this theory: learners that have been pre-exposed to simple tasks and with more compressed initial representations benefit more from curricula. These findings suggest that curricula help with ``unlearning'' of simple rules.

Figures & tables

Explore similar work

CardsList
  1. A Compression Perspective on Simplicity Bias

    Mar 26, 2026Tom Marty, Eric Elmoznino, Leo Gagnon +5Neural Network GeneralizationFeature Selection

  2. Effectiveness of Curriculum Learning Depends on Reward Sparsity and Competing Optima

    Oct 22, 2026John J. Vastola, Ann Huang, Satpreet H. Singh +2Curriculum RLReinforcement Learning

  3. Learning to Reason with Curriculum II: Compositional Generalization

    Jun 26, 2026Nived Rajaraman, Audrey Huang, Miroslav Dudik +3Curriculum LearningAutomata Learning