Curriculum unlearning in humans and neural networks
Authors: Jirko Rubruck, Alice Zhang, Andrew Saxe, Christopher Summerfield
Organizations: Department of Experimental Psychology, University of Oxford, Oxford, UK · Gatsby Unit & Sainsbury Wellcome Centre, University College London, London, UK
During learning, humans often benefit from curricula that teach simple rules before exceptions. However, we lack a computational explanation for this phenomenon. Here, using a canonical behavioural task and modelling with neural networks, we develop and test a theory for why simple-to-complex curricula succeed. We show that during learning, both humans and neural networks show simplicity biases, a tendency to learn simple rules that are underfit to the data. In neural networks this bias is accompanied by compression of neural representations. Simple-to-complex curricula help both types of learners overcome this compression, developing representations (and policy) with appropriate dimensionality for the task. We test and confirm a further prediction from this theory: learners that have been pre-exposed to simple tasks and with more compressed initial representations benefit more from curricula. These findings suggest that curricula help with ``unlearning'' of simple rules.
Figures & tables
Figure 1: a , Participants learned to classify bug stimuli as either “good” or “bad” in a rule plus exception (R+E) task. Most bugs can be correctly classified based on a single feature (here wings) but there are two exceptions to this rule. b , Participants learned the task in a trial-by-trial manner with fully supervised feedback. After the training phase, they were assessed in a test phase without feedback. c , We modelled the task using a two-layer neural network with stimuli ( d ) which similarly varied along three binary dimensions. The eight stimuli can be depicted as the corners of a cube in a three-dimensional binary feature space. Colour denotes rule (purple) versus exception (blue) stimuli and shape denotes the category label (circle = Class 0, triangle = Class 1). The network (bottom) contained H ReLU hidden units and a sigmoid output unit and was trained using trial-wise gradient descent, we trained one network per participant on that participant's own sequence of trials, so that each network receives identical stimuli in identical order to human participants e , Training consisted of five learning blocks of 48 trials each in which rule and exception stimuli were randomly interleaved. In the “no curriculum” condition, stimuli were sampled pseudorandomly on each block (36 rule stimuli and 12 exceptions). In the simple-to-complex condition, the fraction of exception stimuli in each block increased over the course of five blocks from 0 to 0.5. In the “complex-to-simple” curriculum, the fraction of exception stimuli decreased over learning blocks. All curriculum conditions are matched for the overall number of exposures to each stimulus.
Figure 2: a, Proportion of participants (n = 207) classified into each of five strategies at the end of each learning block and at test, separately by curriculum. Strategies were assigned by fitting nine deterministic policies plus a lapse rate and selecting by BIC against an unpenalised random-responding null. b, The same classification applied to networks, one per participant, trained on that participant's exact trial sequence. Icons mark which rows show people and which show networks. c, Block-to-block flow of human strategies, with ribbon width proportional to the number of participants making each transition; the dotted vertical line marks the test phase. d, Transition matrices summarising the transitions in c, across training blocks giving the percentage of participants in each state at block b who occupied each state at block b+1. Rows sum to 100%. Pooled across curricula, the odds of moving from rule to rule + exceptions were 2.23 times higher under the simple-to-complex curriculum than under the other two (95% CI [1.18, 4.24], p = .014; Mantel–Haenszel stratified by block, participant-clustered bootstrap [1.21, 4.51]).
Figure 3: Simple-to-complex curricula boost exception learning in humans and neural networks. a, Human accuracy over training, binned into ten 24-trial bins, separately for rule (purple) and exception (blue) stimuli and for each curriculum. Shading gives ± 1 SEM across participants. b, Human test accuracy, averaged over two presentations of each of the eight stimuli. c , d The same for the networks, measured identically, per-trial accuracy on the presented stimulus during training, and noise-marginalised accuracy at test. Networks inherit each participant's trial sequence, so the two rows are matched trial for trial and differ only in the learner. Error bars give ± 1 SEM. e, f Hyperparameter sweep for the networks. Colour gives the curriculum benefit — simple-to-complex minus no curriculum — in end-of-training accuracy on exception stimuli ( e ) and rule stimuli ( f ). The black box marks the configuration used throughout the paper. The exception benefit is positive in 34 of 35 configurations, so the effect does not depend on the exact hyperparameters selected.
Figure 4: Uncompression in networks, and the reaction-time signature of the strategies. a, Participation ratio of the eight hidden-layer stimulus representations at the end of training, by curriculum. b, Final participation ratio against final exception accuracy, one point per network ( r = +0.70, p < .001). c, d Separation of the hidden representations along the exception axis and the rule axis over training. The rule axis is the class-1 minus class-0 mean over the six rule stimuli; the exception axis is the direction separating the two exceptions, orthogonalised against the rule axis. Both axes have a fixed sign in every network, so projections can be averaged. e, The eight stimuli projected onto those axes early in training (trial 48) and at the end (trial 240), for no curriculum and simple-to-complex . Colour denotes stimulus type and shape the category label, as in Figure 1D. By the end of training the exceptions are pulled further apart under the curriculum ( simple-to-complex minus no curriculum : +1.58, t( 122) = 4.82, p < .001), and the final participation ratio is higher (+0.104, t( 127) = 3.52, p < .001). f–h In human participants, the reaction-time cost of exception stimuli relative to rule stimuli: over blocks and by curriculum ( f ), by the strategy the participant was in ( g , raw), and from a mixed model with participant random effects ( h , estimates relative to random responding). The cost is largest for participants following rule + exceptions, as expected if they are retrieving an exception rather than applying the rule. Stars give p < .05, .01, .001; error bars ± 1 SEM; n = 891 participant-blocks from 207 participants.
Figure 5: Pretraining on a 1-D task strengthens simplicity biases and leads to greater curriculum benefit. Error bars and shaded ribbons represent the standard error of the mean (SEM). a, Exp. 2 task design. Participants learn to classify a different set of bugs in pretraining using a 1-D rule based on a single feature or an XOR rule over two features. In the learning phase, participants learn the rule plus exception (R+E) task under a simple-to-complex curriculum or no curriculum . Finally, participants complete a test phase where they classify the stimuli from learning without feedback. b, c, Pretraining accuracy for participants and for networks, binned into 24-trial bins. One network was trained per participant, on that participant's own trial sequence and in their own physical feature coding, through both phases. Every hyperparameter is inherited unchanged from the Exp. 1 model; one lapse rate per phase is the only quantity fitted. Inset: participation ratio of the eight hidden representations at the end of pretraining, 1.09 after one-dimensional pretraining against 2.93 after XOR — the compression manipulation works on the networks as intended. d, e, Test accuracy for rule (purple) and exception (blue) stimuli, by curriculum, with pretraining shown by marker shape. f–i Accuracy over the R+E phase by curriculum, people and networks. Under simple-to-complex no exception trials occur in the first block, so those bins are empty by design. j, Participation ratio at the end of R+E learning. The curriculum raises it for one-dimensionally pretrained networks (+0.062, p < .001) with no detectable effect after XOR pretraining (+0.020, p = .57). k, Exception accuracy at test against participation ratio after pretraining, one point per network, with the curriculum benefit marked at each cluster. The benefit narrows for the most compressed networks.
Figure 6: RT-based measure of compression in pretraining predicts later curriculum benefit. a. Proportion of participants in each strategy over the five R+E learning blocks, split by pretraining (columns) and curriculum (rows), classified as in Figure 2. b. Proportion reaching the optimal rule + exceptions strategy by block 5. With 1-D pretraining fewer participants reach this state without a curriculum but more with one, giving a significant pretraining × curriculum interaction (b = − 0.686, z = − 2.50, p = .012); the bracket marks that interaction. Error bars are Wilson intervals. c . We compute a reaction-time measure of selective attention to each feature dimension during 1-D pretraining. For consecutive trial pairs, each feature either switches or matches; we define each feature's switch-match reaction time (SMRT) as the ratio of switch to match RTs, where SMRT > 1 indicates attention to that feature. Selective attention to a feature is its SMRT normalized by the average SMRT across features. d. During 1-D pretraining, selective attention to the relevant 1-D feature increases while selective attention to the two irrelevant features decreases. Pretraining is split into five bins for computing selective attention, and error bars are standard errors of participant means. e. Participants who display greater selective attention to the relevant feature in 1-D pretraining benefit more from simple-to-complex curricula. Without a curriculum, exception accuracy falls as participants compress more and under the curriculum it does not. Selective attention is binned into quintiles for visualization. Error bars give ± 1 SEM.
Figure S1: Balanced washout control. We continued training networks for an additional 48-trial block at the no-curriculum composition (36 rule, 12 exception) appended after block 5, so that all three conditions end on the same local statistics; the test probe follows that block. a, Test accuracy on exception stimuli by curriculum, without (open) and with (filled) the washout block. Points are means over 207 networks and error bars are 95% confidence intervals; accuracy is averaged over 20 simulation seeds before plotting. b, The same for rule stimuli. c, Curriculum contrasts against no curriculum , estimated within stimulus type from the same mixed model used for the network test-phase contrasts in the main text (estimate and 95% CI). The exception benefit of simple-to-complex is reduced but remains reliable after the washout, whereas the complex-to-simple exception cost and the simple-to-complex rule cost are both abolished, identifying them as recency effects.
Figure S2: Frozen representation control. Experiment 2 repeated with the hidden layer frozen at its initialisation and only the readout trained (a fixed random-feature network), alongside the trained-representation model of Figure 5. a, Participation ratio of the eight stimuli at the end of pretraining, by pretraining task. One-dimensional pretraining compresses the trained network and leaves the frozen network unchanged by construction. b, Curriculum benefit on exception stimuli at test, within each pretraining group (estimate and 95% CI, from the same mixed model used for the Experiment 2 test-phase contrasts in the main text). c, The three-way pretraining × curriculum × stimulus-type interaction for each network. The frozen network shows a larger overall curriculum benefit than the trained one, but it is the same after one-dimensional and XOR pretraining and the interaction is absent. n = 1027 networks, three simulation seeds.
Department of Neurobiology, Harvard Medical School · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University · Department of Psychology and Center for Brain Science, Harvard University