Language models acquire their capabilities from training data, and developers withhold data during training to keep models from learning capabilities that could be misused. How long this delays a capability, and how much data is needed to acquire it later, is unknown. Here we show that the general progress a model makes on the rest of its training data substitutes for the data that was withheld. We withheld the training signal for individual skills in models of chess, Othello, molecular strings, bracket languages and web text. The delay of a withheld skill in larger models was predicted from smaller ones. A chess model never explicitly trained to respond to check ranked the legal responses almost as well as a model that was. When models were then trained on the withheld data, they acquired the skill with a small fraction of the data needed from initialization, and this fraction decreased as the models progressed. A control rule of equal difficulty was learned much more slowly. A programming language withheld from the training text of a 7-billion-parameter model was regained in the same way. Data filtering therefore delays a capability, and the data required to regain it decreases as models improve.
Figures & tables
Figure 1Figure 2Figure 3Figure 4Figure 5
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Size
Trained
Tokens
Loss
Legal
Ranked first
Quiet block
Shift
Both boards
Disjoint
9.7M
Games without a check
600
1.803
0.453
0.652
0.772
0.905
0.646 (0.601–0.687)
0.566
Without in-check targets
302
1.778
0.426
0.672
0.666
0.882
0.624 (0.584–0.658)
0.571
Fully supervised
200
1.831
0.847
0.678
0.963
0.996
0.933 (0.908–0.956)
0.927
24M
Games without a check
600
1.747
0.477
0.661
0.796
0.928
0.697 (0.657–0.736)
0.656
Without in-check targets
200
1.760
0.437
0.645
0.682
0.886
0.642 (0.603–0.678)
0.592
Fully supervised
200
1.763
0.887
0.710
0.973
0.998
0.960 (0.943–0.973)
0.947
Appendix
Table 6
Fit with source-run intercepts
Least squares
Domain
Runs
Source
Below acq.
Intercept a
Rate b
Residual
Intercept a
Rate b
Residual
runs
loss
s.d.
s.d.
Chess
159
12
129
− 0.53 ± 0.05
1.12 ± 0.04
0.25
− 0.52
1.14 ± 0.04
0.25
Web text
88
11
70
− 0.62 ± 0.05
0.91 ± 0.04
0.10
− 0.66
0.89 ± 0.06
0.17
SMILES
65
4
51
− 0.42 ± 0.09
0.59 ± 0.07
0.32
− 0.41
0.61 ± 0.09
0.37
Single line
312
27
250
− 0.53
0.97 ± 0.07
0.36
Appendix
Table 7
Standard learning rate
One third of it
Size
Checkpoint
Background
Logit
16 seq.
64 seq.
256 seq.
64 seq.
16 seq.
(tokens)
loss (nats)
distance
Cost, % of the tokens from initialization at the same setting