Children, but not language models, show accelerating returns in word learning

Michael C. Frank

cs.CLAug 17, 2026

Children, but not language models, show accelerating returns in word learning

Authors: Michael C. Frank

Abstract

Children learn hundreds of words over the first years of their lives, in a process that begins slowly but quickly picks up speed. Prior models describe vocabulary growth as evidence accumulation over time. Here we show that the process is best characterized as accelerating accumulation: children learn more from each additional unit of linguistic experience than they did from the one before. In contrast to children, language models -- even those trained on child-directed speech -- do not accelerate. Instead, they show constant proportional returns on new data, consistent with scaling laws. Children learn using many orders of magnitude less training data than language models; their increasingly efficient use of their learning input is a candidate explanation.

Figures & tables

Figure 1: The accelerating accumulator provides a psychometric model of vocabulary growth, describing the children’s word learning across datasets and languages. (A) Model predictions in latent-ability (

\theta

) space and, projected through the logistic item response model, in words-produced (CDI) space. Both pure accumulator (

\kappa=1

, dashed) and accelerating accumulator predictions are plotted. Greater efficiency (

\xi

) lifts the curve’s level, while greater acceleration (

\kappa

) fans it open. (B) Individual children’s observed trajectories (gray) for each of the five longitudinal datasets, with quantiles computed across the per-child parameters of the fitted models (colored percentile bands for M2 and M3; black lines for M0 and M1, which do not vary across individuals).

Figure 2: Under the accelerating accumulator model, the number of exposures needed to learn a word decreases with age. Plots show estimated cumulative exposures to a word, plotted by the age at which 50% of the children in our three English-language datasets were estimated to produce the word. Subplots show different word classes, and solid and dashed lines show simple linear fits from the accelerating (M3) and pure accumulator (M0) models, respectively; gray dots show the full distribution of words across all word classes.

Figure 3: Language models (LMs) show no acceleration in their vocabulary growth. (A) Density of the acceleration exponent

\kappa

for both children (per child, English and Norwegian) and LMs (by word). (B) Variation in acceleration (kappa) across children and LMs. Numbers show coefficients of variation and 95% CIs. (C) Proportional return on training (the ratio of decrease in loss to excess loss) for both children and LMs. Blue lines show LM trajectories; red lines show children’s fitted model trajectories from M3. Three children are highlighted in dark red with horizontal uncertainty bands indicating +/- 1 SD in calibration of their input. Logistic acquisition alone can produce some increase in proportional return due to ceiling effects; these asymptote at 1 (gray dashed line).

Figure 4: Non-identifiability of growth models in cross-sectional data. Panels show the same simulated data, reflecting the possibility of stable between-child intercept differences (A) vs. varying growth slopes (B).

				Age (months)
Citation	Language	N (3+/2+)	Admins	Mean	Min	Max	Adm/Chi
Thal (Wordbank)	English (American)	639 / 653	1,919	19.0	12	29	3
Smith (Wordbank)	English (American)	152 / 316	607	22.2	16	30	4
Marchman (Wordbank)	English (American)	162 / 2,136	499	20.3	8	30	3
Simonsen et al. (2014)	Norwegian	792 / 1,630	4,770	24.1	8	36	5
Hagihara et al. (2023)	Japanese	96 / 187	289	16.5	11	33	3

Table 1: Characteristics of all datasets used in the paper. N and ages reflect the analysis samples (post-exclusion) for the two exclusion criteria (3+ and 2+ administrations). Adm/Chi gives the median administrations per child.

Figure 5: Data-quality exclusions. Red lines show children excluded from the analysis entirely, with red dots marking flagged administrations; gray lines are a random sample of retained children, shown for reference.

	$\Delta$ ELPD vs M3 (SE)
Model	Thal	Smith	Marchman	Norwegian	Japanese
$\geq$ 3 administrations (main text)
M0. accumulator ( $\kappa=1$ )	-154,678 (350)	-102,675 (340)	-83,954 (285)	-130,396 (369)	-24,628 (186)
M1. + acceleration	-46,214 (289)	-62,282 (309)	-40,868 (258)	-70,888 (336)	-7,923 (126)
M2. + efficiency var.	-7,880 (133)	-6,542 (121)	-4,878 (110)	-5,702 (111)	-761 ( 40)
M3. + acceleration var.	0	0	0	0	1 0

Table 2: Bayesian leave-one-out (LOO) model comparison. Each cell is the difference in expected log predictive density (ELPD) relative to the best model (M3), with the standard error of the difference in parentheses; more negative is worse.

$k$	horizon (mo)	$\Delta$ ELPD/child	SE	$z$	% children better	$\sigma_{b}$
2	3	-32.41	6.77	-4.8	39	12.56
3	3	-5.32	4.10	-1.3	49	7.86
4	3	-0.33	3.64	-0.1	48	6.20
5	3	+0.02	3.51	+0.0	51	5.17

Table 3: Prospective cross-validation for Norwegian children with at least six administrations (n = 371). Each row describes a model fit to k administrations and scored on the child’s held-out final sitting. dELPD/child is the per-child difference in expected log predictive density, full model minus shared-slope model; negative favors the shared slope. sigma_b is the between-child acceleration SD in the training fit, against a full-data value of 5.65.

$k$	horizon (mo)	$\Delta$ ELPD/child	SE	$z$	% children better
2	3	+91.87	6.97	+13.2	84
3	3	+143.21	8.90	+16.1	88
4	3	+197.27	10.33	+19.1	90
5	3	+248.10	11.22	+22.1	92

Table 4: Prospective cross-validation of acceleration, on the same Norwegian children and held-out administrations as Table 3 . M2 retains a free population acceleration exponent; M2-zero removes it. dELPD/child is the per-child difference in expected log predictive density, M2 minus M2-zero; positive favors acceleration.

Figure 6: M3 fits (blue, drawn from fitted per-child acceleration) vs. non-parametric GAMLSS beta-regression (orange), with empirical 10th/50th/90th percentiles (open circles). Lines are the 10/50/90 percentiles of each model. Color denotes model.

Setting	N	$\kappa$	$\sigma_{b}$	$\Delta$ ELPD/obs (M2 $\to$ M3)
English (Thal)
$\geq$ 2 separate	653	11.5	3.19	0.0154
$\geq$ 3 separate (main text)	639	11.5	3.19	0.0158
pooled ( $\geq$ 3)	639	11.5	3.22 [3.07, 3.38]	—
English (Smith)
$\geq$ 2 separate	316	12.8	8.06	0.0156

Table 5: For each dataset and setting: number of children (N), population acceleration (

\kappa

), between-child acceleration SD (

\sigma_{b}

), and the per-observation LOO advantage of the full model M3 over M2 (

\Delta

ELPD per item response).

Filter	Removed	$\kappa$	$\sigma_{b}$
English (Marchman)
none	0	9.37	8.33
loose	29	11.17	8.30
main	32	10.65	6.60
tight	34	10.44	5.55
Norwegian

Table 6: Population acceleration and between-child acceleration SD by data-quality filter setting, for the two datasets in which the filter removes observations. The loose filter removes children with a decline of more than 40% below a child’s running peak or a rise of more than 60 percentage points per month; the main filter uses a decline of 25% and a rise of 40%, as reported in the Methods; and the tight filter uses 15% and 25%.

Dataset	$\Delta$ ELPD (log $-$ linear)
Thal	267 (31)
Smith	249 (23)
Marchman	281 (26)
Norwegian	323 (39)
Japanese	39 ( 8)

Table 7: Log-age vs linear-age accumulation. LOO ELPD advantage of the log-age model (M3) over its linear-age counterpart, with SE; positive favors log-age.

	Discrimination			Acceleration $\kappa$			Word slopes $\lambda_{j}\kappa_{i}$		Age of acquisition		LOO
Dataset	SD $\log\lambda$	$\lambda$ p10–p90	$r(\lambda,\delta)$	1PL	2PL	$\Delta\log\kappa$	med.	% $>1$	$r(t_{50})$	med. $\|\Delta t_{50}\|$	$\Delta$ ELPD (SE)
English (Thal)	0.25	0.73–1.34	+0.13 [+0.11, +0.15]	11.5	12.4	+0.072	12.2	99.8	0.999	0.13	-2,897 ( 82)
English (Smith)	0.27	0.71–1.37	+0.10 [+0.07, +0.13]	12.9	13.7	+0.053	12.9	99.3	0.996	0.13	-2,726 ( 79)
English (Marchman)	0.24	0.74–1.33	+0.05 [+0.02, +0.09]	10.6	11.6	+0.089	11.1	98.4	0.998	0.25	-1,412 ( 59)
Norwegian	0.29	0.70–1.36	+0.22 [+0.21, +0.23]	13.3	14.1	+0.065	13.7	99.4	0.995	0.16	-4,426 (108)
Japanese	0.34	0.65–1.51	+0.07 [+0.02, +0.12]	11.6	15.3	+0.279	14.7	100.0	0.986	1.47	-641 ( 41)

Table 8: Rasch (1PL) versus two-parameter logistic (2PL) item models, M3 refit on the same data. Discrimination: SD of

\log\lambda_{j}

, its 10th-90th percentile range on the natural scale, and its correlation with item difficulty (90% interval). Acceleration: population

\kappa

under each item model and the difference in

\log\kappa

. Age of acquisition: correlation and median absolute difference in per-word

t_{50}

, which is invariant to the discrimination scale. LOO: expected log predictive density of the 1PL relative to the 2PL, with SE; negative favors the 2PL.

Figure 7: Acceleration under deliberately narrowed item sets, plotted as the ratio to the full-data fit against the ratio of retained item-difficulty spread. Each point is one refit on the same children and administrations. If acceleration rescaled with the instrument, points would fall on the diagonal; if it is a property of the children, they fall on the horizontal line at one. Open points are random subsets of matched size, which discard the same number of items without narrowing the spread.

Figure 8: Histograms of per-child BLUP estimates, z-scored within each dataset, for both efficiency and acceleration parameters. Dashed lines indicate the standard normal reference distribution.

Dataset	N	$\kappa$	$\sigma_{a}$	$\sigma_{b}$	$\rho$	% $\kappa_{i}>1$
English (Thal)	639	11.5 [11.5, 11.6]	1.56 [1.49, 1.64]	3.19 [3.05, 3.35]	+0.10 [+0.03, +0.17]	99.8
English (Smith)	152	12.9 [12.8, 13.1]	2.08 [1.89, 2.31]	5.14 [4.69, 5.68]	-0.26 [-0.39, -0.13]	99.3
English (Marchman)	162	10.6 [10.5, 10.8]	1.80 [1.65, 1.99]	6.60 [5.90, 7.32]	+0.17 [+0.04, +0.29]	97.5
Norwegian	792	13.3 [13.2, 13.3]	2.71 [2.60, 2.83]	5.65 [5.41, 5.88]	-0.62 [-0.66, -0.58]	99.5
Japanese	96	11.6 [11.1, 12.1]	1.95 [1.71, 2.25]	5.45 [4.74, 6.32]	+0.26 [+0.07, +0.43]	100.0

Table 9: Random-effect parameters for M3 fits, by dataset: population acceleration

\kappa

, between-child SD in efficiency (

\sigma_{a}

) and acceleration (

\sigma_{b}

), their correlation (

\rho

), and the percentage of children estimated above the pure-accumulator value

\kappa=1

. Brackets give 90% posterior intervals.

Figure 9: Input rate does not change appreciably across development. Each panel plots log adult input rate against child age, with one point per recording, faint gray lines joining recordings from the same child, and the fitted within-child trend (black) with its 95% confidence band. Dashed lines show zero slope. Trends are fitted over 8-36 months (shaded), the range over which the constant-rate assumption is applied; gray points fall outside that window and are excluded from the fit.

Figure 10: Estimated cumulative exposures at age of acquisition under the accelerating accumulator (M3, left, reproduced from main text) and the fitted pure accumulator (M0, right), using identical corpus frequencies. Points are words, colored by class, with per-class linear fits.

		M3 (accelerating)		M0 (pure accumulator)
Word class	$n$	slope	$R^{2}$	slope	$R^{2}$
nouns	293	-4.77	0.38	-0.67	0.11
action words	103	-3.47	0.07	-0.06	0.00
descriptive words	62	-1.68	0.03	+0.49	0.03
function words	90	-3.18	0.13	-0.22	0.01

Table 10: Slope of estimated cumulative exposures with respect to age of acquisition, by word class, under each item model. Negative values mean fewer exposures are needed for later-acquired words.

Figure 11: Per-word acceleration for children and language models, both estimated with the same four-parameter logistic. Points are words, with the median marked; language model values are medians across the ten training seeds. The dashed line at one is the pure-accumulator value. Child estimates are attenuated by pooling across children and so understate the separation; the correction factors are given in the table.

Sample	words fitted	$\kappa$ raw [IQR]	recovery	$\kappa$ corrected	$\kappa$ (M3)
English (Thal)	135/681	10.18 [4.32, 14.84]	98%	10.42	11.51
English (Smith)	673/680	8.21 [6.04, 10.67]	62%	13.21	12.95
English (Marchman)	624/681	6.34 [3.12, 9.29]	59%	10.66	10.65
Norwegian	713/733	7.87 [6.97, 8.69]	57%	13.81	13.25
Japanese	343/447	10.22 [7.66, 15.56]	86%	11.92	11.57
Language models	609/609	1.16 [0.93, 1.46]

Table 11: Per-word acceleration estimated with the language models’ own estimator. For each word we fit the same four-parameter logistic used on LM surprisal to the proportion of children not yet producing that word, and take kappa = 0.434/scale. Recovery is the fraction of the population kappa that the same pipeline returns when run on data generated from the fitted M3, and is the factor by which pooling across children attenuates the estimate; the corrected estimate is the raw estimate divided by this factor. The final column is the kappa M3 model reports for that dataset.

Figure 12: Per-word acquisition slopes for the four architectures studied in Chang & Bergen (2022) – BERT, GPT-2, BiLSTM, LSTM – trained on BookCorpus + WikiText-103. CHILDES-trained GPT-2 models (training axis) are overlaid in gray.

Explore similar work

Baby Scale: Investigating Models Trained on Individual Children's Language Input
Mar 31, 2026Steven Y. Feng, Alvin W. M. Tan, Michael C. FrankLanguage Model Scaling Small Language Models
Stanford University
Is Child-Directed Language Optimized for Word Learning? A Computational Study of Verb Meaning Acquisition
May 12, 2026Francesca Padovani, Jaap Jumelet, Yevgen Matusevych +1Child Language Acquisition Computational Psycholinguistics
Center for Language and Cognition (CLCG), University of Groningen
Early Language Learning via Spreading Activation and Category Exploration in Complex Networks
Jul 7, 2026Salvatore CitraroChild Language Acquisition Computational Psycholinguistics
Institute of Information Science and Technologies “A. Faedo” (ISTI) National Research Council (CNR) Via G. Moruzzi, 1, 56124 Pisa, Italy · National Research Council (CNR) Via G. Moruzzi, 1, 56124 Pisa, Italy

Mar 31, 2026J/K move · Enter open · S save

Baby Scale: Investigating Models Trained on Individual Children's Language Input

Steven Y. Feng, Alvin W. M. Tan, Michael C. Frank

Stanford University

Language Model Scaling Small Language Models Child Language Acquisition

Open paper AskPDF arXiv

Modern language models (LMs) must be trained on many orders of magnitude more words of training data than human children receive before they begin to produce useful behavior. Assessing the nature and origins of this "data gap" requires benchmarking LMs on human-scale datasets to understand how linguistic knowledge emerges from children's natural training data. Using transcripts from the BabyView dataset (videos from children ages 6-36 months), we investigate (1) scaling performance at child-scale data regimes, (2) variability in model performance across datasets from different children's experiences and linguistic predictors of dataset quality, and (3) relationships between model and child language learning outcomes. LMs trained on child data show acceptable scaling for grammar tasks, but lower scaling on semantic and world knowledge tasks than models trained on synthetic data; we also observe substantial variability on data from different children. Beyond dataset size, performance is most associated with a combination of distributional and interactional linguistic features, broadly consistent with what makes high-quality input for child language development. Finally, model likelihoods for individual words correlate with children's learning of those words, suggesting that properties of child-directed input may influence both model learning and human language development. Overall, understanding what properties make language data efficient for learning can enable more powerful small-scale language models while also shedding light on human language acquisition.

Abstract

Explore similar work

Baby Scale: Investigating Models Trained on Individual Children's Language Input

Is Child-Directed Language Optimized for Word Learning? A Computational Study of Verb Meaning Acquisition

Early Language Learning via Spreading Activation and Category Exploration in Complex Networks