Baby Scale: Investigating Models Trained on Individual Children's Language Input

Steven Y. Feng; Alvin W. M. Tan; Michael C. Frank

cs.CLMar 31, 2026

Baby Scale: Investigating Models Trained on Individual Children's Language Input

Authors: Steven Y. Feng, Alvin W. M. Tan, Michael C. Frank

Organizations: Stanford University

Abstract

Modern language models (LMs) must be trained on many orders of magnitude more words of training data than human children receive before they begin to produce useful behavior. Assessing the nature and origins of this "data gap" requires benchmarking LMs on human-scale datasets to understand how linguistic knowledge emerges from children's natural training data. Using transcripts from the BabyView dataset (videos from children ages 6-36 months), we investigate (1) scaling performance at child-scale data regimes, (2) variability in model performance across datasets from different children's experiences and linguistic predictors of dataset quality, and (3) relationships between model and child language learning outcomes. LMs trained on child data show acceptable scaling for grammar tasks, but lower scaling on semantic and world knowledge tasks than models trained on synthetic data; we also observe substantial variability on data from different children. Beyond dataset size, performance is most associated with a combination of distributional and interactional linguistic features, broadly consistent with what makes high-quality input for child language development. Finally, model likelihoods for individual words correlate with children's learning of those words, suggesting that properties of child-directed input may influence both model learning and human language development. Overall, understanding what properties make language data efficient for learning can enable more powerful small-scale language models while also shedding light on human language acquisition.

Figures & tables

Figure 1: Broader performance scaling with data quantity across all four benchmarks. Each color denotes a separate dataset, with different markers showing different models. Gray curves show spline-smoothed average relationships across all data.

Model	Condition	# of Fams	Tokens	TTR	Zorro	WordSim	COMPS	EWoK
GPT-2 small	Mean of fams	1	142,094	–	55.19 $\pm$ 2.89	0.049 $\pm$ 0.007	50.09 $\pm$ 0.14	50.18 $\pm$ 0.15
GPT-2 small	Top-1	1	725,600	0.026	58.09 $\pm$ 3.36	0.042 $\pm$ 0.025	50.19 $\pm$ 0.49	50.07 $\pm$ 0.36
GPT-2 small	Top-2	2	727,986	0.025	59.58 $\pm$ 1.85	0.064 $\pm$ 0.022	50.37 $\pm$ 0.33	50.12 $\pm$ 0.13
GPT-2 small	Top-5	5	727,883	0.026	56.70 $\pm$ 2.88	0.062 $\pm$ 0.008	50.22 $\pm$ 0.27	50.32 $\pm$ 0.05
GPT-2 small	Top-10	10	735,309	0.027	57.14 $\pm$ 2.79	0.048 $\pm$ 0.020	50.22 $\pm$ 0.20	50.13 $\pm$ 0.21
GPT-2 small	Top-20	20	734,234	0.026	56.59 $\pm$ 2.18	0.044 $\pm$ 0.015	50.03 $\pm$ 0.23	50.31 $\pm$ 0.29

Table 1: Major BabyView results (averaged across 5 seeds per experiment). Mean of fams reports mean and standard deviation across the 20 individual-family datasets. Pooled rows (Top-2/5/10/20) are proportional mixtures with seed-level variability. Top-1 is the largest single family; All fams is the all-families model trained on the concatenated corpus. Zorro, COMPS, and EWoK are percentages (higher is better); WordSim is Spearman correlation (higher is better). TTR is type–token ratio (unique word types divided by total tokens).

Figure 2: Performance scaling of (A) BabyView experiments and (B) TinyDialogues experiments across all four benchmarks. Each color denotes a separate model; different markers show different experimental conditions. Lines of best fit show average linear relationships.

Feature	Top10	Meth.	Spear.	Lasso	XGB	Mean rank	Mean $\|\rho\|$	Mean $\|\beta\|$	Mean imp.
Bigram mutual information	9	3	3	1	5	5.56	0.72	0.153	0.060
Caregiver POS-tagged token count	7	3	4	1	2	4.29	0.74	0.016	0.062
Child POS-tagged token count	7	3	1	4	2	2.00	0.86	0.604	0.356
Child-to-caregiver semantic pair count	6	3	2	2	2	6.17	0.80	0.005	0.050
POS bigram entropy	6	3	1	1	4	3.33	0.56	0.263	0.069
Mean KL divergence from other datasets	5	3	3	1	1	3.60	0.71	0.500	0.053

Table 2: Top linguistic predictors across statistical analyses, identified and sorted by Top10 hit. Top10 counts the number of times a feature appears in top-10 lists across all methods and experimental conditions. Meth. is the number of distinct methods (Spearman, Lasso, XGBoost) that select the feature at least once. Spear. , Lasso , and XGB report method-specific top-10 counts. The following mean columns compute average across top-10 appearances. Mean rank is the average within-list rank when selected. Mean

|\rho|

is the average absolute Spearman correlation. Mean

|\beta|

is the average absolute lasso coefficient (linear effect size under regularization). Mean imp. is the average XGBoost feature importance (nonlinear contribution to predictive performance). Full results (all linguistic features) in Table 9 .

Figure 3: (A) Relationship between mean token NLL and AoA for CDI words by lexical category, for models trained on data from all families. (B) AIC of the regressions predicting AoA from mean token NLL. Dashed line indicates AIC of the base frequency model.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Age (months)	Tokens	Zorro	WordSim	COMPS	EWoK
BV-fam-S00400001	31	725,600	58.09 $\pm$ 3.36	0.042 $\pm$ 0.025	50.19 $\pm$ 0.49	50.07 $\pm$ 0.36
BV-fam-S00510002	11	569,788	55.95 $\pm$ 2.74	0.054 $\pm$ 0.015	50.05 $\pm$ 0.42	50.14 $\pm$ 0.20
BV-fam-S00360001	8	257,817	59.53 $\pm$ 1.83	0.048 $\pm$ 0.022	50.07 $\pm$ 0.15	50.17 $\pm$ 0.30
BV-fam-S00220001	5	242,954	58.02 $\pm$ 2.19	0.058 $\pm$ 0.029	50.12 $\pm$ 0.34	50.18 $\pm$ 0.36
BV-fam-S00400002	28	181,317	57.95 $\pm$ 2.19	0.054 $\pm$ 0.031	50.07 $\pm$ 0.35	50.42 $\pm$ 0.80
BV-fam-S00320002	7	171,656	55.98 $\pm$ 2.31	0.028 $\pm$ 0.024	50.25 $\pm$ 0.32	50.21 $\pm$ 0.27

Appendix

Table 3: Individual-family BabyView results for GPT-2 small with seed-level variability, sorted by training tokens. Age is months at onboarding time of that family’s given child.

Family	Age (months)	Tokens	Zorro	WordSim	COMPS	EWoK
BV-fam-S00400001	31	725,600	57.54 $\pm$ 1.98	0.044 $\pm$ 0.018	50.10 $\pm$ 0.16	50.03 $\pm$ 0.26
BV-fam-S00510002	11	569,788	57.65 $\pm$ 2.52	0.054 $\pm$ 0.027	49.97 $\pm$ 0.20	49.91 $\pm$ 0.10
BV-fam-S00360001	8	257,817	59.29 $\pm$ 0.95	0.032 $\pm$ 0.033	50.13 $\pm$ 0.29	49.94 $\pm$ 0.36
BV-fam-S00220001	5	242,954	59.43 $\pm$ 1.08	0.058 $\pm$ 0.018	49.74 $\pm$ 0.45	50.19 $\pm$ 0.30
BV-fam-S00400002	28	181,317	56.25 $\pm$ 3.03	0.014 $\pm$ 0.024	50.07 $\pm$ 0.32	50.16 $\pm$ 0.25
BV-fam-S00320002	7	171,656	56.49 $\pm$ 2.74	0.014 $\pm$ 0.019	50.42 $\pm$ 0.27	50.17 $\pm$ 0.38

Appendix

Table 4: Individual-family BabyView results for GPT-2 mini with seed-level variability, sorted by training tokens. Age is months at onboarding time of that family’s given child.

Family	Age (months)	Tokens	Zorro	WordSim	COMPS	EWoK
BV-fam-S00400001	31	725,600	59.10 $\pm$ 1.58	0.068 $\pm$ 0.026	50.22 $\pm$ 0.27	50.31 $\pm$ 0.35
BV-fam-S00510002	11	569,788	57.13 $\pm$ 1.69	0.062 $\pm$ 0.013	50.17 $\pm$ 0.20	50.13 $\pm$ 0.17
BV-fam-S00360001	8	257,817	58.26 $\pm$ 0.85	0.058 $\pm$ 0.015	50.18 $\pm$ 0.29	50.08 $\pm$ 0.27
BV-fam-S00220001	5	242,954	59.88 $\pm$ 1.82	0.056 $\pm$ 0.009	49.93 $\pm$ 0.23	50.10 $\pm$ 0.17
BV-fam-S00400002	28	181,317	57.57 $\pm$ 1.58	0.054 $\pm$ 0.015	49.82 $\pm$ 0.28	49.80 $\pm$ 0.27
BV-fam-S00320002	7	171,656	56.68 $\pm$ 0.94	0.026 $\pm$ 0.011	50.24 $\pm$ 0.34	50.14 $\pm$ 0.35

Appendix

Table 5: Individual-family BabyView results for GPT-BERT base with seed-level variability, sorted by training tokens. Age is months at onboarding time of that family’s given child.

Family	Age (months)	Tokens	Zorro	WordSim	COMPS	EWoK
BV-fam-S00400001	31	725,600	54.37 $\pm$ 1.63	0.072 $\pm$ 0.016	50.10 $\pm$ 0.10	50.00 $\pm$ 0.11
BV-fam-S00510002	11	569,788	55.65 $\pm$ 0.85	0.068 $\pm$ 0.047	50.12 $\pm$ 0.33	49.96 $\pm$ 0.20
BV-fam-S00360001	8	257,817	57.97 $\pm$ 1.34	0.066 $\pm$ 0.021	50.22 $\pm$ 0.23	49.88 $\pm$ 0.39
BV-fam-S00220001	5	242,954	57.24 $\pm$ 1.12	0.074 $\pm$ 0.018	49.98 $\pm$ 0.36	50.17 $\pm$ 0.34
BV-fam-S00400002	28	181,317	59.01 $\pm$ 1.77	0.058 $\pm$ 0.016	49.77 $\pm$ 0.23	50.06 $\pm$ 0.28
BV-fam-S00320002	7	171,656	56.21 $\pm$ 1.77	0.056 $\pm$ 0.019	50.07 $\pm$ 0.27	49.99 $\pm$ 0.26

Appendix

Table 6: Individual-family BabyView results for GPT-BERT small with seed-level variability, sorted by training tokens. Age is months at onboarding time of that family’s given child.

Model	Target	$\alpha$	NZ feat.	Lasso $R^{2}$	Lasso RMSE	XGB $R^{2}$	XGB RMSE
GPT-2 Mini	Zorro	0.0488	21	0.657	1.784	0.395	2.369
GPT-2 Mini	WordSim	0.0026	13	-0.208	0.029	-0.166	0.029
GPT-2 Mini	COMPS	0.0668	0	-0.144	0.166	-1.076	0.223
GPT-2 Small	Zorro	0.5230	6	0.289	2.974	0.172	3.208
GPT-2 Small	WordSim	0.0173	1	-1.047	0.031	-1.943	0.037
GPT-2 Small	COMPS	0.0713	3	-1.479	0.257	-0.601	0.207

Appendix

Table 7: Cross-validated predictive performance of lasso regression and XGBoost models for each model–target pair.

\alpha

denotes the lasso regularization strength selected via cross-validation. NZ feat. is the number of features with nonzero coefficients in the fitted lasso model, reflecting model sparsity. Lasso

R^{2}

and XGB

R^{2}

report the proportion of variance explained on held-out data (higher is better), while Lasso RMSE and XGB RMSE report root mean squared error (lower is better). Together, these metrics characterize both predictive performance and model complexity across methods.

Feature name	Brief description
Lexical / Distributional / Scale
avg_conversation_len_tokens	This feature measures the average number of tokens per conversation, capturing how much language is typically contained in a single interaction.
avg_sentence_len_tokens	This feature measures the average number of tokens per sentence, providing a simple proxy for sentence-level length and complexity.
avg_utterance_len_tokens	This feature measures the average number of tokens per utterance, summarizing how long individual turns tend to be.
bigram_entropy	This feature measures the entropy of the bigram distribution, so higher values indicate greater diversity in adjacent token sequences.
bigram_mutual_information	This feature measures the average pointwise mutual information of adjacent token pairs, so higher values indicate that neighboring words are more strongly associated than expected by chance.

Appendix

Table 8: Comprehensive catalog of the 175 linguistic features considered in our BabyView analysis. The table is grouped by feature family and provides a one-sentence description of each feature. Note that all 175 features were kept after restricting to numeric variables and applying missingness (

\leq 0.4

across the 25 experiments) and zero-variance filtering. Hence, all 175 features were used in the final statistical analyses (Spearman, lasso, and XGBoost).

Feature	Top10	Meth.	Spear.	Lasso	XGB	Mean rank	Mean $\|\rho\|$	Mean $\|\beta\|$	Mean imp.
Lexical / Distributional / Scale
bigram_mutual_information	9	3	3	1	5	5.56	0.72	0.153	0.060
hapax_ratio	5	3	1	1	3	6.60	0.48	0.264	0.067
trigram_entropy	5	2	4	0	1	5.00	0.74	0.000	0.075
num_conversations	5	1	0	5	0	2.00	0.00	0.178	0.000
avg_conversation_len_tokens	4	3	2	1	1	8.25	0.57	0.005	0.036

Appendix

Table 9: Full linguistic feature results grouped by category, sorted by Top10 hit count within each category. Top10 is the total number of model–target analyses in which a feature appeared in a method’s top-10 list; Meth. is the number of methods (Spearman, lasso, XGBoost) for which the feature appeared at least once; Spear. , Lasso , and XGB give the corresponding method-specific hit counts; Mean rank is the average within-method top-10 rank across all hits; Mean

|\rho|

is the mean absolute Spearman correlation across the feature’s Spearman top-10 appearances; Mean

|\beta|

is the mean absolute lasso coefficient across the feature’s lasso top-10 appearances; Mean imp. is the mean XGBoost feature-importance score across the feature’s XGBoost top-10 appearances.

Explore similar work

Children, but not language models, show accelerating returns in word learning
Aug 17, 2026Michael C. FrankChild Language Acquisition
Learning from Child-Directed Speech in Two-Language Scenarios: A French-English Case Study
Mar 13, 2026Liel Binyamin, Elior SulemLanguage Model Pretraining Language Modeling
Faculty of Computer and Information Science, Institute for Applied AI Research Data Science Research Center Ben-Gurion University of the Negev
Child-directed speech facilitates production, not comprehension, in BabyLMs
May 31, 2026Bastian Bunzeck, Sina ZarrießLanguage Model Pretraining LLM Evaluation
Computational Linguistics, Department of Linguistics Bielefeld University, Germany

Aug 17, 2026J/K move · Enter open · S save

Children, but not language models, show accelerating returns in word learning

Michael C. Frank

Child Language Acquisition

Open paper AskPDF arXiv

Children learn hundreds of words over the first years of their lives, in a process that begins slowly but quickly picks up speed. Prior models describe vocabulary growth as evidence accumulation over time. Here we show that the process is best characterized as accelerating accumulation: children learn more from each additional unit of linguistic experience than they did from the one before. In contrast to children, language models -- even those trained on child-directed speech -- do not accelerate. Instead, they show constant proportional returns on new data, consistent with scaling laws. Children learn using many orders of magnitude less training data than language models; their increasingly efficient use of their learning input is a candidate explanation.

Abstract

Explore similar work

Children, but not language models, show accelerating returns in word learning

Learning from Child-Directed Speech in Two-Language Scenarios: A French-English Case Study

Child-directed speech facilitates production, not comprehension, in BabyLMs