Language models generalize beyond their training corpus to learn linguistic phenomena (Patil et al., 2024; Misra and Mahowald, 2024), showing that there exists indirect evidence driving learning. Prior work has inferred that indirect evidence exists without identifying its character or considered specific hypotheses about what it might be. We explore an alternative bottom-up method for identifying indirect evidence. We use influence functions (Koh and Liang, 2017) to measure how individual training sentences affect model predictions in 15 grammatical phenomena from BLiMP (Warstadt et al., 2020). We filter out a phenomenon's direct evidence using hand-constructed rules and take the 60,000 most influential remaining sentences as indirect evidence. This indirect evidence substantially affects learning: retraining without it lowers held-out accuracy more than removing a matched random group for every phenomenon across five models (by a median of 12 points). We characterize the indirect evidence by its similarity to our direct evidence filters: it (partially) meets the filters' structural conditions at a median of 4.7 times the rate of a matched random group. Finally, clustering phenomena by indirect-evidence overlap recovers three of the five BLiMP categories that include multiple phenomena, suggesting that this method can reveal linguistic relationships.
Figures & tables
Figure 1: Schematic overview of our methodology, shown for one phenomenon ( c-command ) and one reference model. (A) We rank every sentence in the training corpus using influence functions with an attribution objective capturing the model’s grammatical preferences in the phenomenon’s BLiMP minimal pairs. We filter out the phenomenon’s direct evidence and take the 60,000 most influential remaining sentences as indirect evidence. (B) We remove the indirect evidence or an equally sized control group, retrain without the group, and then evaluate on held-out minimal pairs to validate our methodology. (C) We characterize the indirect evidence.
Figure 2: Accuracy on held-out BLiMP minimal pairs after removing each group. Extra drop is the additional drop in accuracy, averaged over the five initializations, when removing indirect evidence compared to matched random. Points show means over the five models; bars are the range. Ranges narrower than a marker are hidden behind it. Comparing against direct evidence (a random 60,000 of its sentences) was done only for the five phenomena with at least 60,000 direct-evidence sentences.
Figure 3: Held-out accuracy lost (points) after removing each phenomenon’s indirect evidence (rows), evaluated on every phenomenon (columns), relative to the reference model and averaged over five initializations. Numbers give each phenomenon’s own accuracy drop. Black boxes outline the BLiMP categories.
Figure 4: Direct evidence among the 60,000 most influential sentences compared to its proportion of the entire training corpus. Direct share is how much of the top 60,000 is direct evidence. Enrichment is direct share divided by how much of the corpus is direct evidence.
Figure 5: Lexical similarity of the indirect evidence, direct evidence, and the BM25 group, relative to the matched random group’s lexical similarity. Points show means over the five models; bars are the range.
Figure 6: Structural coverage of the indirect evidence and the BM25 group relative to the matched random group (log scale), for the 11 phenomena whose coverage varies among non-direct sentences; Appendix E reports the other four. Points are means over five models and bars are ranges; the right columns give the matched random group’s mean coverage and the indirect evidence’s enrichment.
Binding C-command
Minimal pair
A lot of patients who can sell some couch didn’t investigate themselves / * itself .
Indirect Evidence
Several of ESPN’s personalities also portray themselves .
Structural
the reflexive’s verb has a subject
relative clause inside the subject
Lexical
reflexive ( themselves )
Binding domain
Table 1: Examples of indirect evidence, with a minimal pair for each phenomenon (* marks the ungrammatical sentence). Each is among the 60,000 most influential non-direct sentences of all five reference models. / indicate the filter conditions it satisfies or misses. Structural conditions count toward structural coverage, while lexical (word-list) conditions do not.
Figure 7: Average-linkage clustering of the 15 phenomena by shared indirect evidence. Distance between two phenomena is one minus overlap. The horizontal axis (log scale) gives the average overlap where two branches join. Colors are added after clustering and mark branches whose phenomena all belong to one BLiMP category. The “x/5” numbers at joins show how many of the five reference models produce the same cluster when they are clustered separately.
Table 2: Training settings of the reference and validation models.
Phenomenon
Ours
Published
Diff.
Determiner–noun
94.1 ±
0.5
93.5 ±
0.8
+0.5
Determiner–adj.–noun
86.3 ±
0.4
86.2 ±
2.1
+0.1
Regular/irregular plural
92.6 ±
0.6
92.5 ±
0.3
+0.2
Relative-clause distractor
73.6 ±
2.5
72.9 ±
2.7
+0.7
Relational-noun distractor
90.1 ±
1.7
89.7 ±
1.3
+0.4
Passive
86.9 ±
0.7
86.7 ±
0.5
+0.2
Appendix
Table 3: Held-out BLiMP accuracy (%) of our reference models and Patil et al. (2024) ’s published full-corpus models (mean ± standard deviation over five models). Note that differences are computed between means before rounding occurs.
Phenomenon ( Patil et al., 2024 )
BLiMP paradigm
Example
Query
Held-out
Determiner–noun agreement
Determiner–noun
det_noun_agr_1
Craig explored that grocery store/*stores.
102
706
det-noun
det_noun_agr_2
Carl cures those/*that horses.
108
732
det_noun_agr_irr_1
Phillip was lifting this mouse/*mice.
84
608
det_noun_agr_irr_2
Those ladies walk through those/*that oases.
96
665
Determiner–adj.–noun
det_noun_agr_with_adj_1
Tracy praises those lucky guys/*guy.
104
725
Appendix
Table 4: The fifteen phenomena, grouped by BLiMP category, with the specific names used in Patil et al. (2024) , BLiMP paradigms, and example pairs taken exactly from Patil et al. (2024, Table 1) (* marks the ungrammatical sentence). Paradigm names are abbreviated: agr = agreement, adj = adjective, det = determiner, irr = irregular, sv = subject–verb.
Direct share of the top k (%)
Whole
Phenomenon
100
1k
3k
10k
20k
40k
60k
120k
corpus (%)
Determiner–noun
24.8
17.9
12.9
8.3
6.2
4.6
3.8
2.7
0.47
Determiner–adj.–noun
40.0
25.9
19.0
13.4
10.4
7.7
6.4
4.5
1.15
Regular/irregular plural
39.2
28.6
25.6
23.0
21.4
19.7
18.7
17.0
11.62
Relative-clause distractor
66.2
54.3
45.5
32.3
24.8
18.3
15.1
10.5
2.82
Relational-noun distractor
71.0
51.5
46.1
41.6
38.8
36.0
34.1
30.4
18.30
Appendix
Table 5: Share of direct evidence among each phenomenon’s k most influential sentences, as a mean over five reference models. Bold numbers indicate a majority. The last column is the share of all training sentences in the corpus that are marked as direct evidence. This last column figure is used as the baseline number for the enrichment calculation in Figure 4 .
Phenomenon
Indirect
BM25
Matched random
Regular/irregular plural
0.988
0.990
0.962
Relative-clause distractor
0
0
0
Relational-noun distractor
0
0
0
Reconstruction
0
0
0
Appendix
Table 6: Structural coverage for the four phenomena omitted from Figure 6 , as means computed over 5 reference models.
Phenomenon
Structural conditions (counted)
Lexical conditions (not counted)
Max.
Determiner–noun agreement
Determiner–noun
demonstrative determiner ( this, that, these, those )
its noun is on the BLiMP noun list (450 words)
1
Determiner–adj.–noun
demonstrative determiner ( this, that, these, those ); the demonstrative is directly followed by a non-noun (e.g., an adjective)
–
0.5
Subject–verb agreement
Regular/irregular plural
the sentence has a subject
a subject is on the BLiMP subject list (729 words, including names)
1
Relative-clause distractor
a relative clause modifies the subject or a noun modifying it
–
0
Appendix
Table 7: Conditions for each direct-evidence filter. A sentence is considered direct evidence by the filter if it meets all conditions in at least one branch (a, b). The structural coverage of a sentence is the fraction of a branch’s structural conditions it meets, taking the best branch. Lexical conditions (BLiMP and NPI word lists, and whether a word is a pronoun or a reflexive) do not count towards structural coverage. Max. is the highest coverage of any sentence outside the direct-evidence filter, and thus indicates an upper limit for the structural coverage metric. We reproduce the parse-based filters from Patil et al. (2024) and split them into conditions. The three NPI filters used are surface filters following Patil et al. (2024) , and their structural coverage is the presence of the licensing cue.
After removing
Phenomenon
Reference
Matched random
Indirect
BM25
Direct
Extra drop
Determiner–noun
94.1 ± 0.5
94.2 ± 0.5
85.6 ± 1.1
94.2 ± 0.7
–
8.6 ±
0.6
Determiner–adj.–noun
86.3 ± 0.4
86.0 ± 1.5
78.3 ± 1.5
85.6 ± 0.9
–
7.7 ±
1.6
Regular/irregular plural
92.6 ± 0.6
93.1 ± 0.5
89.9 ± 0.6
93.0 ± 0.5
92.3 ± 0.6
3.2 ±
0.9
Relative-clause distractor
73.6 ± 2.5
76.5 ± 1.4
64.2 ± 1.9
78.2 ± 2.4
69.1 ± 3.4
12.3 ±
2.3
Relational-noun distractor
90.1 ± 1.7
90.1 ± 2.1
80.4 ± 2.6
90.0 ± 1.5
90.0 ± 1.9
9.7 ±
3.2
Appendix
Table 8: Held-out BLiMP accuracy (%) of the reference models and of the models retrained without each group, as mean ± standard deviation over the five models. Direct evidence is a random 60,000 of its sentences, removed only for the five phenomena with at least that many. Extra drop is the accuracy lost by removing the indirect evidence relative to the matched random group: mean ± standard deviation of the paired difference over the five models.
Extra harm
Phenomenon
Removed
Loss
Acc. (pts)
Determiner–noun
20,000
+0.130
5.4
60,000
+0.217
8.6
Regular/irregular plural
20,000
+0.041
1.9
60,000
+0.102
3.2
Appendix
Table 9: A comparison of the effect of removing the most influential indirect evidence at two different sizes, as means over five models. The displayed extra drop for held-out loss and held-out accuracy are relative to the matched random control group of the same size.
Figure 8: Overlap between phenomena’s indirect evidence. Each cell in the heatmap is the share of one phenomenon’s 60,000 indirect evidence sentences that are also in the other’s indirect evidence, averaged over five reference models. Black outlines mark BLiMP categories.
Setting
Value
Curvature
EK-FAC, empirical Fisher
Tracked layers
Query, key, value and output projections, both feed-forward layers in the 8 blocks
Not tracked
Token and position embeddings, layer norms, output layer
Factor fitting
All 2,442,181 training sentences
Damping
0.1× the mean eigenvalue of each layer
Loss for training gradient gi
Summed next-token cross-entropy of training sentence i
Appendix
Table 10: EK-FAC settings, shared by all five reference models.
How do learners avoid overgeneralizations such as Tom laughed me without explicit negative evidence? Constructionists have posited two proposals that describe indirect negative evidence against overgeneralizations: preemption (which privileges exposure to near-synonymous construction---e.g., she made him laugh) vs. entrenchment (all exposures to a verb's grammatical usages, including cases like He laughed). We disentangle these hypotheses by running controlled rearing experiments on LMs trained on child-caregiver conversations, where we systematically remove preemptive vs. non-preemptive evidence. We find that while LMs avoid overgeneralizations, they do not show preemption at a verb-specific level, instead showing weak but non-zero evidence of abstract preemption. Combined with results from analyzing the LMs' training dynamics, we find that LMs treat competing structures as indirect positive---as opposed to negative---evidence in the verb-specific condition. Insofar as preemption is the more plausible route to avoiding overgeneralizations in humans, our results point the need for there to be sensitivities to indirect negative evidence in neural network learners, and suggest new human experiments to test abstract preemption.
Yixuan Wang, Freda Shi, Kanishka Misra
University of Waterloo Vector Institute · Department of Linguistics University of Texas at Austin
Many of the thousands of attested languages share common configurations of features, creating a spectrum from typologically very rare (e.g., object-verb-subject word order) or impossible languages to very common combinations of features (e.g., subject-object-verb word order). One central question is under what conditions such typological tendencies can be predicted, and specifically whether the learning bias of language models (LMs) is sufficient to reproduce such patterns. In this study, we add one dimensionality to such analysis -- the learning scenario for LMs -- to explore its interaction with the inductive bias of LMs. Specifically, as a first study, we examine the effect of curriculum learning (CL), as a developmentally motivated learning scenario, i.e., starting with simpler sentences rather than randomly-ordered input. We expand existing LM-based exploration (El-Naggar et al., 2025a,b) with a simple CL variant and find that CL substantially impacts the apparent inductive bias of LMs.
Grammaticality and likelihood are distinct notions in human language. Pretrained language models (LMs), which are probabilistic models of language fitted to maximize corpus likelihood, generate grammatically well-formed text and discriminate well between grammatical and ungrammatical sentences in tightly controlled minimal pairs. However, their string probabilities do not sharply discriminate between grammatical and ungrammatical sentences overall. But do LMs implicitly acquire a grammaticality distinction distinct from string probability? We explore this question through studying internal representations of LMs, by training a linear probe on a dataset of grammatical and (synthetic) ungrammatical sentences obtained by applying perturbations to a naturalistic text corpus. We find that this simple grammaticality probe generalizes to human-curated grammaticality judgment benchmarks and outperforms LM probability-based grammaticality judgments. When applied to semantic plausibility benchmarks, in which both members of a minimal pair are grammatical and differ in only plausibility, the probe however performs worse than string probability. The English-trained probe also exhibits nontrivial cross-lingual generalization, outperforming string probabilities on grammaticality benchmarks in numerous other languages. Additionally, probe scores correlate only weakly with string probabilities. These results collectively suggest that LMs acquire to some extent an implicit grammaticality distinction within their hidden layers.