Lexical representations can encode both abstract information and information specific to particular contexts. We investigate how these forms of information are organized in language models by decomposing contextual word representations into item, class, and word-class interaction components. Across six pretrained transformers and two linguistic categories, we find that the interaction is consistently present, grows across layers, and contains information associated with both item and class. Unlike the nearly orthogonal item and class effects, the interaction overlaps with both and encodes class information beyond the shared class effect. An interaction also emerges in artificial-language learners even when the input contains only additive structure. Together, these findings show that lexical representations combine abstract information with structure specific to particular word-class combinations.
Figures & tables
Figure 1: Structure of our model. Every form is observed 24 times at each class, each time in a different context; that variation across contexts is what the measurement later splits in two.
Figure 2: The four item-by-class means formed by two items in two classes. The arrow is γ(i,c) .
Figure 3: Ground-truth effect size of each component in the language (x) against its size in the MLP’s representations (y). Points are runs; lines are fitted values from the size model (Equation 3 ) with 95% intervals, colored by hidden width.
Figure 4: Fraction of the representation (ignoring context) accounted for by each component, summing to 1, against relative depth from the embedding layer (0) to the final layer (1). Each line is one model; color is the training corpus, linetype the hidden width, shading the 95% interval.
model
accuracy
null
part of speech
BabyLM 1.3B
0.90 [0.88, 0.91]
0.51 [0.49, 0.54]
Pythia 1.4B
0.94 [0.92, 0.95]
0.50 [0.48, 0.53]
grammatical role
BabyLM 1.3B
0.80 [0.77, 0.83]
0.50 [0.47, 0.53]
Pythia 1.4B
0.84 [0.81, 0.86]
0.52 [0.49, 0.55]
Table 1: Accuracy of recovering the class from the interaction, at the deepest layer of the largest model in each family, compared to the null; brackets are bootstrapped intervals.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
factor
levels
language
item weight
0, 0.5, 1, 2
class weight
0, 0.5, 1, 2
interaction weight
0, 0.5, 1, 2
context weight
0.5, 2, 6
interaction rank
4, 16, 64, 177
Appendix
Table 2: The Experiment 1 language parameters. The first five rows define each artificial language, the next two the learner MLP model. Item and class ranks are held fixed so that the interaction’s rank is the only one that varies. Crossing every level gives 38,400 cells; the three languages in which all three effect weights are zero contain no item-by-class structure at all and are excluded, leaving the reported total.
Figure 5: Fit and recovery against hidden width, for languages whose planted structure is more than 30% interaction. Left: Kullback-Leibler divergence from the learner to the ground-truth distribution, so zero is a perfect fit. Right: the interaction share measured in the hidden layer as a fraction of the share planted in the language, so one would be full recovery and the plotted values are the fraction of it reached. Lines are medians over runs with interquartile bands, which show how much the languages in the grid differ from one another rather than measurement error.
Figure 6: Each word in each class is observed in many contexts, and those observations are averaged to give that word’s representation in that class. Every one of these averages is a vector in Rd , and every item appears in every class.
Figure 7: The three components across the grid of words and classes. An item effect is one value per word, the same in every class; a class effect is one value per class, the same for every word; and the interaction differs for every word in every class.
specified
runs
measured
flagged
0.000
6300
0.0000 [0.0000, 0.0094]
5%
0.030
900
0.0323 [0.0199, 0.0418]
100%
0.048
1800
0.0495 [0.0385, 0.0688]
100%
0.056
1800
0.0573 [0.0458, 0.0918]
100%
0.059
1800
0.0607 [0.0495, 0.1041]
100%
0.100
1800
0.1007 [0.0911, 0.1089]
100%
Appendix
Table 3: Recovery of the interaction at 24 observations per item and class in 768 or more dimensions, pooled over noise levels, item counts, and widths. Specified is the interaction share built into the generated representations and measured the median share recovered, with the 2.5th and 97.5th percentiles across runs in brackets; flagged is the share of runs whose re-split interval excludes zero.
effect
observations
median
95th pct
flagged
class effect
10
0.00000
0.0081
6.1%
class effect
24
0.00000
0.0034
6.5%
class effect
100
0.00000
0.0009
6.3%
interaction
10
0.00009
0.0644
6.2%
interaction
24
0.00004
0.0309
5.9%
interaction
100
0.00002
0.0083
4.9%
Appendix
Table 4: What the measure reports when an effect is set to exactly zero, by observations per item and class, pooled over every other axis of the validation grid. Median and 95th pct summarize the measured share of that effect across runs, and flagged is the share of runs whose re-split interval excludes zero, which should sit near the nominal 5%.
width d
item
class
interaction
8
3%
14%
7%
16
7%
3%
5%
32
4%
4%
10%
128
6%
4%
5%
768
2%
8%
3%
1024
3%
5%
3%
Appendix
Table 5: How often a component set to exactly zero is flagged, by width and by effect.
width d
measured
analytic chance
8
0.17501 [0.06900, 0.40018]
0.25000
16
0.10678 [0.03744, 0.20884]
0.12500
32
0.05609 [0.01864, 0.10454]
0.06250
128
0.01714 [0.00450, 0.02611]
0.02344
768
0.00261 [0.00079, 0.00472]
0.00391
1024
0.00217 [0.00065, 0.00330]
0.00293
Appendix
Table 6: Item-class overlap when the two effects are constructed orthogonal, at 24 observations per item and class. Measured is the median overlap across the runs at that width with the 2.5th and 97.5th percentiles in brackets; chance is the overlap expected if the component were randomly oriented, r/d for a target subspace of r dimensions in a representation of width d .
width d
measured
chance
detected
8
0.1688 [0.0824, 0.3998]
0.12500
9%
16
0.0951 [0.0414, 0.2026]
0.06250
22%
32
0.0593 [0.0225, 0.1119]
0.03125
36%
128
0.0213 [0.0062, 0.0517]
0.00781
63%
768
0.0065 [0.0011, 0.0298]
0.00260
83%
1024
0.0056 [0.0008, 0.0331]
0.00098
82%
Appendix
Table 7: Measured item-class overlap when a fraction of the class direction is placed inside the item subspace (non-orthogonal), at 24 observations per item and class. Measured is the median overlap across the runs at that width with the 2.5th and 97.5th percentiles in brackets; chance is the overlap expected if the component were randomly oriented, r/d for a target subspace of r dimensions in a representation of width d ; detected is the fraction of runs whose overlap falls outside that null’s 95% interval.
specified
noise
runs
measured
flagged
1x
6300
0.0000
6%
10x
6300
0.0000
6%
0.00
50x
6300
0.0001
6%
1x
1800
0.1012
100%
10x
1800
0.1002
100%
0.10
50x
1800
0.1001
100%
Appendix
Table 8: Recovery of the interaction across simulations with different amounts of noise, in 768 or more dimensions, pooled over item counts, widths and sample sizes. Specified is the interaction share built into the representations and noise the variance across contexts per observation as a multiple of the variance between words and classes, so at 1x the two are equal. Measured is the median interaction share recovered across runs and flagged the share whose re-split interval excludes zero, which should sit at 5% in the rows where nothing was planted.
property
level
runs
item
class
interaction
under 30%
3600
0.490
0.399
0.028
30 to 70%
2871
0.538
0.371
0.008
context in the language
over 70%
3886
0.578
0.350
0.004
item < class
3525
0.428
0.480
0.017
item = class
3427
0.569
0.359
0.015
item against class
item > class
3405
0.644
0.294
0.011
Appendix
Table 9: How two other properties of the artificial language move the composition of the item-by-class structure, under ReLU, pooled over widths. Rows within a property partition the same runs. Values are medians of the three shares over the runs in the row.
component
term
estimate
error
95% CI
P(b > 0)
item
item
1.68
0.04
[1.61, 1.75]
1.00
class
item
-1.63
0.04
[-1.70, -1.56]
0.00
interaction
item
0.09
0.04
[0.02, 0.16]
0.99
item
class
-2.67
0.04
[-2.75, -2.59]
0.00
class
class
2.77
0.04
[2.69, 2.85]
1.00
interaction
class
0.24
0.04
[0.16, 0.31]
1.00
Appendix
Table 10: Coefficient values of the size model (Equation 3 ) with error and 95% CIs, ReLU arm.
component
term
estimate
error
95% CI
P(b > 0)
item
item
2.36
0.04
[2.29, 2.43]
1.00
class
item
-2.29
0.03
[-2.35, -2.22]
0.00
interaction
item
0.48
0.04
[0.40, 0.55]
1.00
item
class
-3.52
0.04
[-3.60, -3.45]
0.00
class
class
3.66
0.04
[3.58, 3.74]
1.00
interaction
class
0.08
0.04
[-0.00, 0.16]
0.97
Appendix
Table 11: Coefficient values of the size model (Equation 3 ) with error and 95% CIs, linear arm.
Figure 8: Coefficient estimates from the size model (Equation 3 ), one panel per measured component. The upper block holds the planted effect sizes and the lower the architecture terms, which are on a much smaller scale and so are drawn on their own axis. Points are posterior medians with 95% intervals.
arm
measure
term
estimate
error
95% CI
P(b > 0)
item_class
item
-0.51
0.04
[-0.59, -0.42]
0.00
int_margins
item
-0.96
0.04
[-1.05, -0.88]
0.00
item_class
class
-1.62
0.05
[-1.71, -1.53]
0.00
int_margins
class
-0.62
0.05
[-0.71, -0.53]
0.00
item_class
interaction
0.31
0.04
[0.23, 0.39]
1.00
int_margins
interaction
-0.57
0.04
[-0.65, -0.49]
0.00
Appendix
Table 12: Every coefficient of the overlap model (Equation 3 ), with Estimate, Estimated error, and 95% CIs.
Figure 9: Model results for the size model (Equation 3 ). Columns are the three measured components, rows are the planted interaction rank, and lines are hidden widths, with 95% intervals. Holding rank fixed within a row is what separates the width effect from the width-by-rank term; the near-identical rows show how little the rank changes it.
Figure 10: Two overlap measurements as a function of width of the hidden layer. Left: the item effect projected onto the class direction. Right: the interaction projected onto the span of both margins. The dashed line is chance.
Figure 11: Fraction of the representation by each component (such that they sum to 1), against hidden width, with one line per rank of the interaction planted in the language and 95% intervals.
class
item types
median tokens
part of speech (noun / verb)
78
18 / 18
grammatical role (subject / object)
49
15 / 19
Appendix
Table 13: Stimuli for the two linguistic categories, with the two classes of each in parentheses. Item types is how many word forms occur in both of them at least ten times; forms occurring in only one are excluded. Median tokens is how often a typical form occurs in each of its two classes, in the order they are listed.
family
model
params
width
layers
training data
tokens
opt-babylm-125m
125M
768
12
BabyLM (child-available)
~3B
opt-babylm-350m
350M
1024
24
BabyLM (child-available)
~3B
BabyLM
opt-babylm-1.3B
1.3B
2048
24
BabyLM (child-available)
~3B
pythia-160m
160M
768
12
the Pile
~300B
pythia-410m
410M
1024
24
the Pile
~300B
Pythia
pythia-1.4b
1.4B
2048
24
the Pile
~300B
Appendix
Table 14: The six models, paired by width. Training-token counts are approximate; the OPT-BabyLM models train for 20 epochs over a 100M-word corpus, which is roughly 150M tokens, so their effective token exposure is about twenty times that.
interaction
class effect
item-class overlap
model
std
raw
std
raw
std
raw
BabyLM 125M
0.095
0.095
0.072
0.074
0.0052
0.0053
BabyLM 350M
0.122
0.122
0.066
0.068
0.0056
0.0061
BabyLM 1.3B
0.104
0.104
0.050
0.051
0.0035
0.0036
Pythia 160M
0.231
0.228
0.078
0.089
0.0058
0.0081
Pythia 410M
0.244
0.242
0.098
0.123
0.0067
0.0195
Appendix
Table 15: Part of speech at the deepest layer, measured with and without the per-dimension rescaling. Sizes are shares of the item-by-class structure; overlaps are the raw fractions, not multiples of chance.
class
model
item-class overlap
its null
interaction overlap
its null
BabyLM 125M
0.0052 (4.2×)
0.0012
0.178 (3.5×)
0.051
BabyLM 350M
0.0056 (5.9×)
0.0010
0.166 (4.3×)
0.039
BabyLM 1.3B
0.0035 (7.6×)
0.0005
0.106 (4.8×)
0.022
Pythia 160M
0.0058 (4.6×)
0.0013
0.231 (4.2×)
0.055
Pythia 410M
0.0067 (6.9×)
0.0010
0.239 (6.1×)
0.039
part of speech
Pythia 1.4B
0.0055 (11.3×)
0.0005
0.209 (9.7×)
0.021
Appendix
Table 16: Overlap measurement and the null values at the final layer of each model. Each overlap is the fraction of one component lying inside another’s subspace: the item effect inside the class subspace, the interaction inside the span of the two main effects. Each is followed by the overlap divided by its own null, so 4.2x means the overlap is 4.2 times what random orientation gives.
item effect
class effect
interaction
class
model
hidden
output
hidden
output
hidden
output
BabyLM 125M
0.832
0.751
0.069
0.142
0.099
0.107
BabyLM 350M
0.810
0.721
0.064
0.146
0.126
0.134
BabyLM 1.3B
0.843
0.757
0.049
0.123
0.108
0.120
Pythia 160M
0.688
0.577
0.077
0.143
0.235
0.280
Pythia 410M
0.656
0.619
0.095
0.121
0.248
0.260
Appendix
Table 17: Each component’s share of the item-by-class structure at the final hidden layer and after the unembedding, measured on identical tokens. Both members of a pair are point estimates from one run, which is what makes them comparable to each other; they therefore sit about 0.003 from the re-split medians quoted elsewhere.
class
model
word
class effect
interaction
BabyLM 125M
36.3
3.1
4.2
BabyLM 350M
31.2
2.5
4.7
BabyLM 1.3B
25.8
1.5
3.2
Pythia 160M
22.4
2.5
7.5
Pythia 410M
20.3
3.0
7.6
part of speech
Pythia 1.4B
18.8
2.4
6.8
Appendix
Table 18: How much of a token’s hidden state each component accounts for, at the deepest layer, as the percentage reduction in squared error against guessing the grand mean.
model
word-built
context
null
POS (noun/verb)
BabyLM 125M
0.85 [0.82, 0.88]
0.15
-0.00
Pythia 160M
0.86 [0.83, 0.87]
0.14
-0.00
BabyLM 350M
0.89 [0.86, 0.91]
0.11
-0.00
Pythia 410M
0.81 [0.77, 0.83]
0.19
-0.00
BabyLM 1.3B
0.89 [0.87, 0.91]
0.11
-0.00
Appendix
Table 19: How much of the deepest-layer interaction is attributable to the left context, by model and category. Word-built is the share requiring word-dependent processing, with its bootstrap interval; context is the share an item-independent carry-over of the context predicts; null mismatches items before the comparison.
Neural language models are typically trained on next-token prediction, although linguistic structure spans multiple temporal scales. Successor representations (SRs) make this horizon explicit by encoding discounted distributions over future states. Here, we ask whether such predictive representations can recover not only word classes, but also finer functional and construction-like structure from natural language. A residual network trained on WikiText-103 predicts SR distributions at three horizons without part-of-speech supervision. At the shortest horizon, unsupervised clustering robustly recovers nouns, verbs, and adjectives, while directed inter-cluster transitions reproduce familiar syntactic asymmetries. At finer resolutions and across 13 part-of-speech categories, the same geometry reveals semantic-functional groupings that cross category boundaries and directed relations tracing candidate date, measurement, and title-name constructions. Part-of-speech agreement declines as the predictive horizon lengthens. These results suggest that word classes are coarse regions within a richer predictive geometry in which categorical and construction-like linguistic structure emerge from future-word distributions.
Mathis Immertreu, Achim Schilling, Thomas Kinfe +1
Cognitive Computational Neuroscience Group, Friedrich-Alexander-Universität Erlangen–Nürnberg (FAU), Germany. · Mannheim Center for Neuromodulation and Neuroprosthetics (MCNN), University Hospital Mannheim, University Heidelberg, Germany.
Whether idiosyncratic, item-specific knowledge is learned before abstract class-level generalizations, or vice versa, is a central question in language learning, with exemplar and abstraction-based theories making opposite predictions. Recent methods have claimed to show that, at least for large language models, abstract knowledge is learned first. We show that these methods fall short: pure memorizer models with no abstract representations can appear, by the same criteria, to learn either item-specific or class-level knowledge first, depending on their sensitivity to individual observations, with the transition point governed by the distributional properties of the input. We further argue that the distinction between item-specific and abstract knowledge may be ill-defined for distributed representations, as a word's class-level properties may not be separable from its item-specific properties.
Representations extracted from large language models (LLMs) play an important role in many downstream applications. However, the structure of these representations is often influenced by lexical overlap rather than semantic content. Our understanding of the relationship between this lexical influence and semantic content, and its implications for downstream tasks, remains limited. In this work, we investigate representations to quantify the effect of lexical overlap relative to semantic content. We consider several adversarial semantic stress tests and further connect our findings to the information theory perspective. We find that lexical influence extends across the depth of models, consistently across architectures, training regimes, and objective functions, including the models trained for semantic similarity. Moreover, we observe a mid-depth region in which both lexical and semantic signals degrade simultaneously, indicating a transitional regime where representations are poor for both surface form and meaning. We further demonstrate the effect of lexical influence on downstream uses of LLMs using summarization and model editing as a case study.
Hammad Rizwan, Muhammad Umair Haider, Nishant Subramani +3
Dalhousie University Halifax, NS · University of Kentucky Lexington, KY · Carnegie Mellon University Pittsburgh, PA