Language models suffer from a curse of ambiguity

Nicolas Zucchet; Hyun Dong Lee; Scott Linderman

Figure 6: The Hebbian model captures the same dependency on ambiguity as learned linear heads. The left and middle panels show the number of contexts whose top-

k

predictions exactly match the ground-truth support, as a function of the dataset size

n

, for a linear head trained with Adam (left) and the closed-form Hebbian construction of Section 3.1 (middle). The right panel shows the corresponding capacity

n^{\ast}

as a function of the support size

k

, where

n^{\ast}

is the largest

n

for which the median top-

k

accuracy stays above

0.99

. Both models exhibit similar dependencies on ambiguity and only approximately differ by a constant factor.

Figure 7: The capacity of a trained linear head grows proportionally to the number of parameters

hd

, with the

1/k

offset predicted by Theorem 1 . Same capacity definition and learning-rate selection as Figure 6 , up to the reductions described in the experimental setup below, with the capacity

n^{\ast}

reported as a function of the number of parameters

hd

for

k\in\{1,4,16\}

, sweeping

h=d\in\{32,64,128,256\}

. The gray dotted line indicates a slope of

1

, that is, a capacity directly proportional to

hd

.

Architecture	linear head, $h=d=128$ ( $hd=16{,}384$ parameters)
Loss	exact KL divergence to the target distributions
Optimizer	Adam, batch size $2{,}048$ , constant learning rate, $150{,}000$ steps
Support size $k$	$\{1,\,2,\,4,\,8,\,16,\,32\}$
Number of contexts $n$	$21$ values, log-spaced between $32$ and $32{,}768$
Learning rate	$\{10^{-3},\,3{\cdot}10^{-3},\,10^{-2},\,3{\cdot}10^{-2},\,10^{-1},\,3{\cdot}10^{-1},\,1\}$
Seeds	$5$

Appendix

Table 1: Sweep configuration for the capacity validation experiment of Figure 6 .

\{\cdots\}

indicate the values that we sweep over.

Figure 8: The tetrahedron is

3

-neighborly: every subset of its four vertices is a face. The cube is only

1

-neighborly: antipodal vertices are not connected by an edge, so not every pair of vertices forms a face.

Figure 9: Gradient-flow theory qualitatively captures learning below the capacity threshold, even with mixed ambiguity within the data. Each panel shows the mean KL divergence on the size-

k

subset of contexts, against normalized step

\mathrm{step}/n

, for one value of the number of contexts

n

. Below the capacity threshold (top row,

n\in\{32,64,128\}

), the curves stack in increasing order of

k

and decay at similar rate, mirroring the theoretical prediction that more ambiguous distributions are harder to learn but share the same asymptotic slope; see Figure 2 . As we approach and cross the capacity threshold (bottom row,

n\in\{256,512,1024\}

), the dynamics flatten and the simplified theory no longer applies: ambiguous distributions plateau at high loss while less ambiguous ones are still learned, consistent with the capacity-based picture of Appendix A .

Figure 10: Local log-log slope

\Delta\log\mathrm{loss}/\Delta\log t

for each panel of Figure 9 . Below the capacity threshold, all

k

converge to slopes close to the dashed reference at

-1

, matching the gradient-flow prediction

D_{\mathrm{KL}}\sim k(d-k)/(td)\approx k/t

for small

k

values.

Architecture	linear head, $h=d=256$
	( $hd=65{,}536$ parameters)
Optimizer	stochastic gradient descent, batch size $1{,}024$ ,
	constant learning rate $1$ , $6.4{\cdot}10^{6}$ steps
Support size $k$	$\{1,\,2,\,4,\,8,\,16,\,32\}$ , mixed in equal proportion
Contexts per ambiguity level $n$	$\{32,\,64,\,128,\,256,\,512,\,1024\}$
Seeds	$5$

Appendix

Table 2: Sweep configuration for the gradient-flow validation experiment of Figures 9 and 10 . Each ambiguity level forms a group of

n

contexts, for a total of

6n

context-distribution pairs per run.

Figure 11: The empirical sampling gap grows with

k

asymptotically, in line with the theoretical prediction of Section 4.2 . Top row: evolution of the mean per-

k

KL divergence under stochastic gradient descent on the cross-entropy loss (with sampled targets), against the training step normalized by the number of contexts per ambiguity level

n

, for

n\in\{32,256\}

. Bottom row: gap to the matched distributional-target reference (full-batch gradient descent on the same task);

k=1

is omitted as its gap is below the seed-noise floor. The late-time gap increases with

k

, matching the asymptote of Equation 4.2 .

Figure 12: Same as Figure 3 , but using the raw support size of the ground-truth distribution as the measure of ambiguity instead of the effective support size.

Figure 13: Visual representation of how we summarize the per-context loss landscape of Figure 3 (left) into the iso-loss line we use in Figure 4 .

	Default	Capacity	Embedding	Optimization	Sampling
Architecture	$4$ -layer transformer, $4$ attention heads
Vocabulary	$2{,}000$ most frequent words of TinyStories
Data	$256$ -token sequences, batch size $128$ , one seed
Optimizer	AdamW, cosine schedule, weight decay $0.01$
Learning rate	$\{10^{-4},\,3\times 10^{-4},$ $10^{-3},\,3\times 10^{-3}\}$	$\{3\times 10^{-4},\,10^{-3},\,3\times 10^{-3}\}$
$d_{\mathrm{model}}$	$256$	$\mathbf{\{32,\,64,}$ $\mathbf{128,\,256,}$ $\mathbf{512,\,1024\}}$	$256$	$256$	$256$

Appendix

Table 3: Configuration of ablation experiments on the synthetic

3

-gram task. Each column corresponds to a sweep that varies a single parameter (highlighted in bold) while holding all others at the default value. Learning rate is tuned for every configuration in every sweep: the number of runs is the number of swept values times the size of the learning-rate grid, and the reported curves use the best learning rate (

3\times 10^{-3}

for the default configuration).

Figure 14: Per-context losses across the capacity ablation. Each row corresponds to one value of

d_{\mathrm{model}}\in\{32,\,64,\,128,\,256,\,512,\,1024\}

(annotated as parameter count in millions). All other parameters are held at the defaults of Table 3 : embedding dimension

256

,

3000

training steps, sampled cross-entropy loss, with learning rate selected per configuration as the best of

\{3\times 10^{-4},\,10^{-3},\,3\times 10^{-3}\}

.

Figure 15: Per-context losses across the embedding-dimension ablation. Each row corresponds to one value of

d_{\mathrm{embed}}\in\{32,\,64,\,128,\,256,\,512,\,1024\}

. All other parameters are held at the defaults of Table 3 :

d_{\mathrm{model}}=256

,

3000

training steps, sampled cross-entropy loss, with learning rate selected per configuration as the best of

\{3\times 10^{-4},\,10^{-3},\,3\times 10^{-3}\}

.

Figure 16: Per-context losses across the training-duration ablation. Each row corresponds to one number of training steps in

\{356,\,725,\,1500,\,3000,\,6000,\,12000\}

. All other parameters are held at the defaults of Table 3 :

d_{\mathrm{model}}=d_{\mathrm{embed}}=256

, sampled cross-entropy loss, with learning rate selected per configuration as the best of

\{3\times 10^{-4},\,10^{-3},\,3\times 10^{-3}\}

.

Figure 17: Per-context losses across the loss ablation. The two rows compare optimizing the full KL divergence

D_{\mathrm{KL}}(p_{x}\,\|\,\hat{p}_{x})

(top) against the sampled cross-entropy with one label drawn per step (bottom). All other parameters are held at the defaults of Table 3 :

d_{\mathrm{model}}=d_{\mathrm{embed}}=256

,

3000

training steps, with learning rate selected per configuration as the best of

\{3\times 10^{-4},\,10^{-3},\,3\times 10^{-3}\}

.

Figure 18: Effect of changing the kernel bandwidth

\lambda

on the results of Figure 5 . Same setup as Figure 5 , varying the bandwidth

\lambda

of the exponential kernel

\exp(-d_{i}/\lambda)

used to compute

n_{\mathrm{est}}

from kNN distances. Larger

\lambda

shifts the x-axis to the right (more neighbors are weighted in), but the cloud shape and the per-context KL coloring are essentially invariant under

\lambda

.

Figure 19: Joint distribution of semantic context frequency and estimated effective support size, alongside the average position in the sequence at which each context occurs. Both panels share axes with Figure 5 . Left: each hex is colored by the geometric-mean position (in tokens) of the contexts that fall in it, with opacity modulated by the log of the per-hex count so sparse cells fade out. Low-frequency, low-effective-support cells are dominated by early-sequence positions, where local context is short and the kNN density estimate is least reliable. Right: log-density of contexts in the same plane. Most of the mass concentrates in a narrow band where

n_{\mathrm{est}}

and effective support size are positively correlated.

Figure 20: The kNN density proxy correlates with the true context frequency on the

3

-gram task. Log-density (hexbin,

10^{6}

positions subsampled uniformly from the held-out evaluation set) of the true frequency of each context in the training corpus (x-axis) against its kernel density

\mathrm{e}^{-d_{i}/\lambda}

averaged over its retrieved nearest neighbors in the last-layer embedding space of the trained

4

-layer transformer (y-axis), at

\lambda=2

. On this task we plot the average rather than the sum of Appendix G.1 , which is why the y-axis lies below one; the two differ by the constant number of retrieved neighbors, so both correlations below are identical for either choice. The two quantities are strongly rank-correlated across positions (Spearman correlation of

0.92

), supporting the use of

n_{\mathrm{est}}

as a context-frequency proxy on natural data, where the true count is not observable.

Figure 21: The curse of ambiguity persists when ambiguity is measured by raw support size instead of effective support size. Same setup as Figure 5 , except that the y-axis now shows the raw support size of the OLMo-2 13B reference distribution. The qualitative picture is unchanged.

Figure 22: Effect of varying the size of the evaluated model on the results of Figure 5 . Same setup as Figure 5 , with OLMo-2 1B (left) and OLMo-2 7B (middle) as the model evaluated against the OLMo-2 13B reference. The 7B model tracks the reference more closely (KL is lower across the cloud), mirroring the improvement observed when increasing model size in the synthetic

3

-gram task; cf. Figures 4 and 14 .

population (1994 est.) Net migration rate: -12.05 migrant(s)/ 1, 000 population (1994 est.) Infant mortality rate: 18.5 deaths/1 ,

RESSIDA. I will not, uncle. I have forgot my father; I know no touch of consanguinity, No kin, no love, no blood, no soul so near me As the sweet Troil us. O you gods divine, Make Cressid’s name the very crown of falsehood, If ever she leave Tro il

from the men of Shechem, and from the house of Millo, and devour Abimele ch. 9:21 And Jotham ran away, and fled, and went to Beer, and dwelt there, for fear of Abimelech his brother. 9:22 When Abime le

:014:017 And I, behold, I will harden the hearts of the Egyptians, and they shall follow them: and I will get me honour upon Ph araoh, and upon all his host, upon his chariots, and upon his horsemen. 02:014:018 And the Egyptians shall know that I am the LORD, when I have gotten me honour upon Ph

/1, 000 population (1994 est.) Net migration rate: -1.78 migrant(s)/1,000 population (1994 est.) Infant mortality rate: 8.9 deaths/1 ,

God, that I may be at once avenged of the Philistines for my two eyes. 16:29 And Sam son took hold of the two middle pillars upon which the house stood, and on which it was borne up, of the one with his right hand, and of the other with his left. 16:30 And Sam

the tone of fairy tales, and it is certainly not lawlessness or even liberty, though men under a mean modern tyranny may think it liberty by comparison. People out of Portland Gaol might think Fleet Street free; but closer study will prove that both fairies and journalists are the slaves of duty. Fairy godmo thers seem at least as strict as other god mo

Appendix

Table 4: Examples of low-loss tokens from the band highlighted in Figure 5 (

n_{\mathrm{est}}<10

). The token to predict is shown in italics and the matching

n

-gram earlier in the same sequence is shown in dark red (a trigram when a trigram match exists, a bigram otherwise).

Figure 23: Per-context loss and effective support size are correlated at fixed semantic context frequency. Each panel slices the data of Figure 5 by a log-spaced bin of semantic context frequency, indicated in brackets in the title, and shows the 2D log-density of the KL divergence between the predictions of the 13B reference model and the 1B probe against the estimated effective support size; the value next to each title is the squared Pearson correlation

R^{2}

between

\log D_{\mathrm{KL}}

and

\log

effective support size within the bin. The correlation is strongest for both small (rare contexts,

n_{\mathrm{est}}\lesssim 10

) and large (common contexts,

n_{\mathrm{est}}\gtrsim 100

) semantic context frequencies. It drops for intermediate values of

n_{\mathrm{est}}

, which is partially explained by the high overall loss in that regime: when most contexts already sit near the maximum KL, effective support size has little remaining variance to predict.

Figure 24: Per-context loss vs. raw support size at fixed semantic context frequency. Same setup as Figure 23 , with raw support size in place of effective support size on the x-axis.

R^{2}

is higher than in the effective-support-size version, partly because the relationship is more linear in this plane.

Language models suffer from a curse of ambiguity

Abstract

Figures & tables

Appendix

Explore similar work

Probabilistic Calibration Is a Trainable Capability in Language Models

The Probabilistic Structure of Large Language Models

Tokens, the oft-overlooked appetizer: Large language models, the distributional hypothesis, and meaning