Expect the Unexpected? Testing the Surprisal of Salient Entities

Jessica Lin; Amir Zeldes

cs.CLApr 12, 2026

Expect the Unexpected? Testing the Surprisal of Salient Entities

Authors: Jessica Lin, Amir Zeldes

Organizations: Department of Linguistics, Georgetown University

Abstract

Previous work examining the Uniform Information Density (UID) hypothesis has shown that while information as measured by surprisal metrics is distributed more or less evenly across documents overall, local discrepancies can arise due to functional pressures corresponding to syntactic and discourse structural constraints. However, work thus far has largely disregarded the relative salience of discourse participants. We fill this gap by studying how overall salience of entities in discourse relates to surprisal using 70K manually annotated mentions across 16 genres of English and a novel minimal-pair prompting method. Our results show that globally salient entities exhibit significantly higher surprisal than non-salient ones, even controlling for position, length, and nesting confounds. Moreover, salient entities systematically reduce surprisal for surrounding content when used as prompts, enhancing document-level predictability. This effect varies by genre, appearing strongest in topic-coherent texts and weakest in conversational contexts. Our findings refine the UID competing pressures framework by identifying global entity salience as a mechanism shaping information distribution in discourse.

Figures & tables

Genre	Docs	Tokens	Modality	Mentions	Entities	% Top1	% Top2	% Top3	% Salient
Academic writing	18	17,169	Written	5,055	3,001	1.90	3.35	5.00	15.65
Biographies	20	18,213	Written	5,772	3,052	2.32	3.71	5.57	16.02
Vlog	15	16,864	Spoken	4,499	1,256	1.63	3.44	5.61	14.83
Conversations	15	17,932	Spoken	4,531	1,166	1.12	2.49	4.64	16.24
Courtroom transcripts	9	11,148	Spoken	2,938	1,053	1.80	3.46	5.82	14.68
Essays	9	10,842	Written	3,061	1,672	0.93	1.74	2.89	10.88

Table 1: Overview of GUM-SAGE genres with document counts, token counts, modality, mention and entity counts, and proportion of salient entities by salience level (Top1: score=5, Top2: score

\geqslant

4, Top3: score

\geqslant

3). Percentages are calculated over all entities per genre.

Figure 1: Change in mean surprisal for salience scores 1–5 relative to score 0 in Experiment 1.

Predictor	Coef.	95% CI	$\Delta$ AIC
Intercept	0.8205	[0.764, 0.877]	–
Position	0.0002	[0.000, 0.000]	2.68
Salience score	-0.0471	[-0.070, -0.025]	14.86
Mention length	-0.0535	[-0.059, -0.048]	352.11
Is Nesting	-0.8493	[-0.903, -0.796]	886.27

Table 2: Results from an OLS regression model Surprisal ~ Salience score + Mention length + Is Nested + Position , with predictors ordered by

\Delta

AIC from single-term deletions (least to most informative): larger

\Delta

AIC indicates a larger degradation in model fit when that predictor is removed. Significance codes:

{}^{*}p<0.05

{}^{**}p<0.01

{}^{***}p<0.001

Figure 2: Decrease in mean surprisal relative to salience score 0 in Experiment 2. Magnitude of surprisal reduction largely grows with salience, indicating that more salient entities facilitate sentence-level predictability.

Figure 3: Mean surprisal scores across genres for Experiment 2 (salience score>0 in orange, score=0 in green).

Figure 4: Decrease in mean surprisal relative to salience score 0 in Experiment 3.

Figure 5: Mean surprisal scores across genres for Experiment 3 (salience > 0 in orange, salience=0 in green).

Figure 6: Mean surprisal (z-score) for salient vs. non-salient entities across genres. Bars show mean surprisal for salient (orange) and non-salient (blue) entities, with genres sorted by surprisal difference. Error bars represent 95% confidence intervals. Asterisks indicate significant differences based on adjusted t-tests (* $p$ < .05, ** $p$ < .01, *** $p$ < .001). Genres shaded gray show no significant difference between salient and non-salient entities.

Panel of Figure 7 — Figure 7: Surprisal contour in different genres. Blue lines represent the surprisal contour of the sentence followed by salient entities; Red lines represent the surprisal contour of the sentence followed by non salient entities. Each stimulus is distinguished by different markers.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Change in mean surprisal for salience scores 1–5 relative to score 0 in Experiment 1 using GPT2-Small .

Figure 9: Change in mean surprisal for salience scores 1–5 relative to score 0 in Experiment 2 using GPT2-Small .

Figure 10: Change in mean surprisal for salience scores 1–5 relative to score 0 in Experiment 3 using GPT2-Small .

Surprisal Theory

See all 16 papers

On the Proper Treatment of Units in Surprisal Theory
Apr 30, 2026Samuel Kiegeland, Vésteinn Snæbjarnarson, Tim Vieira +1Surprisal Theory Computational Psycholinguistics
ETH Zürich · CHI-FRO · University of Copenhagen
Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?
Feb 16, 2026Matteo Gay, Coleman Haley, Mario Giulianelli +1Computational Psycholinguistics
KU Leuven · University of Edinburgh · University College London
From Exposure to Expectation: Frequency, Surprisal, and Language Across Development in Spanish
Aug 23, 2026Francisco Portillo LópezSurprisal Theory Child Language Acquisition
Universidad de Navarra, Pamplona, Spain

Apr 30, 2026J/K move · Enter open · S save

On the Proper Treatment of Units in Surprisal Theory

Samuel Kiegeland, Vésteinn Snæbjarnarson, Tim Vieira, Ryan Cotterell

ETH Zürich · CHI-FRO · University of Copenhagen

Surprisal Theory Computational Psycholinguistics

Open paper AskPDF arXiv

Surprisal theory links human processing effort to the predictability of an upcoming linguistic unit, but empirical work often leaves the notion of a unit underspecified. In practice, experimental stimuli are segmented into linguistically motivated units (e.g., words), while pretrained language models assign probability mass to a fixed token alphabet that typically does not align with those units. As a result, surprisal-based predictors depend implicitly on ad hoc procedures that conflate two distinct modeling choices: the definition of the unit of analysis and the choice of regions of interest over which predictions are evaluated. In this paper, we disentangle these choices and give a unified framework for reasoning about surprisal over arbitrary unit inventories. We argue that surprisal-based analyses should make these choices explicit and treat tokenization as an implementation detail rather than a scientific primitive.

Abstract

Surprisal Theory

Explore similar work

On the Proper Treatment of Units in Surprisal Theory

Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?

From Exposure to Expectation: Frequency, Surprisal, and Language Across Development in Spanish