Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?

Matteo Gay; Coleman Haley; Mario Giulianelli; Edoardo Ponti

cs.CLFeb 16, 2026

Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?

Authors: Matteo Gay, Coleman Haley, Mario Giulianelli, Edoardo Ponti

Organizations: KU Leuven · University of Edinburgh · University College London

Abstract

The Uniform Information Density (UID) hypothesis posits that speakers are subject to a communicative pressure to distribute information evenly within utterances, minimising surprisal variance. While this hypothesis has been tested empirically, prior studies are limited exclusively to text-only inputs, abstracting away from the perceptual context in which utterances are produced. In this work, we present the first computational study of UID in visually grounded settings. We estimate surprisal using multilingual vision-and-language models over image-caption data in 30 languages and visual storytelling data in 13 languages, together spanning 11 families. We find that grounding on perception consistently smooths the distribution of information, increasing both global and local uniformity across typologically diverse languages compared to text-only settings. In visual narratives, grounding in both image and discourse contexts has additional effects, with the strongest surprisal reductions occurring at the onset of discourse units. Overall, this study takes a first step towards modelling the temporal dynamics of information flow in ecologically plausible, multimodal language use, and finds that grounded language exhibits greater information uniformity, supporting a context-sensitive formulation of UID.

Figures & tables

Figure 1: Global and local UID values under the U condition (caption in isolation) and P condition (caption with visual context) across the 30 typologically diverse languages, from 10 language families, in Ground-XM3600 dataset. Points below the dashed diagonal line indicate increased uniformity (i.e., decreased global or local variability) when the caption’s surprisal is conditioned on the image context.

Figure 2: Kernel density estimation of global UID values for captions in Ground-XM3600 , conditioned either on their respective image ( P ) or without any contextual image ( U ). Curves are truncated at the 99th percentile to enhance visual clarity.

Figure 3: Each box represents the distribution of paragraph-level

\mathrm{UID}_{v}

values across languages under four conditions: no context ( U ); paired image as context ( P ); all preceding paragraphs as context ( D ); all preceding paragraphs and interleaved images as context (

\text{{\color[rgb]{0.4688,0.3672,0.9414}P} }+\text{{\color[rgb]{0.8633,0.1484,0.4961}D} }

). Dashed horizontal lines indicate the mean

\mathrm{UID}_{v}

per condition, averaged across all languages.

Figure 4: Paragraph-level UID values averaged across BloomVIST stories (truncated at 20 paragraphs) under the utterance only ( U ), perceptual context ( P ), discourse context ( D ), and multimodal context (

\text{{\color[rgb]{0.4688,0.3672,0.9414}P} }+\text{{\color[rgb]{0.8633,0.1484,0.4961}D} }

) conditions.

Appendix figures & tables26 assets

Supplementary material from the paper’s appendix.

Appendix

Lang	Caps	Words	Mean	Std
arb	6928	50225	7.2	3.3
ces	5867	38299	6.5	5.0
dan	6696	58845	8.8	4.4
deu	8615	93753	10.9	3.8
ell	6040	48281	8.0	4.0
eng	7179	67956	9.5	3.9

Appendix

Table 1: Summary statistics of caption lengths across languages in the Ground-XM3600 dataset. Caps : number of captions; Words : total word count; Mean and Std : average and standard deviation of caption lengths (in words).

Lang	Stories	Pars	Words	W/P	W/S
ben	217	1919	58198	30.33	268.19
deu	21	220	10827	49.21	515.57
eng	1997	19602	766901	39.12	384.03
fas	125	599	8128	13.57	65.02
fra	294	3444	136148	39.53	463.09
ind	225	1774	30659	17.28	136.26

Appendix

Table 2: BloomVist data: number of stories, paragraphs (Pars), words, mean words per paragraph (W/P), and words per story (W/S).

Macro-area	WALS Family	ISO
Africa (+ Eurasia)	Afro-Asiatic	ARB
Eurasia	Afro-Asiatic	HEB
	Indo-European	BEN, CES, DAN, DEU, ELL, ENG, FAS, FRA, HIN, HRV, ITA, NLD, NOR, POL, POR, RON, RUS, SPA, SWE, UKR
	Uralic	FIN, HUN
	Turkic	TUR, KIR
	Sino-Tibetan	ZHO

Appendix

Table 3: Classification of the 33 analysed languages by macro-area and family, as defined in WALS ( Dryer and Haspelmath, 2013 ).

Figure 5: Surprisal contours for the first three paragraphs of a BloomVIST story (ID: dd0b0ef6-3889-47c7-b5ca-93e18e76aba8) across all four experimental conditions. Dashed vertical lines denote paragraph boundaries.

Lang	UID v ( U )	UID v ( P )	UID lv ( U )	UID lv ( P )	$\Delta_{v}$ (%) ( $d$ )	$\Delta_{lv}$ (%) ( $d$ )
arb	8.92	8.79	21.50	21.87	-1.44 * (-0.02)	1.75 *** (0.02)
ces	18.48	11.89	39.96	25.64	-35.67 *** (-0.72)	-35.83 *** (-0.54)
dan	22.81	14.61	45.06	29.19	-35.93 *** (-0.93)	-35.22 *** (-0.64)
deu	17.55	10.51	40.13	22.80	-40.09 *** (-0.95)	-43.18 *** (-0.87)
ell	16.30	10.13	29.88	20.42	-37.89 *** (-0.79)	-31.66 *** (-0.53)
eng	18.25	15.57	32.14	25.83	-14.70 *** (-0.35)	-19.63 *** (-0.31)

Appendix

Table 4: UID values of Ground-XM3600 across languages under U and P conditions. The

\Delta

columns report the relative change in UID from U to P , computed as

\frac{(\text{{\color[rgb]{0.4688,0.3672,0.9414}P}{}}-\text{{\color[rgb]{0.9961,0.3789,0}U}{}})}{\text{{\color[rgb]{0.9961,0.3789,0}U}{}}}

and Cohen’s

d

effect size in parentheses. Bold values mark languages where UID is higher under P , indicating lower uniformity in textual information when grounded in perception. Statistical significance is based on paired Wilcoxon signed-rank tests across sentences, with Benjamini–Hochberg FDR correction across languages. Significance thresholds: ∗

q<0.05

, ∗∗

q<0.01

, ∗∗∗

q<0.001

, n.s. (not significant).

Lang	Global CV			Local CV
	U	P	$\Delta\%$	U	P	$\Delta\%$
arb	0.52	0.75	$\textbf{43.45}^{***}$	0.73	1.05	$\textbf{45.16}^{***}$
ces	0.72	0.89	$\textbf{23.68}^{***}$	0.91	1.13	$\textbf{24.53}^{***}$
dan	0.80	0.98	$\textbf{22.49}^{***}$	1.00	1.23	$\textbf{23.38}^{***}$
deu	0.71	0.86	$\textbf{21.73}^{***}$	1.00	1.19	$\textbf{18.50}^{***}$
ell	0.71	0.81	$\textbf{12.91}^{***}$	0.87	1.03	$\textbf{18.64}^{***}$

Appendix

Table 5: Mean Global CV (Coefficient of Variation) and Local CV across languages and conditions for captions in GROUND-XM3600 .

Lang	U	P	D	$[\text{{\color[rgb]{0.4688,0.3672,0.9414}P} }+\text{{\color[rgb]{0.8633,0.1484,0.4961}D} }]$
ben	1.05	0.99	0.94	0.93
deu	1.11	1.07	1.08	1.08
eng	1.06	1.01	1.13	1.13
fas	0.87	0.83	0.93	0.91
fra	1.04	1.01	1.07	1.08
ind	0.91	0.88	0.99	1.01

Appendix

Table 6: Mean Global CV across languages and conditions for paragraph in BLOOM-Vist .

Figure 6: Average change in variance contribution ( $\overline{\Delta C}_{\text{POS}}$ ) by POS across languages for UID reduction failure cases in Ground-XM3600 . Rows are ordered, from top to bottom, by descending cross-linguistic mean. Intuitively, red cells denote an increased divergence from the sequence mean under visual grounding ( P ) relative to text-only ( U ) conditions.

Figure 7: Cross-linguistic dynamics of UID reduction failure across 30 languages in Ground-XM3600 . The bar charts represent the average change in variance contribution (

\overline{\Delta C}_{\text{POS}}

) per Part-of-Speech in sentences where visual grounding increases global variance (

\text{UID}_{v}(\text{{\color[rgb]{0.4688,0.3672,0.9414}P} })>\text{UID}_{v}(\text{{\color[rgb]{0.9961,0.3789,0}U} })

) (see also Figure 6 ). Line plots illustrate the percentage of words per POS category exhibiting surprisal increases (red circles) versus decreases (blue squares).

Lang	Unit	U	P	D	$[\text{{\color[rgb]{0.4688,0.3672,0.9414}P} }+\text{{\color[rgb]{0.8633,0.1484,0.4961}D} }]$
ben	paragraph	0.04 n.s	0.01 n.s	-1.37 ***	-1.19 ***
ben	sentence	-4.13 ***	-3.27 ***	-0.81 ***	-0.56 ***
deu	paragraph	-0.12 n.s	-0.08 n.s	-0.98 ***	-0.71 **
deu	sentence	-3.25 ***	-2.09 ***	-0.51 ***	-0.31 ***
eng	paragraph	0.24 ***	0.27 n.s	-1.00 ***	-0.67 ***
eng	sentence	-2.93 ***	-1.92 ***	-0.64 ***	-0.39 ***

Appendix

Table 7: Fixed-effect slope estimates over relative position for mean surprisal , from mixed-effects models (with random effects by story), computed separately per language and discourse unit (sentence, paragraph). Slopes reflect condition-specific trajectories via interaction terms, with U as the baseline. Chinese was excluded from sentence-level analysis due to insufficient longer sentences for reliable estimation. Significance thresholds: ∗

p<0.05

, ∗∗

p<0.01

, ∗∗∗

p<0.001

, n.s. (not significant).

Lang	Unit	U	P	D	$[\text{{\color[rgb]{0.4688,0.3672,0.9414}P} }+\text{{\color[rgb]{0.8633,0.1484,0.4961}D} }]$
ben	paragraph	0.74 n.s	-0.46 n.s	-17.31 ***	-11.90 ***
ben	sentence	-66.38 ***	-44.07 ***	-9.45 ***	-6.01 ***
deu	paragraph	-1.48 n.s	-0.56 n.s	-9.49 ***	-5.66 *
deu	sentence	-31.17 ***	-16.52 ***	-3.22 ***	-1.27 ***
eng	paragraph	1.97 ***	2.18 n.s	-8.09 ***	-3.99 ***
eng	sentence	-25.15 ***	-12.32 ***	-4.40 ***	-1.93 ***

Appendix

Table 8: Fixed-effect slope estimates over relative position for

\bm{\mathrm{UID}}_{v}

, from mixed-effects models (with random effects by story), computed separately per language and discourse unit (sentence, paragraph). Slopes reflect condition-specific trajectories via interaction terms, with U as the baseline. Chinese was excluded from sentence-level analysis due to insufficient longer sentences for reliable estimation. Significance thresholds: ∗

p<0.05

, ∗∗

p<0.01

, ∗∗∗

p<0.001

, n.s. (not significant).

Figure 8: Surprisal reduction density for ben .

Figure 9: Surprisal reduction density for deu .

Figure 10: Surprisal reduction density for eng .

Figure 11: Surprisal reduction density for fas .

Figure 12: Surprisal reduction density for fra .

Figure 13: Surprisal reduction density for ind .

Figure 14: Surprisal reduction density for kir .

Figure 15: Surprisal reduction density for kor .

Figure 16: Surprisal reduction density for por .

Figure 17: Surprisal reduction density for rus .

Figure 18: Surprisal reduction density for spa .

Figure 19: Surprisal reduction density for tha .

Figure 20: Surprisal reduction density for zho .

Lang	$\Delta_{w}$	U	P	D	$[\text{{\color[rgb]{0.4688,0.3672,0.9414}P} }+\text{{\color[rgb]{0.8633,0.1484,0.4961}D} }]$
ben	$\Delta_{1}$	-33.798	-27.867	-7.777	-7.036
	$\Delta_{2}$	-21.496	-17.803	-5.861	-5.360
	$\Delta_{3}$	-15.853	-13.089	-4.450	-3.980
deu	$\Delta_{1}$	-18.408	-14.799	-4.418	-4.002
	$\Delta_{2}$	-15.674	-12.237	-3.351	-3.141
	$\Delta_{3}$	-12.439	-9.234	-2.398	-2.246

Appendix

Table 9: Mean surprisal difference across sentence boundaries (

\Delta_{w}

) for window sizes

w=1,2,3

, reported per language. For each sentence transition, we subtract the average surprisal of the final

w

words of the preceding sentence from that of the first

w

words of the following one. More negative values indicate sharper surprisal spikes at sentence onsets, reflecting stronger deviations from uniform information flow across sentence boundaries.

Lang	$\Delta_{w}$	U	P	D	$[\text{{\color[rgb]{0.4688,0.3672,0.9414}P} }+\text{{\color[rgb]{0.8633,0.1484,0.4961}D} }]$
ben	$\Delta_{1}$	-33.927	-27.903	-7.909	-7.095
	$\Delta_{2}$	-21.433	-17.691	-5.919	-5.350
	$\Delta_{3}$	-15.800	-12.983	-4.470	-3.947
deu	$\Delta_{1}$	-18.890	-15.328	-4.508	-4.064
	$\Delta_{2}$	-15.885	-12.417	-3.393	-3.175
	$\Delta_{3}$	-12.516	-9.246	-2.368	-2.196

Appendix

Table 10: Mean surprisal difference across paragraph boundaries (

\Delta_{w}

) for window sizes

w=1,2,3

, reported per language. For each transition, we subtract the average surprisal of the final

w

words of a paragraph from that of the first

w

words of the following one. More negative values indicate larger spikes in surprisal at paragraph onsets, reflecting greater non-uniformity in information flow across discourse boundaries.

Computational Psycholinguistics

Computational psycholinguistics models how people learn, produce, and understand language using formal theories, algorithms, and behavioral data.

See all 86 papers

Expect the Unexpected? Testing the Surprisal of Salient Entities
Apr 12, 2026Jessica Lin, Amir ZeldesSurprisal Theory Computational Psycholinguistics
Department of Linguistics, Georgetown University
Uniform Information Density-Based Preference Optimization
Oct 10, 2026Yongyuan Li, Yang XuPreference Optimization LLM Alignment
Department of Computer Science and Engineering Southern University of Science and Technology Shenzhen, Guangdong, China
Estimating Surprisal from the Speech Signal
Oct 13, 2026Zachary Hopton, Samuel Kiegeland, Francesco Ignazio Re +4Surprisal Theory Computational Psycholinguistics
University of Zurich · ETH Zürich · CHI-FRO +1

Apr 12, 2026J/K move · Enter open · S save

Expect the Unexpected? Testing the Surprisal of Salient Entities

Jessica Lin, Amir Zeldes

Department of Linguistics, Georgetown University

Surprisal Theory Computational Psycholinguistics

Open paper AskPDF arXiv

Previous work examining the Uniform Information Density (UID) hypothesis has shown that while information as measured by surprisal metrics is distributed more or less evenly across documents overall, local discrepancies can arise due to functional pressures corresponding to syntactic and discourse structural constraints. However, work thus far has largely disregarded the relative salience of discourse participants. We fill this gap by studying how overall salience of entities in discourse relates to surprisal using 70K manually annotated mentions across 16 genres of English and a novel minimal-pair prompting method. Our results show that globally salient entities exhibit significantly higher surprisal than non-salient ones, even controlling for position, length, and nesting confounds. Moreover, salient entities systematically reduce surprisal for surrounding content when used as prompts, enhancing document-level predictability. This effect varies by genre, appearing strongest in topic-coherent texts and weakest in conversational contexts. Our findings refine the UID competing pressures framework by identifying global entity salience as a mechanism shaping information distribution in discourse.

Abstract

Computational Psycholinguistics

Explore similar work

Expect the Unexpected? Testing the Surprisal of Salient Entities

Uniform Information Density-Based Preference Optimization

Estimating Surprisal from the Speech Signal