Linguistic boundaries emerge from pressure-constrained processing dynamics

Rui He; Philipp Homan; Wolfram Hinzen

Figure 1: Pressure-constrained simulation and parameter selection. (A) Simulation model. Dependency-locality costs and contextual surprisal jointly contribute to processing pressure. Accumulated pressure is transformed by a sigmoid function into a probabilistic boundary drive. When open relations remain unresolved, pressure promotes the resolution of open relations. When no relation remains open, pressure increases the probability of a boundary, which resets accumulated pressure and the local contextual states. (B) Language-likeness score. Parameter regimes were evaluated using dependency-length decay, Zipf-like lexical rank-frequency structure, and anti-collapse lexical quality control. (C) Top 10% language-like regimes. Model families were compared within the subset of parameter combinations retained by the language-likeness score.

Figure 2: Boundary-aligned profiles in the best-performing pressure-constrained model. Analyses were performed on the highest-scoring DLT-surprisal parameter regime, rerun across 500 random seeds. Model variables were aligned to emitted boundaries and compared with non-boundary control positions sampled from the same simulations. The vertical dashed line marks the boundary crossing. Six variables of interest: (A) Processing pressure; (B) Boundary probability; (C) Contextual surprisal costs; (D) Memory costs; (E) Integration costs; and (F) Semantic mismatch.

Figure 3: Temporal scales of continuous dynamics and discrete structural events in the selected model. Analyses were performed on the highest-scoring DLT–surprisal parameter regime rerun across 500 random seeds. Points show mean period or scale across seeds. Error bars indicate 95% bootstrap confidence intervals. (A) Dominant periods of continuous model variables estimated from Welch peak periods. Boundary probability and integration cost showed similar segment-scale periods, whereas memory cost varied over a longer scale and contextual surprisal over a much slower scale. (B) Discrete structural scales estimated from median inter-event intervals or relation spans. Relation openings, relation closures, and relation spans occurred at short local scales, whereas actual boundaries occurred at a longer segmental scale.

Figure 4: Processing costs and local memory history in human boundary judgments. (A) Adjusted boundary-choice probabilities across standardized memory cost after the current word, integration cost, and local semantic mismatch. Shading indicates 95% confidence intervals. (B) Odds ratios for memory cost at the preceding word and mean memory cost across the preceding three words. Error bars indicate 95% confidence intervals; the dashed line marks an odds ratio of 1.

Figure 5: Sentence-boundary-aligned profiles of human reading time, model-derived surprisal, and LLM measures. All measures were aligned to sentence boundaries in the naturalistic stories. The vertical dashed line marks the sentence boundary position. (A) Mean log reading-time residuals, including spillover correction. (B) Memory cost from dependency parsing. (C) Integration cost from dependency parsing. (D) Semantic mismatch increased at the boundary. (E) LLM surprisal. (F) Cosine distance between consecutive hidden states. (G) Local intrinsic dimensionality. (H) Trajectory curvature among tokens. (I) Averaged layer curvature. (J) Length-to-chord ratio of layer curvature.

Figure 6: (A) Constraint sensitivity across behavioral and LLM outcomes. Each panel shows the Wald

z

-statistics for memory cost, integration cost, and static semantic mismatch as predictors of one outcome. Outcomes included spillover-adjusted reading-time residuals, Qwen surprisal, cosine update, local intrinsic dimensionality, trajectory curvature, mean layer curvature, and layer length-to-chord ratio (LCR). Circles show models without explicit sentence-boundary control. Squares show models including sentence-boundary status as an additional categorical predictor. Stars indicate statistical significance corrected by False Discovery Rate. (B) Boundary-control gain. Points show

\Delta\mathrm{AIC}

, defined as AIC with boundary control minus AIC without boundary control. More negative values indicate better model fit after adding explicit sentence-boundary status.

Parameter	Value	Interpretation
$k$	$1.2$	Steepness of the pressure-to-boundary sigmoid
$N^{\mathrm{open}}_{\max}$	$6$	Maximum number of simultaneously open relations
$L_{\min}$	$3$	Minimum segment length
$L_{\max}$	$99$	Maximum segment length
$P_{\mathrm{reset}}$	$0.0$	Pressure value after a boundary reset
$\sigma_{\epsilon}$	$0.05$	Standard deviation of pressure noise

Table 1: Fixed architectural parameters used in the parameter scan.

Parameter	Candidate values	Interpretation	Applied family
$\theta$	5.0, 8.0, 12.0	Boundary threshold; controls how much pressure is required before boundary drive increases.	All
$\rho$	0.3, 0.5, 0.8, 0.95	Pressure carry-over; controls how strongly pressure persists across events.	All
$(b_{\mathrm{open}},b_{\mathrm{close}})$	(0.15, 0.15), (0.30, 0.30), (0.50, 0.50), (0.15, 0.50), (0.50, 0.15)	Baseline tendency to open and close dependency-like relations.	All
$(a_{\mathrm{mem}},b_{\mathrm{int}})$	(0.2, 0.0), (0.0, 0.2), (0.2, 0.2), (0.5, 0.2), (0.2, 0.5), (0.5, 0.5)	DLT-inspired memory and integration cost weights.	DLT-only; DLT-surprisal
$\gamma_{\mathrm{surp}}$	0.0, 0.02, 0.05, 0.1, 0.3, 0.6, 0.9	Weight of lexical surprisal in the pressure equation.	Surprisal-only
$\gamma_{\mathrm{surp}}$	0.02, 0.05, 0.1, 0.3, 0.6, 0.9	Non-zero lexical surprisal weight; ensures that lexical surprisal is active.	DLT-surprisal

Table 2: Tuned parameters used in the parameter scan.

Parameter	Value	Interpretation
Model family	DLT–surprisal	Combined structural and contextual pressure model
Combination ID	13777	Parameter combination selected by language-likeness score
$b_{\mathrm{open}}$	0.50	Baseline tendency to open dependency-like relations
$b_{\mathrm{close}}$	0.50	Baseline tendency to close dependency-like relations
$a_{\mathrm{mem}}$	0.50	Weight of memory cost from currently open relations
$b_{\mathrm{int}}$	0.20	Weight of integration cost from resolved relations

Table 3: The best parameter regime.

Figure 7: Model-family differences in component language-likeness metrics. Model families were compared among the retained top 10% parameter regimes: DLT-only, surprisal-only, and combined DLT-surprisal models. DLT stands for dependency locality theory. Panels show group differences in dependency-distance exponential fit (

R^{2}

; A), dependency-distance power-law fit (

R^{2}

; B), dependency-distance decay score (C), lexical Zipf fit (

R^{2}

; D), deviation of the Zipf exponent from the ideal value of 1 (E), lexical Zipf score (F), quality-control pass rate (G), and the overall language-likeness score (H). Omnibus differences were tested using Kruskal–Wallis tests, followed by Dwass–Steel–Critchlow–Fligner pairwise comparisons. Significance brackets indicate pairwise comparisons: ns, not significant; *

p<0.05

; **

p<0.01

; ***

p<0.001

. Quality-control pass rate was constant across retained regimes and was therefore not tested.

Figure 8: Parameter sensitivity of language-likeness across model families among the retained top 10% parameter regimes. Mean language-likeness scores are shown as a function of individual simulation parameters, separately for DLT-only, surprisal-only, and combined DLT-plus-surprisal models. For each panel, scores were averaged over all other parameter settings within the scanned parameter space. Panels show (A) the boundary threshold

\theta

, (B) pressure carry-over

\rho

, (C) context strength

\beta_{\mathrm{ctx}}

, (D) baseline opening tendency, (E) baseline closure tendency, (F) total DLT strength

a_{\mathrm{mem}}+b_{\mathrm{int}}

, (G) DLT balance

a_{\mathrm{mem}}-b_{\mathrm{int}}

, (H) memory-cost weight

a_{\mathrm{mem}}

, (I) integration-cost weight

b_{\mathrm{int}}

, (J) surprisal weight

\gamma

, and (K) effective surprisal strength

\gamma\times\beta_{\mathrm{ctx}}

. The analyses show that language-like organization was not restricted to a single parameter setting. Models containing DLT-inspired structural costs generally achieved higher language-likeness scores than surprisal-only models across a broad range of thresholds, carry-over values, opening/closure tendencies, and structural-cost weights. Contextual parameters showed a more nonlinear profile, with strongest performance at intermediate context strengths and reduced scores when surprisal pressure became too dominant.

Figure 9: Actual sentence boundaries occur at a characteristic but variable timescale. The plot shows the empirical distribution of token intervals between consecutive sentence boundaries. Most intervals clustered around short-to-intermediate spans, while the distribution showed a pronounced right tail, indicating that sentence lengths are variable rather than periodic. This empirical distribution was used as a reference for assessing whether emergent boundaries in the simulation occurred at naturalistic sentence-like intervals.

Figure 10: Pressure-dependent versus completion-only boundary formation. (A) Survival distributions of generated segment lengths,

P(L\geq x)

, for the pressure-driven and completion-only models, calibrated to produce approximately matched mean segment lengths. (B–E) Boundary-aligned trajectories of accumulated processing pressure (B), memory cost (C), integration cost (D), and lexical surprisal (E). Position 0 denotes the final token before boundary emission, and position +1 denotes the first token of the following segment, with the vertical dashed line marking the boundary crossing. The pressure-driven model exhibited stronger pre-boundary pressure accumulation, memory-cost reduction, and integration-cost increases, while post-boundary trajectories converged across models. Blue solid lines represent the pressure-driven model, and orange dashed lines represent the completion-only control. Shading indicates mean

\pm

1.96 SEM across 500 random seeds.

Figure 11: Sensitivity of processing-cost effects to unit endings. Odds ratios and 95% confidence intervals for memory cost after the current word, integration cost, and semantic mismatch in the full sample and after excluding 300 top-level unit-ending positions. The dashed vertical line indicates an odds ratio of 1.

Figure 12: Sensitivity of processing-cost effects to grammatical coding. Odds ratios and 95% confidence intervals for the three processing-cost predictors under numeric and categorical representations of grammatical structure. The dashed vertical line indicates an odds ratio of 1.

Figure 13: Sentence-boundary sensitivity differs across human behavioral and LLM representational measures. For each outcome, the plot shows the

z

value associated with the sentence-boundary coefficient from an ordinary least-squares regression model controlling for pressure-derived predictors, word position, and item fixed effects, with cluster-robust standard errors at the sentence level. Lines extend from zero to the estimated boundary effect. Positive values indicate increased outcome values at sentence boundaries, whereas negative values indicate decreased outcome values at sentence boundaries. Reading-time residuals and LLM local intrinsic dimensionality increased at boundaries, while LLM surprisal, cosine update, trajectory curvature, and layer-curvature measures decreased at boundaries. Reported

P_{\mathrm{FDR}}

values are corrected across outcomes using False Discovery Rate (FDR).

Linguistic boundaries emerge from pressure-constrained processing dynamics

Abstract

Figures & tables

Computational Psycholinguistics

STRUCTURALCOST: A controlled reading time dataset for modeling human sentence processing difficulty

Trajectory Dynamics in Language Model Hidden States Predict Human Processing Costs Beyond Surprisal

Dual Alignment Between Language Model Layers and Human Sentence Processing

Abstract

Computational Psycholinguistics

Explore similar work

STRUCTURALCOST: A controlled reading time dataset for modeling human sentence processing difficulty

Trajectory Dynamics in Language Model Hidden States Predict Human Processing Costs Beyond Surprisal

Dual Alignment Between Language Model Layers and Human Sentence Processing