Modeling the Arrival Time of Information-Bearing Events in Natural Language

Leyi Lao; Yulin Ou; Yang Xu; Hao An

cs.CLOct 9, 2026

Modeling the Arrival Time of Information-Bearing Events in Natural Language

Authors: Leyi Lao, Yulin Ou, Yang Xu, Hao An

Organizations: Computational Linguistics and Consciousness Sciences Lab, Southern University of Science and Technology, Shenzhen, China

Abstract

Understanding how information is distributed over time is fundamental to modeling natural language. While existing empirical studies primarily rely on post-hoc analysis of information-theoretic metrics, such as surprisal, little is known about the generation processes under the surface. We propose a novel approach for addressing this gap using the tool of neural temporal point processes (TPPs) by modeling the key information-bearing events in text sequences with learnable event intensity functions. Our model takes the history of event intervals and the contextual semantic representations from LLMs as input, and predicts the future arrival times for two types of events: new-word and high-surprisal-token. The experimental results reveal an interesting distinction in event intensities: LLMs tend to produce new words at significantly higher rates than humans, while high-surprisal tokens do not exhibit the difference. A decaying pattern of intensity over time is found in new-word events for both humans and LLMs, but not in high-surprisal-token events. Our findings revisit and extend previous studies that use simpler methods; provide supplementary evidence for theories about constrained context representation inherent in human language production; and find a new angle to view human and LLM generation differences.

Figures & tables

Figure 1: Overview of our RNN-based neural TPP framework. (a) Event extraction from text: new-word or high-surprisal-token. (b)Neural TPP architectures that encode inter-event intervals, semantic embeddings, and positional encodings to estimate the cumulative hazard function.

Dataset	Event Type	Text Source	Conventional TPP	Neural Baseline w/o LLM	Neural TPP w/o LLM	Neural Baseline w/ LLM	Neural TPP w/ LLM
Speech	New-word	Human	1.890	2.135	1.779	1.867	1.726
		Claude-Sonnet-4.5	1.256	1.226	1.086	1.069	1.019
		GPT-4.1	1.299	1.286	1.118	1.043	1.009
		Qwen-Plus	1.162	1.154	1.000	0.878	0.859
	High-surprisal- token	Human	-	2.202	1.888	1.919	1.780
		Claude-Sonnet-4.5	-	2.201	1.785	1.764	1.624

Table 1: Comparison of predicting performance (MAE

\downarrow

) among conventional TPPs, neural baselines, and neural TPPs, with and without LLaMA-3.1-8B semantic embeddings. Conventional TPPs for high-surprisal-token are left empty due to the low rate of successful fit.

Panel of Figure 2 — Figure 2: Estimated conditional intensity against time step for new-word events from conventional and neural TPPs. Both TPPs rely solely on temporal information without semantic embedding input. Shadow areas are 95% bootstrap confidence intervals.

Panel of Figure 3 — Figure 3: Conditional intensity curves across event types and datasets, estimated from neural TPPs with semantic embeddings (LLaMA-3.1-8B). Shadow areas are 95% bootstrap confidence intervals.

Model	Accuracy	Macro F1	AUROC
Speech Dataset
Claude-sonnet-4.5	0.9450	0.9450	0.9913
GPT-4.1	0.9650	0.9650	0.9932
Qwen-plus	0.9800	0.9800	0.9988
Writing Prompt Dataset
Claude-sonnet-4.5	0.8535	0.8535	0.9263

Table 2: Human vs. LLM classification results based on features extracted from the intensity curves.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Model	RMS $\downarrow$			R $\uparrow$
Model	Mean	Min	Max	Mean	Min	Max
Speech Dataset
Human	0.001191	0.000191	0.004748	0.991702	0.963751	0.997403
Claude-Sonnet-4.5	0.001753	0.000364	0.005538	0.990342	0.974717	0.996734
GPT-4.1	0.001415	0.000199	0.004075	0.990706	0.974341	0.997319
Qwen-Plus	0.001392	0.000191	0.005521	0.989525	0.974030	0.997438

Appendix

Table 3: Goodness-of-fit evaluation of the Conventional TPP model when events are defined as new words. The metrics include the Residual Mean Square (RMS) of the regression and the correlation (

R

) between fitted and observed data.

Figure 4: Estimated conditional intensity functions with LLaMA-3.1-8b-derived high-surprisal-token events on the both datasets. Both subplots utilize RNN-based neural TPPs without LLM embeddings (w/o LLM). Shadow areas are 95% bootstrap confidence intervals.

Figure 5: Estimated conditional intensity functions with new-word events without lemmatization on the both datasets. Both subplots utilize RNN-based neural TPP incorporating semantic embeddings extracted via LLaMA-3.1-8B. Shadow areas are 95% bootstrap confidence intervals.

Dataset	Event Type	Text Source	RNN-based			Transformer-based
Dataset	Event Type	Text Source	LLaMA- 3.1-8B	LLaMA- 3.2-1B	Qwen- 3-0.6B	LLaMA- 3.1-8B	LLaMA- 3.2-1B	Qwen- 3-0.6B
Speech	New-word	Human	1.726	1.718	1.729	1.605	1.594	1.611
		Claude-Sonnet-4.5	1.019	1.018	1.024	0.972	0.976	0.981
		GPT-4.1	1.009	1.005	1.011	0.962	0.966	0.966
		Qwen-Plus	0.859	0.863	0.871	0.828	0.832	0.834
	High-surprisal- token	Human	1.780	1.865	1.820	2.070	2.010	1.802

Appendix

Table 4: Comparison of predicting performance (MAE

\downarrow

) for RNN-based and Transformer-based neural TPPs with semantic embeddings from LLaMA-3.1-8B, LLaMA-3.2-1B, and Qwen-3-0.6B.

Panel of Figure 6 — Figure 6: Estimated conditional intensity functions across different linguistic event types. All subplots utilize RNN-based neural TPP incorporating semantic embeddings extracted via LLaMA-3.2-1B . Shadow areas are 95% bootstrap confidence intervals.

Panel of Figure 7 — Figure 7: Estimated conditional intensity functions across different linguistic event types. All subplots utilize RNN-based neural TPP incorporating semantic embeddings extracted via Qwen-3-0.6B . Shadow areas are 95% bootstrap confidence intervals.

Panel of Figure 8 — Figure 8: Estimated conditional intensity functions across different linguistic event types. All subplots utilize Transformer-based neural TPP incorporating semantic embeddings extracted via LLaMA-3.1-8B. Shadow areas are 95% bootstrap confidence intervals.

Panel of Figure 9 — Figure 9: Estimated conditional intensity functions across different linguistic event types. All subplots utilize Transformer-based neural TPP incorporating semantic embeddings extracted via LLaMA-3.2-1B. Shadow areas are 95% bootstrap confidence intervals.

Panel of Figure 10 — Figure 10: Estimated conditional intensity functions across different linguistic event types. All subplots utilize Transformer-based neural TPP incorporating semantic embeddings extracted via Qwen-3-0.6B. Shadow areas are 95% bootstrap confidence intervals.

Panel of Figure 11 — Figure 11: Mean values and 95% bootstrap confidence intervals of the five parameters ( $\lambda_{1}$ to $\lambda_{5}$ ) estimated by the Conventional TPP model across different text sources on the Speech and Writing Prompt datasets.

Model	Accuracy	Macro F1	Machine (Generated)			Human
Model	Accuracy	Macro F1	Precision	Recall	F1	Precision	Recall	F1
Speech Dataset
Claude-sonnet-4.5	0.763	0.760	0.700	0.800	0.750	0.820	0.730	0.780
GPT-4.1	0.725	0.720	0.570	0.820	0.680	0.880	0.670	0.760
Qwen-plus	0.725	0.720	0.620	0.780	0.690	0.820	0.690	0.750
Writing Prompt Dataset

Appendix

Table 5: Human vs. LLM classification results based on features extracted from the intensity functions estimated by the Conventional TPP model. We run separate binary classifications for each LLM (Claude-Sonnet-4.5, GPT-4.1, Qwen-Plus) against human texts.

Model	w/o LLM		LLaMA-3.1-8B		LLaMA-3.2-1B		Qwen-3-0.6B
Model	Acc	AUROC	Acc	AUROC	Acc	AUROC	Acc	AUROC
Speech Dataset
Claude-sonnet-4.5	0.9750	0.9964	0.9450	0.9913	0.9500	0.9896	0.9600	0.9914
GPT-4.1	0.9800	0.9994	0.9650	0.9932	0.9600	0.9939	0.9750	0.9967
Qwen-plus	0.9950	0.9999	0.9800	0.9988	0.9800	0.9978	0.9800	0.9990
Writing Prompt Dataset

Appendix

Table 6: Classification performance (Accuracy and AUROC) of the RNN-based neural TPP model across four embedding configurations (without-LLM, LLaMA-3.1-8B, LLaMA-3.2-1B, Qwen-3-0.6B) for each LLM vs. Human binary classification.

Figure 12: Feature importance heatmap for the Logistic Regression classifiers applied to the RNN-based neural TPP across all combinations of embedding types (without-LLM and LLaMA-3.1-8B, LLaMA-3.2-1B, Qwen-3-0.6B) and generation models (Claude-Sonnet-4.5, GPT-4.1, Qwen-Plus) on both datasets. The importance scores are estimated using the permutation importance method. The color intensity represents the importance of each feature in distinguishing human from LLM-generated texts.

Figure 13: Heatmap of new-word occurrences in a Writing-Prompts sample (Example 1). Darker orange gradients indicate higher intensity of each word estimated by the RNN-based neural TPP model, whereas new words (appearing for the first time in the context) are denoted by blue bounding boxes. The visualizations demonstrate the bursty, non-uniform distribution of new-word events across the generation sequence.

Figure 14: Heatmap visualizations of new-word occurrence patterns in a second human-written Writing-Prompts sample (Example 2), with intensity levels estimated by the RNN-based neural TPP model.

Event Time Prediction

Event time prediction estimates when future events will occur from event histories, time series, and contextual information.

See all 47 papers

Temporal Flattening in LLM-Generated Text: Comparing Human and LLM Writing Trajectories
Apr 13, 2026Zhanwei Cao, YeoJin Go, Yifan Hu +1Language Model Generation Evaluation AI-Generated Text Detection
Northeastern University, Seattle, WA, USA
Probabilistic Attribution For Large Language Models
May 20, 2026Shilpika Shilpika, Carlo Graziani, Bethany Lusch +2LLM Interpretability LLM Uncertainty Estimation
Argonne Leadership Computing Facility Argonne National Laboratory Lemont, IL, USA · Mathematics and Computer Science Division Argonne National Laboratory Lemont, IL, USA · Department of Computer Science University of Illinois Chicago Chicago, IL, USA
Measuring Temporal Linguistic Emergence in Diffusion Language Models
Apr 25, 2026Harry LuDiffusion Language Models
School of Mathematics, University of Minnesota, Minneapolis MN, United States.

Apr 13, 2026J/K move · Enter open · S save

Temporal Flattening in LLM-Generated Text: Comparing Human and LLM Writing Trajectories

Zhanwei Cao, YeoJin Go, Yifan Hu, Shanu Sushmita

Northeastern University, Seattle, WA, USA

Language Model Generation Evaluation AI-Generated Text Detection

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used in daily applications, from content generation to code writing, where each interaction treats the model as stateless, generating responses independently without memory. Yet human writing is inherently longitudinal: authors' styles and cognitive states evolve across months and years. This raises a central question: can LLMs reproduce such temporal structure across extended time periods? We construct and publicly release a longitudinal dataset of 412 human authors and 6,086 documents spanning 2012--2024 across three domains (academic abstracts, blogs, news) and compare them to trajectories generated by three representative LLMs under standard and history-conditioned generation settings. Using drift and variance-based metrics over semantic, lexical, and cognitive-emotional representations, we find temporal flattening in LLM-generated text. LLMs produce greater lexical diversity but exhibit substantially reduced semantic and cognitive-emotional drift relative to humans. These differences are highly predictive: temporal variability patterns alone achieve 94% accuracy and 98% ROC-AUC in distinguishing human from LLM trajectories. Our results demonstrate that temporal flattening persists regardless of whether LLMs generate independently or with access to incremental history, revealing a fundamental property of current deployment paradigms. This gap has direct implications for applications requiring authentic temporal structure, such as synthetic training data and longitudinal text modeling.

Abstract

Event Time Prediction

Explore similar work

Temporal Flattening in LLM-Generated Text: Comparing Human and LLM Writing Trajectories

Probabilistic Attribution For Large Language Models

Measuring Temporal Linguistic Emergence in Diffusion Language Models