Although linguistic differences between human-written and Large Language Model (LLM)-generated texts have been extensively studied, their implications for the cognitive effort involved in reading remain less explored. In psycholinguistics, reading time is commonly used as an indirect measure of cognitive processing effort. Building on this perspective, we investigate whether human-written texts and their LLM-generated rewrites exhibit differences in predicted reading time that may reflect differences in processing demands for readers. To this end, we introduce RoBERTa-RT, a model trained on human eye-tracking data to predict word-level reading times. We evaluate the model on both internal and external test sets, construct a paired dataset of 496 human-written articles and their LLM-generated rewrites, and apply RoBERTa-RT to compare their predicted reading times. Our findings show that LLM-generated rewrites tend to be shorter and have slightly lower predicted total reading times, while exhibiting higher predicted reading times per word. These differences are associated with systematic linguistic changes, including longer and less frequent words, greater lexical diversity, and higher contextual predictability. From a cognitive perspective, our findings suggest that LLM-generated writing may redistribute reading demands: reducing the amount of text to process while increasing the predicted processing effort associated with individual words. This highlights the importance of considering not only the linguistic characteristics of machine-generated content, but also its potential implications for human cognitive processing.
Figures & tables
Dataset
Split
Word occurrences
OneStop
Training
20,364
GECO
Training
56,410
MECO
Training
2,107
Total training
78,881
OneStop
Validation
7,975
Total validation
7,975
TABLE I: Eye-tracking datasets used for model training, validation, and evaluation. Counts refer to word occurrences.
Feature
Description
Feature
Description
word_len_clean
Number of characters in the word.
prev_word_len_clean
Length of the preceding word.
contains_digit
Presence of numerical digits (binary).
next_word_len_clean
Length of the following word.
has_hyphen
Presence of a hyphen (binary).
prev_word_freq_zipf
Zipf frequency of the preceding word.
ends_with_punctuation
Ends with punctuation (binary).
next_word_freq_zipf
Zipf frequency of the following word.
word_index_in_context
Word index within the paragraph.
prev_ends_with_punctuation
Preceding word ends with punctuation (binary).
relative_position_in_context
Word position relative to paragraph length.
next_ends_with_punctuation
Following word ends with punctuation (binary).
TABLE II: Linguistic features used for word-level reading-time prediction in feature-based models.
Model
Hyperparameter
Search range
Selected
Learning rate
[2,8]×10−5
2.05×10−5
Weight decay
[0.02,0.12]
0.0493
RoBERTa-base
Warmup ratio
[0.10,0.30]
0.2284
RoBERTa-large
Learning rate
[1,6]×10−5
5.66×10−5
Weight decay
[0.04,0.12]
0.0405
Warmup ratio
[0.15,0.35]
0.3439
TABLE III: Hyperparameter search ranges and selected values for the evaluated models.
Model
OneStop Val
OneStop Test
Provo Test
RoBERTa-base
15.76
15.00
16.09
RoBERTa-large
17.41
15.65
17.00
DeBERTa-v3-base
17.55
18.29
20.36
DeBERTa-v3-large
18.50
17.61
17.85
MLP
23.77
18.72
16.74
Ridge Regression
24.63
19.29
21.10
TABLE IV: Text-level MAE (ms/word) for the best configuration of each model, selected on the OneStop validation set.
Measure
Human mean ± SD
LLM mean ± SD
RC± SD (%)
padj
Total predicted RT (s)
117.10±25.89
115.58±25.48
−1.19±4.06
1.45×10−9
Mean predicted RT/word (ms)
229.91±15.27
233.39±14.77
+1.57±2.91
2.33×10−25
Word count
511.19±115.34
496.39±109.40
−2.66±4.49
7.76×10−34
TABLE V: Reading-Time and Text-Length Differences Between Human-Written and LLM-Generated Texts
Measure
Human mean ± SD
LLM mean ± SD
RC± SD (%)
padj
Mean word length
4.949±0.176
5.248±0.195
+6.06±2.33
4.08×10−82
Mean word frequency
0.007262±0.000874
0.006686±0.000790
−7.61±7.34
1.48×10−61
MTLD
103.02±21.11
134.67±29.89
+31.83±20.37
1.50×10−78
Mean surprisal
2.952±0.228
2.755±0.188
−6.51±4.36
4.80×10−77
TABLE VI: Linguistic Differences Between Human-Written and LLM-Generated Texts
Linguistic measure
ρ with RCi(RT/word)
padj
Mean word length
+.588
1.47×10−46
Mean word frequency
−.405
4.61×10−20
MTLD
+.292
2.97×10−10
Mean surprisal
+.438
1.18×10−23
TABLE VII: Relationship Between Linguistic Changes and Predicted Reading Time
Measure
ρ
padj
Mean predicted RT/word
−.458
3.46×10−26
Mean word length
−.303
4.93×10−11
Mean word frequency
+.364
4.47×10−16
MTLD
−.269
1.10×10−8
Mean surprisal
−.362
6.93×10−16
TABLE VIII: Spearman correlations between relative changes in word count and corresponding changes in predicted reading time and linguistic characteristics across Human–LLM pairs ( N=496 ).
A recent study (Kuribayashi et al., 2025) has shown that human sentence processing behavior, typically measured on syntactically unchallenging constructions, can be effectively modeled using surprisal from early layers of large language models (LLMs). This raises the question of whether such advantages of internal layers extend to more syntactically challenging constructions, where surprisal has been reported to underestimate human cognitive effort. In this paper, we begin by exploring internal layers that better estimate human cognitive effort observed in syntactic ambiguity processing in English. Our experiments show that, in contrast to naturalistic reading, later layers better estimate such a cognitive effort, but still underestimate the human data. This dual alignment sheds light on different modes of sentence processing in humans and LMs: naturalistic reading employs a somewhat weak prediction akin to earlier layers of LMs, while syntactically challenging processing requires more fully-contextualized representations, better modeled by later layers of LMs. Motivated by these findings, we also explore several probability-update measures using shallow and deep layers of LMs, showing a complementary advantage to single-layer's surprisal in reading time modeling.
Tatsuki Kuribayashi, Alex Warstadt, Yohei Oseki +1
Large language models (LLMs) are increasingly used in daily applications, from content generation to code writing, where each interaction treats the model as stateless, generating responses independently without memory. Yet human writing is inherently longitudinal: authors' styles and cognitive states evolve across months and years. This raises a central question: can LLMs reproduce such temporal structure across extended time periods? We construct and publicly release a longitudinal dataset of 412 human authors and 6,086 documents spanning 2012--2024 across three domains (academic abstracts, blogs, news) and compare them to trajectories generated by three representative LLMs under standard and history-conditioned generation settings. Using drift and variance-based metrics over semantic, lexical, and cognitive-emotional representations, we find temporal flattening in LLM-generated text. LLMs produce greater lexical diversity but exhibit substantially reduced semantic and cognitive-emotional drift relative to humans. These differences are highly predictive: temporal variability patterns alone achieve 94% accuracy and 98% ROC-AUC in distinguishing human from LLM trajectories. Our results demonstrate that temporal flattening persists regardless of whether LLMs generate independently or with access to incremental history, revealing a fundamental property of current deployment paradigms. This gap has direct implications for applications requiring authentic temporal structure, such as synthetic training data and longitudinal text modeling.
Human perception of time is subjective. Temporal illusions show that the brain relies on context and relational cues when judging duration. Prior studies established these effects with visual and auditory stimuli. We ask whether written narratives alone can cause these illusions, and whether LLMs respond as human participants do. We introduce a benchmark of 6,684 paired scenarios spanning five illusions, each judged from the character's and the reader's perspective. In our human study (60 participants), participants' judgments matched the predicted directions for all five illusions in at least one perspective (7 of 10 conditions). Across 14 LLMs, models reproduce every effect humans show in the same direction, but on average more strongly. Most effects persisted even after cue words were removed and after controlling for 15 lexical and semantic features. These results show that models and humans agree in direction but not in strength, which cautions against treating LLM judgments as substitutes for human responses for subjective temporal reasoning.