Although linguistic differences between human-written and Large Language Model (LLM)-generated texts have been extensively studied, their implications for the cognitive effort involved in reading remain less explored. In psycholinguistics, reading time is commonly used as an indirect measure of cognitive processing effort. Building on this perspective, we investigate whether human-written texts and their LLM-generated rewrites exhibit differences in predicted reading time that may reflect differences in processing demands for readers. To this end, we introduce RoBERTa-RT, a model trained on human eye-tracking data to predict word-level reading times. We evaluate the model on both internal and external test sets, construct a paired dataset of 496 human-written articles and their LLM-generated rewrites, and apply RoBERTa-RT to compare their predicted reading times. Our findings show that LLM-generated rewrites tend to be shorter and have slightly lower predicted total reading times, while exhibiting higher predicted reading times per word. These differences are associated with systematic linguistic changes, including longer and less frequent words, greater lexical diversity, and higher contextual predictability. From a cognitive perspective, our findings suggest that LLM-generated writing may redistribute reading demands: reducing the amount of text to process while increasing the predicted processing effort associated with individual words. This highlights the importance of considering not only the linguistic characteristics of machine-generated content, but also its potential implications for human cognitive processing.
Figures & tables
Dataset
Split
Word occurrences
OneStop
Training
20,364
GECO
Training
56,410
MECO
Training
2,107
Total training
78,881
OneStop
Validation
7,975
Total validation
7,975
TABLE I: Eye-tracking datasets used for model training, validation, and evaluation. Counts refer to word occurrences.
Feature
Description
Feature
Description
word_len_clean
Number of characters in the word.
prev_word_len_clean
Length of the preceding word.
contains_digit
Presence of numerical digits (binary).
next_word_len_clean
Length of the following word.
has_hyphen
Presence of a hyphen (binary).
prev_word_freq_zipf
Zipf frequency of the preceding word.
ends_with_punctuation
Ends with punctuation (binary).
next_word_freq_zipf
Zipf frequency of the following word.
word_index_in_context
Word index within the paragraph.
prev_ends_with_punctuation
Preceding word ends with punctuation (binary).
relative_position_in_context
Word position relative to paragraph length.
next_ends_with_punctuation
Following word ends with punctuation (binary).
TABLE II: Linguistic features used for word-level reading-time prediction in feature-based models.
Model
Hyperparameter
Search range
Selected
Learning rate
[2,8]×10−5
2.05×10−5
Weight decay
[0.02,0.12]
0.0493
RoBERTa-base
Warmup ratio
[0.10,0.30]
0.2284
RoBERTa-large
Learning rate
[1,6]×10−5
5.66×10−5
Weight decay
[0.04,0.12]
0.0405
Warmup ratio
[0.15,0.35]
0.3439
TABLE III: Hyperparameter search ranges and selected values for the evaluated models.
Model
OneStop Val
OneStop Test
Provo Test
RoBERTa-base
15.76
15.00
16.09
RoBERTa-large
17.41
15.65
17.00
DeBERTa-v3-base
17.55
18.29
20.36
DeBERTa-v3-large
18.50
17.61
17.85
MLP
23.77
18.72
16.74
Ridge Regression
24.63
19.29
21.10
TABLE IV: Text-level MAE (ms/word) for the best configuration of each model, selected on the OneStop validation set.
Measure
Human mean ± SD
LLM mean ± SD
RC± SD (%)
padj
Total predicted RT (s)
117.10±25.89
115.58±25.48
−1.19±4.06
1.45×10−9
Mean predicted RT/word (ms)
229.91±15.27
233.39±14.77
+1.57±2.91
2.33×10−25
Word count
511.19±115.34
496.39±109.40
−2.66±4.49
7.76×10−34
TABLE V: Reading-Time and Text-Length Differences Between Human-Written and LLM-Generated Texts
Measure
Human mean ± SD
LLM mean ± SD
RC± SD (%)
padj
Mean word length
4.949±0.176
5.248±0.195
+6.06±2.33
4.08×10−82
Mean word frequency
0.007262±0.000874
0.006686±0.000790
−7.61±7.34
1.48×10−61
MTLD
103.02±21.11
134.67±29.89
+31.83±20.37
1.50×10−78
Mean surprisal
2.952±0.228
2.755±0.188
−6.51±4.36
4.80×10−77
TABLE VI: Linguistic Differences Between Human-Written and LLM-Generated Texts
Linguistic measure
ρ with RCi(RT/word)
padj
Mean word length
+.588
1.47×10−46
Mean word frequency
−.405
4.61×10−20
MTLD
+.292
2.97×10−10
Mean surprisal
+.438
1.18×10−23
TABLE VII: Relationship Between Linguistic Changes and Predicted Reading Time
Measure
ρ
padj
Mean predicted RT/word
−.458
3.46×10−26
Mean word length
−.303
4.93×10−11
Mean word frequency
+.364
4.47×10−16
MTLD
−.269
1.10×10−8
Mean surprisal
−.362
6.93×10−16
TABLE VIII: Spearman correlations between relative changes in word count and corresponding changes in predicted reading time and linguistic characteristics across Human–LLM pairs ( N=496 ).