Accelerated Drift of Scientific Prose in the LLM Era: Evidence from an Interrupted Time-Series Analysis

Hanna Hubarava; Benjamin V. Ineichen

cs.CLOct 19, 2026

Accelerated Drift of Scientific Prose in the LLM Era: Evidence from an Interrupted Time-Series Analysis

Authors: Hanna Hubarava, Benjamin V. Ineichen

Organizations: Department of Clinical Research Department of Digital Medicine University of Bern, Switzerland

Abstract

Every scientific text is embedded in the writing norms of its time: it reflects earlier publications while also shaping the texts that follow. The growing use of LLM-based writing tools introduces a new factor in this evolving system, but the linguistic changes coinciding with their emergence remain insufficiently understood. We investigate whether the trajectories of scientific-preprint writing changed in rate or direction after conversational LLMs became publicly accessible, using the release of ChatGPT in November 2022 as the observational breakpoint. We compare pre- and post-2022 trajectories across vocabulary, punctuation, readability, and syntax in abstracts from arXiv's artificial intelligence, statistics, and quantitative biology categories and from medRxiv. Using interrupted time-series regression, we find substantial post-2022 changes across multiple feature families and research domains, including increases in average word length, reduced formula-based readability, and declines in maximum dependency-tree depth and sentence-level depth variation. Individual features follow different dynamics: some lexical markers accelerate persistently, while others rise and then recede. Our findings show that isolated marker frequencies capture only part of the shift: changes in the rate, direction, and structure of linguistic trajectories provide complementary evidence of broader post-2022 change in scientific-preprint writing. More broadly, these findings motivate monitoring scientific writing as a multidimensional, evolving system rather than through a fixed set of LLM-associated markers.

Figures & tables

Panel of Figure 1 — Figure 1: Selected yearly trajectories illustrating sustained increases ( across and syllables per word), transient peaks ( insight and delve ), and trajectory reversal (max tree depth). Lexical frequencies are reported per 1,000 words; the dashed line marks ChatGPT’s release.

Feature	arXiv AI	arXiv q-bio	arXiv stat	medRxiv
$\it{across}$	6.10 ∗∗∗ [5.51, 6.69]	3.88 ∗∗∗ [3.02, 4.73]	3.88 ∗∗∗ [3.41, 4.35]	3.10 ∗∗∗ [2.30, 3.90]
$\it{align}$	3.14 ∗∗∗ [2.71, 3.57]	1.51 ∗∗∗ [1.20, 1.82]	1.92 ∗∗∗ [1.62, 2.22]	2.71 ∗∗∗ [2.20, 3.22]
$\it{while}$	3.24 ∗∗∗ [2.67, 3.82]	1.80 ∗∗∗ [1.47, 2.13]	1.80 ∗∗∗ [1.42, 2.17]	2.07 ∗∗∗ [1.69, 2.44]
$\it{exhibit}$	2.19 ∗∗∗ [1.44, 2.93]	0.76 ∗∗∗ [0.52, 1.00]	0.95 ∗∗∗ [0.64, 1.25]	1.37 ∗∗∗ [0.75, 1.99]
em dash	2.35 ∗∗∗ [1.45, 3.24]	1.45 ∗∗∗ [0.82, 2.07]	1.46 ∗∗∗ [0.76, 2.17]	2.84 ∗∗∗ [2.15, 3.54]
syllables/word	4.15 ∗∗∗ [3.90, 4.41]	3.78 ∗∗∗ [3.48, 4.08]	2.75 ∗∗∗ [2.48, 3.02]	1.24 ∗∗∗ [0.67, 1.81]

Table 1: Largest consistently significant post-intervention trajectory changes by feature group. Cells report standardized interrupted-time-series slope changes per year with 95% confidence intervals. Within each prespecified group—lexical markers, punctuation, readability, syntax, and rhetorical stance—we report up to four individual features with the largest mean absolute standardized slope change across the four corpora. Eligible features required estimable effects and family-level FDR-adjusted

q<.05

in every corpus; aggregate group measures were excluded.

{}^{*}q<.05

{}^{**}q<.01

, and

{}^{***}q<.001

, after Benjamini–Hochberg correction within each feature family and corpus.

Panel of Figure 2 — Figure 2: Standardized post-intervention slope changes for punctuation features across the four abstract corpora, with 95% confidence intervals.

Panel of Figure 3 — Figure 3: Standardized post-intervention slope changes for readability features across the four abstract corpora, with 95% confidence intervals. Negative FRE changes indicate reduced reading ease.

Panel of Figure 4 — Figure 4: Decomposition of the annual post-intervention slope changes in ARI (left) and FKGL (right) into sentence- and word-length components across the four abstract corpora.

Panel of Figure 5 — Figure 5: Standardized post-intervention slope changes for syntactic features across the four abstract corpora, with 95% confidence intervals.

Panel of Figure 6 — Figure 6: Dependency-context trajectories across the four abstract corpora. The left panel shows determiners attached to prepositional objects per 1,000 words. The right panel shows the ratio of adjectival and compound prenominal modifiers to prepositional modifiers. The dashed line marks the public release of ChatGPT.

Panel of Figure 7 — Figure 7: Topic-level heterogeneity in post-2022 abstract trajectories in arXiv (AI, quantitative biology, statistics) and medRxiv (medicine). Cells show standardized ITS slope changes for corpus-specific topics. We report up to the two largest individual-feature effects per prespecified group among features significant in all four corpora; aggregate measures are excluded. Green indicates positive and orchid negative slope changes. Topic identifiers are corpus-specific and are not matched across panels.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Feature(s)	Operational definition	Unit
Lexical and discourse features
Single-word markers	Whole-token matches in lowercased lemmatized text	per 1,000 words
Multiword phrases	Whole-phrase matches in lowercased normalized text	per 1,000 words
Punctuation	Counts of colons, semicolons, and normalized em dashes	per 1,000 words
Certainty / hedging	Token matches against the corresponding lemmatized lexicons	proportion
Readability features

Appendix

Table 2: Operational definitions of the analyzed linguistic features. Features were computed at document level and aggregated as monthly arithmetic means; monthly paper counts were used as weights in the inferential models. Frequency features were normalized as

1000c/(N_{\alpha}+1)

and certainty and hedging as

c/(N_{\alpha}+1)

, where

c

is the document-level count and

N_{\alpha}

is the number of alphabetic spaCy tokens.

Panel of Figure 8 — Figure 8: Yearly dependency-role proportions in arXiv AI abstracts for the roles with the largest absolute post-2022 changes in slope.

Panel of Figure 9 — Figure 9: Post-2022 changes in the annual slopes of directed dependency-edge proportions across the four abstract corpora. A bigram A $\rightarrow$ B denotes an edge from a head to its child. Bars report the change in slope after 2022 relative to the pre-2022 trajectory. Positive values indicate a more positive post-2022 trend; negative values indicate a more negative trend.

Figure 10: Pairwise similarity of post-2022 linguistic-change profiles across the abstract corpora. Cells report Pearson correlations between standardized ITS slope-change estimates across all features jointly estimable in each corpus pair.

Panel of Figure 11 — Figure 11: Directional agreement between abstract and title trajectories. Arrows show the standardized ITS slope-change direction for abstracts and titles. In each cell, the first arrow indicates the standardized ITS slope-change direction for abstracts and the second for titles.

	AI	q-bio	stat	medRxiv
Complete pairs	57	53	50	55
Inestimable in titles	29	33	36	31
Profile correlation, $r$	.49	.59	.34	.45
Median $\|\Delta_{\mathrm{title}}-\Delta_{\mathrm{abstract}}\|$	1.60	0.75	1.05	0.90
Larger title effect	3.5%	5.7%	4.0%	23.6%

Appendix

Table 3: Complementary title–abstract comparison across 86 configured features. Complete pairs are features with estimable standardized ITS slope changes in both abstract and title scopes. The median difference is measured on the standardized slope-change scale. Larger title effect is the percentage of complete feature pairs for which the absolute standardized ITS slope change is larger in titles than in abstracts.

Panel of Figure 12 — Figure 12: Topic composition over time in the four abstract corpora. Stacked areas show the annual prevalence of corpus-specific BERTopic clusters. The combined legend reports topic identifiers and shortened c-TF-IDF labels separately for each corpus. Topic identifiers are corpus-specific and are not comparable across panels. Values for 2026 include records through April.

Panel of Figure 13 — Figure 13: Topic-level yearly trajectories for the five representative features shown in Figure 1 . Rows correspond to the four abstract corpora and lines to corpus-specific BERTopic clusters. Lexical frequencies are reported per 1,000 words; the dashed line marks ChatGPT’s release. Clause depths stands for maximum tree depth. Topic identifiers are corpus-specific and are not comparable across rows.

Corpus	Feature	2018	2019	2020	2021	Actual
AI	$\it{across}$	2.98	3.65	4.19	4.80	6.10
	em dash	0.47	0.81	0.96	1.18	2.35
	syll./word	2.80	2.77	3.25	3.75	4.15
	max tree depth	-0.51	-0.58	-0.80	-1.20	-2.13
	$\it{moreover}$	-0.17	-0.21	-0.48	-0.85	-1.54
q-bio	$\it{across}$	1.39	1.45	1.63	2.30	3.88

Appendix

Table 4: Temporal placebo checks for selected main-result features. Cells report standardized annual ITS slope changes for January placebo interventions and the prespecified November 2022 intervention (Actual, in bold). Dashes indicate unavailable estimates.

Feature	AI	q-bio	stat	medRxiv
Adjectives
$\it{comprehensive}$	-0.20 [-1.83, 1.43]	-0.07 [-1.30, 1.16]	-0.07 [-0.87, 0.73]	0.15 [-0.61, 0.91]
$\it{crucial}$	-1.30 [-2.92, 0.32]	-0.60 [-1.83, 0.62]	-0.67 [-1.72, 0.39]	0.16 [-1.01, 1.33]
$\it{intricate}$	-1.26 [-3.96, 1.44]	-0.71 [-2.31, 0.88]	-0.45 [-1.93, 1.02]	-0.45 [-1.94, 1.04]
$\it{pivotal}$	-1.17 [-3.98, 1.65]	-0.83 [-2.31, 0.65]	-0.24 [-0.86, 0.38]	-0.01 [-0.24, 0.21]
$\it{potential}$	-1.39 [-3.18, 0.41]	-0.43 [-1.21, 0.36]	-0.21 [-0.92, 0.51]	0.08 [-0.71, 0.86]

Appendix

Table 5: Abstract-level interrupted-time-series results. Cells report the annualized post-intervention change in slope (

\beta_{3}

), standardized by the pre-intervention monthly standard deviation, with 95% confidence intervals. Positive values indicate a shift toward a more positive trajectory; negative values indicate a shift toward a more negative trajectory.

{}^{*}q<.05

{}^{**}q<.01

{}^{***}q<.001

after within-family Benjamini–Hochberg correction.

Feature	AI	q-bio	stat	medRxiv
Adjectives
$\it{comprehensive}$	3.84 ∗∗∗ [2.08, 5.59]	2.69 ∗∗ [1.13, 4.24]	2.12 ∗∗∗ [1.09, 3.15]	2.30 ∗∗∗ [1.24, 3.35]
$\it{crucial}$	3.39 ∗∗∗ [1.91, 4.87]	2.45 ∗∗ [1.04, 3.85]	1.95 ∗∗∗ [0.95, 2.96]	3.84 ∗∗∗ [2.11, 5.58]
$\it{intricate}$	7.25 ∗∗∗ [3.41, 11.08]	3.99 ∗∗∗ [2.31, 5.68]	3.67 ∗∗∗ [1.80, 5.55]	4.51 ∗∗∗ [2.76, 6.26]
$\it{pivotal}$	6.87 ∗∗ [2.39, 11.36]	3.12 ∗∗ [1.19, 5.05]	2.03 ∗∗∗ [1.16, 2.90]	0.54 ∗∗ [0.15, 0.92]
$\it{potential}$	4.79 ∗∗∗ [3.34, 6.23]	1.20 ∗∗ [0.46, 1.95]	1.56 ∗∗∗ [0.71, 2.40]	2.98 ∗∗∗ [1.77, 4.20]

Appendix

Table 6: Abstract-level counterfactual-excess results for 2023–2024. Cells report the paper-count-weighted mean difference between observed monthly values and projections from the pre-intervention trend, standardized by the pre-intervention monthly standard deviation, with 95% confidence intervals.

{}^{*}q<.05

{}^{**}q<.01

{}^{***}q<.001

after within-family Benjamini–Hochberg correction.

Scientific Communication

Scientific communication studies how research is written, shared, translated, visualized, and understood by specialist and public audiences.

See all 50 papers

What Are LLMs Doing to Scientific Communication? Measuring Changes in Writing Practices and Reading Experience
May 19, 2026Filip Miletić, Neele FalkScientific Communication Text Generation
Institute for Natural Language Processing, University of Stuttgart, Germany
Large language models reshape the language of science
Apr 10, 2025Dingkang Lin, Naixuan Zhao, Dan Tian +1AI-Assisted Scientific Research Scientific Communication
School of Information Management, Nanjing University, Nanjing, China, 210032 · School of Economics and Management, Shanxi University, Taiyuan, China, 030031
SciZoom: A Large-scale Benchmark for Hierarchical Scientific Summarization across the LLM Era
Mar 17, 2026Han Jang, Junhyeok Lee, Kyu Sung ChoiLLM Evaluation Scientific Communication
Seoul National University · Seoul National University Hospital · Seoul National University College of Medicine

May 19, 2026J/K move · Enter open · S save

What Are LLMs Doing to Scientific Communication? Measuring Changes in Writing Practices and Reading Experience

Filip Miletić, Neele Falk

Institute for Natural Language Processing, University of Stuttgart, Germany

Scientific Communication Text Generation

Open paper AskPDF arXiv

Has the style of scientific communication changed due to the growing use of large language models in the writing process? We address this question in the domain of Natural Language Processing by leveraging two data resources we create: a naturalistic corpus of over 37,000 papers from the ACL Anthology (2020-2024); and a synthetic dataset of 3,000 human-written passages and their LLM-generated improvements. We first implement a series of diachronic lexical analyses, showing that both word frequency and usage contexts have changed significantly over time, indicating semantic specialization in some cases and generalization in others. Broadening our perspective, we then model a range of more complex stylistic features and find that LLM-modified texts more frequently contain certain syntactic constructions, more complex and longer words and a lower lexical diversity. Finally, we connect these changes in writing practices to subjective reading experience through a pilot annotation study with 20 domain experts. They overall rate LLM-improved texts as more understandable and exciting, but also express negative qualitative attitudes towards LLMs, highlighting the strongly subjective effect of AI-assisted writing on reading experience.

Abstract

Scientific Communication

Explore similar work

What Are LLMs Doing to Scientific Communication? Measuring Changes in Writing Practices and Reading Experience

Large language models reshape the language of science

SciZoom: A Large-scale Benchmark for Hierarchical Scientific Summarization across the LLM Era