Accelerated Drift of Scientific Prose in the LLM Era: Evidence from an Interrupted Time-Series Analysis

Hanna Hubarava; Benjamin V. Ineichen

cs.CLOct 19, 2026

Accelerated Drift of Scientific Prose in the LLM Era: Evidence from an Interrupted Time-Series Analysis

Authors: Hanna Hubarava, Benjamin V. Ineichen

Organizations: Department of Clinical Research Department of Digital Medicine University of Bern, Switzerland

Abstract

Every scientific text is embedded in the writing norms of its time: it reflects earlier publications while also shaping the texts that follow. The growing use of LLM-based writing tools introduces a new factor in this evolving system, but the linguistic changes coinciding with their emergence remain insufficiently understood. We investigate whether the trajectories of scientific-preprint writing changed in rate or direction after conversational LLMs became publicly accessible, using the release of ChatGPT in November 2022 as the observational breakpoint. We compare pre- and post-2022 trajectories across vocabulary, punctuation, readability, and syntax in abstracts from arXiv's artificial intelligence, statistics, and quantitative biology categories and from medRxiv. Using interrupted time-series regression, we find substantial post-2022 changes across multiple feature families and research domains, including increases in average word length, reduced formula-based readability, and declines in maximum dependency-tree depth and sentence-level depth variation. Individual features follow different dynamics: some lexical markers accelerate persistently, while others rise and then recede. Our findings show that isolated marker frequencies capture only part of the shift: changes in the rate, direction, and structure of linguistic trajectories provide complementary evidence of broader post-2022 change in scientific-preprint writing. More broadly, these findings motivate monitoring scientific writing as a multidimensional, evolving system rather than through a fixed set of LLM-associated markers.

Figures & tables

Panel of Figure 1 — Figure 1: Selected yearly trajectories illustrating sustained increases ( across and syllables per word), transient peaks ( insight and delve ), and trajectory reversal (max tree depth). Lexical frequencies are reported per 1,000 words; the dashed line marks ChatGPT’s release.

Feature	arXiv AI	arXiv q-bio	arXiv stat	medRxiv
$\it{across}$	6.10 ∗∗∗ [5.51, 6.69]	3.88 ∗∗∗ [3.02, 4.73]	3.88 ∗∗∗ [3.41, 4.35]	3.10 ∗∗∗ [2.30, 3.90]
$\it{align}$	3.14 ∗∗∗ [2.71, 3.57]	1.51 ∗∗∗ [1.20, 1.82]	1.92 ∗∗∗ [1.62, 2.22]	2.71 ∗∗∗ [2.20, 3.22]
$\it{while}$	3.24 ∗∗∗ [2.67, 3.82]	1.80 ∗∗∗ [1.47, 2.13]	1.80 ∗∗∗ [1.42, 2.17]	2.07 ∗∗∗ [1.69, 2.44]
$\it{exhibit}$	2.19 ∗∗∗ [1.44, 2.93]	0.76 ∗∗∗ [0.52, 1.00]	0.95 ∗∗∗ [0.64, 1.25]	1.37 ∗∗∗ [0.75, 1.99]
em dash	2.35 ∗∗∗ [1.45, 3.24]	1.45 ∗∗∗ [0.82, 2.07]	1.46 ∗∗∗ [0.76, 2.17]	2.84 ∗∗∗ [2.15, 3.54]
syllables/word	4.15 ∗∗∗ [3.90, 4.41]	3.78 ∗∗∗ [3.48, 4.08]	2.75 ∗∗∗ [2.48, 3.02]	1.24 ∗∗∗ [0.67, 1.81]

Table 1: Largest consistently significant post-intervention trajectory changes by feature group. Cells report standardized interrupted-time-series slope changes per year with 95% confidence intervals. Within each prespecified group—lexical markers, punctuation, readability, syntax, and rhetorical stance—we report up to four individual features with the largest mean absolute standardized slope change across the four corpora. Eligible features required estimable effects and family-level FDR-adjusted

q<.05

in every corpus; aggregate group measures were excluded.

{}^{*}q<.05

{}^{**}q<.01

, and

{}^{***}q<.001

, after Benjamini–Hochberg correction within each feature family and corpus.

Panel of Figure 2 — Figure 2: Standardized post-intervention slope changes for punctuation features across the four abstract corpora, with 95% confidence intervals.

Panel of Figure 3 — Figure 3: Standardized post-intervention slope changes for readability features across the four abstract corpora, with 95% confidence intervals. Negative FRE changes indicate reduced reading ease.

Panel of Figure 4 — Figure 4: Decomposition of the annual post-intervention slope changes in ARI (left) and FKGL (right) into sentence- and word-length components across the four abstract corpora.

Panel of Figure 5 — Figure 5: Standardized post-intervention slope changes for syntactic features across the four abstract corpora, with 95% confidence intervals.

Panel of Figure 6 — Figure 6: Dependency-context trajectories across the four abstract corpora. The left panel shows determiners attached to prepositional objects per 1,000 words. The right panel shows the ratio of adjectival and compound prenominal modifiers to prepositional modifiers. The dashed line marks the public release of ChatGPT.

Panel of Figure 7 — Figure 7: Topic-level heterogeneity in post-2022 abstract trajectories in arXiv (AI, quantitative biology, statistics) and medRxiv (medicine). Cells show standardized ITS slope changes for corpus-specific topics. We report up to the two largest individual-feature effects per prespecified group among features significant in all four corpora; aggregate measures are excluded. Green indicates positive and orchid negative slope changes. Topic identifiers are corpus-specific and are not matched across panels.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Feature(s)	Operational definition	Unit
Lexical and discourse features
Single-word markers	Whole-token matches in lowercased lemmatized text	per 1,000 words
Multiword phrases	Whole-phrase matches in lowercased normalized text	per 1,000 words
Punctuation	Counts of colons, semicolons, and normalized em dashes	per 1,000 words
Certainty / hedging	Token matches against the corresponding lemmatized lexicons	proportion
Readability features

Appendix

Table 2: Operational definitions of the analyzed linguistic features. Features were computed at document level and aggregated as monthly arithmetic means; monthly paper counts were used as weights in the inferential models. Frequency features were normalized as

1000c/(N_{\alpha}+1)

and certainty and hedging as

c/(N_{\alpha}+1)

, where

c

is the document-level count and

N_{\alpha}

is the number of alphabetic spaCy tokens.

Panel of Figure 8 — Figure 8: Yearly dependency-role proportions in arXiv AI abstracts for the roles with the largest absolute post-2022 changes in slope.

Panel of Figure 9 — Figure 9: Post-2022 changes in the annual slopes of directed dependency-edge proportions across the four abstract corpora. A bigram A $\rightarrow$ B denotes an edge from a head to its child. Bars report the change in slope after 2022 relative to the pre-2022 trajectory. Positive values indicate a more positive post-2022 trend; negative values indicate a more negative trend.

Figure 10: Pairwise similarity of post-2022 linguistic-change profiles across the abstract corpora. Cells report Pearson correlations between standardized ITS slope-change estimates across all features jointly estimable in each corpus pair.

Panel of Figure 11 — Figure 11: Directional agreement between abstract and title trajectories. Arrows show the standardized ITS slope-change direction for abstracts and titles. In each cell, the first arrow indicates the standardized ITS slope-change direction for abstracts and the second for titles.

	AI	q-bio	stat	medRxiv
Complete pairs	57	53	50	55
Inestimable in titles	29	33	36	31
Profile correlation, $r$	.49	.59	.34	.45
Median $\|\Delta_{\mathrm{title}}-\Delta_{\mathrm{abstract}}\|$	1.60	0.75	1.05	0.90
Larger title effect	3.5%	5.7%	4.0%	23.6%

Appendix

Table 3: Complementary title–abstract comparison across 86 configured features. Complete pairs are features with estimable standardized ITS slope changes in both abstract and title scopes. The median difference is measured on the standardized slope-change scale. Larger title effect is the percentage of complete feature pairs for which the absolute standardized ITS slope change is larger in titles than in abstracts.

Panel of Figure 12 — Figure 12: Topic composition over time in the four abstract corpora. Stacked areas show the annual prevalence of corpus-specific BERTopic clusters. The combined legend reports topic identifiers and shortened c-TF-IDF labels separately for each corpus. Topic identifiers are corpus-specific and are not comparable across panels. Values for 2026 include records through April.

Panel of Figure 13 — Figure 13: Topic-level yearly trajectories for the five representative features shown in Figure 1 . Rows correspond to the four abstract corpora and lines to corpus-specific BERTopic clusters. Lexical frequencies are reported per 1,000 words; the dashed line marks ChatGPT’s release. Clause depths stands for maximum tree depth. Topic identifiers are corpus-specific and are not comparable across rows.

Corpus	Feature	2018	2019	2020	2021	Actual
AI	$\it{across}$	2.98	3.65	4.19	4.80	6.10
	em dash	0.47	0.81	0.96	1.18	2.35
	syll./word	2.80	2.77	3.25	3.75	4.15
	max tree depth	-0.51	-0.58	-0.80	-1.20	-2.13
	$\it{moreover}$	-0.17	-0.21	-0.48	-0.85	-1.54
q-bio	$\it{across}$	1.39	1.45	1.63	2.30	3.88

Appendix

Table 4: Temporal placebo checks for selected main-result features. Cells report standardized annual ITS slope changes for January placebo interventions and the prespecified November 2022 intervention (Actual, in bold). Dashes indicate unavailable estimates.

Feature	AI	q-bio	stat	medRxiv
Adjectives
$\it{comprehensive}$	-0.20 [-1.83, 1.43]	-0.07 [-1.30, 1.16]	-0.07 [-0.87, 0.73]	0.15 [-0.61, 0.91]
$\it{crucial}$	-1.30 [-2.92, 0.32]	-0.60 [-1.83, 0.62]	-0.67 [-1.72, 0.39]	0.16 [-1.01, 1.33]
$\it{intricate}$	-1.26 [-3.96, 1.44]	-0.71 [-2.31, 0.88]	-0.45 [-1.93, 1.02]	-0.45 [-1.94, 1.04]
$\it{pivotal}$	-1.17 [-3.98, 1.65]	-0.83 [-2.31, 0.65]	-0.24 [-0.86, 0.38]	-0.01 [-0.24, 0.21]
$\it{potential}$	-1.39 [-3.18, 0.41]	-0.43 [-1.21, 0.36]	-0.21 [-0.92, 0.51]	0.08 [-0.71, 0.86]

Appendix

Table 5: Abstract-level interrupted-time-series results. Cells report the annualized post-intervention change in slope (

\beta_{3}

), standardized by the pre-intervention monthly standard deviation, with 95% confidence intervals. Positive values indicate a shift toward a more positive trajectory; negative values indicate a shift toward a more negative trajectory.

{}^{*}q<.05

{}^{**}q<.01

{}^{***}q<.001

after within-family Benjamini–Hochberg correction.

Feature	AI	q-bio	stat	medRxiv
Adjectives
$\it{comprehensive}$	3.84 ∗∗∗ [2.08, 5.59]	2.69 ∗∗ [1.13, 4.24]	2.12 ∗∗∗ [1.09, 3.15]	2.30 ∗∗∗ [1.24, 3.35]
$\it{crucial}$	3.39 ∗∗∗ [1.91, 4.87]	2.45 ∗∗ [1.04, 3.85]	1.95 ∗∗∗ [0.95, 2.96]	3.84 ∗∗∗ [2.11, 5.58]
$\it{intricate}$	7.25 ∗∗∗ [3.41, 11.08]	3.99 ∗∗∗ [2.31, 5.68]	3.67 ∗∗∗ [1.80, 5.55]	4.51 ∗∗∗ [2.76, 6.26]
$\it{pivotal}$	6.87 ∗∗ [2.39, 11.36]	3.12 ∗∗ [1.19, 5.05]	2.03 ∗∗∗ [1.16, 2.90]	0.54 ∗∗ [0.15, 0.92]
$\it{potential}$	4.79 ∗∗∗ [3.34, 6.23]	1.20 ∗∗ [0.46, 1.95]	1.56 ∗∗∗ [0.71, 2.40]	2.98 ∗∗∗ [1.77, 4.20]

Appendix

Table 6: Abstract-level counterfactual-excess results for 2023–2024. Cells report the paper-count-weighted mean difference between observed monthly values and projections from the pre-intervention trend, standardized by the pre-intervention monthly standard deviation, with 95% confidence intervals.

{}^{*}q<.05

{}^{**}q<.01

{}^{***}q<.001

after within-family Benjamini–Hochberg correction.

Scientific Communication

Scientific communication studies how research is written, shared, translated, visualized, and understood by specialist and public audiences.

See all 50 papers

May 19, 2026

What Are LLMs Doing to Scientific Communication? Measuring Changes in Writing Practices and Reading Experience

Has the style of scientific communication changed due to the growing use of large language models in the writing process? We address this question in the domain of Natural Language Processing by leveraging two data resources we create: a naturalistic corpus of over 37,000 papers from the ACL Anthology (2020-2024); and a synthetic dataset of 3,000 human-written passages and their LLM-generated improvements. We first implement a series of diachronic lexical analyses, showing that both word frequency and usage contexts have changed significantly over time, indicating semantic specialization in some cases and generalization in others. Broadening our perspective, we then model a range of more complex stylistic features and find that LLM-modified texts more frequently contain certain syntactic constructions, more complex and longer words and a lower lexical diversity. Finally, we connect these changes in writing practices to subjective reading experience through a pilot annotation study with 20 domain experts. They overall rate LLM-improved texts as more understandable and exciting, but also express negative qualitative attitudes towards LLMs, highlighting the strongly subjective effect of AI-assisted writing on reading experience.

Filip Miletić, Neele Falk

Institute for Natural Language Processing, University of Stuttgart, Germany

Scientific Communication Text Generation

Ask PDF

Apr 10, 2025

Large language models reshape the language of science

Scientific language is a central infrastructure of knowledge production, but it remains unclear whether large language models (LLMs) are altering not only how scientists write, but also how scientific knowledge is communicated and accessed. Here we analyze 21.36 million scientific abstracts published between 2020 and 2024, together with historical records from major journals, to trace recent changes in the language of science. We identify a marked turning point in 2024, when scientific writing shows a sharp increase in lexical complexity alongside a decline in syntactic complexity. This shift is pervasive across disciplines and journal tiers, and is more pronounced in texts by scholars working in non-native English contexts, especially those from language backgrounds that differ more typologically from English. Controlled polishing experiments confirm that LLMs reproduce this pattern by favoring more lexically dense and syntactically compressed expression. We further show why this linguistic shift matters: it may widen the distance between scientific discourse and public-facing language, while also helping scholars in non-native English contexts navigate English-language publishing requirements. These findings suggest that LLMs may broaden participation in scientific authorship while narrowing the accessibility of scientific communication, making them a new force in the linguistic infrastructure of science.

Dingkang Lin, Naixuan Zhao, Dan Tian +1

School of Information Management, Nanjing University, Nanjing, China, 210032 · School of Economics and Management, Shanxi University, Taiyuan, China, 030031

AI-Assisted Scientific Research Scientific Communication

Ask PDF

Mar 17, 2026

SciZoom: A Large-scale Benchmark for Hierarchical Scientific Summarization across the LLM Era

The explosive growth of AI research has created unprecedented information overload, increasing the demand for scientific summarization at multiple levels of granularity beyond traditional abstracts. While LLMs are increasingly adopted for summarization, existing benchmarks remain limited in scale, target only a single granularity, and predate the LLM era. Moreover, since the release of ChatGPT in November 2022, researchers have rapidly adopted LLMs for drafting manuscripts themselves, fundamentally transforming scientific writing, yet no resource exists to analyze how this writing has evolved. To bridge these gaps, we introduce SciZoom, a benchmark comprising 44,946 papers from four top-tier ML venues (NeurIPS, ICLR, ICML, EMNLP) spanning 2020 to 2025, explicitly stratified into Pre-LLM and Post-LLM eras. SciZoom provides three hierarchical summarization targets (Abstract, Contributions, and TL;DR) achieving compression ratios up to 600:1, enabling both multi-granularity summarization research and temporal mining of scientific writing patterns. Our linguistic analysis reveals striking shifts in phrase patterns (up to 10x for formulaic expressions) and rhetorical style (23% decline in hedging), suggesting that LLM-assisted writing produces more confident yet homogenized prose. SciZoom serves as both a challenging benchmark and a unique resource for mining the evolution of scientific discourse in the generative AI era. Our code and dataset are publicly available on GitHub (https://github.com/janghana/SciZoom) and Hugging Face (https://huggingface.co/datasets/hanjang/SciZoom), respectively.

Han Jang, Junhyeok Lee, Kyu Sung Choi

Seoul National University · Seoul National University Hospital · Seoul National University College of Medicine

LLM Evaluation Scientific Communication Text Summarization

Ask PDF

Abstract

Scientific Communication

Explore similar work

What Are LLMs Doing to Scientific Communication? Measuring Changes in Writing Practices and Reading Experience

Large language models reshape the language of science

SciZoom: A Large-scale Benchmark for Hierarchical Scientific Summarization across the LLM Era