Retractions Revisited: A Replication, Extension, and Mechanistic Evaluation of Linguistic Obfuscation Effects in Fraudulent Science

David M. Markowitz

cs.CLOct 9, 2026

Retractions Revisited: A Replication, Extension, and Mechanistic Evaluation of Linguistic Obfuscation Effects in Fraudulent Science

Authors: David M. Markowitz

Organizations: Associate Professor, Department of Communication, Michigan State University, East Lansing, MI USA

Abstract

Prior work suggests fraudulent scientists write their articles with higher rates of linguistic obfuscation than genuine scientists, with a possible mechanism suggesting obfuscation makes research more costly for a reader to analyze. This result has not undergone additional empirical scrutiny in the open science era, nor have other psychological mechanisms been evaluated. The current replication and extension addressed this opportunity by comparing 800 fraudulent and 800 genuine control papers (2013-2025). The results supported the linguistic obfuscation hypothesis, suggesting fraudulent papers were more obfuscated than genuine ones. A parallel mediation model evaluated three competing mechanisms, finding that reality monitoring, as measured through decreased rates of adjectives, best explained the link between fraudulent science and linguistic obfuscation. Cross-validated detection accuracy using obfuscation features (60.7%) and an elastic net of over 100 language dimensions (71.6%) were significantly greater than chance. Implications for scientific fraud and psychology of language research are discussed.

Scientific Communication

Scientific communication studies how research is written, shared, translated, visualized, and understood by specialist and public audiences.

See all 50 papers

TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs
Aug 11, 2026Valentin Rodionov, Shamil AssylbekovLLM Evaluation LLM Reliability
Intellicat Cleveland, OH 44106
Large language models reshape the language of science
Apr 10, 2025Dingkang Lin, Naixuan Zhao, Dan Tian +1AI-Assisted Scientific Research Scientific Communication
School of Information Management, Nanjing University, Nanjing, China, 210032 · School of Economics and Management, Shanxi University, Taiyuan, China, 030031
War in the Abstract: The Rise and Consequences of Militarized Language in Scientific Communication
Jun 22, 2026Sovesh Mohapatra, David Lydon-Staley, Dani S. BassettScientific Communication
Department of Bioengineering, School of Engineering and Applied Science, University of Pennsylvania, PA, USA · Annenberg School of Communication, University of Pennsylvania, PA, USA · Departments of Electrical & Systems Engineering and Physics & Astronomy, University of Pennsylvania, PA, USA +2

Aug 11, 2026J/K move · Enter open · S save

TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs

Valentin Rodionov, Shamil Assylbekov

Intellicat Cleveland, OH 44106

LLM Evaluation LLM Reliability Scientific Reasoning in Language Models Scientific Claim Verification

Open paper AskPDF arXiv

Large language models are being proposed as agents in scientific workflows, in domains where no downstream verifier exists. Such deployment assumes the model can distinguish reliable scientific literature from unreliable literature, a capability that has not yet been directly measured. Existing benchmarks evaluate factuality on questions with known answers; the failure mode we target here is different. We introduce a probe corpus of 42 retracted, fraudulent, and pseudoscientific papers, paired with a methodology for eliciting and scoring single-shot model engagement with each paper's framing. Each probe pairs a preamble extracted near-verbatim from the target paper with a scientifically plausible study-design request. The probes span five claim types: fabricated observation, pseudophysical mechanism, magical premise, legitimization bridge, and cargo-cult experiment. Two complementary scores measure whether a model rejects the flawed premise outright (IFR-a) and whether it recognizes the unreliability while still engaging (IFR-i). A depth score, the Engagement Depth Index (EDI), quantifies reproduction of paper- or field-specific withheld details. Across 30 models and 10 repeated runs, aggregate IFR-a is 0.93

\pm

0.004 and aggregate IFR-i is 0.809

\pm

0.009. Models engaged with untenable premises in 95% of all non-empty responses. Every evaluated model fails more than 71% of agentic probes, and 22 of 30 models fail more than 90% of the time. Rejections are concentrated on a small number of high-notoriety topics and specific probes, and disappear under matched-structure controls. These results are consistent with topic-keyed safety behavior rather than robust epistemic competence, and indicate an urgent need for guardrail infrastructure for scientific deployment of language models.

Abstract

Scientific Communication

Explore similar work

TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs

Large language models reshape the language of science

War in the Abstract: The Rise and Consequences of Militarized Language in Scientific Communication