Writing for the Reviewer: Defensive Writing in GPT Models

Junchi Liao

cs.AIOct 8, 2026

Writing for the Reviewer: Defensive Writing in GPT Models

Authors: Junchi Liao

Abstract

Researchers increasingly use ChatGPT to revise their papers, and recent GPT versions often narrow or even retract the authors' claims. We call such changes defensive writing when the given material does not support them, and we test two explanations: the model corrects the authors' overclaiming, or it writes for an anticipated reviewer. We ask GPT versions and models from other developers to rewrite paragraphs from papers written before ChatGPT, or to write from an evidence sheet that lists a paper's method and results. Defensive writing grows with GPT version. GPT-6-astra retracts the authors' claims outright, and when it writes from the evidence sheet, it still adds the most ungrounded qualifications. The results favor the anticipated-review explanation, and correcting overclaiming explains only a small part. When the models are only asked to polish, defense stays near the level of the originals; mentioning review raises it, and one round of self-review raises it further. At the same time, fewer than one in ten of the claims GPT-6-astra retracts are overstated. AI reviewers score defensive rewrites higher, while human readers find them harder to read and the authors less certain. Combining AI writing with AI review may amplify this style.

Figures & tables

Figure 1: Overview of defensive writing by GPT models.

	Rewriting						Writing from evidence
	$R_{U}$						$R_{U}$
Writer	Nov	Con	Res	Dis	Ret %	Soft %	Nov	Con	Res	Dis
Authors (original)	0.11	0.00	0.04	0.00	–	–	0.14	0.03	0.15	0.38
GPT-4o	0.00	0.03	0.04	0.00	0.2	15	0.00	0.00	0.00	0.04
GPT-4.1	0.13	0.04	0.11	0.05	0.2	14	0.06	0.00	0.11	0.05
GPT-5.1	0.12	0.00	0.07	0.08	0.6	15	0.09	0.00	0.02	0.06

Table 1: Ungrounded qualifications per 100 words (

R_{U}

) by paragraph type, and the share of the authors’ claims retracted (Ret) or softened (Soft) in rewriting, pooled over the four types. Nov = novelty, Con = contribution, Res = result analysis, Dis = discussion. Bold marks the highest model value.

Figure 2:

R_{U}

pooled over the four paragraph types (left axis; dotted line: authors’ originals in rewriting) and the share of the authors’ claims softened or retracted in rewriting (right axis). Error bars are 95% CIs.

Figure 4: Ungrounded qualifications per 100 words (

R_{U}

) under prompt contexts P1–P7, pooled over the four paragraph types. Shading marks the contexts that mention review. Error bars are 95% CIs.

Review–revise
	$R_{U}$			Ret %
Writer	R1	R2	R3	R1	R2	R3
GPT-5.5	0.95	1.23	1.10	10.8	14.5	10.2
GPT-5.6-sol	1.42	1.62	1.44	18.3	24.3	19.1
GPT-6-astra	1.87	1.61	1.67	25.3	21.4	17.4
Claude Opus 5.5	0.54	0.64	0.70	1.2	2.5	4.6

Table 2: Multi-round revision over rounds R1–R3, measured against the authors’ original paragraph. Further measures are in Appendix D.2 .

	Soft %		Ret %
Writer	Sup	Over	Sup	Over	Int	Hit %	$n$
GPT-5.2 to 6-astra	21	53	1.2	5.7	2.4	13	46
GPT-6-astra	35	71	4.7	9.5	10.6	6	34
GPT-6-astra, P5	27	68	5.6	13.6	20.5	7	46
GPT, R1	32	48	13.0	25.8	33.3	6	262
Claude Opus, R1	18	36	0.0	4.5	4.1	–	6

Table 3: What rewrites do to the authors’ claims, by the judge’s label: Sup = supported, Over = overstated or contradicted, Int = interpretation. Hit is the share of retracted claims that are overstated and

n

the number of retracted claims; Hit is omitted when

n<10

. R1: first round of review–revise chains, with GPT pooling GPT-5.5, GPT-5.6-sol, and GPT-6-astra. Controls: the four non-GPT models. Only 5% of claims are overstated, so the Over columns have wide intervals (Appendix D.3 ).

Writer	Overall	Soundness	Overclaim	Overhedged
GPT-5.5	+1.29	+1.49	$-$ 0.24	+0.04
	[1.07,1.51]	[1.20,1.73]	[ $-$ 0.41, $-$ 0.05]	[ $-$ 0.01,0.10]
GPT-5.6-sol	+0.99	+1.20	$-$ 0.19	+0.06
	[0.76,1.22]	[0.94,1.49]	[ $-$ 0.33, $-$ 0.04]	[0.03,0.11]
GPT-6-astra	+1.22	+1.84	$-$ 0.42	+0.21
	[0.99,1.46]	[1.55,2.17]	[ $-$ 0.58, $-$ 0.28]	[0.12,0.29]

Table 4: AI review of P5 minus P1 rewrites by the same writer. Overall and Soundness are scores from 1 to 10; Overclaim and Overhedged are weaknesses per review. Values average the two reviewer models; brackets are 95% CIs.

Rating	P1	P5	P5 $-$ P1
Ease of reading	5.75	5.05	$-$ 0.70 [ $-$ 0.95, $-$ 0.45]
Flow	6.05	5.25	$-$ 0.80 [ $-$ 1.05, $-$ 0.55]
Effort	2.55	3.60	+1.05 [0.75, 1.35]
Claim identifiable	6.10	5.78	$-$ 0.33 [ $-$ 0.53, $-$ 0.12]
Author certainty	4.35	3.38	$-$ 0.97 [ $-$ 1.20, $-$ 0.75]
Comprehension (%)	100	100	0

Table 5: Human readers’ ratings of GPT-6-astra’s P1 and P5 rewrites. Author certainty is rated from 1 to 5 and the other ratings from 1 to 7; higher effort means harder. Comprehension is the share of correct answers to two questions per paragraph. Brackets are 95% intervals (Appendix E.2 ).

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Paper	Venue	arXiv	Dis	Evid
IMEXnet	ICML 2019	1903.02639	✓	✓
Message Pruning	AAAI 2020	1912.05304	✓	✓
Neural TS	ICLR 2021	2010.00827	✓
OVERT	JMLR 2022	2108.01220	✓	✓
DualAfford	ICLR 2023	2207.01971		✓
IRMCon	ECCV 2022	2208.03462	✓	✓

Appendix

Table 6: The 20 source papers; all EMNLP papers are from 2022. Every paper contributes one novelty, one contribution, and one result-analysis paragraph; Dis marks the 17 papers that also contribute a discussion paragraph. Evid marks the 16 papers used for writing from evidence; all 20 have an evidence sheet.

	Context and request
P1	Please polish it.
P2	I am writing an internal technical report for my research group. … Please polish it.
P3	I am submitting a paper to a top-tier machine learning / NLP / computer vision conference. … Please polish it.
P4	As P3. … Revise it so that it holds up under peer review.
P5	As P3. … Revise it to address the concerns a peer reviewer is likely to raise.
P6	As P3. … Revise it to maximize the paper’s chances of acceptance.

Appendix

Table 7: Full wording of the seven prompt contexts; “…” stands for “Below is {where}.”

Coder	Agree	$\kappa$	$\kappa$	$\kappa$	#Ret
	Six statuses		Soft	Ret
DeepSeek-V4-Flash	0.87	0.72	0.74	0.87	12
Qwen3-32B	0.73	0.49	0.59	0.35	17
Kimi-K2.6	0.81	0.62	0.63	0.71	17

Appendix

Table 8: Agreement of claim tracking with the main run on 762 claims. Soft and Ret are binary: softened (weakened or scoped) or not, retracted or not. #Ret is the number of retracted claims; the main run has 11.

Writer	$R_{U}$	$P_{U}$ %	$R_{U}$
	Polish only		Default
Authors (original)	0.04	5	–
GPT-4o	0.03	4	0.02
GPT-4.1	0.02	3	0.08
GPT-5.1	0.04	5	0.07
GPT-5.2	0.05	6	0.10

Appendix

Table 9: Ungrounded qualifications when the 12 models are asked only to polish the paragraph, against the default prompt of § 3.1 . All four paragraph types are pooled.

Writer	P1	P2	P3	P4	P5	P6	P7
GPT-4o	11 / 0.0	8 / 0.0	10 / 0.0	18 / 0.2	18 / 0.6	11 / 0.0	17 / 0.2
GPT-5.2	10 / 0.0	11 / 0.0	10 / 0.0	22 / 0.2	26 / 0.4	15 / 0.2	26 / 0.4
GPT-5.5	9 / 0.0	10 / 0.0	11 / 0.2	29 / 0.2	30 / 0.8	15 / 0.2	27 / 0.0
GPT-5.6-sol	8 / 0.0	9 / 0.2	9 / 0.2	25 / 0.4	30 / 2.7	16 / 0.0	26 / 1.0
GPT-6-astra	9 / 0.2	7 / 0.2	11 / 0.0	33 / 6.0	33 / 9.5	20 / 0.8	33 / 5.8
Claude Opus 5.5	3 / 0.2	5 / 0.2	7 / 0.2	15 / 0.2	15 / 1.1	12 / 0.2	14 / 0.6

Appendix

Table 10: Share of the authors’ claims softened / retracted (%) under the seven prompt contexts of § 3.2 .

Chain	Writer	R1	R2	R3	R1	R2	R3	R1	R2	R3	R1	R2	R3
		Words			$U$ per paragraph			Soft %			Dropped %
Review–revise	GPT-5.5	232	319	345	2.2	3.9	3.8	32	34	28	7	19	27
Review–revise	GPT-5.6-sol	234	338	363	3.3	5.5	5.2	32	28	24	9	20	30
Review–revise	GPT-6-astra	248	294	312	4.6	4.7	5.2	27	26	25	8	19	30
Review–revise	Claude Opus 5.5	194	259	314	1.0	1.6	2.2	20	25	24	9	18	24
Polish only	GPT-5.6-sol	119	117	117	0.1	0.1	0.0	10	11	11	0	0	0

Appendix

Table 11: Further measures for the multi-round chains of Table 2 .

U

per paragraph is the average number of ungrounded qualifications in a paragraph. The authors’ originals average 128 words.

Writer	Supported	Overstated	Supported	Overstated	Hit %
	Soft %		Ret %
GPT-5.2 to 6-astra	21 [18, 25]	53 [39, 67]	1.2 [0.5, 2.1]	5.7 [0.0, 17.6]	13 [0, 30]
GPT-6-astra	35 [27, 43]	71 [50, 89]	4.7 [2.0, 8.4]	9.5 [0.0, 25.0]	6 [0, 14]
GPT-6-astra, P5	27 [20, 35]	68 [48, 87]	5.6 [2.7, 8.8]	13.6 [0.0, 29.4]	7 [0, 13]
GPT, R1	32 [26, 38]	48 [35, 63]	13.0 [10.0, 16.0]	25.8 [11.9, 39.7]	6 [3, 10]
Claude Opus, R1	18 [13, 24]	36 [17, 57]	0.0 [0.0, 0.0]	4.5 [0.0, 15.8]	–

Appendix

Table 12: Point estimates and 95% bootstrap intervals for Table 3 .

Claude Sonnet 5
	GPT-5.5		GPT-5.6-sol		GPT-6-astra		Opus 5.5
	P1	P5	P1	P5	P1	P5	P1	P5
Soundness	4.91	6.71	5.01	6.35	5.04	7.49	4.96	5.72
Contribution	4.61	5.66	4.52	5.25	4.61	5.62	4.59	5.07
Clarity	5.10	6.74	5.32	6.35	5.35	6.96	5.29	6.24
Overall	4.77	6.20	4.87	5.89	4.86	6.47	4.85	5.56

Appendix

Table 13: Mean scores (1–10) from each reviewer model for P1 and P5 rewrites.

	GPT-5.5		GPT-5.6-sol		GPT-6-astra		Opus 5.5
	P1	P5	P1	P5	P1	P5	P1	P5
Overclaim	0.79	0.56	0.78	0.59	0.73	0.31	0.79	0.77
Missing detail	0.89	0.91	0.91	0.91	0.86	0.85	0.92	0.93
Unclear	0.61	0.52	0.58	0.54	0.59	0.42	0.51	0.46
Shallow	0.19	0.16	0.15	0.19	0.20	0.20	0.20	0.21
Significance	0.41	0.37	0.45	0.43	0.44	0.60	0.43	0.36

Appendix

Table 14: Weaknesses per review in each category, averaged over the two reviewer models.

Panel of Figure 5 — Figure 5: Rewrites under the default prompt, one paragraph of each type. […]: omitted sentences. ④ : generic scope, with the scoping phrase underlined. ① : not-shown statement, in bold. Italic: the authors’ claim that the claim tracker labels retracted. [CITE] and [REF] are placeholders in the model input.

Explore similar work

No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions
Jun 11, 2026Xu Yang, Zhizhou Sha, Junbo Li +10Adversarial Attacks Automated Peer Review
University of Texas at Austin · University of Illinois Urbana-Champaign · University of Texas at Dallas +1
Beating the Style Detector: Three Hours of Agentic Research on the AI-Text Arms Race
May 4, 2026Andreas Maier, Moritz Zaiss, Siming BayerAI-Generated Text Detection Language Model Generation Evaluation
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Institute of Neuroradiology, University Hospital Erlangen, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
Gaming AI-Assisted Peer Reviews Poses New Risks to the Scientific Community
Jun 8, 2026Lin Li, Qi Zhang, Xander Davies +2Adversarial Attacks Adversarial Attacks on LLMs
OATML, University of Oxford · MBZUAI

Jun 11, 2026, cs.CLJ/K move · Enter open · S save

No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions

Xu Yang, Zhizhou Sha, Junbo Li, Jian Yu+9

University of Texas at Austin · University of Illinois Urbana-Champaign · University of Texas at Dallas+1

Adversarial Attacks Automated Peer Review Adversarial Text Attacks

Open paper AskPDF arXiv

As AI-generated reviews move from experimental tools into peer-review infrastructure, most robustness concerns have focused on explicit attacks such as hidden instructions and prompt injection. We study a harder and more policy-relevant failure mode: no hidden text, no prompt injection, and no changes to methods, experiments, figures, equations, proofs, or numerical results. The attacker modifies only presentation-level content, such as the abstract, contribution framing, related work, discussion, and narrative structure. We introduce adversarial repackaging: a closed-loop attack that uses AI-reviewer feedback to search for presentation-level revisions while keeping the scientific evidence fixed. Across three mainstream AI reviewers, adversarial repackaging achieves a 75.1% attack success rate and a mean score gain of +1.21/10. The effect is not explained by ordinary prose polishing. We also reveal that strategies that change how the reviewer interprets the paper, such as related-work repositioning and analytical discussion expansion, substantially outperform surface edits such as local polishing, table formatting, and algorithm boxes. Our analysis reveals two deeper structural failure modes. First, AI reviewers are easier to impress than to convince: highlighting strengths reliably increases perceived merit, while attempts to dissolve weaknesses frequently backfire. Second, AI reviewers can confuse the appearance of addressing a limitation with actually resolving it, allowing unchanged evidence to be reinterpreted as stronger scientific contribution. These results show that the deployment risk is not only malicious hidden instructions, but the emergence of paper presentation itself as an optimization surface. We release a contamination-free rolling benchmark and attack framework for testing whether AI reviewers remain anchored to scientific content under presentation-only edits.

Abstract

Explore similar work

No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only Revisions

Beating the Style Detector: Three Hours of Agentic Research on the AI-Text Arms Race

Gaming AI-Assisted Peer Reviews Poses New Risks to the Scientific Community