ChatGPT Images 2.5 on Forgery Tasks: Testing Advertised Improvements Against Known Answers

Ankit Raj; Yuxin Zhang; Kidus Zewde; Tommy Duong; Jiaqi Gan; Xingyu Shen; Yuchen Zhou; Huaiyu Guo

cs.CVSep 12, 2026

ChatGPT Images 2.5 on Forgery Tasks: Testing Advertised Improvements Against Known Answers

Authors: Ankit Raj, Yuxin Zhang, Kidus Zewde, Tommy Duong, Jiaqi Gan, Xingyu Shen, Yuchen Zhou, Huaiyu Guo,

Abstract

OpenAI released ChatGPT Images 2.5 on 8 September 2026, advertising more precise local edits, better consistency across edits, more faithful reference products and sharper detail. We evaluate these claims on four forgery tasks with answers fixed in advance: receipt-field alteration, repeated editing, product placement and small-print rendering. GPT-Image-2 provides same-week baselines at a cheaper and a more expensive tier. A limited improvement appears in receipt editing. After alignment, OCR detects changes to surrounding text in 31.7% of Flare outputs, against 44.2% for the cheaper baseline. This gain is concentrated on CORD receipts and sensitive to shifts of a pixel or less; the forged value itself is no more often correct. Repeated editing and fine print show no measurable gain. Product codes become more legible mainly because Images 2.5 draws the product larger. Defence outcomes change little: localisation remains weak for both generations. A detector that flags 68.6% of controlled benchmark images flags only 35.9% of images posted online. Advertised improvements therefore transfer unevenly to the tested forgery capabilities, while substantial detection limitations remain.

Figures & tables

Figure 1: Known-answer tasks and measurement layers. Each panel shows the predetermined answer, a Flare output and the automatic check. We measure edit correctness (L1, capability), preservation of review-relevant content (L2, an automated review check), and refusal or forensic detection (L3, defence). Inputs and prompts are matched within each task; E1 also uses a matched mask. On E1, DocTamper localises edited pixels and TruFor tests sensitivity to the paste-back seam. Community Forensics classifies whole images. These examples explain the method; results appear in § 4 (L1–L2) and § 5 (L3).

Figure 2: Capability outcomes across the four tasks. Flare: blue circles; Sunburst: red triangles; GPT-Image-2: green squares. Whiskers show 95% intervals;

*

marks a Holm-corrected difference from GPT-Image-2. Grey labels summarise the findings, with qualifications discussed in this section. (a) Surrounding-text changes after one receipt edit; dashed lines show calibration false-change rates. (b) Receipt edits still readable after 1, 2 or 3 further turns. The narrow right-hand axes show final-turn survival, the primary endpoint. Filled markers and solid lines denote CORD; hollow markers and dashed lines denote WildReceipt. (c) Photo edits at the final turn: retained in place, retained elsewhere, or lost, summing to 100%. Fresh requests are primary; chat sessions are exploratory and carry no significance markers. (d) Exact product-code reads by glyph height. The histogram gives sample counts; the dashed OCR ceiling marks the limit below which failures cannot be attributed to the generator. (e) Exact small-print reads by quality tier, an exploratory within-model comparison. Tier names are not price-matched: GPT-Image-2 medium costs as much as Images 2.5 high . The primary equal-cost contrasts are in Table 1 .

Claim $\to$	Exp.	Layer	Metric (unit, n)	GPT-Image-2		Flare	Sunburst	$\Delta$ Images 2.5 $-$ GPT-Image-2 [95% CI]
fraud task								Flare		Sunburst
Precise local editing $\to$ alter one field on a receipt	E1	L1	Target value read (%)	low	67.7	71.9	77.1	$+4.2$	[ $\mbox{\textendash}5.2$ , $14.6$ ]	$+9.4$	[ $0.0$ , $19.8$ ]
			n = 96 readable fields	med.	74.0			$\mbox{\textendash}2.1$	[ $\mbox{\textendash}12.5$ , $8.3$ ]	$+3.1$	[ $\mbox{\textendash}7.3$ , $13.5$ ]
		L2	Other text changed, aligned (%)	low	44.2	31.7	31.2	$\mbox{\textendash}12.6$ ∗ †	[ $\mbox{\textendash}19.1$ , $\mbox{\textendash}6.5$ ]	$\mbox{\textendash}13.1$ ∗ †	[ $\mbox{\textendash}19.6$ , $\mbox{\textendash}6.5$ ]
			n = 199; calibration 12.1	med.	44.2			$\mbox{\textendash}12.6$ ∗	[ $\mbox{\textendash}20.1$ , $\mbox{\textendash}5.5$ ]	$\mbox{\textendash}13.1$ ∗ †	[ $\mbox{\textendash}20.6$ , $\mbox{\textendash}6.0$ ]
Multi-turn consistency $\to$ keep editing the receipt	E2a	L1	Edit success, CORD (%)	med.	92	75	87	$\mbox{\textendash}17$	[ $\mbox{\textendash}28$ , $\mbox{\textendash}8$ ]	$\mbox{\textendash}5$	[ $\mbox{\textendash}13$ , $3$ ]

Table 1: Task outcomes and matched GPT-Image-2 comparisons. Flare and Sunburst are the Images 2.5 models. L1 measures task correctness; L2 is an automated preservation proxy, not human acceptance. Interpretation and task-specific limitations are in § 4 .

\Delta

: Images 2.5

-

GPT-Image-2 in pp (label area: ratio), unadjusted 95% bootstrap CI clustered by specification, receipt, source photo, reference or prompt. low , med. , high : GPT-Image-2 tiers. E1–E3 compare Images 2.5 medium with GPT-Image-2 low and medium , which cost less and four times more (§ 3.1 ); the low runs of E2a, E2b and E3 were added post hoc (Appendix C.2 ). E4 compares settings of equal cost: GPT-Image-2 med. with Images 2.5 high , and GPT-Image-2 high with Images 2.5 max . Rows marked expl. are exploratory; every other

\Delta

is a primary endpoint. ∗ significant after Holm correction within the claim; † also after correction across all 56 primary tests (Appendix C ). WildReceipt and conversational chains: Table 4 ; L3: Table 5 .

Figure 3: Defence outcomes. Panel subtitles summarise Table 5 . (a) First-pass refusal rates with Wilson 95% intervals. The hatched band is the earlier GPT-Image-2 result [ 32 ] . (b) DocTamper’s per-image localisation AUC; chance is 0.5. (c) Community Forensics whole-image AUC against real photographs, by quality setting. (d) Flag rates at a 5% false-positive rate. Controlled cells show their mean and min–max range; posted images show a Wilson 95% interval.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Out. tokens	$/image	Images 2.5 tier	GPT-Image-2 tier	Flare s	Sunburst s	GPT-Image-2 s
196	0.006	low	low	$13.4{}\pm 3.9{}$	$16.7{}\pm 2.7{}$	$21.2{}\pm 5.7{}$
439	0.013	medium	—	$16.3{}\pm 6.3{}$	$20.7{}\pm 2.5{}$	—
1,756	0.053	high	medium	$21.4{}\pm 2.9{}$	$37.1{}\pm 4.5{}$	$51.9{}\pm 5.2{}$
3,122	0.094	xhigh	—	$31.6{}\pm 3.3{}$	$58.5{}\pm 7.8{}$	—
7,024	0.211	max	high	$51.7{}\pm 2.7{}$	$105.9{}\pm 12.9{}$	$151.3{}\pm 14.7{}$
3,336	0.100	4K high	—	29.7	38.9	—

Appendix

Table 2: Measured tier grid at 1024

\times

1024 (5 calls per cell, direct OpenAI API), with the 4K rows at their billed token counts. Output tokens are deterministic per tier and identical across models at 1024 px, so one row is one price and GPT-Image-2’s tiers are printed on the rows whose token count they match. 1024-px latency is the smoke-grid mean

\pm

SD; 4K latency is the mean over the photoreal-core 4K cells. The 4K rows bill about 1.9

\times

the 1024 high price of the same family.

				Batch/sync rows
Cell	Claim, layer	Units	Ref.	Flare	Sunburst	GPT-Image-2	GPT-Image-2 low
smoke	tiers, cost, latency	65 calls	—	sync
E1 documents	local editing, L1–L3	1,996 edits	4	sync
GPT-Image-2 control	E1 control, L1–L3	399 edits	1	sync
E2a receipt chains	multi-turn, L1–L2	60 receipts	0	228/12	228/12	228/12	240/0
E2b photo chains, stateless	multi-turn, L1–L2	157 chains	6	36/200	216/18	216/18	237/3

Appendix

Table 3: Benchmark cells. E1 counts are distinct requests by latest status (the documents rows print persistent refusals; first-pass counts are in § 5.1 ). E2–E4 give the analysed units; the E2b refusals are chain turns blocked by moderation, all on one seed. Route columns: accepted ledger rows sent through the Batch API / synchronously, per model and configuration; the GPT-Image-2 low configuration is the baseline added after review, which ran about 11 hours after the others. E4 Images 2.5 counts pool the five quality tiers; its GPT-Image-2 column is the high baseline (the medium and low tiers ran 42/2 and 42/2).

Claim $\to$	Exp.	Layer	Metric (unit, n)	GPT-Image-2		Flare	Sunburst	$\Delta$ Images 2.5 $-$ GPT-Image-2 [95% CI]
fraud task								Flare		Sunburst
Precise local editing $\to$ alter one field on a receipt	E1	L2	Numeric text changed, v1 coordinates	low	22.2	18.8	15.3	$\mbox{\textendash}3.5$	[ $\mbox{\textendash}11.1$ , $4.2$ ]	$\mbox{\textendash}6.9$	[ $\mbox{\textendash}15.3$ , $0.7$ ]
			(%, n = 199)	med.	31.9			$\mbox{\textendash}13.2$	[ $\mbox{\textendash}21.5$ , $\mbox{\textendash}5.6$ ]	$\mbox{\textendash}16.7$	[ $\mbox{\textendash}25.0$ , $\mbox{\textendash}7.6$ ]
			Tolerant locality, median	low	3.3	2.5	1.6	$\mbox{\textendash}0.48$	[ $\mbox{\textendash}0.92$ , $\mbox{\textendash}0.29$ ]	$\mbox{\textendash}1.4$	[ $\mbox{\textendash}1.7$ , $\mbox{\textendash}1.1$ ]
			(% crop, n = 192; floor 0.15)	med.	3.9			$\mbox{\textendash}1.1$	[ $\mbox{\textendash}1.6$ , $\mbox{\textendash}0.6$ ]	$\mbox{\textendash}1.9$	[ $\mbox{\textendash}2.3$ , $\mbox{\textendash}1.4$ ]
Multi-turn consistency $\to$ keep editing the receipt	E2a	L1	Edit success, WildReceipt (%)	med.	91	82	91	$\mbox{\textendash}9$	[ $\mbox{\textendash}18$ , $\mbox{\textendash}2$ ]	$\mbox{\textendash}1$	[ $\mbox{\textendash}8$ , $6$ ]

Appendix

Table 4: Secondary capability rows, WildReceipt and conversational chains, continuing Table 1 . Columns and marks as in Table 1 . E2a WildReceipt rows are primary endpoints; other rows are exploratory. Conversational E2b was scored before its analysis freeze. Older chains start from each model’s own seed. Defence results are in Table 5 .

Claim $\to$	Exp.	Layer	Metric (unit, n)	GPT-Image-2		Flare	Sunburst	$\Delta$ Images 2.5 $-$ GPT-Image-2 [95% CI]
fraud task								Flare		Sunburst
Faster on the forgery task	E1	L3	Median s per edit; $\Delta$ GPT-Image-2 / Images 2.5	med.	30.9	10.8	13.5	$\times$ 2.75	[2.59, 2.86]	$\times$ 2.26	[2.20, 2.31]
			(n = 199–200 specs)	low	13.0			$\times$ 1.19	[1.15, 1.21]	$\times$ 0.97	[0.94, 0.99]
Refusal	E1	L3	Refused of 200 shared specs	low 1, med. 0		1	0
			exact McNemar $p$ vs low , med.					1, 1		1, 1
Localisation	E1	L3	DocTamper per-image AUC	low	0.613	0.588	0.599

Appendix

Table 5: Defence and receipt-edit latency. Columns as in Table 1 . Speed: ratio of median synchronous latency, with 95% intervals clustered by specification; arms were not interleaved. Refusal: first-pass counts on shared specifications. DocTamper: mean per-image localisation AUC; the margin is the smallest symmetric band around chance containing the arm’s 90% interval. Tier results do not establish equivalence. Wild membership is self-reported; detection rates use one fixed threshold and are not content-matched. Detection-rate intervals are reported in § 5.4 .

Contrast	all rows	$\leq$ 2 px	$\leq$ 1 px	$\leq$ 1 px	$\leq$ 1 px	$n$
	Registered	Registered	Registered	Old token set	Unregistered
Flare $-$ low	$-12.6$ % ( $1.5\times 10^{-4}$ )	$-14.4$ % ( $3.5\times 10^{-4}$ )	$-16.7$ % (0.0019)	$-5.6$ % (0.46)	$-4.2$ % (0.63)	72
Flare $-$ medium	$-12.6$ % (0.0015)	$-15.6$ % (0.011)	$-16.7$ % (0.07)	$-2.8$ % (1)	$-5.6$ % (0.73)	36
Sunburst $-$ low	$-13.1$ % ( $2.0\times 10^{-4}$ )	$-12.2$ % (0.0011)	$-17.9$ % (0.002)	$-10.4$ % (0.09)	$-10.4$ % (0.12)	67
Sunburst $-$ medium	$-13.1$ % ( $8.5\times 10^{-4}$ )	$-9.0$ % (0.14)	$-14.7$ % (0.062)	$-5.9$ % (0.5)	$-8.8$ % (0.25)	34

Appendix

Table 6: E1 other text changed, Images 2.5

-

GPT-Image-2 in pp (unadjusted permutation

p

), with both arms restricted by locality registration shift. Registered: tokens re-read at registered coordinates, the primary metric. Old token set: the unregistered analysis’s stable tokens re-read at registered coordinates. Unregistered: tokens read at identical coordinates, without registration.

n

: specifications with both arms at

\leq

1 px.

Arm	$n$	Tolerant median [95% CI]	Pixel-exact	Inside	Shift (px)	Registered	Excl.
GPT-Image-2 low	192	3.3% [3.0%, 3.9%]	11.7%	25.0%	0.88	97%	7
GPT-Image-2 medium	193	3.9% [3.3%, 4.6%]	15.2%	24.5%	2.22	98%	7
Flare	950	2.5% [2.3%, 2.7%]	10.5%	25.1%	0.66	96%	49
Sunburst	948	1.6% [1.4%, 1.7%]	8.6%	22.3%	0.75	94%	49
Resampling floor	999	0.15% [0.12%, 0.22%]	7.3%	—	—	—	—

Appendix

Table 7: E1 edit locality, context-crop denominator: medians over specifications with 95% bootstrap intervals clustered by specification. “Tolerant” is the share of off-field crop pixels changed beyond 12/255 after registration and a

\pm

1 px tolerance; “pixel-exact” is the superseded metric on the same rows, shown only for reconciliation; “inside” is the tolerant share within the field; “shift” is the median registration displacement. Size-mismatched rows are excluded (

n

excluded in the last column).

Figure 4: Three authentic–edited pairs of context crops from AIForge-Doc round v3, produced by gpt-image-2.5-flare at medium for a median of $0.0111 and 12.3 s each. In each pair one panel is authentic and one has a single field edited; which panel varies by row and is not marked. Key in Appendix D.6 . The pairs illustrate the edits; they are not a human discrimination test.

Variant	Model	Chains	Success	Present [95% CI]	In place [95% CI]	Relocated	Drift	Strength
Stateless	GPT-Image-2 med.	39	93.2%	95.6% [92.7%, 98.2%]	98.3% [95.9%, 100.0%]	1.6%	18.0%	44.3%
	GPT-Image-2 low	40	—	94.6% [91.4%, 97.7%]	98.9% [97.4%, 100.0%]	—	—	—
	Flare	39	94.0%	94.1% [90.5%, 97.3%]	100.0% [100.0%, 100.0%]	0.0%	25.6%	41.4%
	Sunburst	39	93.2%	95.6% [92.6%, 98.0%]	100.0% [100.0%, 100.0%]	0.0%	15.7%	39.6%
Conversational	GPT-Image-2 med.	19	88.6%	98.8% [96.4%, 100.0%]	38.1% [29.4%, 47.6%]	61.2%	22.2%	49.1%
	Flare	19	86.0%	95.0% [88.5%, 100.0%]	48.7% [39.2%, 57.7%]	48.8%	5.5%	43.7%

Appendix

Table 8: E2b endpoints per variant and model, CIs clustered by source photo. Success: edit detected at its own turn. Present: among successes, present anywhere at the last turn (L1 survival; lost is its complement). In place: among survivors, still in place (L2). Relocated: share of successes present only elsewhere. Drift: changed share outside all edit boxes, final turn against

t_{0}

; strength: changed share inside the edit boxes. The GPT-Image-2 low arm, added post hoc, was scored on the primary endpoints only. Primary contrasts are in Table 1 ; the conversational variant is exploratory.

Corpus	Endpoint	GPT-Image-2 medium	GPT-Image-2 low	Flare	Sunburst
CORD	success	92% (106/115)	94% (108/115)	75% (86/115)	87% (100/115)
	survival	98% (80/82)	94% (75/80)	89% (56/63)	97% (74/76)
	joint	93% (80/86)	87% (75/86)	65% (56/86)	86% (74/86)
	never-edited kept	93% (84/90)	97% (87/90)	82% (74/90)	94% (85/90)
WildReceipt	success	91% (107/117)	80% (94/117)	82% (96/117)	91% (106/117)
	survival	96% (81/84)	94% (68/72)	97% (71/73)	98% (78/80)

Appendix

Table 9: E2a endpoints by arm: rate (count). Success: exact target on its own line at its own turn. Survival: among edits that succeeded at turns 1–3, still read at the last turn. Joint: success and survival together, without conditioning. Never-edited kept: control fields still read their original value at the last turn. GPT-Image-2 low was added post hoc.

	Metric	GPT-Image-2 med.	GPT-Image-2 low	Flare	Sunburst
In scene	Area	2.4%	2.7%	4.1%	3.9%
	Width	14%	14%	18%	18%
	Wider	85%	94%	100%	100%
	Rel. px	17.3	17.4	17.1	17.0
	IoU	0.908	0.904	0.902	0.932
Hero	Area	12.3%	12.0%	12.7%	12.4%

Appendix

Table 10: E3 placement per arm. Area: label share of the frame. Width: product width as a share of the frame. Wider: share of outputs wider than a tenth of the frame. Rel. px: SKU glyph height per unit label scale. IoU: logo mask IoU. Medians except for shares.

Arm (tokens)	Above ceiling	Intention to treat	No line
GPT-Image-2 medium (1,756)	88% [78%, 96%]	28% [22%, 35%]	6 of 40
Flare high (1,756)	91% [85%, 95%]	63% [58%, 68%]	0 of 40
Sunburst high (1,756)	83% [73%, 90%]	37% [31%, 42%]	0 of 40
GPT-Image-2 high (7,024)	82% [71%, 91%]	23% [18%, 29%]	5 of 40
Flare max (7,024)	95% [92%, 98%]	67% [60%, 74%]	1 of 40
Sunburst max (7,024)	82% [73%, 90%]	30% [26%, 35%]	0 of 40

Appendix

Table 11: E4 per arm: exact share of code lines above the OCR ceiling and the intention-to-treat share of all planned lines (sub-ceiling lines failed), with 95% intervals clustered by prompt, and images with no line above the ceiling.

Round (arm)	Generator	Route	Mask	Prompt	Run	Attempts	Refused (95% CI)	Isolates
v2	GPT-Image-2	gateway	drawn box	5 clauses	Apr 2026	4,062	$\sim$ 10%	—
v3 (Flare arm)	2.5-flare	direct	alpha	4 sentences	Sep 2026	1,000	0.40% [0.16%, 1.02%]	—
v3 (Sunburst arm)	2.5-sunburst	direct	alpha	4 sentences	Sep 2026	1,000	0.30% [0.10%, 0.88%]	—
v3 (control arm)	GPT-Image-2	direct	alpha	4 sentences	Sep 2026	400	0.25% [0.04%, 1.40%]	generation

Appendix

Table 12: Refusal behaviour on the same specification catalogue, with the axes on which the measurements differ. Round v2 is quoted from [ 32 ] ; round-v3 rows are first-pass rates measured here with Wilson 95% intervals (persistent rates in the text). The control row runs round v2’s generator, GPT-Image-2, through the round-v3 request in the same week, so route, mask, prompt, corpora and date are fixed and only the generation moves. Its rate is of the same order as the Images 2.5 arms; the comparison cannot resolve small differences. Date and request format remain confounded with each other.

Figure 5: Per-image DocTamper localisation (pixel) AUC by AIForge-Doc round, paired over the 30 specifications scored in all three rounds, with 95% intervals; dotted line at chance. These round-v3 values are the paired subset, not the full-arm values of § 5.2 : the denominators differ, the two arms’ order reverses between them, and the Sunburst interval contains 0.5. Suggestive only.

DocTamper	AUC [90% CI]	Margin	$\pm$ 0.05	$\pm$ 0.10
Flare	0.588 [0.578, 0.597]	0.097	no	yes
Sunburst	0.599 [0.588, 0.610]	0.110	no	no
GPT-Image-2 low	0.613 [0.590, 0.635]	0.135	no	no
GPT-Image-2 med.	0.615 [0.592, 0.636]	0.136	no	no
Flare $-$ low	$-0.023$ [ $-0.044$ , $-0.002$ ]	0.044
Flare $-$ med.	$-0.025$ [ $-0.044$ , $-0.005$ ]	0.044

Appendix

Table 13: Equivalence tests. Left: DocTamper per-image localisation AUC against chance per arm, and paired Images 2.5

-

GPT-Image-2 differences against zero, with 90% intervals clustered by specification. Right: Community Forensics AUC across price tiers; range is the largest minus smallest cell AUC, with its bootstrap upper bound. Margin: the smallest symmetric margin the test supports.

Out. tokens	$/image	GPT-Image-2	Flare	Sunburst
196	0.006	0.881 [0.852, 0.909] (21.2 s)	0.903 [0.878, 0.927] (13.4 s)	0.893 [0.864, 0.919] (16.7 s)
439	0.013	—	0.890 [0.861, 0.917] (16.3 s)	0.918 [0.894, 0.939] (20.7 s)
1,756	0.053	0.880 [0.850, 0.908] (51.9 s)	0.888 [0.859, 0.914] (21.4 s)	0.917 [0.895, 0.937] (37.1 s)
3,122	0.094	—	0.889 [0.861, 0.915] (31.6 s)	0.914 [0.889, 0.936] (58.5 s)
7,024	0.211	0.897 [0.869, 0.921] (151.3 s)	0.916 [0.886, 0.941] (51.7 s)	0.935 [0.912, 0.958] (105.9 s)
4K high		0.898 [0.850, 0.940]	0.823 [0.762, 0.873]	0.775 [0.714, 0.832]

Appendix

Table 14: Community Forensics image-level AUC on the photoreal core by billed output tokens (price proxy) and model, with 95% bootstrap intervals and smoke-grid mean latency in parentheses. The negative class is 544 real photographs from IMD2020 and Columbia, JPEG and uncompressed TIFF against PNG generations, so part of any AUC may be format rather than generator. The 4K row is 3840

\times

2160 at high , billed at about 1.9

\times

the 1024 high price (Table 2 ); its cells hold 20 prompts and are not prompt-matched to the 1024 cells. Equivalence tests are in Appendix I.3 .

Task	Control	$n$	GPT-Image-2 s	Flare s	Sunburst s	Ratio, Flare	Ratio, Sunburst
E1 edits	low	199 (199)	13.0	10.8	13.5	1.19 [1.15, 1.21]	0.97 [0.94, 0.99]
E1 edits	medium	200 (200)	30.9	10.8	13.5	2.75 [2.59, 2.86]	2.26 [2.20, 2.31]
E2a receipt chains	medium	12 (3)	31.6	11.7	15.5	2.86 [2.63, 3.15]	2.03 [1.91, 2.21]
E2b stateless	medium	18 (3)	39.3	13.3	19.1	2.96 [2.85, 3.13]	2.13 [2.05, 2.21]
E2b conversational	medium	115 (20)	41.2	16.0	21.2	2.57 [2.56, 2.64]	1.97 [1.93, 2.00]
E3 references	medium	9 (6)	40.4	12.9	19.0	3.15 [2.83, 3.42]	2.17 [1.97, 2.18]

Appendix

Table 15: Forgery-task latency from synchronous rows: median seconds per accepted call and the GPT-Image-2 / Images 2.5 ratio of medians with 95% bootstrap intervals.

n

: paired units (clusters).

Figure 6: Community Forensics score against turn index, all three arms, means with 95% intervals over 50 complete eight-turn chains per arm at turns 1/3/5/8. GPT-Image-2’s mean rises from turn 1 to turn 8, but the rise is not significant after Holm correction across the three arms; both Images 2.5 arms are flat within their intervals. Turn-1 levels do not differ reliably between arms, and each arm starts from its own seed.

Component	Version
Image models	gpt-image-2 , gpt-image-2.5-flare , gpt-image-2.5-sunburst
Relay (E2b conv.)	gpt-5.4-nano-2026-03-17
OCR	easyocr 1.7.2, torch 2.10.0+cu128, English, GPU; E4 mag_ratio=1.0 ; allowlist ACDEFHKLMNPQRTUWXYZ
Grounding (E2b)	OWLv2 google/owlv2-base-patch16-ensemble
TruFor	trufor.pth.tar , grip-unina/TruFor release v1.0
DocTamper	released checkpoint dtd_doctamper

Appendix

Table 16: Model and instrument versions. The image API returns no dated snapshot; the call window is in the ledger.

Image Editing Evaluation

Image editing evaluation measures whether edits follow instructions while preserving identity, context, and visual quality.

See all 92 papers

Apr 28, 2026

When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents

OpenAI's GPT-Image-2 has effectively erased the visual boundary between authentic and AI-edited document images: a single number on a receipt can be replaced in under a second for a few cents. We release AIForge-Doc v2, a paired dataset of 3,066 GPT-Image-2 document forgeries with pixel-precise masks in DocTamper-compatible format, and benchmark four lines of defence: human inspectors (N=120, n=365 pair-votes via the public 2AFC site CanUSpotAI.com), TruFor (generic forensic), DocTamper (qcf-568, document-specific), and the same GPT-Image-2 model as a zero-shot self-judge -- asked, to avoid the trivial "image is mostly real" reading, whether any region was generated or edited by an AI image model. Human 2AFC accuracy is 0.501, indistinguishable from chance: even side-by-side, inspectors cannot tell GPT-Image-2 receipt forgeries from authentic counterparts. The three computational judges sit only modestly above (TruFor 0.599, DocTamper 0.585, self-judge 0.532). The self-judge fails consistently, not by chance: across five prompt strategies and four policies for handling ambiguous responses, AUC never rises above 0.59. To rule out the possibility that the two forensic detectors are broken on our source domain rather than blind to AI inpainting, we calibrate each on a same-domain traditional-tampering set built for its training distribution: TruFor reaches AUC 0.962 on cross-camera splicing of our dataset, DocTamper reaches 0.852 on cross-document OCR-token splicing with two-pass JPEG re-encoding. Both retain near-published performance on traditional tampering; switching to GPT-Image-2 inpainting drops AUC by 0.27-0.36 (0.962->0.599 TruFor; 0.852->0.585 DocTamper), isolating a detection gap specific to GPT-Image-2 inpainting. We release the dataset, pipeline, four-judge protocol, and calibration sets.

Jiaqi Wu, Yuchen Zhou, Dennis Tsang Ng +5

arXiv:2604.25213v1

Image Editing Evaluation Image Forgery Detection AI-Generated Image Detection

Ask PDF

Sep 14, 2026

ChatGPT Images 2.5 in the Wild: A Launch-Period Dataset and Detector Evaluation

An image tool can change its underlying generator while retaining its public name, making version attribution from online posts ambiguous. We study this problem after the ChatGPT Images 2.5 launch. Our frozen collection contains 3,478 images from 2,440 posts across 8 sources. Recorded posting times fall within the first 51.1 hours after the announcement. It records three attribution tiers and retains standalone images after image-form filtering and targeted review. Caption claims and host records provide admission evidence, not independently verified generator identity. The observed content profile depends on the source mixture: NightCafe supplies 39.0% of images but 77.0% of CLIP-assigned fantasy scenes. We then evaluate six frozen detectors at thresholds calibrated to a 5% flag rate on reference photographs. Collection flag rates range from 3.7 to 56.4%, falling 42-81 percentage points below GenImage recall. Held-out artwork false-positive rates range from 1.5 to 96.5%, so a higher collection flag rate does not by itself establish better detection. An exploratory X-only comparison with our April collection finds a higher September flag rate for Effort, and a suggestive difference for DoU, under fixed-threshold post-clustered bootstrap intervals. Attribution, content and processing differences prevent a causal interpretation of these contrasts. The collection supports analysis of reported model use during a product transition, with source and attribution evidence retained for interpretation. The collection is released at https://scam.ai/research.

Dennis Ng, Xingyu Shen, Ankit Raj +6

AI-Generated Image Detection Generative AI Evaluation

Ask PDF

May 31, 2026

TextFake: Benchmarking AI-Generated Image Detection on Text-Rich Images

Recent AI-generated image (AIGI) detectors perform well on natural-image benchmarks, but their behavior on text-rich forgeries, such as fabricated screenshots, documents, and news pages prevalent in misinformation, remains untested. We introduce TextFake, a 20,000-image benchmark for text-rich AIGI detection spanning 28 languages, 4 topic categories, and 2 scene modalities. Fake images are synthesized via a four-stage pipeline that annotates real images along three controlled dimensions and generates counterparts through distribution-aligned structured prompting, ruling out covariate shortcuts. Zero-shot evaluation of 14 specialized detectors and 3 frontier VLM APIs reveals a large systematic gap: no method exceeds 80% accuracy, with some dropping over 60% from natural-image benchmarks. Diagnostic evaluations identify three failure modes: the Text Density Curse, where dense glyphs overwhelm low-level detectors; Cloaking via Rendering Fidelity, where stronger text rendering suppresses enerative artifacts; and Threshold Collapse, where routine perturbations drive detectors toward chance-level performance.

Yuning Zhang, Changtao Miao, Mingyu Liao +5

School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Individual Researcher

AI-Generated Image Detection Adversarial Attacks on AI-Generated Image Detectors

Ask PDF

Abstract

Image Editing Evaluation

Explore similar work

When the Forger Is the Judge: GPT-Image-2 Cannot Recognize Its Own Faked Documents

ChatGPT Images 2.5 in the Wild: A Launch-Period Dataset and Detector Evaluation

TextFake: Benchmarking AI-Generated Image Detection on Text-Rich Images