Mandela-Bench: Multimodal Models Remember Canonical Images Instead of Seeing Them

Yicheng Bao; Zhenkun Gao; Xiahui Guo; Mingqian Yang; Xueheng Li; Bangwei Liu; Mingang Chen; Lijun Li

cs.AISep 26, 2026

Mandela-Bench: Multimodal Models Remember Canonical Images Instead of Seeing Them

Authors: Yicheng Bao, Zhenkun Gao, Xiahui Guo, Mingqian Yang, Xueheng Li, Bangwei Liu, Mingang Chen, Lijun Li,

Organizations: East China Normal University · University of Science and Technology of China · Shanghai Development Center of Computer Software Technology · Shanghai Artificial Intelligence Laboratory

Abstract

Historical photographs and other canonical images can now be edited seamlessly with a single instruction, often leaving no reliable pixel-level trace. In such cases, the only evidence of manipulation may be a fact about what the image depicts. Existing benchmarks instead rely on generator artefacts, image-caption inconsistencies, visual implausibilities, or external references, and therefore do not test whether a model can use its own world knowledge to verify a recognized image. We introduce Mandela-Bench, containing 1,507 edits of canonical images: 1,359 knowledge-only forgeries, each contradicting one verifiable fact, and 148 anchor-free controls that preserve the editing process without introducing a factual contradiction, together with 474 untouched originals. We score not only whether a model detects a forgery, but whether its explanation identifies the inserted entity or the fact being violated. Across 36 multimodal models, from 0.8B parameters to frontier scale, we find a consistent failure mode. When a public figure is removed from a familiar photograph, models still name that person in up to 72.7% of responses. Some models can distinguish the replacement face from the original when shown in isolation, yet still judge the full edited photograph as authentic. Providing the true event and date does not improve knowledge-grounded detection, whereas providing the same information after cropping away the recognizable composition does. Even under explicit verification prompts, only one of the 36 models meets the KGR criterion on at least half of the forged images. These results suggest that the failures cannot be explained by missing knowledge or inadequate perception alone. Instead, they are consistent with recognition biasing verification toward the remembered canonical image rather than the observed edit.

Figures & tables

Figure 1: A canonical photograph of John F. Kennedy with his face replaced by a synthetic one. The model rejects the replacement face in isolation, yet infers Kennedy from the full scene and judges the edited photograph authentic.

K1 People	K5 Event attribute
Karl May (died in 1912) inserted	Zidane’s shirt recolored blue (France wore white)
Scene: Casablanca (1942)	Scene: 2006 World Cup final
K2 Objects	K6 Artwork
Jet airliner (first flew in 1949) inserted	Extra figure added (not in the canonical composition)
Scene: McKinley campaign print (1900)	Scene: Leonardo’s The Last Supper
K3 Place	K0 Control

Table 1: Mandela-Bench item taxonomy. Full visual examples and the six-domain distribution are shown in Appendix Figure 5 .

Figure 2: Construction of Mandela-Bench for one 1945 Yalta Conference seed. Stage 1 anchors the seed to a knowledge-base entry. Stage 2 generates a knowledge-anchored edit and matched K0 control, automatically checks candidates, and retries failures. Stage 3 applies manual review, pixel-detector auditing, and deduplication before release. Here the first proposal failed the face check.

Figure 3: Evaluation overview. Left: 7 probes (§ 4.1 ). Middle: 5 de-recognition conditions under L1 (§ 4.2 ). Right: E0–E4 evidence ladder, with E3+ defining KGR (§ 4.3 ).

Proprietary
	Default (L0)				Prompted (L1)				L-adv	KGR by knowledge family (L0)							Derived
Model	KGR	Det	Auth	Bal	KGR	Det	Auth	Bal	Auth	K0	K1	K2	K3	K4	K5	K6	$\Delta$ KGR
Gemini-3.8-Flash	46.6	66.3	94.7	80.5	55.1	80.5	88.2	84.3	99.4	0.7	27.5	79.2	58.4	56.5	66.2	66.0	+8.5
GPT-5.6	19.8	40.9	95.8	68.3	36.0	61.0	91.6	76.3	99.2	0.0	5.8	54.8	19.1	26.9	20.7	17.9	+16.2
Open weights
GLM-5.3-Flash	29.7	53.8	92.6	73.2	34.8	60.6	85.9	73.2	99.4	2.0	7.6	62.2	40.5	55.6	40.0	41.7	+5.1

Table 2: Main results on the release set (1,507 forged, 474 authentic; strict scoring) for the 15 main-text models; all 36 are in Appendix Table 5 . KGR, Det, Auth, and Bal denote knowledge-grounded rate, forgery detection, authentic-image accuracy, and balanced accuracy. Left: L0, L1, and L-adv results. Right: L0 KGR by knowledge family. Colors show changes from the corresponding baseline (L1 and L-adv vs. L0; K1–K6 vs. K0): blue better, red worse;

\Delta

KGR = L1

-

L0 is defined as elicitation gap; † Judge model. Rows are sorted by L1 KGR.

Proprietary
	Full			Full+Caption			Crop			Crop+Caption			Absent-celebrity (L0)					Derived
Model	Det	Bal	KGR	Det	Bal	KGR	Det	Bal	KGR	Det	Bal	KGR	Erase	Swap	K0	Named	& Real	Cap.	Pen.
Gemini-3.8-Flash	73.1	78.9	35.2	77.9	75.2	33.1	79.3	76.4	46.2	86.2	79.1	44.8	53.0	71.8	2.4	74.2	32.6	+4.8	+0.2
GPT-5.6	53.8	69.2	14.5	57.2	71.5	13.1	56.6	67.3	13.1	69.7	76.8	16.6	17.4	52.6	2.4	23.5	19.7	+3.4	+7.6
Open weights
GLM-5.3-Flash	52.4	67.5	7.6	47.6	68.7	4.8	62.8	68.8	14.5	59.3	74.5	10.3	32.6	65.4	9.8	52.3	37.1	$-$ 4.8	+7.0

Table 3: People-family analyses. Det, Bal, KGR as in Table 2 . Left: 145 peripheral forgeries under L1 in the four de-recognition conditions. Right: 132 absent-celebrity items under the default probe L0. Named denotes answers naming the erased person; Real denotes naming that person while judging the image authentic. Colors show changes from Full (Erase from Swap); bold Det indicates McNemar

p<0.05

. Cap. = Det(Full+Caption)

-

Det(Full); Pen. = Bal(Crop+Caption)

-

Bal(Full). ¶ Recognizes fewer than half of the images. All models: Appendix Table 6 .

Figure 4: Results across all 36 models. (a) Prompting increases detection more than KGR. (points above

y=x

) (b) Recognition penalty is positive for models recognizing at least half of the de-recognition images and near zero otherwise (median +0.4). (c) Models that often name the erased person show larger erase–swap gaps. Wedges mark off-axis points.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Mandela-Bench examples for each knowledge family, an anchor-free control (K0), and an untouched authentic image. Center: distribution of 1,507 forged items across six image domains.

Figure 6: One seed and its derived benchmark items (top), and the de-recognition conditions applied to the knowledge-anchored forgery (bottom). Top: (a) authentic original; (b) knowledge-anchored forgery with George Orwell inserted, for which the ideal verdict is fake, grounded in the recorded factual contradiction; (c) matched anchor-free control (K0) with an unnamed face inserted at the same position by the same editor; (d) absent-celebrity item with Maradona replaced by a synthetic face. Bottom: (e) Full; (f) Full+Caption; (g) Crop; (h) Crop+Caption. The crop preserves the edited region while removing much of the recognizable composition, and the box marks the crop region. (f) and (h) receive the same event-and-date context.

Human	E0	E1	E2	E3	E4	$n$	exact	$\pm 1$
E0	46	1	2	0	2	51	90.2	92.2
E1	0	29	5	0	0	34	85.3	100.0
E2	0	9	30	0	0	39	76.9	100.0
E3	0	0	4	30	1	35	85.7	100.0
E4	0	2	2	8	35	47	74.5	91.5

Appendix

Table 4: Human label (rows) against the judge’s level (columns) on the 206 calibration answers; the right-hand columns give exact and within-one agreement by human level.

Proprietary
	Default (L0)				Prompted (L1)				L-adv	KGR by knowledge family (L0)							Derived
Model	KGR	Det	Auth	Bal	KGR	Det	Auth	Bal	Auth	K0	K1	K2	K3	K4	K5	K6	$\Delta$ KGR
Gemini-3.8-Flash	46.6	66.3	94.7	80.5	55.1	80.5	88.2	84.3	99.4	0.7	27.5	79.2	58.4	56.5	66.2	66.0	+8.5
GPT-5.6	19.8	40.9	95.8	68.3	36.0	61.0	91.6	76.3	99.2	0.0	5.8	54.8	19.1	26.9	20.7	17.9	+16.2
Open weights
GLM-5.3-Flash	29.7	53.8	92.6	73.2	34.8	60.6	85.9	73.2	99.4	2.0	7.6	62.2	40.5	55.6	40.0	41.7	+5.1

Appendix

Table 5: Table 2 for all thirty-six models (same columns and notation). Family sizes: K0 148, K1 567, K2 294, K3 89, K4 108, K5 145, K6 156. Colour saturates at a change of 20 points (L1 vs. L0), 50 (L-adv vs. L0), and 60 (families vs. K0). Rows sorted by KGR under L1 within each group.

Proprietary
	Full			Full+Caption			Crop			Crop+Caption			Absent-celebrity (L0)					Derived
Model	Det	Bal	KGR	Det	Bal	KGR	Det	Bal	KGR	Det	Bal	KGR	Erase	Swap	K0	Named	& real	Cap.	Pen.
Gemini-3.8-Flash	73.1	78.9	35.2	77.9	75.2	33.1	79.3	76.4	46.2	86.2	79.1	44.8	53.0	71.8	2.4	74.2	32.6	+4.8	+0.2
GPT-5.6	53.8	69.2	14.5	57.2	71.5	13.1	56.6	67.3	13.1	69.7	76.8	16.6	17.4	52.6	2.4	23.5	19.7	+3.4	+7.6
Open weights
GLM-5.3-Flash	52.4	67.5	7.6	47.6	68.7	4.8	62.8	68.8	14.5	59.3	74.5	10.3	32.6	65.4	9.8	52.3	37.1	$-$ 4.8	+7.0

Appendix

Table 6: Table 3 for all thirty-six models (same columns and notation; ¶ recognizes fewer than half of the images under Full). Colour saturates at a change of 25 points from Full (40 for Erase vs. Swap). Authentic accuracy under every condition is in Table 17 ; parsed-only rates in Table 18 .

Model	Full	Mirror	Crop	$n$	Model	Full	Mirror	Crop	$n$
Qwen3.5-122B-A10B	88.5	85.6	48.8	902	Qwen3-Omni-30B-A3B	66.3	64.6	33.6	902
Kimi-K3	87.8	86.9	31.3	902	Qwen3.6-35B-A3B	62.0	58.2	25.2	902
GPT-5.6	87.8	86.8	42.1	902	Qwen3.8-27B †	60.2	51.4	17.0	902
Gemini-3.8-Flash	87.4	87.5	43.2	902	Gemma-4-31B	57.5	52.7	19.4	902
Qwen3.5-35B-A3B	86.3	85.8	48.4	902	Qwen3-VL-8B	56.4	52.5	11.8	902
Qwen3.5-27B	83.6	80.5	47.1	902	Qwen3-VL-235B-A22B	54.4	46.3	13.3	902

Appendix

Table 7: Self-reported recognition (%, RECOG probe) of the 902 peripheral forgeries with a crop variant, under Full, Mirror (the full image flipped horizontally), and Crop (the edited region and its neighbours). Rows sorted by recognition under Full;

n=902

for every model (release set, complete sweeps). Detection, KGR, and authentic accuracy under the same conditions are in Table 19 .

Proprietary
	TEXT (%)	L2 (%)		L0 detection by L2 answer (%)		FACE
Model	knows	names	cannot	named	cannot	pass	full real (%)	named (%)
Gemini-3.8-Flash	83.0	68.2	12.1	53.3 (90)	62.5 (16)	81/119	29.6	49.4
GPT-5.6	80.0	38.6	42.4	17.6 (51)	16.1 (56)	4/119 r	75.0	75.0
Open weights
GLM-5.3-Flash	83.5	43.2	13.6	32.7 (55)	27.8 (18)	—	—	—

Appendix

Table 8: Text-only, identity, and face probes for all models. TEXT reports the share of 435 named-insertion forgeries for which the model, given only the name, event, and date, correctly rules out the person’s presence. The remaining columns use the 132 verified absent-celebrity items. L2 reports whether the erased person is named or whether the model says someone cannot be identified; L0 detection is then stratified by those L2 responses. FACE reports, for five models, how often the original face is accepted and the replacement rejected when shown in isolation (119 pairs), followed by full-image authenticity and naming rates on the pass set. r GPT-5.6 declines isolated-face identification, so its FACE row reflects refusal rather than perception; — indicates that the probe was not run. † Also the judge.

Proprietary
Model	pairs	Full	Crop	only Full / only Crop	$p$
Gemini-3.8-Flash	116	92.2	44.0	56/0	$<$ 0.0001
GPT-5.6 §	66	56.0	7.8	37/1	$<$ 0.0001
Open weights
GLM-5.3-Flash	95	56.0	6.9	52/1	$<$ 0.0001
Kimi-K3	116	84.5	18.1	78/1	$<$ 0.0001

Appendix

Table 9: The remembered people follow the composition. On the 116 peripheral people-family forgeries whose scene has named principal figures, the identity probe (L2) is asked on the full image and on the crop around the edited region. Full / Crop: share of items whose answer names at least one principal figure of the scene (strict denominators); only Full / only Crop: items named under one condition only;

p

: exact McNemar test. Rows in the order of Table 2 . † Also the judge.

Proprietary
	RECOG	People family (K1)					Class-level families (K2–K6)
Model	Full (%)	recognized	not	$\Delta$	OR	$p$	recognized	not	OR	$p$
Gemini-3.8-Flash	78.2	75.4 (391)	87.7 (203)	$-$ 12.2	0.67	0.1766	79.7 (788)	89.6 (125)	0.29	0.0529
GPT-5.6	76.8	37.9 (116)	63.8 (69)	$-$ 25.8	0.58	0.3597	66.7 (129)	40.0 (5)	3.36	0.0860
Open weights
GLM-5.3-Flash	74.2	60.0 (365)	56.3 (229)	+3.7	1.15	0.5612	61.9 (753)	61.9 (160)	1.30	0.5536

Appendix

Table 10: Prompted-verification detection as a function of self-reported scene recognition (RECOG under Full), shown per model on the release set with strict scoring. For the people family (K1), we report detection on recognized and unrecognized scenes, their difference, and the recognition coefficient from a logistic regression controlling for saliency, family, and domain with scene-clustered errors. The same quantities are reported for class-level families (K2–K6). Anchor-free controls are assigned to the family of their edit specification. ¶ Recognizes fewer than half of the images and is excluded from the pooled model; — denotes separated or empty cells. † Also the judge.

	Full				Full+Caption				Full+Wrong caption				$p$ vs. Full		Wrong caption
Model	Det	Auth	Bal	KGR	Det	Auth	Bal	KGR	Det	Auth	Bal	KGR	Caption	Wrong	uptake	dispute
Gemini-3.8-Flash	73.1	84.9	79.0	35.2	77.9	73.1	75.5	33.1	93.8	19.4	56.6	31.0	0.1671	$<$ 0.0001	80.8	30.1
GPT-5.6	53.8	84.9	69.4	14.5	57.2	86.0	71.6	13.1	91.0	16.1	53.6	9.7	0.4421	$<$ 0.0001	84.1	10.9

Appendix

Table 11: Wrong-caption control on the 145 peripheral people-family forgeries and 93 authentic originals under prompted verification and strict scoring. We compare Full, Full+Caption, and Full+Wrong caption, where the wrong caption names a different same-era event from the same domain. Det, Auth, Bal, and KGR are defined as in the main text;

p

is the exact McNemar test against Full. Uptake and dispute report whether the model adopts or explicitly rejects the wrong event.

Model	Fam.	$n$	F	FC	C	CC	F	FC	C	CC	$\Delta$	$p$	$p_{\mathrm{CC}}$
			Det				KGR				Cap.
Gemini-3.8-Flash	K1	145	73.1	77.9	79.3	86.2	35.2	33.1	46.2	44.8	+4.8	0.1671	0.0009
	K2	294	96.3	96.3	94.6	100.0	84.7	83.3	85.4	91.5	0.0	1.00	0.0010
	K3	89	85.4	88.8	91.0	97.8	69.7	62.9	79.8	88.8	+3.4	0.4531	0.0010
	K4	108	90.7	96.3	75.9	90.7	87.0	94.4	66.7	84.3	+5.6	0.1094	1.00
	K5	145	86.2	90.3	83.4	95.9	77.9	85.5	73.1	87.6	+4.1	0.1796	0.0013

Appendix

Table 12: Detection and KGR under the four de-recognition conditions by knowledge family for the two proprietary models (peripheral forgeries on the release set; prompted verification; strict scoring).

n

: forgeries; F, FC, C, CC: Full, Full+Caption, Crop, Crop+Caption; Cap.: Det(FC)

-

Det(F) with its exact McNemar

p

;

p_{\mathrm{CC}}

: McNemar

p

of Det(CC) against Det(F).

Proprietary
	Authentic accuracy, L1 (%)							By recognition (L1)
Model	All	A1	A2	A3	A4	A5	A6	RECOG	recognized	not	$\Delta$
Gemini-3.8-Flash	88.2	93.3	83.3	88.6	91.5	89.6	41.7	91.4	89.4	75.6	+13.8
GPT-5.6	91.6	95.2	95.0	91.8	91.5	91.0	41.7	86.1	91.9	89.4	+2.5
Open weights
GLM-5.3-Flash	85.9	92.3	86.7	83.7	85.1	82.1	83.3	83.3	89.4	68.4	+21.0

Appendix

Table 13: Accuracy on authentic images (%, prompted verification, strict scoring) by domain (A1 historical photographs 104, A2 film stills 60, A3 sports 184, A4 paintings 47, A5 political news 67, A6 meme templates 12 authentic images), and by whether the model recognizes the image (RECOG under Full: share recognized, accuracy on recognized and unrecognized images, difference). † Also the judge.

Proprietary
	Detection				KGR
Model	C1	C2	C3	C1 $-$ C3	C1	C2	C3
Gemini-3.8-Flash	81.5	98.6	78.2	+3.3	42.0	93.1	61.4
GPT-5.6	56.4	97.2	61.2	$-$ 4.8	22.6	81.9	41.7
Open weights
GLM-5.3-Flash	58.2	90.3	59.8	$-$ 1.6	20.2	79.2	41.6

Appendix

Table 14: Detection and KGR (%, prompted verification, strict scoring) by saliency of the edit over all families: C1 main figure (605 forgeries), C2 secondary figure (72), C3 background (830). † Also the judge.

Proprietary
	L0		L1		K1 Det (L1)		K1 KGR (L1)
Model	E3+	E4	E3+	E4	all	no border	all	no border
Gemini-3.8-Flash	46.6	28.4	55.1	37.2	81.1	80.0	33.7	33.7
GPT-5.6	19.8	8.3	36.0	26.3	62.1	61.4	18.7	18.6
Open weights
GLM-5.3-Flash	29.7	22.5	34.8	29.6	60.0	59.1	11.1	10.8

Appendix

Table 15: E4 rate next to KGR (E3+), and the people family without threshold-borderline items (%, strict scoring). E3+ / E4 under the default (L0) and prompted (L1) probes over all forgeries; K1 Det / KGR under L1 on all people-family forgeries and after dropping the 41 items whose face-similarity score lies within

\pm 0.05

of the acceptance threshold. † Also the judge.

Model	K0	K1	K2	K3	K4	K5	K6
Gemini-3.8-Flash	11	65	92	74	62	70	69
GPT-5.6	5	40	73	43	40	30	28
GLM-5.3-Flash	13	56	75	64	61	50	53
Kimi-K3	4	48	67	45	67	37	29
Kimi-K2.6	5	40	65	33	54	39	18
Qwen3.5-122B-A10B	22	49	69	52	45	48	39

Appendix

Table 16: Detection rate (%) on forgeries under the default probe by knowledge family, over items with a parsed verdict. Family sizes (forged items): K0 148, K1 567, K2 294, K3 89, K4 108, K5 145, K6 156. Rows in the order of Table 2 . KGR by family is in Table 5 .

	Detection on forgeries (%)				Authentic accuracy (%)				Balanced accuracy (%)				KGR (%)
Model	F	FC	C	CC	F	FC	C	CC	F	FC	C	CC	F	FC	C	CC
Gemini-3.8-Flash	73.1	77.9	79.3	86.2	84.7	72.4	73.5	72.0	78.9	75.2	76.4	79.1	35.2	33.1	46.2	44.8
GPT-5.6	53.8	57.2	56.6	69.7	84.7	85.7	78.1	83.9	69.2	71.5	67.3	76.8	14.5	13.1	13.1	16.6
GLM-5.3-Flash	52.4	47.6	62.8	59.3	82.7	89.8	74.9	89.7	67.5	68.7	68.8	74.5	7.6	4.8	14.5	10.3
Kimi-K3	35.9	26.9	82.8	62.8	92.9	95.9	57.1	86.5	64.4	61.4	69.9	74.6	3.4	2.1	12.4	8.3
Kimi-K2.6	28.3	29.7	67.6	63.4	89.8	87.8	75.0	89.2	59.0	58.7	71.3	76.3	4.1	2.8	20.0	15.2

Appendix

Table 17: All four de-recognition conditions for every model with condition variants, using the same population, probe, and strict scoring as Table 3 . We report detection, authentic accuracy, balanced accuracy, and KGR under Full (F), Full+Caption (FC), Crop (C), and Crop+Caption (CC). Each sweep covers 145 forgeries and the 98 authentic originals from the same scenes.

	Detection on forgeries (%)				Authentic accuracy (%)				Balanced accuracy (%)
Model	F	FC	C	CC	F	FC	C	CC	F	FC	C	CC
Gemini-3.8-Flash	73.1	77.9	79.3	86.2	84.7	72.4	73.5	72.0	78.9	75.2	76.4	79.1
GPT-5.6	53.8	57.2	56.6	69.7	84.7	85.7	78.1	83.9	69.2	71.5	67.3	76.8
GLM-5.3-Flash	59.4	50.0	72.2	61.0	92.0	96.7	83.1	93.0	75.7	73.4	77.7	77.0
Kimi-K3	35.9	27.3	82.8	63.6	92.9	98.9	57.3	86.8	64.4	63.1	70.0	75.2
Kimi-K2.6	28.3	29.9	68.1	63.9	89.8	87.8	75.7	89.4	59.0	58.8	71.9	76.6

Appendix

Table 18: De-recognition conditions of Table 3 under parsed-only scoring (items without a parsed verdict excluded from the denominator). Conditions abbreviated as in Table 17 . KGR is unchanged by the convention and is omitted.

	Detection					KGR			Authentic accuracy
Model	Full	Mirror	Crop	$b$ / $c$	$p$	Full	Mirror	Crop	Full	Mirror	Crop
Qwen3.5-122B-A10B	59.5	57.6	66.1	193/117	$<$ 0.0001	38.6	29.5	40.5	77.4	60.5	55.7
Kimi-K3	61.2	57.8	81.4	275/62	$<$ 0.0001	43.7	36.6	51.9	92.2	84.2	74.3
GPT-5.6	64.1	62.6	66.3	153/120	0.0526	44.9	40.5	45.0	91.6	82.1	82.4
Gemini-3.8-Flash	79.8	80.9	80.7	91/93	0.9413	64.0	55.5	65.5	88.2	68.1	80.2
Qwen3.5-35B-A3B	45.6	47.0	50.2	180/151	0.1237	28.8	24.3	30.7	58.4	35.0	28.5

Appendix

Table 19: Detection, KGR, and authentic accuracy under Full, Mirror, and Crop for the 902 peripheral forgeries in Table 7 and all 474 authentic images.

b/c

counts forgeries detected only under Crop or only under Mirror, and

p

is the exact McNemar test between them. Because this population includes visible-text forgeries (K4), for which mirroring itself changes the text, the paired recognition analysis in Table 3 is restricted to the people family.

		Contrast, recorded		Contrast, measured		Swap coef. vs. K0
Model	$n$	log-odds	$p$	log-odds	$p$	base	full	shrink
Gemini-3.8-Flash	248	0.688	0.1188	0.946	0.0450	4.580	4.623	$-$ 0.9%
GLM-5.3-Flash	239	1.449	0.0012	1.605	0.0020	3.072	3.056	0.5%
Kimi-K3	248	1.430	0.0019	1.552	0.0021	4.556	4.165	8.6%
Kimi-K2.6	248	1.596	$<$ 0.0001	1.644	0.0001	3.581	3.260	9.0%
Qwen3.5-122B-A10B	242	1.520	0.0004	1.548	0.0005	2.046	2.132	$-$ 4.2%

Appendix

Table 20: Per-model logistic regressions predicting detection from edit condition, with domain fixed effects, log footprint, and both pixel-detector scores; standard errors are clustered by seed. We report the swap-versus-erase log-odds contrast under both footprint definitions, and the swap coefficient relative to K0 before and after adding the covariates. “Shrink” is the change in that coefficient from the base to the full specification; negative values indicate an increase. Rows follow Table 2 .

Model	Erase (%)	Swap (%)	Ratio	E4 (%)	Outcome
Gemini-3.8-Flash	53.0	71.8	0.739	7.6	intermediate
GPT-5.6	17.4	52.6	0.331	0.0	collapse
GLM-5.3-Flash	32.6	65.4	0.498	2.3	collapse
Kimi-K3	30.3	70.5	0.430	2.3	collapse
Kimi-K2.6	21.2	59.0	0.360	1.5	collapse
Qwen3.5-122B-A10B	29.5	61.5	0.480	0.0	collapse

Appendix

Table 21: Application of the preregistered rule (default probe, strict scoring). Ratio: erase detection over swap detection. E4: share of the erase items whose verdict states the contradiction. Models detecting fewer than one swap in ten are listed without a ratio. Rows in the order of Table 2 .

Image Forgery Detection

Image Forgery Detection studies how to identify manipulated or synthetic images and locate the visual evidence of tampering.

See all 127 papers

EditSleuth: A Dataset of Grounded Reasoning Chains for Image-Edit Forensics
May 9, 2026Van-Loc Nguyen, AprilPyone MaungMaung, Minh-Triet Tran +1Image Forgery Detection Digital Image Forensics
University of Science, Vietnam National University Ho Chi Minh City · National Institute of Informatics
ManipShield: A Unified Framework for Image Manipulation Detection, Localization and Explanation
Nov 18, 2025Zitong Xu, Huiyu Duan, Xiaoyu Wang +4Image Forgery Detection Image Forgery Localization
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University · University of Electronic and Science Technology of China
Beyond Plausibility: Verifiable Fine-Grained Image Editing on Structured Assets
Oct 3, 2026Muyao Wang, Chen Zhu, Shiqi Yang +2Image Editing Image Editing Evaluation
SB Intuitions · University of Tokyo

May 9, 2026J/K move · Enter open · S save

EditSleuth: A Dataset of Grounded Reasoning Chains for Image-Edit Forensics

Van-Loc Nguyen, AprilPyone MaungMaung, Minh-Triet Tran, Isao Echizen

University of Science, Vietnam National University Ho Chi Minh City · National Institute of Informatics

Image Forgery Detection Digital Image Forensics Image Forgery Localization VLM Reasoning AI-Generated Image Detection

Open paper AskPDF arXiv

Forensic analysis of AI-edited images requires more than binary real-versus-fake prediction: a useful system should localize the edit, identify its semantic type, and ground its decisions in visual evidence. Existing image-forensics datasets typically emphasize detection or localization, while reasoning-supervised vision-language datasets rarely target image manipulation and often rely on LLM-generated rationales whose faithfulness is difficult to verify. We introduce EditSleuth, a dataset of 257,725 image-edit triplets constructed from existing image-editing corpora for grounded image-edit forensic reasoning. Each example includes an edited image, its source image, a binary edit mask, a 12-class edit taxonomy label, a difficulty score, and a six-step reasoning chain. EditSleuth chains are generated deterministically from triplet-grounded upstream artifacts, with each statement tied to a specific computable source of evidence. Our analysis reveals that a naive four-component difficulty formulation suffers from a rank-2 correlation collapse among magnitude features; a simplified three-component formulation substantially increases score dispersion on both Pico-Banana and MagicBrush. Difficulty also varies meaningfully within most edit categories, indicating that the score is not a proxy for edit type. As an initial learning study, we fine-tune Qwen2-VL-2B with LoRA and find that chain-as-target supervision matches a label-only baseline on classification accuracy among parseable answers, while additionally yielding grounded explanatory prose that label-only supervision cannot produce. We release the dataset, the deterministic construction pipeline, and pilot training scripts.

Abstract

Image Forgery Detection

Explore similar work

EditSleuth: A Dataset of Grounded Reasoning Chains for Image-Edit Forensics

ManipShield: A Unified Framework for Image Manipulation Detection, Localization and Explanation

Beyond Plausibility: Verifiable Fine-Grained Image Editing on Structured Assets