EviScene: Beyond "Is It AI?" for Agentic Image Factuality Assessment

ZiAn Wang; Ruiqi Liu; Zhengbo Zhang; Yi Han; Jialiang Shen; Ruoxin Chen; Manni Cui; Ximo Zhu

cs.CVOct 10, 2026

EviScene: Beyond "Is It AI?" for Agentic Image Factuality Assessment

Authors: ZiAn Wang, Ruiqi Liu, Zhengbo Zhang, Yi Han, Jialiang Shen, Ruoxin Chen, Manni Cui, Ximo Zhu,

Organizations: Jilin University · Institute of Automation, Chinese Academy of Sciences · Southwest University · University of Sydney ·

Abstract

Determining whether an image is AI-generated does not establish whether the situation it depicts is factually supported. We formulate Image Factuality Assessment: given an image and a generic verification instruction, a system returns a supported or refuted verdict with an evidence-grounded explanation. Without a supplied textual claim, the system must progressively refine factual questions during investigation and align external evidence with the specific situation depicted. We introduce EviLens, an expert-reviewed benchmark of 10,174 supported and refuted images from controlled construction and web collection, covering twelve factual error types and evaluating both verdict accuracy and evidence sufficiency. We develop EviScene, a multimodal ReAct agent that interleaves visual inspection, external retrieval, and evidence analysis, using new findings to refine its questions and revisit relevant image details. We train a compact policy with supervised fine-tuning on reviewed teacher trajectories, followed by privileged on-policy self-distillation applied to verified decisions from the policy's own investigations. On the 1,527-image test set, the trained Qwen3.5-9B agent achieves 79.05% balanced accuracy and a Strict Evidence Sufficiency Rate (SESR) of 52.26%, improving over the same-backbone base agent by 6.71 and 9.37 percentage points, respectively. Its SESR is the highest among the evaluated agents.Code, models, and data will be released at https://github.com/wangzian828/EVISCENE and https://huggingface.co/ZiAnwang/EviScene.

Figures & tables

Figure 1: Claim-guided versus image-centered verification. EviScene alternates evidence retrieval and visual reinspection to refine questions about the depicted event and its context.

Figure 2: Factual errors in EviLens: (a) constructed and collected examples with corresponding evidence; (b) the twelve error types grouped into three families.

Figure 3: Construction and collection of EviLens. The curation agent builds and checks candidates, uses feedback to guide later rounds, and submits them for expert review.

Figure 4: EviScene investigation and reporting. (a) Three tool groups available at every investigation turn. (b) An illustrative ReAct sequence: observations guide question refinement and tool selection, while each interaction is appended to the history. (c) At reporting turn $T$ , the policy uses the input image and full history of turns $1,\ldots,T-1$ to generate a verdict, explanation, and cited observations.

Method	BAcc	Macro F1	Supported recall	Refuted recall	SESR
Direct visual judgment
GPT-5.5	75.87	70.51	79.58	72.17	3.67
GPT-5.2	65.84	55.41	85.94	45.74	2.95
Gemini 3.7 Flash	75.64	72.80	71.62	79.65	9.10
Gemini 3.1 Pro	62.58	64.54	28.65	96.52	5.57
Claude Opus 4.8	72.18	68.67	69.76	74.61	6.75

Table 1: Results (%) on 1,527 test images. Best and second-best results among EviScene agents are bold and underlined, respectively; trained Qwen policies are shaded. Full denotes the three-epoch SFT policy after PSD.

Figure 5: Qualitative examples from EviLens. Each column pairs an image and its label with the GPT-5.5 and EviScene verdicts and the decisive factual comparison.

	Acquisition source		Controlled test generator
Policy	Controlled	Web	Nano Banana Pro	GPT Image 2	Seedream 5.0 Lite
Direct	61.11 / 1.29	74.46 / 3.46	62.54 / 1.17	52.95 / 0.71	49.39 / 2.32
Base agent	67.87 / 43.94	73.34 / 38.41	69.43 / 32.36	66.36 / 55.56	60.20 / 47.35
SFT	76.39 / 50.97	76.17 / 48.44	77.36 / 39.18	73.74 / 61.47	67.63 / 56.29
Full	76.75 / 52.75	77.21 / 50.17	75.87 / 46.98	78.75 / 58.39	69.68 / 54.64

Table 2: Results (%) by image source and generator for Qwen3.5-9B. Each entry reports BAcc / SESR; generator columns cover controlled-construction images. Best and second-best values for each column and metric are bold and underlined.

Variant	BAcc	SESR
All tools (Full)	79.05	52.26
w/o web search	74.37	35.89
w/o image retrieval	74.52	40.01
w/o evidence inspection	76.55	46.04

Table 3: Tool-family ablations (%).

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	No claim	World fact	Web evidence	Evidence output	Origin-independent factuality	Size
Generated-image detection and visual reasoning
GenImage [ 2023 ]	✓	✗	✗	✗	✗	2.68M images
WHOOPS! [ 2023 ]	✓	✗	✗	✗	✗	500 images
T2I-FactualBench [ 2025 ]	✗	✓	✗	✗	✓	3K prompts
Image-text and out-of-context verification
NewsCLIPpings [ 2021 ]	✗	✗	✗	✗	✗	988,283 pairs

Appendix

Table 4: Comparison with representative benchmarks.

Pathway	Label	Cases	Factual basis
Synthesis from factual references	Supported	2,073	Reference material establishes the situation that the generated image is intended to depict.
Supported web images	Supported	1,041	Source information corroborates the depicted situation.
Counterfactual construction	Refuted	3,106	A documented situation is modified through a selected visible detail or relation.
Fabrication from real entities	Refuted	3,067	A constructed occurrence or relation conflicts with records or constraints applying to the depicted subjects.
Images from fact-checking sources	Refuted	887	A circulating image is paired with source material explaining its factual error.
Total		10,174	3,114 supported and 7,060 refuted images.

Appendix

Table 5: Acquisition pathways, retained examples, and the basis for their factual interpretation.

Error type	Definition
Event and relation errors
Invented action	A real subject performs an action or participates in an event that did not occur.
Invented interaction	Real subjects are depicted meeting, collaborating, or participating together when that interaction did not occur.
Impossible state	The depicted state contradicts physics, science, technology, or commonsense.
False official record	A visible roster, certificate, scoreboard, directory, or similar display presents an incorrect official role, result, ranking, or entry.
Attribute and detail errors

Appendix

Table 6: Definitions of the twelve factual error types in EviLens.

	Training		Test		Total
Source	Supported	Refuted	Supported	Refuted
Controlled construction	1,840	5,168	233	1,005	8,246
Web collection	897	742	144	145	1,928
Total	2,737	5,910	377	1,150	10,174

Appendix

Table 7: Dataset composition by acquisition source, split, and factual label.

Error type	Cases	Error type	Cases
Invented action	513	Numerical alteration	520
Invented interaction	512	Identity substitution	504
Impossible state	502	Spatial alteration	514
False official record	516	Text or mark alteration	521
Temporal displacement	515	Source impersonation	510
Event recontextualization	522	Geographic displacement	524

Appendix

Table 8: Error-type distribution of 6,173 refuted images from controlled construction.

Hyperparameter	Value
Model and supervision
Policy backbone	Qwen3.5-9B [ Qwen Team, 2026 ]
Policy parameters	9,409,813,744
SFT demonstration model	Qwen3.5-397B-A17B [ Qwen Team, 2026 ]
Model-mediated tool calls	Same checkpoint as the investigation policy; separate requests
OCR service	Baidu general OCR API

Appendix

Table 9: Implementation and training settings for the Qwen3.5-9B full policy.

Operation	Role and returned information	Limit
perceive_scene	Scene description, entities, attributes, relations, and visible uncertainties.	Global
ocr_with_position	Text transcription with positions for later cropping and inspection.	3
crop_and_inspect	Deterministic crop and focused description of a selected region.	4
focused_visual_inspection	Original image and derived views assessed against one evidence-motivated visual question.	2
analyze_visual_anomalies	Locatable physical, logical, relation, or text observations relevant to the current target.	3
check_consistency	Lighting, geometry, scale, boundary, and physical-relation checks for a specified aspect.	3

Appendix

Table 10: Native operations available to the investigation policy. Numeric limits are per case and remain subject to the 24-action global budget. “Global” means that only the global budget applies.

Variant	Disabled native operations
Full	None
w/o web search	text_search
w/o image retrieval	text_image_search , reverse_image_search
w/o evidence inspection	visit , compare_with_reference

Appendix

Table 11: Native-operation settings for the tool ablations in Table 3 .

1	Freeze the current policy as $\pi_{\bar{\theta}}$ and collect fresh investigations.
2	Select a decision $t$ in an eligible student investigation.
3	Construct a procedural hint $h_{t}$ for that decision.
4	Continue $\pi_{\bar{\theta}}$ from $\operatorname{insert}(c_{t},h_{t})$ through terminal judgment.
5	Review the selected decision, completed investigation, evidence use, and trajectory structure.
6	Add an accepted completion $u_{d}$ with teacher context $c_{d}^{\mathrm{T}}=\operatorname{insert}(c_{t},h_{t})$ and student context $c_{d}^{\mathrm{S}}=c_{t}$ to $\mathcal{D}_{\mathrm{rep}}$ .
7	Extract preservation decisions from successful investigations with $c_{d}^{\mathrm{T}}=c_{d}^{\mathrm{S}}$ into $\mathcal{D}_{\mathrm{pres}}$ .

Appendix

Table 12: One PSD round. Symbols follow the formulation in the main text.

Setting	Value
Test images	1,527: 377 supported and 1,150 refuted
Evaluator	Gemini 3.7 Flash ( gemini-3.7-flash ) [ Google DeepMind, 2026a ]
Classification statistics	BAcc, macro F1, and class recalls
Evidence evaluation	Stated grounds checked against private reference material
Evaluation population	All 1,527 test images
Execution attempts	Up to three per case; stop at the first binary verdict

Appendix

Table 13: Test set and evidence evaluation settings.

Figure 6: Performance on the test set. (a) BAcc and SESR for direct models and agent policies. (b) Supported and refuted recall, with equal-recall and equal-BAcc guides. The Qwen3.5-9B comparison includes the base agent, three-epoch SFT, and full SFT+PSD policies.

Figure 7: Development of the Qwen3.5-9B policy across training stages. Panels separate overall decision quality, recall for the two factual labels, and evidence sufficiency. Full denotes the policy trained with SFT and PSD. Values are percentages; vertical scales differ across panels.

Automated Fact-Checking

Automated fact-checking detects checkable claims, retrieves evidence, and assesses whether that evidence supports or refutes them.

See all 139 papers

WorldFact-Bench: Beyond Image-Internal Plausibility to Image-World Consistency
Oct 8, 2026Zhuohong Chen, Zhengxian Wu, Yunyao Yu +7Factual Consistency Evaluation Image Forgery Detection
Ant Group · Tsinghua University
FACT: A Forensic Agent with Compiled Tool-Use Trajectories for AI-Generated Image Detection
Sep 5, 2026Jiaoyang Chen, Bin Hu, Jingyu Hu +3AI-Generated Image Detection Image Forgery Detection
Lingnan University, Hong Kong · Harbin Institute of Technology, Shenzhen · The University of Hong Kong +2
MIC: Explaining Image-Claim Inconsistencies in AI-Generated Multimodal Misinformation
Sep 27, 2026Ruihong Zeng, Jonathan Tonglet, Preslav Nakov +1Factual Consistency Evaluation Automated Fact-Checking
Mohamed bin Zayed University of Artificial Intelligence, UAE · UKP Lab · Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, TU Darmstadt and National Research Center for Applied Cybersecurity ATHENE, Germany +2

Oct 8, 2026J/K move · Enter open · S save

WorldFact-Bench: Beyond Image-Internal Plausibility to Image-World Consistency

Zhuohong Chen, Zhengxian Wu, Yunyao Yu, Hangrui Xu+6

Ant Group · Tsinghua University

Factual Consistency Evaluation Image Forgery Detection Automated Fact-Checking

Open paper AskPDF arXiv

Advances in image generation have made visual authenticity increasingly difficult to assess. Although image forensics now examines both generation artifacts and higher-level visual inconsistencies, a plausible image can still contradict real-world facts or rules. We introduce WorldFact-Bench to evaluate image-world consistency from a single image, without a predefined claim or verification target. The benchmark contains 1,274 source-aligned real-fake pairs across four verification regimes and ten semantic domains. Each pair introduces a specific, evidence-supported factual conflict while seeking to preserve non-target content and visual plausibility. Images are evaluated independently, and pair accuracy requires both members of a pair to be classified correctly. We further propose PERSIST-Agent, which organizes iterative verification around a persistent state linking candidate facts, visual observations, evidence, and verification statuses. This state guides subsequent inspection and retrieval while retaining unresolved candidates. With backbone weights fixed, harness self-optimization refines the agent's prompts and execution rules through validation feedback. Experiments reveal strong label biases in several detectors and uneven gains from retrieval. On the evaluated 8B backbones, PERSIST-Agent improves pair accuracy over both direct judgment and retrieval-augmented baselines, while ablations support the role of persistent verification state. These findings highlight the value of state-guided verification and the remaining gap between visual plausibility and factual correctness.

Abstract

Automated Fact-Checking

Explore similar work

WorldFact-Bench: Beyond Image-Internal Plausibility to Image-World Consistency

FACT: A Forensic Agent with Compiled Tool-Use Trajectories for AI-Generated Image Detection

MIC: Explaining Image-Claim Inconsistencies in AI-Generated Multimodal Misinformation