Forensic verification has converged on a binary "real vs. fake" label that confates fully synthetic, tampered, and AI-retouched images despite their very different consequences. We argue that GenAI manipulations decompose along two complementary channels: a camera channel of sensor fingerprints (destroyed by neural processing), and a semantic channel of scene-level coherence (disrupted by content edits). Each manipulation type leaves a distinctive two-channel signature. We instantiate this taxonomy in 2CAP (2-Channel Authenticity Protocol), pairing a contrastively-trained camera encoder with a frozen semantic encoder to serve two applications on a shared backbone: (i) four-class authenticity classification via cross-attention fusion with learnable reliability weights; and (ii) evidence generation, where an authentic image serves as a reference for a query and the two encoders produce per-channel alignment scores together with a patch-level saliency map. These signals are supplied as privileged context to a frozen Vision-Language Model (VLM) through an Observe-Generate-Refine loop, yielding explanations with localization without forensic fine-tuning. On a multi-source benchmark, 2CAP attains the best classification performance, including strong retouching detection where existing methods fail, and improves explanation quality.
Figures & tables
Figure 1: 2CAP overview. (a) Two-channel taxonomy. Authenticity cues are decoupled into a camera channel Ccam (sensor traces) and a semantic channel Csem (scene coherence); each manipulation type leaves a distinct signature: footprint-preserved transformations preserve both channels, GenAI retouching corrupts only Ccam , GenAI tampering corrupts Ccam globally and Csem locally, and fully synthetic images corrupt both. (b) Authenticity classification (Q1). A camera encoder (Sec. 3.2 ) and a frozen semantic encoder feed a cross-attention fusion module with learnable reliability weights (Sec. 3.3 ) that produces a four-class verdict (Sec. 4.2 ). (c) Reference-based evidence generation (Q2). Given a query xq and a camera-verified reference xref , the two encoders yield channel alignment scores scam,ssem (Eq. 4 ) and a patch-level saliency map (Eq. 5 ); these serve as privileged two-channel context to a frozen VLM in an Observe-Generate-Refine loop (Sec. 3.4 ) that returns an explanation and localization, revising whenever the VLM’s initial assessment is inconsistent with the channel evidence (Sec. 4.3 ).
Figure 2: Camera encoder pretraining. Left: Training data composition: 80,465 pairs, 32,242 unpaired camera entries, and 15,000 unpaired edited entries, totaling 208,172 image entries (Table 3 ); unique images have not been deduplicated. Center: DCT-domain encoder with frequency-based patch selection and dual-branch feature extraction. Right: InfoNCE training where camera image embeddings are pulled together while edited image embeddings are pushed apart.
Method
Overall
Camera
Tampered
Synthetic
Retouched
Flickr8K ∗
VISION ∗
Prec.
Rec.
F1
AUC
OvR
F1
OvR
F1
OvR
F1
OvR
F1
AUC
Brier ↓
TruFor [ 8 ]
.100
.254
.123
.559
.618
.410
.724
.000
.497
.000
.396
.082
.637
.123
D3 [ 22 ]
.755
.754
.750
.928
.864
.591
.899
.680
.973
.863
.975
.868
.671
.179
CNNSpot [ 10 ]
.370
.361
.359
.631
.601
.349
.565
.316
.685
.430
.672
.344
.583
.237
AIDE [ 11 ]
.767
.770
.767
.934
.872
.638
.946
.777
.963
.835
.952
.816
.870
.228
VIBAIGCD [ 13 ]
.752
.740
.730
.925
.863
.523
.890
.673
.975
.871
.974
.851
.879
.253
Table 1: Four-class authenticity classification. Overall and per-class performance on the common 3,200-image test set, plus held-out datasets ( ∗ ). Per-class metrics report one-vs-rest AUC (OvR) and F1. Best in bold , second underlined (ranking computed over baselines and 2CAP only). The bottom block uses a common group-filtered training/validation protocol for all six variants and is excluded from the ranking; the full row uses paired, neutral gate initialization and baseline training populations differ.
Figure 3: Left: learned reliability weights αcam and αsem shown by category as descriptive post-attention diagnostics. Right: score distribution. Retouching: low scam , high ssem ; camera: high/high; tampering: higher variance.
Figure 4: 2CAP evidence generation case study. For each query-reference pair, 2CAP performs camera and semantic alignment (Pass 1), applies a reliability cross-check (camera-, semantic-, and heatmap consistency), and optionally uses Pass 2 to finalize the edit type, separating augmentation, tampered removal, and retouching.
Forensic evidence
Policy review
Model
Config.
ChgType ↑
Recall ↑
Hit ↑
IoU ↑
Acc. ↑
Prec. ↑
Recall ↑
Rev (%)
GPT-4o-mini
Baseline
66.1±0.5
21.3±0.9
32.5±1.1
25.6±0.6
95.5
94.7
51.4
–
+OGR
72.1±0.5
29.3±0.6
39.7±1.6
36.1±1.5
96.2
83.3
71.4
76.4±0.3
GPT-4o
Baseline
84.4±0.3
35.6±0.7
59.4±1.2
60.9±3.7
97.8
86.1
88.6
–
+OGR
89.8±0.2
36.4±0.2
60.3±0.3
61.3±0.3
98.2
86.8
94.3
15.2±0.8
Gemini 2.5 Flash
Baseline
71.0±1.0
41.8±2.1
64.0±2.3
57.4±1.4
92.8
56.8
71.4
–
Table 2: VLM evidence generation and downstream policy review. Baseline versus +OGR on 400 pairs. All values are percentages. Forensic evidence and Rev report mean ± standard deviation over three runs; policy columns retain separate point estimates. Bold marks the better result within each model; ↑ indicates higher is better. IoU is a localization proxy. Policy metrics use reject as the positive class. Rev is revision frequency and has no preferred direction.
Appendix figures & tables23 assets
Supplementary material from the paper’s appendix.
Appendix
Source
Type
Size
MagicBrush [ 37 ] + SAGI-D [ 36 ]
Paired (camera-edit)
80,465 pairs
Dresden [ 40 ]
Unpaired camera
17,242
SID-Set [ 29 ]
Unpaired camera
15,000
SID-Set [ 29 ]
Unpaired edited
15,000
Total image entries
208,172
Appendix
Table 3: Training data for contrastive learning. Paired sources provide camera-edit correspondences; unpaired sources contribute additional camera and edited images. The total counts image entries ( 2×80,465+17,242+15,000+15,000 ), not deduplicated unique images. The unique-image count requires a per-image manifest.
Dataset
Historical training pool
Evaluation inventory
SAGI-D [ 36 ]
2,240
3,838
Defactify* [ 31 ]
3,840
22,000
SID-Set [ 29 ]
640
16,000
MagicBrush [ 37 ]
2,240
0
Dresden [ 40 ]
640
0
Generative Photography* [ 38 ]
3,200
800
Appendix
Table 4: Historical downstream training pool and source-level evaluation inventory. The historical training pool contains 12,800 images (3,200 per class); the controlled fusion experiments instead use the filtered 9,877/2,451 training/validation split described in Sec. 4.2 . The common four-class test split in Table 1 samples 800 images per class from the evaluation pools, for 3,200 images total; the inventory counts below describe the larger pools rather than the size of that test split. * indicates not seen in contrastive learning.
Figure 5: Visualization of camera-captured images from the source collections shown. For Dresden and VISION, we show images from diverse camera models (Agfa, Canon, Nikon, Samsung, Huawei, iPhone, Sony, etc). The source names identify the illustrated collections; split membership is specified separately in the evaluation protocol.
Figure 6: Visualization of fully synthetic images generated by different text-to-image models. These images contain no camera traces and may show subtle semantic inconsistencies in lighting, geometry, or physics.
Figure 7: Visualization of tampered images with their camera-captured references. Tampered images contain localized semantic edits; their effects on semantic similarity and camera-related cues depend on the edit and can overlap with other categories.
Figure 8: Visualization of GenAI-retouched images with their original camera captures. These examples show retouching with low camera-feature similarity and high semantic similarity; actual responses may overlap with tampering.
Figure 9: Cross-channel attention fusion for authenticity classification. Camera features zcam and semantic features zsem are projected to a shared 256-d space, then refined via bidirectional 4-head cross-attention. Learnable reliability weights (αcam,αsem) adaptively combine the channel representations based on input characteristics. The fused representation is classified as camera-captured or edited.
Stage
Tensor / shape
Operation and normalization
Patch tokens
Zcam:B×Ncam×128Zsem:B×Nsem×768
Retain the patch sequence in each channel; no global pooling before attention.
Linear projections
Zc:B×Nc×256
Project camera and semantic features independently to 256 dimensions.
Camera-query attention
Acam:B×Ncam×NsemZcam′:B×Ncam×256
Camera queries; semantic keys and values. Softmax over the Nsem keys for each query and head.
Semantic-query attention
Asem:B×Nsem×NcamZsem′:B×Nsem×256
Semantic queries; camera keys and values. Softmax over the Ncam keys for each query and head.
Channel pooling
zc′:B×256
Pool over each channel’s patch axis after residual cross-attention.
Reliability gate
[zcam′;zsem′]:B×512ℓ,α:B×2
Two-layer MLP produces two logits. Channel-axis softmax gives per-image weights with αcam+αsem=1 .
Appendix
Table 5: Cross-channel fusion architecture and tensor shapes. B is batch size; Ncam and Nsem are channel-specific patch counts. Shapes include the batch axis. Attention uses four heads and two layers; the attention-weight shapes below are per head. The key-patch, channel, and class softmax operations normalize over different axes.
Component
AIDE
2CAP
Patch selection
DCT energy ranking
DCT energy ranking (same)
Filter bank
30 SRM filters
30 SRM filters (same)
Backbone
Dual-branch ResNet-50
Dual-branch ResNet-50 (same)
Training objective
Cross-entropy (binary)
InfoNCE contrastive
Training data
Single-image labels
Paired camera-edit + unpaired
Embedding space
Class logits
128-d unit-norm (metric space)
Appendix
Table 6: Architectural comparison: 2CAP camera encoder vs. AIDE.
Collection
Comparison
Metric
Result
SID-Set
Camera vs. tampered
AUC
95.5%
Defactify
Camera vs. fully synthetic
Accuracy
90.4%
Appendix
Table 7: Within-source diagnostics. Values are retained in the metric originally reported for each comparison. These are not matched-original or independent-generator evaluations.
Pipeline
Effect
Camera
Retouched
Tampered
Synthetic
GP
Color temperature
1.0
97.0
2.0
0.0
GP
Bokeh
1.5
94.5
4.0
0.0
GP
Focal length
5.5
84.5
10.0
0.0
GP
Shutter speed
1.0
97.0
1.5
0.5
OpenCV
Color temperature
3.2
90.6
5.4
0.8
OpenCV
Bokeh
2.7
81.2
15.3
0.8
Appendix
Table 8: Cross-pipeline diagnostic. Row-normalized predicted-class percentages. “Retouched” is a model prediction, not proof of GenAI processing or an established correctness label for OpenCV operations.
Transformation
Camera
Tampered
Synthetic
Retouched
No processing
62.5
26.5
9.5
1.5
Resize
60.5
29.0
9.0
1.5
Denoise
58.5
29.5
10.0
2.0
HDR tonemap
59.5
27.5
11.0
2.0
Social recompress
60.5
27.0
11.0
1.5
Computational photo
62.0
26.0
10.5
1.5
Appendix
Table 9: Mild benign-processing diagnostic. Predicted-class percentages on camera-derived images. All rows should retain the camera label under this diagnostic’s policy.
Method / AUC
Defactify Images
Flickr8K*
Avg.
SD2.1
SDXL
SD3
DALL-E 3
Midjourney 6
SDAbso.
SDEpic.
SDv1.5
SDv6.0
SDXL
TruFor [ 8 ]
.455
.407
.382
.452
.516
.656
.668
.660
.550
.689
.544
D3 [ 22 ]
.613
.715
.618
.778
.762
.723
.693
.668
.643
.635
.685
CNNSpot [ 10 ]
.561
.547
.542
.606
.560
.625
.574
.598
.550
.533
.570
AIDE [ 11 ]
.626
.701
.633
.567
.862
.929
.905
.895
.835
.796
.775
VIBAIGCD [ 13 ]
.757
.808
.740
.844
.820
.932
.924
.922
.903
.698
.835
Appendix
Table 10: Camera authenticity across generators (AUC). Defactify and Flickr8K*, each with five generators. This breakdown covers eight baselines; SAFE and GAPL are included only in the main four-class comparison. Flickr8K* is held-out. Best in bold , second underlined .
Frozen model
Acc.
Macro-F1
Macro-AUC
Full 2CAP
46.25
39.14
71.99
Camera only
28.75
23.88
64.33
Semantic only
44.50
36.51
66.42
Appendix
Table 11: Matched-source four-class diagnostic. All values are percentages.
Baseline
Ours
Metric
Mean
Std
Mean
Std
IoU
.500
.241
.597
.228
Precision
.722
.244
.823
.203
Recall
.638
.297
.690
.254
Appendix
Table 12: Proxy GT validation. Agreement between VLM-derived bounding boxes and human-annotated bounding boxes on 50 tampered pairs.
Property
FakeVLM
FakeShield
ForgeryGPT
SIDA
2CAP
Input type
Single
Single
Single
Single
Pair
VLM training
SFT
SFT
Staged tuning
SFT
Frozen
Channel decomp.
×
×
×
×
✓
Retouch vs. tamp.
×
×
×
×
✓
Forgery localization
×
Mask
Mask
Mask
Saliency
Output type
Binary+text
Binary+mask
Desc.+mask
Label+mask+desc.
Scores+sal.+OGR
Appendix
Table 13: Design-space comparison with VLM-based forensic methods. 2CAP is the only method that (i) uses reference-based verification, (ii) decomposes camera and semantic channels, (iii) distinguishes retouching from tampering, and (iv) uses the tested VLMs without forensic instruction tuning.
Configuration
ChgType ↑
Recall ↑
Hit ↑
IoU ↑
Rev (%)
GPT-4o-mini
Baseline
66.1±0.5
21.3±0.9
32.5±1.1
25.6±0.6
–
Ours (+OGR)
72.1±0.5
29.3±0.6
39.7±1.6
36.1±1.5
76.4±0.3
w/o semantic
67.2±1.6
27.7±0.5
38.2±1.1
32.8±0.0
71.3±2.8
w/o camera
69.8±0.6
28.4±0.6
38.0±1.6
35.4±0.3
71.5±0.0
w/o heatmap
72.7±0.6
25.5±0.6
35.5±0.8
34.3±1.2
69.2±0.3
Appendix
Table 14: VLM evidence generation and component ablation. All values are percentages; smaller gray values denote ± standard deviation over three runs. Shaded rows: full OGR; indented rows: removal of one OGR input. Bold marks the highest mean per model (including ties). ↑ : higher is better. IoU is a localization proxy; revision frequency (Rev) has no preferred direction.
Figure 10: OGR example: benign augmentation. The refinement retracts the initial style-transfer assessment and reports no object changes. The pair illustrates a camera-derived augmentation rather than localized tampering.
Figure 11: OGR example: localized tampering by replacement. Broccoli on the right side of the plate is replaced with strawberries. The object-change description identifies a replacement rather than a global retouch.
Figure 12: OGR example: retouching by style transfer. A global watercolor/sketch-like aesthetic filter changes appearance while the displayed output reports no object changes. This is a paired retouching example, not text-to-image full synthesis.
Figure 13: OGR example: localized tampering by replacement. A seagull is replaced with a butterfly. The paired reference makes the changed object and operation explicit.
Figure 14: OGR example: localized tampering by addition. A boat is added to the water. The displayed cross-check reports high camera alignment, illustrating that localized tampering can coexist with preserved camera-related cues.
Figure 15: Failure cases of 2CAP. Three challenging examples where the two-pass pipeline produces incorrect conclusions. Left: retouching misclassified due to VLM hallucination under ambiguous saliency. Middle: a building-region replacement missed in Pass 1, with Pass 2 incorrectly describing the edit as a removal. Right: augmentation misclassified as tampering due to VLM hallucinating a person removal.
Advances in generative AI have made image falsification highly realistic, demanding trustworthy authentication systems. Existing forensic detectors can target certain forgery types but lack interpretability, while vision-language models (VLMs) provide explanations but cannot exploit forensic traces for reliable detection. We propose Forensic Knowledge Graphs (FKGs), a unified framework that integrates forensic evidence extraction, structured reasoning, and human-interpretable explanation. Our FKG structure encodes forensic traces along with their causal dependencies and links to scene content. To generate accurate FKGs, we introduce a novel forensic authentication network and an Iterative Context Refinement strategy that guides VLMs to produce faithful, grounded explanations. We also present FKG-50K, a dataset of 50,000 realistic forgeries with ground-truth FKGs. Experiments demonstrate that FKG outperforms both forensic detectors and VLMs in detection, forgery identification and localization, and forensic justification.
A marketplace review photograph is a document: platforms approve refunds on it, and generative models drove the cost of forging one to zero. We study that detection problem, so we build a detector and attach an attribution map as its evidence, then measure what that pair delivers on 186,527 images under controls designed to change our conclusions when something is wrong. Compression history, not synthesis, drives naive evaluation: our strongest model reaches 0.9999 PR-AUC (area under the precision-recall curve) on a product-disjoint split, yet falls to 0.7254 once we re-encode synthetics into the real class's format, while five public detectors move by at most 0.07. Aligning one class relocates the cue rather than removing it, and the repaired model then assigns native files a median probability of synthesis of 0.0004. One identical final encode for both classes repairs that, and a three-seed factorial credits the encoding change with the whole gain (+0.176 +- 0.009 PR-AUC). That encode equalises the last stage only: forensic features alone still separate the classes at 0.7145 against a base rate of 0.254. For evidence we test maps causally, against controls that never consult the detector. Whether an attribution ranking exists at all depends on whether the detector reacts to the image. On our first-fix detector, which calls 96 of 100 edited frames real, no map beats a random one. On the detector we selected, twelve of seventeen maps clear that control on edited images and eight on generated ones; perturbation leads both axes and no gradient-CAM variant shows a positive advantage. The trivial controls never clear it, and on generated images the centre prior is worse than random. Our ensembled regional map clears both axes and takes the top pixel AP at 12.4 s per map against 44.9 for occlusion. Clearing a detector-blind control is not yet a faithful explanation, and we demonstrate none.
Leonid Kuturin, Ilya Sotnikov, Mark Khusnutdinov +4
Sirius Educational Centre, Big Challenges programme · HSE University, Faculty of Computer Science
Recent generative models have largely closed the gap on low-level artifacts - pixel fingerprints, frequency anomalies, upsampling traces - particularly in person-centric and partial-edit settings where the manipulated region is small and surrounded by photometrically authentic content. We introduce Social Gaze Consistency, a high-level semantic cue defined as the mutual coherence of gaze direction, head-eye alignment, and pupil placement between interacting individuals, and show that it constitutes a previously underutilized detection axis orthogonal to existing low-level paradigms. We instantiate this insight through three coupled mechanisms: (i) a controlled diagnostic dataset with region-specific perturbations of gaze-consistent imagery, where strict pair-level grouping forecloses generator-fingerprint memorization as an optimization-time shortcut rather than relying on augmentation; (ii) Block-Compositional Caption Supervision, which holds a single 5-block reasoning skeleton invariant across 1,250 macro-combined captions, decoupling reasoning consistency from surface diversity; (iii) Cross-architecture validation showing the same supervision improves a vision-language backbone (FakeVLM) by +3.7 pp on the COCOAI Interaction subset (balanced accuracy 67.8 -> 71.5) and +1.3 pp on the COCOAI Person subset (83.0 -> 84.3), with consistent gains on a vision-only backbone (Effort), evidencing a backbone-agnostic cue. Real- and fake-class recalls rise simultaneously, ruling out a "predict-all-fake" artifact. A four-step mechanistic account - paired-edit shortcut blocking, hard-to-easy difficulty transfer, CLIP prior preservation, and diffusion-family shared spectral weakness in periocular structure - explains why training on a single inpainter (FLUX.1-Fill) transfers to multi-generator suites. We will release the code upon acceptance to facilitate reproducibility.
Jihyeon Kim, Sohee Kim, Soosan Lee +3
School of Computer Engineering Hoseo University · School of Electronic Engineering Soongsil University · School of Computer Science University of Illinois at Urbana-Champaign