Localized AI manipulations in long-form videos often affect brief events within otherwise authentic content, making video-level classification insufficient for forensic analysis. Existing audio-visual deepfake benchmarks largely focus on short talking-head clips, while long-form temporal forensics remains predominantly visual. We introduce AV-FABLE, a dual-timeline benchmark for sparse manipulations in open-scene long-form audio-visual videos. AV-FABLE contains 16,650 pristine--manipulated twin pairs (998 hours) created with 27 editing configurations, with separate audio and visual timelines that enable modality-specific temporal localization and expose errors hidden by joint-timeline evaluation. On AV-FABLE, zero-shot prompting of audio-visual large language models (AV-LLMs) yields near-chance detection and poor temporal localization. In contrast, supervised readouts of their frozen perception features recover strong forensic signals, suggesting manipulation evidence is encoded in these representations but not effectively exposed through language outputs. Motivated by this observation, we propose NFSR(Native Forensics via Separated Readouts), which reads frozen AV-LLM features with separate audio and visual temporal decoders. NFSR achieves the best audio-visual localization performance in-domain and across all three out-of-distribution settings, including transfer to held-out videos from an external long-video forensic benchmark. AV-FABLE, code, and model weights will be publicly released.
Figures & tables
Figure 1: Overview of SEE-Forge, the event-driven pipeline for constructing AV-FABLE. Given an event in a pristine video, SEE-Forge plans audio and/or visual manipulations, executes them with editors, and reinserts them into context to create pristine–manipulated twin pairs. The executed interval provides temporal ground truth, while human-reviewed annotations record forensic cues within and around the manipulated event.
Dataset
Scale
Scene
Length
Manip.
Loc.
Expl.
Pair
OOD shift
A
V
Src
Gen
Type
FaceForensics++ ( Rössler et al., 2019 )
6,000 v
Faces
Clip
×
✓
×
×
( ✓ )
×
( ✓ )
( ✓ )
Celeb-DF (v2) ( Li et al., 2020 )
6,529 v
Celebrity faces
Clip
×
✓
×
×
( ✓ )
×
×
×
DFDC ( Dolhansky et al., 2020 )
128,154 v
Consented actors
Clip
( ✓ )
✓
×
×
( ✓ )
( ✓ )
( ✓ )
×
FakeAVCeleb ( Khalid et al., 2021 )
20,000 v
Talking heads
Clip
✓
✓
×
×
( ✓ )
×
×
×
ForgeryNet ( He et al., 2021 )
221,247 v
Faces
Clip
×
✓
✓
×
( ✓ )
×
×
×
Table 1: Forensic benchmarks comparison. Length: short clips (Clip) or minute-scale videos (Long). Manip.: audio (A) or visual (V) manipulations; Loc.: temporal localization; Expl.: explanations; Pair: pristine–manipulated twins; OOD shift: held-out sources, generators, or manipulation types/routes. ( ✓ ) : partial support, or shift without a dedicated held-out split.
Figure 2: Overview of NFSR. (a) Frozen intermediate-layer (“detail”) and interface features pass through modality-specific gated projection and host-conditioned FiLM. (b) Independent decoders localize audio and visual manipulations on separate timelines. (c) Source-calibrated thresholds give video-level detection and four-way attribution without changing the intervals. Audio and video excerpts come from the same clip; intervals are schematic.
Method
Training Form
Output
AUROC
FAR
Attr. F1
AV-mAP
A-mAP
V-mAP
Joint-mAP
A-IoU
V-IoU
Joint-IoU
Xception
Official weights
V win
50.71
4.94
17.62
0.01
–
0.02
0.02
–
0.22
0.24
MSLoc-PG
Official weights
V
57.34
3.54
20.37
0.95
–
1.90
1.36
–
4.55
3.40
ActionFormer
TASLE
V
67.94
2.78
20.99
5.06
–
10.13
7.56
–
5.56
4.55
Xception
AV-FABLE
V win
66.27
4.94
23.27
0.04
–
0.08
0.06
–
4.87
3.98
DeMamba (Stage-1 window)
AV-FABLE
V win
74.35
4.26
28.77
24.75
–
49.49
26.83
–
35.21
28.54
ActionFormer
AV-FABLE
V
84.55
5.84
34.95
44.61
–
89.22
59.93
–
80.66
62.71
Table 2: ID main results on AV-FABLE (%). Blocks are visual-only, audio-only, and audio-visual. Outputs are separate audio/visual intervals (A/V), one track (A or V), fixed-window intervals (win), modality-labeled language timestamps (Text), or modality-unlabeled intervals (Unified). – marks unsupported metrics; absent tracks count as misses in AV-mAP and Joint-timeline metrics. NFSR, UMMAFormer, and TASLE-trained ActionFormer are three-seed means (Table 18 ); others are single runs. Bold and underlined values mark the best and second-best results per column; FAR is an operating-point diagnostic and is not ranked.
Method
Training Form
Source transfer
Held-out generators
Held-out TI2V
AUROC
FAR
Attr. F1
AV-mAP
A-mAP
V-mAP
AUROC
FAR
Attr. F1
AV-mAP
A-mAP
V-mAP
AUROC
FAR
Attr. F1
AV-mAP
Xception
Official weights
48.41
3.53
22.27
0.30
–
0.59
50.59
5.47
18.96
0.11
–
0.23
50.18
5.33
37.21
0.25
MSLoc-PG
Official weights
72.37
0.97
47.97
26.51
–
53.01 †
57.86
3.22
22.82
3.82
–
7.65
56.88
5.00
40.18
0.92
ActionFormer
TASLE
73.82
0.00
47.55
37.50
–
75.00 †
77.50
2.10
24.05
10.52
–
21.05
66.78
2.89
44.00
15.15
Xception
AV-FABLE
52.63
1.60
24.24
0.15
–
0.31
53.26
3.86
19.44
0.25
–
0.49
53.91
4.33
38.46
0.67
DeMamba (Stage-1 window)
AV-FABLE
54.94
2.23
25.65
0.12
–
0.23
55.75
1.93
18.16
1.18
–
2.37
59.66
4.33
39.11
2.17
Table 3: OOD performance (%). Metrics, source-calibrated thresholds, rows, –, and bold/underline follow Table 2 . For source transfer, detection averages HDTF/TASLE, attribution and V-mAP pool them, and A-mAP uses HDTF (384 audio events). Held-out TI2V has visual edits only (AV-mAP = V-mAP). Attribution F1 uses three, four, and two classes, respectively. † TASLE overlaps the model’s training distribution. Joint-timeline metrics and Event-IoU: Table 12 .
Variant
ID
OOD-All
AUROC
FAR
Attr. F1
AV-mAP
A-mAP
V-mAP
A-IoU
V-IoU
AUROC
FAR
Attr. F1
AV-mAP
A-mAP
V-mAP
w/o intermediate-layer features
92.73 ± 0.17
4.93 ± 0.69
75.91 ± 0.52
84.75 ± 0.13
88.94 ± 0.12
80.56 ± 0.20
80.05 ± 0.71
65.24 ± 1.21
79.43 ± 0.25
5.29 ± 0.66
60.92 ± 0.69
53.73 ± 0.47
68.53 ± 0.67
42.66 ± 0.65
w/o interface features
95.58 ± 0.16
6.42 ± 0.80
83.35 ± 0.51
90.61 ± 0.10
92.12 ± 0.24
89.10 ± 0.12
84.81 ± 0.30
79.76 ± 1.51
81.83 ± 0.24
8.84 ± 0.16
64.82 ± 1.74
58.38 ± 0.40
72.95 ± 0.32
47.31 ± 0.76
w/o host conditioning
95.41 ± 0.08
5.00 ± 0.93
82.25 ± 0.48
89.86 ± 0.57
91.36 ± 0.86
88.36 ± 0.32
83.77 ± 0.33
74.13 ± 0.78
81.90 ± 0.82
4.45 ± 1.15
62.24 ± 1.05
57.72 ± 1.55
69.24 ± 2.93
47.89 ± 0.83
Full NFSR
96.04 ± 0.10
3.52 ± 0.33
83.61 ± 0.04
91.30 ± 0.13
92.24 ± 0.16
90.36 ± 0.28
84.72 ± 0.55
76.90 ± 0.78
84.61 ± 0.12
3.83 ± 0.44
64.77 ± 0.72
62.45 ± 0.08
73.66 ± 0.79
53.10 ± 0.56
Table 4: NFSR core component ablation. Three-seed mean ± sample standard deviation with matched 12-epoch training, splits, scoring, and thresholds. OOD-All averages Table 3 ; A-mAP uses its two audio-edit settings (Appendix C ). Bold/underline mark the best and second-best means (FAR not ranked). Joint-timeline metrics and OOD-All Event-IoU: Table 13 .
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Scene diversity in AV-FABLE. Example frames span indoor and outdoor environments, human activities, objects, food preparation, sports, and other contexts represented in the twin pairs.
Metric
Value
Retention: raw agreement
96.4%
Retention: Cohen’s κ
0.841
Explanation eligibility: Cohen’s κ
0.823
Overall third-reviewer adjudication rate
7.7%
Appendix
Table 7: Initial agreement on retention and explanation eligibility, and the overall third-reviewer adjudication rate. Rates use the complete reviewed event pool as the denominator.
Control
ID
OOD pool
Multifield probes: AUROC range
53.9–54.6
51.0–53.0
Strongest single field
51.9
–
Appendix
Table 8: Metadata-only authenticity probes: AUROC (%). Ranges summarize the evaluated multifield configurations. The OOD pool, used throughout the appendix analyses, pools all 1,002 AV-FABLE OOD pairs across the source, generator, and TI2V settings, without TASLE or equal weighting of settings.
Figure 4: Statistics and manipulation coverage in AV-FABLE. The six panels show video duration, event duration by modality, manipulated temporal coverage, tampering position, semantic operation, and aggregated generation route. Video-level distributions count each twin pair once; coverage is computed over the union of audio and visual intervals.
Figure 5: Source and generation composition in AV-FABLE. The left donut shows source-domain shares; the right donut shows modality attribution and the aggregated generation methods used to produce the edited events. The full list of editing configurations is given in Table 6 .
Dataset
Scale
Scene
Manip.
Loc.
Expl.
Pair
OOD shift
A
V
Src
Gen
Type/ route
Temporal and audio–visual benchmarks
AV-Deepfake1M ( Cai et al., 2024 )
1,146,760 v
Talking heads
✓
✓
✓
×
( ✓ )
×
×
×
DDL-AV ( Miao et al., 2025 )
330K v
Face-centric AV
✓
✓
✓
×
( ✓ )
( ✓ )
( ✓ )
×
ArEnAV ( Kuckreja et al., 2026 )
387,072 v
Arabic/English speech
✓
✓
✓
×
( ✓ )
×
×
×
X-AVFake (FakeHunter) ( Chen et al., 2025a )
>5,700 v
Open AV scenes
✓
✓
( ✓ )
✓
( ✓ )
×
×
×
Appendix
Table 9: Extended comparison with forensic datasets. Additional video, audio, and image benchmarks. Columns and symbols follow Table 1 .
Figure 6: Layer-wise forensic separability of Qwen3-Omni (a, b) and Qwen2.5-Omni (c, d). Each panel plots cell-level probe AUROC for each layer alone and concatenated with interface features; shading shows cross-validation variation, stars mark the selected layer, and dashed lines mark the fixed audio layer (L16). Visual features are spatially pooled; audio and visual depth indices are separate.
Training fraction
ID A
OOD A
ID V
OOD V
5%
55.7
16.8
5.6
5.1
10%
70.5
50.4
56.1
17.4
25%
83.6
74.7
80.0
39.3
50%
88.3
79.8
85.7
51.2
100%
92.1
84.6
90.7
55.2
Appendix
Table 20: Training-fraction sensitivity: native-track mAP (%). Fractions use the same epoch count, so sample count and optimizer updates vary together. OOD denotes the OOD pool. Entries are single-run results.
Figure 7: Manipulation examples in AV-FABLE. Each row gives a short description of the edit, the annotated interval, frames sampled across the event boundaries, and the local ground-truth (GT) timeline of the manipulated track; audio rows add the manipulated waveform with the event shaded. Green and red denote pristine and manipulated content, and frame borders follow the visual track. Rows (c) and (d) are co-timed events in one both-track video.
Figure 8: Localization successes and errors. (a) A visual-only object replacement and (b) an audio-only speech replacement are localized; (c) a visual object insertion is missed; (d) a pristine video yields a visual false alarm. Each row shows sampled frames, full-video GT and NFSR timelines of the displayed track, the annotated and predicted intervals, and the best tIoU. NFSR intervals are those above the fixed track threshold; frame borders follow the visual-track GT.
Figure 9: Four asynchronous events in one both-track video. Top: sampled frames and full-video GT and NFSR timelines for each track, with numbered GT events. Bottom: each event’s interval and operation/target, pristine and manipulated frames at the same timestamp (visual) or pristine and manipulated waveforms over the event window (audio), and the best tIoU. NFSR matches events 2–4 (tIoU ≥0.5 ) and misses event 1.
Figure 10: Explanation outputs on fixed NFSR candidates. Each example gives the annotated track, operation, target, and interval, the tIoU of the candidate supplied to both modes, and sampled frames. Zero-shot and SFT panels show the support verdict and the generated inside, context, and judgment fields; dashes mark empty fields.
The rapid advancement of AI-driven video generation has transformed content creation, while simultaneously increasing the risk of misinformation through localized manipulations in long-form videos. Existing video forensic methods predominantly operate on short, independent clips, and thus fail to capture realistic scenarios where AI-generated content is sparsely embedded within otherwise authentic footage. To bridge this gap, we formulate the task of Temporal AI-Generated Segment Localization and Explanation, which targets authenticity detection, temporal localization, and interpretable analysis of manipulated segments in untrimmed long videos. We further introduce TASLE, a large-scale benchmark comprising 12,472 untrimmed videos with diverse manipulation patterns and rich annotation signals, including temporal boundaries, authenticity labels, and segment-level rationales. In addition, we propose MSLoc, a coarse-to-fine forensic baseline that combines a boundary-sensitive proposal generation module for efficient long-video scanning with an MLLM-based refinement module for precise boundary localization and interpretable reasoning. Experiments validate the effectiveness of the proposed baseline, highlighting the importance of segment-level explainable forensics for long-form AI-generated video analysis. Our dataset and code are publicly available at https://debby-0527.github.io/TASLE.
Yue Feng, Jingjing Li, Qijia Lu +9
MoE Key Laboratory of Brain-Machine Intelligence Technology, College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics · Dalian University of Technology · Independent Scholar +2
Current Vision-Language Models (VLMs) for deepfake detection excel at identifying spatial artifacts but overlook a critical dimension: temporal inconsistencies in video forgeries. Adapting VLMs to reason about these dynamic cues remains a distinct challenge. To bridge this gap, we propose Forensic Answer-Questioning (FAQ), a large-scale benchmark that formulates temporal deepfake analysis as a multiple-choice task. FAQ introduces a three-level hierarchy to progressively evaluate and equip VLMs with forensic capabilities: (1) Facial Perception, testing the ability to identify static visual artifacts; (2) Temporal Deepfake Grounding, requiring the localization of dynamic forgery artifacts across frames; and (3) Forensic Reasoning, challenging models to synthesize evidence for final authenticity verdicts. We evaluate a range of VLMs on FAQ and generate a corresponding instruction-tuning set, FAQ-IT. Extensive experiments show that models fine-tuned on FAQ-IT achieve advanced performance on both in-domain and cross-dataset detection benchmarks. Ablation studies further validate the impact of our key design choices, confirming that FAQ is the driving force behind the temporal reasoning capabilities of these VLMs.
Zheyuan Gu, Qingsong Zhao, Yusong Wang +6
Institute of Artificial Intelligence, China Telecom (TeleAI) · Peking University · Fudan University
The rapid advancement of video editing and generative artificial intelligence technologies has made realistic video manipulation increasingly accessible. Although existing datasets have significantly advanced research in deepfake detection, object removal, and video inpainting, they do not adequately model scenarios in which a short manipulated segment is inserted into an otherwise authentic video and the original video continues afterward. In this study, we review representative datasets from the literature, analyze their characteristics, and discuss their limitations with respect to temporally localized realistic manipulation detection. Based on this analysis, we motivate the need for a new dataset specifically designed for authentic videos containing short and highly realistic manipulated intervals. Finally, we evaluate two complementary approaches on our custom-curated test set to establish an initial benchmark for this challenging scenario. The first employs a linear probe on DINOv3 features, assessed under three thresholding strategies. The second leverages DINOv3 features with a consecutive frame similarity-based method to detect temporal manipulation boundaries. Together, these experiments provide an initial benchmark for partially manipulated video detection and highlight the need for content-adaptive thresholding mechanisms. The dataset, code, and supplementary materials are publicly available at https://github.com/OkanUmur/temporally-localized-video-manipulation-detection.
Okan Umur, Ali Emre Güşlü, Ibrahim Delibasoglu
Department of Software Engineering Sakarya University Sakarya, Türkiye