AV-FABLE: Audio-Visual Manipulation Forensics in Long-Form Videos

Yuesheng Huang; Hoi Fan Au; Jiaqi Hou; Ruicheng Zhang; Jiarui He; Haocheng Yang; Bin Hu; Kewei Zhang

cs.CVOct 24, 2026

AV-FABLE: Audio-Visual Manipulation Forensics in Long-Form Videos

Authors: Yuesheng Huang, Hoi Fan Au, Jiaqi Hou, Ruicheng Zhang, Jiarui He, Haocheng Yang, Bin Hu, Kewei Zhang,

Organizations: Tsinghua University · Peking University

Abstract

Localized AI manipulations in long-form videos often affect brief events within otherwise authentic content, making video-level classification insufficient for forensic analysis. Existing audio-visual deepfake benchmarks largely focus on short talking-head clips, while long-form temporal forensics remains predominantly visual. We introduce AV-FABLE, a dual-timeline benchmark for sparse manipulations in open-scene long-form audio-visual videos. AV-FABLE contains 16,650 pristine--manipulated twin pairs (998 hours) created with 27 editing configurations, with separate audio and visual timelines that enable modality-specific temporal localization and expose errors hidden by joint-timeline evaluation. On AV-FABLE, zero-shot prompting of audio-visual large language models (AV-LLMs) yields near-chance detection and poor temporal localization. In contrast, supervised readouts of their frozen perception features recover strong forensic signals, suggesting manipulation evidence is encoded in these representations but not effectively exposed through language outputs. Motivated by this observation, we propose NFSR(Native Forensics via Separated Readouts), which reads frozen AV-LLM features with separate audio and visual temporal decoders. NFSR achieves the best audio-visual localization performance in-domain and across all three out-of-distribution settings, including transfer to held-out videos from an external long-video forensic benchmark. AV-FABLE, code, and model weights will be publicly released.

Figures & tables

Figure 1: Overview of SEE-Forge, the event-driven pipeline for constructing AV-FABLE. Given an event in a pristine video, SEE-Forge plans audio and/or visual manipulations, executes them with editors, and reinserts them into context to create pristine–manipulated twin pairs. The executed interval provides temporal ground truth, while human-reviewed annotations record forensic cues within and around the manipulated event.

Dataset	Scale	Scene	Length	Manip.		Loc.	Expl.	Pair	OOD shift
				A	V				Src	Gen	Type
FaceForensics++ ( Rössler et al., 2019 )	6,000 v	Faces	Clip	$\times$	$\checkmark$	$\times$	$\times$	( $\checkmark$ )	$\times$	( $\checkmark$ )	( $\checkmark$ )
Celeb-DF (v2) ( Li et al., 2020 )	6,529 v	Celebrity faces	Clip	$\times$	$\checkmark$	$\times$	$\times$	( $\checkmark$ )	$\times$	$\times$	$\times$
DFDC ( Dolhansky et al., 2020 )	128,154 v	Consented actors	Clip	( $\checkmark$ )	$\checkmark$	$\times$	$\times$	( $\checkmark$ )	( $\checkmark$ )	( $\checkmark$ )	$\times$
FakeAVCeleb ( Khalid et al., 2021 )	20,000 v	Talking heads	Clip	$\checkmark$	$\checkmark$	$\times$	$\times$	( $\checkmark$ )	$\times$	$\times$	$\times$
ForgeryNet ( He et al., 2021 )	221,247 v	Faces	Clip	$\times$	$\checkmark$	$\checkmark$	$\times$	( $\checkmark$ )	$\times$	$\times$	$\times$

Table 1: Forensic benchmarks comparison. Length: short clips (Clip) or minute-scale videos (Long). Manip.: audio (A) or visual (V) manipulations; Loc.: temporal localization; Expl.: explanations; Pair: pristine–manipulated twins; OOD shift: held-out sources, generators, or manipulation types/routes. (

\checkmark

) : partial support, or shift without a dedicated held-out split.

Figure 2: Overview of NFSR. (a) Frozen intermediate-layer (“detail”) and interface features pass through modality-specific gated projection and host-conditioned FiLM. (b) Independent decoders localize audio and visual manipulations on separate timelines. (c) Source-calibrated thresholds give video-level detection and four-way attribution without changing the intervals. Audio and video excerpts come from the same clip; intervals are schematic.

Method	Training Form	Output	AUROC	FAR	Attr. F1	AV-mAP	A-mAP	V-mAP	Joint-mAP	A-IoU	V-IoU	Joint-IoU
Xception	Official weights	V win	50.71	4.94	17.62	0.01	–	0.02	0.02	–	0.22	0.24
MSLoc-PG	Official weights	V	57.34	3.54	20.37	0.95	–	1.90	1.36	–	4.55	3.40
ActionFormer	TASLE	V	67.94	2.78	20.99	5.06	–	10.13	7.56	–	5.56	4.55
Xception	AV-FABLE	V win	66.27	4.94	23.27	0.04	–	0.08	0.06	–	4.87	3.98
DeMamba (Stage-1 window)	AV-FABLE	V win	74.35	4.26	28.77	24.75	–	49.49	26.83	–	35.21	28.54
ActionFormer	AV-FABLE	V	84.55	5.84	34.95	44.61	–	89.22	59.93	–	80.66	62.71

Table 2: ID main results on AV-FABLE (%). Blocks are visual-only, audio-only, and audio-visual. Outputs are separate audio/visual intervals (A/V), one track (A or V), fixed-window intervals (win), modality-labeled language timestamps (Text), or modality-unlabeled intervals (Unified). – marks unsupported metrics; absent tracks count as misses in AV-mAP and Joint-timeline metrics. NFSR, UMMAFormer, and TASLE-trained ActionFormer are three-seed means (Table 18 ); others are single runs. Bold and underlined values mark the best and second-best results per column; FAR is an operating-point diagnostic and is not ranked.

Method	Training Form	Source transfer						Held-out generators						Held-out TI2V
Method	Training Form	AUROC	FAR	Attr. F1	AV-mAP	A-mAP	V-mAP	AUROC	FAR	Attr. F1	AV-mAP	A-mAP	V-mAP	AUROC	FAR	Attr. F1	AV-mAP
Xception	Official weights	48.41	3.53	22.27	0.30	–	0.59	50.59	5.47	18.96	0.11	–	0.23	50.18	5.33	37.21	0.25
MSLoc-PG	Official weights	72.37	0.97	47.97	26.51	–	53.01 †	57.86	3.22	22.82	3.82	–	7.65	56.88	5.00	40.18	0.92
ActionFormer	TASLE	73.82	0.00	47.55	37.50	–	75.00 †	77.50	2.10	24.05	10.52	–	21.05	66.78	2.89	44.00	15.15
Xception	AV-FABLE	52.63	1.60	24.24	0.15	–	0.31	53.26	3.86	19.44	0.25	–	0.49	53.91	4.33	38.46	0.67
DeMamba (Stage-1 window)	AV-FABLE	54.94	2.23	25.65	0.12	–	0.23	55.75	1.93	18.16	1.18	–	2.37	59.66	4.33	39.11	2.17

Table 3: OOD performance (%). Metrics, source-calibrated thresholds, rows, –, and bold/underline follow Table 2 . For source transfer, detection averages HDTF/TASLE, attribution and V-mAP pool them, and A-mAP uses HDTF (384 audio events). Held-out TI2V has visual edits only (AV-mAP

{}={}

V-mAP). Attribution F1 uses three, four, and two classes, respectively. † TASLE overlaps the model’s training distribution. Joint-timeline metrics and Event-IoU: Table 12 .

Variant	ID								OOD-All
Variant	AUROC	FAR	Attr. F1	AV-mAP	A-mAP	V-mAP	A-IoU	V-IoU	AUROC	FAR	Attr. F1	AV-mAP	A-mAP	V-mAP
w/o intermediate-layer features	92.73 $\pm$ 0.17	4.93 $\pm$ 0.69	75.91 $\pm$ 0.52	84.75 $\pm$ 0.13	88.94 $\pm$ 0.12	80.56 $\pm$ 0.20	80.05 $\pm$ 0.71	65.24 $\pm$ 1.21	79.43 $\pm$ 0.25	5.29 $\pm$ 0.66	60.92 $\pm$ 0.69	53.73 $\pm$ 0.47	68.53 $\pm$ 0.67	42.66 $\pm$ 0.65
w/o interface features	95.58 $\pm$ 0.16	6.42 $\pm$ 0.80	83.35 $\pm$ 0.51	90.61 $\pm$ 0.10	92.12 $\pm$ 0.24	89.10 $\pm$ 0.12	84.81 $\pm$ 0.30	79.76 $\pm$ 1.51	81.83 $\pm$ 0.24	8.84 $\pm$ 0.16	64.82 $\pm$ 1.74	58.38 $\pm$ 0.40	72.95 $\pm$ 0.32	47.31 $\pm$ 0.76
w/o host conditioning	95.41 $\pm$ 0.08	5.00 $\pm$ 0.93	82.25 $\pm$ 0.48	89.86 $\pm$ 0.57	91.36 $\pm$ 0.86	88.36 $\pm$ 0.32	83.77 $\pm$ 0.33	74.13 $\pm$ 0.78	81.90 $\pm$ 0.82	4.45 $\pm$ 1.15	62.24 $\pm$ 1.05	57.72 $\pm$ 1.55	69.24 $\pm$ 2.93	47.89 $\pm$ 0.83
Full NFSR	96.04 $\pm$ 0.10	3.52 $\pm$ 0.33	83.61 $\pm$ 0.04	91.30 $\pm$ 0.13	92.24 $\pm$ 0.16	90.36 $\pm$ 0.28	84.72 $\pm$ 0.55	76.90 $\pm$ 0.78	84.61 $\pm$ 0.12	3.83 $\pm$ 0.44	64.77 $\pm$ 0.72	62.45 $\pm$ 0.08	73.66 $\pm$ 0.79	53.10 $\pm$ 0.56

Table 4: NFSR core component ablation. Three-seed mean

\pm

sample standard deviation with matched 12-epoch training, splits, scoring, and thresholds. OOD-All averages Table 3 ; A-mAP uses its two audio-edit settings (Appendix C ). Bold/underline mark the best and second-best means (FAR not ranked). Joint-timeline metrics and OOD-All Event-IoU: Table 13 .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Scene diversity in AV-FABLE. Example frames span indoor and outdoor environments, human activities, objects, food preparation, sports, and other contexts represented in the twin pairs.

Metric	Value
Retention: raw agreement	96.4%
Retention: Cohen’s $\kappa$	0.841
Explanation eligibility: Cohen’s $\kappa$	0.823
Overall third-reviewer adjudication rate	7.7%

Appendix

Table 7: Initial agreement on retention and explanation eligibility, and the overall third-reviewer adjudication rate. Rates use the complete reviewed event pool as the denominator.

Control	ID	OOD pool
Multifield probes: AUROC range	53.9–54.6	51.0–53.0
Strongest single field	51.9	–

Appendix

Table 8: Metadata-only authenticity probes: AUROC (%). Ranges summarize the evaluated multifield configurations. The OOD pool, used throughout the appendix analyses, pools all 1,002 AV-FABLE OOD pairs across the source, generator, and TI2V settings, without TASLE or equal weighting of settings.

Figure 4: Statistics and manipulation coverage in AV-FABLE. The six panels show video duration, event duration by modality, manipulated temporal coverage, tampering position, semantic operation, and aggregated generation route. Video-level distributions count each twin pair once; coverage is computed over the union of audio and visual intervals.

Figure 5: Source and generation composition in AV-FABLE. The left donut shows source-domain shares; the right donut shows modality attribution and the aggregated generation methods used to produce the edited events. The full list of editing configurations is given in Table 6 .

Dataset	Scale	Scene	Manip.		Loc.	Expl.	Pair	OOD shift
			A	V				Src	Gen	Type/ route
Temporal and audio–visual benchmarks
AV-Deepfake1M ( Cai et al., 2024 )	1,146,760 v	Talking heads	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	( $\checkmark$ )	$\times$	$\times$	$\times$
DDL-AV ( Miao et al., 2025 )	330K v	Face-centric AV	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	( $\checkmark$ )	( $\checkmark$ )	( $\checkmark$ )	$\times$
ArEnAV ( Kuckreja et al., 2026 )	387,072 v	Arabic/English speech	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	( $\checkmark$ )	$\times$	$\times$	$\times$
X-AVFake (FakeHunter) ( Chen et al., 2025a )	$>5,700$ v	Open AV scenes	$\checkmark$	$\checkmark$	( $\checkmark$ )	$\checkmark$	( $\checkmark$ )	$\times$	$\times$	$\times$

Appendix

Table 9: Extended comparison with forensic datasets. Additional video, audio, and image benchmarks. Columns and symbols follow Table 1 .

Figure 6: Layer-wise forensic separability of Qwen3-Omni (a, b) and Qwen2.5-Omni (c, d). Each panel plots cell-level probe AUROC for each layer alone and concatenated with interface features; shading shows cross-validation variation, stars mark the selected layer, and dashed lines mark the fixed audio layer (L16). Visual features are spatially pooled; audio and visual depth indices are separate.

Training fraction	ID A	OOD A	ID V	OOD V
5%	55.7	16.8	5.6	5.1
10%	70.5	50.4	56.1	17.4
25%	83.6	74.7	80.0	39.3
50%	88.3	79.8	85.7	51.2
100%	92.1	84.6	90.7	55.2

Appendix

Table 20: Training-fraction sensitivity: native-track mAP (%). Fractions use the same epoch count, so sample count and optimizer updates vary together. OOD denotes the OOD pool. Entries are single-run results.

Figure 7: Manipulation examples in AV-FABLE. Each row gives a short description of the edit, the annotated interval, frames sampled across the event boundaries, and the local ground-truth (GT) timeline of the manipulated track; audio rows add the manipulated waveform with the event shaded. Green and red denote pristine and manipulated content, and frame borders follow the visual track. Rows (c) and (d) are co-timed events in one both-track video.

Figure 8: Localization successes and errors. (a) A visual-only object replacement and (b) an audio-only speech replacement are localized; (c) a visual object insertion is missed; (d) a pristine video yields a visual false alarm. Each row shows sampled frames, full-video GT and NFSR timelines of the displayed track, the annotated and predicted intervals, and the best tIoU. NFSR intervals are those above the fixed track threshold; frame borders follow the visual-track GT.

Figure 9: Four asynchronous events in one both-track video. Top: sampled frames and full-video GT and NFSR timelines for each track, with numbered GT events. Bottom: each event’s interval and operation/target, pristine and manipulated frames at the same timestamp (visual) or pristine and manipulated waveforms over the event window (audio), and the best tIoU. NFSR matches events 2–4 (tIoU $\geq 0.5$ ) and misses event 1.

Figure 10: Explanation outputs on fixed NFSR candidates. Each example gives the annotated track, operation, target, and interval, the tIoU of the candidate supplied to both modes, and sampled frames. Zero-shot and SFT panels show the support verdict and the generated inside, context, and judgment fields; dashes mark empty fields.

Video Forensics

See all 21 papers

Explainable Forensics of Manipulated Segments in Untrimmed Long Videos
Jun 1, 2026Yue Feng, Jingjing Li, Qijia Lu +9Digital Forensics Deepfake Detection
MoE Key Laboratory of Brain-Machine Intelligence Technology, College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics · Dalian University of Technology · Independent Scholar +2
Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models
Feb 25, 2026Zheyuan Gu, Qingsong Zhao, Yusong Wang +6Deepfake Detection Video-Language Model Evaluation
Institute of Artificial Intelligence, China Telecom (TeleAI) · Peking University · Fudan University
Detecting Temporally Localized Manipulations in Authentic Video Streams
Jun 5, 2026Okan Umur, Ali Emre Güşlü, Ibrahim DelibasogluDeepfake Detection Temporal Video Forgery Localization
Department of Software Engineering Sakarya University Sakarya, Türkiye

Jun 1, 2026J/K move · Enter open · S save

Explainable Forensics of Manipulated Segments in Untrimmed Long Videos

Yue Feng, Jingjing Li, Qijia Lu, Wei Ji+8

MoE Key Laboratory of Brain-Machine Intelligence Technology, College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics · Dalian University of Technology · Independent Scholar+2

Digital Forensics Deepfake Detection Long-Video Understanding Temporal Video Forgery Localization

Open paper AskPDF arXiv

The rapid advancement of AI-driven video generation has transformed content creation, while simultaneously increasing the risk of misinformation through localized manipulations in long-form videos. Existing video forensic methods predominantly operate on short, independent clips, and thus fail to capture realistic scenarios where AI-generated content is sparsely embedded within otherwise authentic footage. To bridge this gap, we formulate the task of Temporal AI-Generated Segment Localization and Explanation, which targets authenticity detection, temporal localization, and interpretable analysis of manipulated segments in untrimmed long videos. We further introduce TASLE, a large-scale benchmark comprising 12,472 untrimmed videos with diverse manipulation patterns and rich annotation signals, including temporal boundaries, authenticity labels, and segment-level rationales. In addition, we propose MSLoc, a coarse-to-fine forensic baseline that combines a boundary-sensitive proposal generation module for efficient long-video scanning with an MLLM-based refinement module for precise boundary localization and interpretable reasoning. Experiments validate the effectiveness of the proposed baseline, highlighting the importance of segment-level explainable forensics for long-form AI-generated video analysis. Our dataset and code are publicly available at https://debby-0527.github.io/TASLE.

Abstract

Video Forensics

Explore similar work

Explainable Forensics of Manipulated Segments in Untrimmed Long Videos

Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models

Detecting Temporally Localized Manipulations in Authentic Video Streams