Temporal Forgery Localization (TFL) aims to precisely identify manipulated segments within videos or audio streams, providing interpretable evidence for multimedia forensics and security. While most existing TFL methods rely on dense frame-level labels in a fully supervised manner, Weakly Supervised TFL (WS-TFL) reduces labeling cost by learning only from binary video-level labels. However, current WS-TFL approaches suffer from mismatched training and inference objectives, limited supervision from binary labels, gradient blockage caused by non-differentiable top-k aggregation, and the absence of explicit modeling of inter-proposal relationships. To address these issues, we propose GEM-TFL (Graph-based EM-powered Temporal Forgery Localization), a two-phase classification-regression framework that effectively bridges the supervision gap between training and inference. Built upon this foundation, (1) we enhance weak supervision by reformulating binary labels into multi-dimensional latent attributes through an EM-based optimization process; (2) we introduce a training-free temporal consistency refinement that realigns frame-level predictions for smoother temporal dynamics; and (3) we design a graph-based proposal refinement module that models temporal-semantic relationships among proposals for globally consistent confidence estimation. Extensive experiments on benchmark datasets demonstrate that GEM-TFL achieves more accurate and robust temporal forgery localization, substantially narrowing the gap with fully supervised methods.
Figures & tables
Figure 1: Comparison between prior and our WS-TFL pipelines. (a) Prior methods [ 40 , 39 ] : trained with clip-level labels but required for temporal localization, leading to noisy proposals due to the training–inference mismatch. (b) Ours: a two-phase design aligning classification and regression to refine coarse predictions for precise boundary localization.
Figure 2: Overview of the classification phase. (a) Label Attribute Decomposition: The feature enhancement module aligns and fuses audio–visual features, after which the attention and attribute branches produce frame-level attention and attribute predictions optimized through EM to capture diverse forgery patterns. (b) Temporal Consistency Refinement: The non-differentiable top- k operation blocks gradient flow between the attention and attribute branches, causing inconsistent temporal responses. To address this, frame-level attribute predictions are alternately projected onto the row constraint (attention-weighted alignment between frame- and clip-level predictions) and the column constraint (categorical distribution). The refined attribute predictions are then used to generate initial pseudo proposals. (c) Graph-based Proposal Refinement: Proposals are mapped into a unified space, where a proposal graph is constructed and confidence values are diffused to obtain fusion weights. These are integrated and thresholded at zero to produce the final pseudo proposals, merging fragmented proposals (e.g., p1 , p2 , p3 ) into continuous ones (e.g., pfuse ) from a global and relational perspective.
Supervision
Method
Modality
mAP@IoU(%)
mAR@Proposals(%)
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Avg.
20
10
5
2
Avg.
Fully
AFormer ECCV’22 [ 44 ]
V
98.0
97.7
97.3
96.8
96.3
95.5
94.6
96.6
99.2
99.0
98.4
96.0
98.1
TriDet CVPR’23 [ 31 ]
V
95.0
94.7
94.4
93.8
93.2
92.2
90.7
93.4
97.1
96.9
96.3
93.7
96.0
UFormer MM’23 [ 46 ]
AV
97.7
97.6
97.4
97.1
96.7
96.0
94.9
96.8
98.6
98.5
98.2
95.1
97.6
MFMS MM’24 [ 47 ]
AV
98.0
97.9
97.8
97.6
97.3
96.7
95.8
97.3
98.9
98.9
98.6
95.6
98.0
Weakly
CoLA CVPR’21 [ 43 ]
V
31.3
25.9
19.4
13.4
8.6
5.2
2.7
15.2
41.2
41.2
40.8
37.8
40.3
Table 1: Temporal forgery localization results of fully and weakly supervised methods on LAV-DF [ 5 ] . ’UFormer’ and ’PFormer’ denote UMMAFormer [ 46 ] and PseudoFormer [ 22 ] . ’V’, ’A’, and ’AV’ indicate visual, audio, and audio–visual modalities. Fully supervised methods use forged-segment timestamps as supervision, while weakly supervised ones use binary clip-level labels, except WMMT [ 39 ] , which adopts a quadruple signal (both real, both fake, visual fake only, and audio fake only). The best mAP/mAR are highlighted in red , and the second-best in blue .
Figure 3: Comparison between prior and our WS-TFL pipelines.
Supervision
Method
Modality
mAP@IoU(%)
mAR@Proposals(%)
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Avg.
20
10
5
2
Avg.
Fully
AFormer ECCV’22 [ 44 ]
V
67.3
67.3
67.2
67.2
67.0
66.7
65.5
66.9
83.1
82.9
82.6
78.7
81.8
TriDet CVPR’23 [ 31 ]
V
55.7
55.6
55.4
55.1
54.7
53.8
51.7
54.6
74.9
74.2
72.9
68.0
72.5
UFormer MM’23 [ 46 ]
AV
91.8
91.7
91.5
91.3
91.0
90.4
89.0
90.9
95.0
94.6
93.9
89.5
93.3
MFMS MM’24 [ 47 ]
AV
94.7
94.6
94.6
94.5
94.3
93.9
92.5
94.1
96.7
96.4
96.0
92.0
95.3
Weakly
CoLA CVPR’21 [ 43 ]
V
3.22
1.09
0.39
0.14
0.05
0.02
0.01
0.70
20.7
20.0
16.1
8.17
16.2
Table 2: Temporal forgery localization results of fully and weakly supervised methods on the AV-Deepfake1M [ 4 ] benchmark.
#
LAD
TCR
GPR
LP
mAP (0.1:0.7)
mAR (0.1:0.7)
Param [M]
1
✗
✗
✗
✗
5.12
35.6
17.1
2
✓
✗
✗
✗
23.8 ↑ 18.7
44.3 ↑ 8.7
17.2
3
✓
✓
✗
✗
27.3 ↑ 22.2
46.7 ↑ 11.1
17.2
4
✓
✓
✓
✗
31.9 ↑ 26.8
51.2 ↑ 15.6
17.2
5
✓
✓
✓
✓
42.7 ↑ 37.6
59.8 ↑ 24.2
62.2
Table 3: Ablation study on the components. Performance gains relative to the base model (row #1) are indicated as subscripts.
Figure 4: T-SNE visualizations of classification features under different latent forgery attribute d m . Overlaps between attributes are highlighted in gray. “binary” denotes the setting without EM optimization, where the model directly performs binary classification.
Method
mAP@IoU(%)
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Avg.
CoLA
1.35
0.13
0.03
0.01
0.01
0.01
0.01
0.22
LOCO
0.27
0.04
0.01
0.00
0.00
0.00
0.00
0.05
MDP
15.5
9.10
7.76
4.01
2.33
1.83
0.42
5.85
PFormer
26.3
22.7
16.6
12.7
9.28
6.44
2.15
13.7
WMMT
21.7
18.5
15.7
13.1
10.2
7.39
4.45
13.0
Table 4: Cross-dataset generalization performance. Models are trained on AV-Deepfake1M [ 4 ] and tested on LAV-DF [ 5 ] .