As the scale of video surveillance data outpaces manual annotation capacities, weakly supervised video anomaly detection (WSVAD) has emerged as a critical research frontier. Most existing approaches formulate WSVAD within a Multiple Instance Learning (MIL) framework that relies on rigid, hand-crafted temporal priors to supervise anomaly scoring. However, such formulations exhibit limited adaptability to the wide variation in anomaly durations and temporal dynamics observed in real-world videos, often leading to unstable or unreliable snippet-level predictions. To address this limitation, we propose an adaptive temporal modeling framework for WSVAD that explicitly accounts for variations in video dynamics across multiple temporal granularities. First, we introduce a Temporal Refinement Module (TRM) that leverages dynamic positional encoding and a learnable class token to model long-range temporal dependencies while distilling a stable global video-level representation. Second, to capture anomalous events with varying frequency and duration, we develop an adaptive Event Segmentation Module (ESM) that identifies event boundaries through temporal discontinuity analysis and aggregates snippet features into discriminative event-level representations. Finally, for snippet-level and event-level predictions, we propose an adaptive similarity-based fusion strategy that dynamically integrates anomaly scores into video-level predictions, replacing fixed top-k aggregation heuristics with global semantic relevance. Extensive experiments on two benchmarks demonstrate that the proposed framework consistently outperforms state-of-the-art methods.
Figures & tables
Figure 1: Overview of the proposed framework. Snippet-level features are first extracted from input videos using backbone networks. These features are then refined by the TRM to capture long-range temporal dependencies and produce a global video-level representation enriched with contextual information. Subsequently, the ESM is adaptively applied to identify event boundaries and aggregate snippet features into event-level representations. Finally, the model is jointly optimized through a multi-scale prediction strategy that integrates snippet-, event-, and video-level outputs.
Supervision
Methods
Feature Encoder
Ground Truth
AUC(%)
FAR(%)
Semi
Object-Centric [ 7 ]
-
N
61.60
-
Conv-AE [ 5 ]
AE
N
50.60
27.2
Weak
HL-Net [ 19 ]
I3D
B
82.44
-
RTFM [ 16 ]
I3D
B
84.30
-
DDL [ 12 ]
I3D
B
85.12
-
MGFN [ 2 ]
I3D
B
86.98
-
Table 1: Performance comparison of SOTA methods on the UCF-Crime dataset.
Supervision
Methods
Feature Encoder
Ground Truth
AP(%)
FAR(%)
Semi
OCSVM [ 13 ]
I+V
N
27.25
-
Conv-AE [ 5 ]
I+V
N
30.77
-
Weak
HL-Net [ 19 ]
I3D
B
75.41
-
RTFM [ 16 ]
I3D
B
77.81
-
DDL [ 12 ]
I3D
B
80.72
-
MGFN [ 2 ]
I3D
B
79.19
-
Table 2: Performance comparison of SOTA methods on the XD-Violence dataset.
No
I3D
TRM
ESM
class token
AUC(%)
FAR(%)
1
✓
-
-
-
82.50
1.84
2
✓
✓
-
-
84.78
0.85
3
✓
✓
-
✓
85.52
0.53
4
✓
✓
✓
✓
87.24
0.34
Table 3: Ablation study of different modules in our method on the UCF-Crime dataset.
Figure 2: Visualized anomaly detection results of our proposed method. Examples in the first row are from the UCF-Crime dataset, while those in the second row are from the XD-Violence dataset. In each plot, the blue curve indicates the anomaly score at each time step, with green backgrounds representing normal segments and red backgrounds denoting anomalous segments.