cs.CVOct 8, 2026

A Unified Score Matching Paradigm for Video Anomaly Detection and Anticipation

Authors: Congqi Cao, Zhenhe Liang, Hanwen Zhang, Yifan Zhao, Qinyi Lv, Lingtong Min, Yanning Zhang

Organizations: National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science, Northwestern Polytechnical University, Xi’an 710129, China · School of Electronics and Information, Northwestern Polytechnical University, Xi’an 710129, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Video anomaly detection (VAD) is a fundamental and safety-critical task in computer vision. Recent generative approaches detect anomalies from a distributional perspective, but remain limited by local anomaly modes. Meanwhile, video anomaly anticipation (VAA), as a proactive extension beyond post-hoc detection, introduces additional challenges. In particular, the contrastive inference paradigm in VAD, which relies on ground-truth frames, is not applicable to VAA, hindering its development. To address these challenges, we propose a unified score-driven framework, termed Uni-DSM, based on denoising score matching (DSM), which models anomaly patterns through likelihood estimation and score functions over the learned data distribution. Within this unified framework, we adopt a shared noise-conditioned score transformer backbone with scene-dependent embeddings and motion-aware weighting for distribution-level modeling. Instead of introducing separate architectures, Uni-DSM unifies VAD and VAA through different inference and supervision paradigms built upon the same score-based formulation. For VAD, we instantiate an autoregressive denoising score matching (ADSM) mechanism, which progressively accumulates anomalous evidence via autoregressive denoising, enabling enhanced perception of local modes beyond visual cues. For VAA, we extend the same architecture by incorporating a lightweight auxiliary decoder and a novel self-distilled denoising score matching (SDSM) mechanism. By constructing supervision from output discrepancies instead of relying on unavailable future ground truth, our method achieves efficient training suitable or early anomaly anticipation. Extensive experiments on multiple benchmark datasets demonstrate state-of-the-art performance in both VAD and VAA while maintaining high efficiency, establishing a unified and scalable pipeline from anomaly detection to anticipation.

Figures & tables

Fig. 1: An illustration of the local modes of anomalies. We explore two regions in which the anomalies are residing. The likelihood-based method is highly sensitive to regions of anomalies with inherently low probability likelihood while blind to anomalies located in local modes near the learned distribution.
Fig. 2: An illustration of three video anomaly anticipation frameworks, where GT denotes ground truth. Compared with semantic memory bank and bidirectional frame prediction frameworks, our self-distilled DSM (SDSM) within Uni-DSM adopts a lightweight unidirectional main-auxiliary architecture, enabling efficient video anomaly anticipation with lower GPU memory consumption and faster inference speed.
Fig. 3: An overview of the Uni-DSM framework, a unified score-driven architecture based on denoising score matching (DSM). The red flow represents autoregressive denoising score matching (ADSM) for video anomaly detection, while the purple flow denotes self-distilled score matching (SDSM) for video anomaly anticipation.
Fig. 4: Schematic illustration of the video anomaly detection (VAD) and video anomaly anticipation (VAA) tasks.
Fig. 5: Overview of the proposed noise-conditional score transformer (NCST) architecture used in Fig. 3 .
Fig. 6: Detailed architecture of the NCST block utilized within the noise-conditional score transformer. MLP, FFN, and MHA stand for the multi-layer perception layer, the feed-forward neural network, and the multi-head attention layer, respectively.
Fig. 7: An overview of the proposed autoregressive denoising score matching process.
MethodAvenueShanghaiTechNWPU
MicroMacroMicroMacroMicroMacro
◇ FFP [ 14 ]84.9-72.8---
◇ AMMC-Net [ 15 ]86.6-73.7-64.5-
◇ HF 2 -VAD [ 16 ]91.193.576.283.163.7-
◇ VABD [ 17 ]86.6-78.2---
+ MTP [ 18 ]82.9-76.0---
TABLE I: Comparison of different methods on the Avenue, ShanghaiTech, and NWPU datasets in AUCs ( %\% ) metric. The best result on each dataset is shown in bold. We also list the inputs of the methods, in which method without marking denotes single modal input.
α\alpha0.5s1s1.5s2s2.5s3s
Chance50.050.050.050.050.050.0
Human–––––90.4
FBAE [ 12 ]65.865.364.964.664.264.0
SSAE [ 13 ]69.569.368.768.267.166.9
Uni-DSM (ours)71.371.170.570.269.569.1
TABLE II: Anticipation Results of different methods on the NWPU Dataset
MethodShanghaiTechNWPU
MicroMacroMicroMacro
L-ADSM77.687.568.173.2
Uni-DSM84.593.276.978.1
TABLE III: AUCs ( %\% ) of our model in latent space and original data space on ShanghaiTech and NWPU datasets.
MethodShanghaiTechNWPU
MicroMacroMicroMacro
MSMA [ 33 ]81.390.472.375.1
MULDE [ 34 ]82.690.473.174.5
Uni-DSM84.593.276.978.1
TABLE IV: AUCs ( %\% ) of different likelihood-based methods in the original data space on ShanghaiTech and NWPU datasets.
 DSMADSMSceneMotionAppearanceShanghaiTech
     MicroMacro
1✓    73.978.5
2✓✓   76.484.1
3✓✓✓  76.986.3
4✓✓ ✓ 78.687.7
5✓✓  ✓82.189.9
TABLE V: AUCs( %\% ) of our model variants on the ShanghaiTech dataset for the VAD task.
 DSMSDSMSceneMotionNWPU datasets
1✓   67.5
2✓✓  68.1
3✓✓✓ 68.6
4✓✓ ✓69.3
5✓✓✓✓70.5
TABLE VI: AUCs(%) of our model variants on the NWPU Dataset for the VAA task.
ModelGPU Memory (G)Running Speed (s)Frame Rate (FPS)Performance (%)
⋄ HF 2 -VAD [ 16 ]–0.066715.076.2
⋄ VABD [ 17 ]–0.014768.078.2
LLSH [ 45 ]–0.039225.577.6
FPDM [ 31 ]–0.12827.878.6
SSAE [ 13 ]240.109610.180.5
Uni-DSM (ours)110.019451.584.5
TABLE VII: Comparison of computational resource consumption during VAD inference on the ShanghaiTech dataset.
ModelGPU Memory (G)Running Speed (s)Frame Rate (FPS)Performance (%)
EPAP-Net [ 36 ]–0.021945.7–
FBAE [ 12 ]210.027436.564.9
SSAE [ 13 ]240.109610.168.7
SDSM (ours)160.016361.370.5
TABLE VIII: Comparison of Computational Resource Consumption During VAA Inference on the NWPU Dataset
Fig. 8: Visualization results on the NWPU dataset for the VAD task. The red boxes indicate that an anomaly is occurring, while the green boxes denote that the anomaly has disappeared.
Fig. 9: Visualization results on the NWPU dataset for the VAA task. The orange boxes indicate that an anomaly is about to occur, the red boxes denote that an anomaly is currently happening, and the yellow boxes signify that the anomaly is disappearing.
Figure 18
Figure 19
Figure 20
Figure 21
Figure 22
Figure 23
Figure 24

Explore similar work

CardsList
Aug 7, 2026cs.AI

Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning

Video Anomaly Detection (VAD) aims to identify anomalous events and localize their temporal intervals. Existing approaches exhibit a "when-what" dissociation: traditional DNN-based methods localize when anomalies occur but lack semantic understanding, whereas LLM-based methods explain what happens but neglect precise temporal grounding. We attribute this to the absence of a unified reasoning paradigm. Inspired by how humans inspect surveillance videos - glancing globally to form temporal hypotheses, scrutinizing suspicious segments, and thinking iteratively to correct errors - we study this global-to-local paradigm from two perspectives. We first propose Glance then Scrutinize (GtS), a training-free framework using static and dynamic textual guidance for coarse-to-fine anomaly grounding and understanding, balancing accuracy and speed. To break the ceiling imposed by frozen external modules, we further propose a tool-augmented agentic VAD method, where a multimodal large language model learns to invoke a video cropping tool, inspect densely resampled frames, and self-correct mislocalized hypotheses, via cold-start supervised fine-tuning followed by reinforcement learning with a joint answer-grounding reward. For training and evaluation, we extend our prior VAGU benchmark into VAGU-T (Video Anomaly Grounding, Understanding, and Thinking), comprising 7,567 real-world videos over 21 anomaly categories with human-validated grounding, explanations, QA pairs, and chain-of-thought tool-calling traces. We further introduce JeAUG, a metric jointly evaluating semantic interpretability and temporal precision. Experiments show that GtS substantially surpasses training-free baselines, while the agentic model delivers both higher accuracy and faster inference.

Shibo Gao, Peipei Yang, Xu-Yao Zhang +1

School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China. · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.

Anomaly LocalizationVideo Anomaly DetectionMultimodal Agents
AskPDF
Sep 15, 2026cs.CV

Probe-VAD: Ordinal Likelihood Probing for Training-Free Video Anomaly Detection

Video anomaly detection (VAD) aims to localize anomalous events in untrimmed videos. Vision-language models (VLMs) provide rich visual understanding for training-free VAD, but existing approaches impose restrictive interfaces between visual understanding and anomaly scoring. Caption-based pipelines compress visual evidence into text, potentially discarding subtle cues, while direct numerical generation forces the model to express its judgment through a small set of predefined scores. Such interfaces can obscure subtle differences in anomaly severity, causing visually distinct clips to receive similar representations or scores and thereby limiting the resolution of anomaly ranking. We propose \textbf{Probe-VAD}, an ordinal binary-probing framework that directly probes severity preferences from a frozen VLM. Given raw video clips, Probe-VAD queries ten ordered severity thresholds and extracts constrained \textit{YES}/\textit{NO} continuation likelihoods. Their normalized preferences form a cumulative severity profile, from which tail evidence is aggregated into a continuous anomaly score, with isotonic projection enforcing ordinal consistency. Experiments on public VAD benchmarks demonstrate superior performance with low computational cost. Probe-VAD provides a simple interface for translating frozen VLM visual understanding into continuous, rank-sensitive anomaly scores without task-specific training or caption-based compression. Code is available at: https://github.com/yvestine/COVAS-VAD.

Jiawei Gu, Qilin Zhao, Tengkuo Guo +6

School of Intelligence Science and Technology, Nanjing University, Suzhou 215163, China · Faculty of Life Science and Medicine, School of Medicine and Health, Harbin Institute of Technology, Harbin 150001, China · School of Instrumentation Science and Engineering, Harbin Institute of Technology, Harbin 150001, China +2

Video Anomaly DetectionZero-Shot Anomaly Detection
AskPDF
Oct 1, 2026cs.CV

Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understanding

Video Anomaly Detection (VAD) aims to temporally localize abnormal events in videos. Most existing approaches rely on dataset-specific training and curated annotations, limiting generalization in open-set scenarios. Recent zero-shot methods based on Large Vision- Language Models (LVLMs) alleviate this dependency but often lack temporal continuity and structured reasoning. We propose Cog-VADU, a fully training-free framework that reformulates VAD as a sequential cognitive reasoning task. Cog-VADU introduces Chain-of- Anomaly Detection Thought Prompting (CoADTP), which unrolls an LVLM into a recurrent reasoning chain across video segments. By propagating structured rationales over time, the model maintains implicit temporal memory, enabling robust discrimination between com- plex anomalies and high-motion normal activities. To improve reliability, we further design a cross-modal re-ranking stage that aligns textual rationales with visual embeddings, enforcing semantic consistency and temporal coherence for refined and stable predictions. Extensive experiments on multiple public VAD benchmarks demonstrate that Cog-VADU achieves competitive zero-shot performance. Moreover, cross-model evaluations show that CoADTP consistently enhances reasoning-based anomaly detection in a model-agnostic manner, pro- viding interpretable and generalizable anomaly understanding for real-world applications.

Mohd Ubaid Wani, Sara Atito, Josef Kittler +1

Centre for Vision, Speech and Signal Processing (CVSSP) University of Surrey, UK · Surrey Institute for People-Centred AI (PAI) University of Surrey, UK

CoT ReasoningInterpretable Anomaly DetectionVideo Anomaly DetectionZero-Shot Anomaly DetectionTemporal Video Understanding
AskPDF