In this paper, we address the challenging problem of single-scene, fully unsupervised video anomaly detection (VAD), where raw videos containing both normal and abnormal events are used directly for training and testing without any labels. This differs sharply from prior work that either requires extensive labeling (fully or weakly supervised) or depends on normal-only videos (one-class classification), which are vulnerable to distribution shifts and contamination. We propose an entropy-guided autoencoder that detects anomalies through reconstruction error by reconstructing normal frames well while making anomalies reconstruct poorly. The key idea is to combine the standard reconstruction loss with a novel Minimal Latent Entropy (MLE) loss in the autoencoder. Reconstruction loss alone maps normal and abnormal inputs to distinct latent clusters due to their inherent differences, but also risks reconstructing anomalies too well to detect. Therefore, MLE loss addresses this by minimizing the entropy of latent embeddings, encouraging them to concentrate around high-density regions. Since normal frames dominate the raw video, sparse anomalous embeddings are pulled into the normal cluster, so the decoder emphasizes normal patterns and produces poor reconstructions for anomalies. This dual-loss design produces a clear reconstruction gap that enables effective anomaly detection. Extensive experiments on two widely used benchmarks and a challenging self-collected driving dataset demonstrate that our method achieves robust and superior performance over baselines.
Figures & tables
Figure 1: Collapse of the anomalous latent distribution. Green: normal frames. Red: abnormal frames. Solid lines: latent distributions of the original inputs. Dashed lines: latent distributions of the reconstructions. The key idea is to pull sparse anomalous embeddings toward the dense normal embeddings, so normal frames remain well reconstructed while anomalies reconstruct poorly.
Figure 2: MLE-UVAD pipeline: unsupervised training and detection through reconstruction gap. Unlabeled videos are directly put into an autoencoder trained with a dual loss: (1) The MSE loss reconstructs all frames accurately. (2) The MLE collapses sparse abnormal embeddings toward the dense normal embedding, making anomaly reconstructions worse relative to normals. An anomaly is detected by setting a threshold on the reconstruction error.
Figure 3: t-SNE visualization of latent embeddings across different methods (blue = normal, orange = abnormal). Baselines, only MSE and GCL [ 32 ] , keep the normal and abnormal embeddings separate for better reconstruction. In contrast, our proposed MLE loss regularizes the latent distribution by collapsing the abnormal distribution into the dominant normal cluster.
Category
Method
Donkeycar
Corridor
UBnormal
OCC (Semi-supervised)
DAST [ 10 ]
0.999
0.996
0.991
ROADMAP [ 29 ]
1.000
0.996
0.991
GAN [ 19 ]
0.559
1.000
0.979
Fully Unsupervised
FUN-AD [ 8 ]
0.633
0.478
0.499
FRD-UVAD [ 24 ]
0.751
1.000
0.935
TMAE [ 7 ]
1.000
0.994
0.901
Table 1: AUC comparison of anomaly detectors across datasets.
Figure 4: Unsupervised anomaly detection performance across three benchmarks. Top row: baselines (TMAE, GCL, Vanilla CAE). Bottom row: our MLE-Guided CAE. Columns: DonkeyCar, UBnormal, Corridor. Our method shows a clear normal–anomaly separation of the PCC value.
Figure 5: Effect of kernel size σ in the MLE loss. Left: PCC trajectories for different σ values; shaded gray regions indicate ground-truth anomalies. Right: AUC heatmap across epochs. Mid-range σ (0.01–0.1) yields stable reconstructions and high AUC, while very small or large σ degrades detection.
Figure 6: Effect of MLE loss weight λ on anomaly detection. Left: PCC trajectories for different λ values; shaded gray regions indicate ground-truth anomalies. Right: AUC heatmap across epochs. Moderate λ (0.1–1.0) achieves the best trade-off between stable reconstruction and clear anomaly sensitivity.
Figure 7: Impact of anomaly ratios on detection performance for the Corridor dataset. The subplots show the PCC trajectories at different anomaly ratios, with the gray regions indicating anomalies.
Figure 8: Test of generalization ability across 10 different anomalies on the Corridor dataset. The model is trained on normal data with only protest abnormal data (gray area: protest) and is tested on the other 10 anomalies.
In this paper, we propose MuST-VAD, a mutual structured learning framework for weakly supervised video anomaly detection (VAD) in which an anomaly detector and a large vision-language model (LVLM) exchange their acquired knowledge. Detectors in weakly supervised VAD learn anomaly scores from features extracted by a fixed, task-agnostic backbone. These fixed features bound the achievable detection accuracy. Recent methods therefore transfer LVLM semantics into the detector as richer features. However, this transfer is one-way: what the detector learns about the target videos never returns to the LVLM. MuST-VAD extends the one-way transfer into a bidirectional learning loop. In this loop, the latest detector predictions supervise the LVLM adaptation, and the adapted LVLM returns updated representations that retrain the detector; the two models alternate these updates over small video groups. Both models train on detector-selected key clips, while confidence weighting and annotation-anchored question answering keep the exchanged supervision reliable. On UCF-Crime, our mutual learning improves the one-pass transfer baseline from 88.15% to 88.63% AUROC and from 37.25% to 42.46% average precision (AP), outperforming the state-of-the-art method in AP by 4.13 points.
Satoshi Hashimoto, Hitoshi Nishimura, Mori Kurokawa
Video anomaly detection (VAD) aims to identify abnormal events in videos. Traditional VAD methods generally suffer from the high costs of labeled data and full training, thus some recent works have explored leveraging frozen multi-modal large language models (MLLMs) in a tuning-free manner to perform VAD. However, their performance is limited as they directly inherit pre-training biases and cannot adapt internal representations to specific video contexts, leading to difficulties in handling subtle or ambiguous anomalies. To address these limitations, we propose a novel intervention framework, termed SteerVAD, which advances MLLM-based VAD by shifting from passively reading to actively steering and rectifying internal representations. Our approach first leverages the gradient-free representational separability analysis (RSA) to identify top attention heads as latent anomaly experts (LAEs) which are most discriminative for VAD. Then a hierarchical meta-controller (HMC) generates dynamic rectification signals by jointly conditioning on global context and these LAE outputs. The signals execute targeted, anisotropic scaling directly upon the LAE representation manifolds, amplifying anomaly-relevant dimensions while suppressing inherent biases. Extensive experiments on mainstream benchmarks demonstrate our method achieves state-of-the-art performance among tuning-free approaches requiring only 1% of training data, establishing it as a powerful new direction for video anomaly detection. The code will be released upon the publication.
Zhaolin Cai, Fan Li, Huiyu Duan +2
Xinjiang University · Xi’an Jiao Tong University · Shanghai Jiao Tong University +1
Video anomaly detection (VAD) aims to automatically identify events that deviate from normal patterns in untrimmed surveillance videos. Existing methods universally depend on large-scale annotations or task-specific training procedures, severely limiting their rapid deployment to novel scenes. We observe that intermediate-layer features of pre-trained multimodal large language models (MLLMs) already encode rich anomaly semantics, yet existing approaches rely on the language output pathway and fail to exploit the geometric discriminability latent in these representations. Based on this finding, we propose SphereVAD, a fully training-free, zero-shot VAD framework that recasts anomaly discrimination as von Mises-Fisher (vMF) likelihood-ratio geodesic inference on the unit hypersphere, unleashing latent discriminability through principled geometric reasoning rather than learning new representations. Specifically, SphereVAD first applies Frechet mean centering to unfold feature distributions and eliminate domain biases, then employs Holistic Scene Attention (HSA) to reinforce feature consistency using cross-video priors, and finally performs vMF-guided Spherical Geodesic Pulling (SGP) to align ambiguous segments with directional prototypes on the spherical manifold. This training-free pipeline requires only minimal synthetic images for calibration. SphereVAD establishes new state-of-the-art results among training-free approaches on three major benchmarks and remains competitive with fully supervised baselines. Code will be available upon acceptance.
Chao Huang, Pengfei Wei, Wei Wang +5
Shenzhen Campus of Sun Yat-sen University · Harbin Institute of Technology, Shenzhen