GridVAD: Open-Set Video Anomaly Detection via Spatial Reasoning over Stratified Frame Grids

Mohamed Eltahir; Ahmed O. Ibrahim; Obada Siralkhatim; Tabarak Abdallah; Sondos Mohamed

cs.CVMar 26, 2026

GridVAD: Open-Set Video Anomaly Detection via Spatial Reasoning over Stratified Frame Grids

Authors: Mohamed Eltahir, Ahmed O. Ibrahim, Obada Siralkhatim, Tabarak Abdallah, Sondos Mohamed

Organizations: King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · African Institute for Mathematical Sciences (AIMS), Cape Town, South Africa · Independent Researcher · Faculty of Computers and Information Technology, University of Tabuk, Tabuk 71491, KSA

Abstract

Vision-Language Models (VLMs) are powerful open-set reasoners, yet their direct use as anomaly detectors in video surveillance is fragile: without calibrated anomaly priors, they alternate between missed detections and hallucinated false alarms. We argue the problem is not the VLM itself but how it is used. VLMs should function as anomaly proposers, generating open-set candidate descriptions that are then grounded and tracked by purpose-built spatial and temporal modules. We instantiate this propose-ground-propagate principle in GridVAD, a training-free pipeline that produces pixel-level anomaly masks without any domain-specific training. A VLM reasons over stratified grid representations of video clips to generate natural-language anomaly proposals. Self-Consistency Consolidation (SCC) filters hallucinations by retaining only proposals that recur across multiple independent samplings. Grounding DINO anchors each surviving proposal to a bounding box, and SAM2 propagates it as a dense mask through the anomaly interval. The per-clip model budget is fixed at M+1 calls regardless of video length, where M can be set according to the proposals needed. On UCSD Ped2, GridVAD achieves the highest Pixel-AUROC (77.59) among all compared methods, surpassing even the partially fine-tuned TAO (75.11) and outperforms other zero-shot approaches on object-level RBDC by over 5x. Ablations reveal that SCC provides a controllable precision-recall tradeoff: filtering improves all pixel level metrics at a modest cost in object-level recall. Efficiency experiments show GridVAD is 2.7x more call-efficient than uniform per-frame VLM querying while additionally producing dense segmentation masks.Code and qualitative video results are available at https://gridvad.github.io.

Figures & tables

Method	Open-set Reasoning	Object-level Localization	Temporal Localization	Hallucination Filtering	Adaptation Strategy
LAVAD [ 33 ]	✓	✗	✗	✗	TF
SUVAD [ 7 ]	✓	✗	✗	✓	TF
Holmes-VAD [ 35 ]	✓	✗	✗	✗	FT
VERA [ 32 ]	✓	✗	✗	✗	PT
TAO [ 11 ]	✗	✓	✓	✗	FT
GridVAD (Ours)	✓	✓	✓	✓	TF

Table 1: Comparison with recent VLM-based VAD methods. The adaptation strategy is denoted as FT (fine-tuning), PT (prompt-tuning), and TF (training-free).

Figure 1: Overview of the GridVAD pipeline. GridVAD decomposes training-free video anomaly detection into a four-stage pipeline: (1) Grid Sampling & Anomaly Proposal: A video clip $\mathcal{V}$ is converted into $M$ stratified frame grids $\{\mathcal{G}^{(m)}\}_{m=1}^{M}$ and processed the VLM to generate $M$ open-set anomaly proposal sets $\{\mathcal{A}^{(m)}\}_{m=1}^{M}$ . (2) Self-Consistency Consolidation: An LLM filters hallucinatory proposals across samples to retain statistically consistent anomaly candidates $a_{j}$ . (3) Spatial Grounding: Grounding DINO anchors candidate description $d_{j}$ to a spatial bounding box $b_{j}$ in keyframe $f_{a_{j}}$ . (4) Temporal Propagation: SAM2 propagates the mask $S_{t}^{j}$ bidirectionally within interval $[t_{s}^{j},t_{e}^{j}]$ to produce temporally consistent pixel-level anomaly maps.

Method	Pixel-AUROC	Pixel-AP	Pixel-AUPRO	Pixel-F1	RBDC	TBDC
AdaCLIP (Fully fine-tuned) [ 4 ]	53.06	4.97	50.66	11.19	12.3	15.5
AnomalyCLIP (Fully fine-tuned) [ 37 ]	54.25	23.73	38.59	7.48	13.1	21.0
DDAD (Fully trained) [ 21 ]	55.87	5.61	15.12	2.67	18.01	13.29
SimpleNet (Fully trained) [ 19 ]	52.49	20.51	44.05	10.71	51.18	27.75
DRAEM (Fully trained) [ 34 ]	69.58	30.63	35.78	10.89	44.26	70.64
TAO (Partially fine-tuned) [ 11 ]	75.11	50.78	72.97	64.12	83.6	93.2

Table 2: Quantitative comparison on UCSD Ped2 . Baseline numbers are transcribed from the TAO paper [ 11 ] ; our results are in the last row.

Method	RBDC	TBDC
OCAD [ 12 ]	20.7	44.5
AED-SSMTL [ 8 ]	43.2	84.1
HF2VAD [ 18 ]	45.4	84.5
STPT [ 22 ]	51.6	84.6
TAO [ 11 ]	62.1	85.4
GridVAD (Ours)	33.32	14.58

Table 3: Object-level comparison on ShanghaiTech Campus . Baseline numbers are transcribed from the TAO paper [ 11 ] .

Config	Px-AUROC	Px-AP	Px-F1	RBDC	TBDC
$M{=}1$ , no SCC	62.71	25.36	29.64	32.31	33.18
$M{=}5$	70.04	37.90	43.43	27.97	25.91

Table 4: Effect of SCC on ShanghaiTech Campus (15 videos). SCC improves all pixel-level metrics while reducing object-level recall, revealing a precision-recall tradeoff.

Paradigm	VLM calls	Frame-AUROC	Time (s)	Fr-AUC / call
Uniform sampling	201	0.6948	640.6	0.0035
GridVAD (Ours)	43	0.4024	276.7	0.0094

Table 5: Efficiency comparison on UCSD Ped2 . GridVAD uses

4.7{\times}

fewer VLM calls and runs

2.3{\times}

faster, and additionally produces dense pixel-level segmentation masks, though at lower absolute Frame-AUROC.

Figure 2: Qualitative comparison on the UCSD Ped2 dataset. We compare our proposed method ( GridVAD ) against state-of-the-art baselines ( SimpleNet , DRAEM , and TAO ) across consecutive video frames.

Explore similar work

Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Models
Apr 7, 2026Sakshi Agarwal, Aishik Konwer, Ankit Parag ShahVideo Anomaly Detection Interpretable Anomaly Detection
Center of Advanced AI, Accenture.
Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning
Aug 7, 2026Shibo Gao, Peipei Yang, Xu-Yao Zhang +1Anomaly Localization Video Anomaly Detection
School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China. · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.
Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understanding
Oct 1, 2026Mohd Ubaid Wani, Sara Atito, Josef Kittler +1CoT Reasoning Interpretable Anomaly Detection
Centre for Vision, Speech and Signal Processing (CVSSP) University of Surrey, UK · Surrey Institute for People-Centred AI (PAI) University of Surrey, UK

Apr 7, 2026J/K move · Enter open · S save

Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Models

Sakshi Agarwal, Aishik Konwer, Ankit Parag Shah

Center of Advanced AI, Accenture.

Video Anomaly Detection Interpretable Anomaly Detection Anomaly Localization VLM Reasoning Vision-Language Grounding Video Surveillance

Open paper AskPDF arXiv

Video Anomaly Detection (VAD) has traditionally been framed as binary classification or outlier detection, providing neither interpretable reasoning nor precise spatial localization of anomalous events. While Vision-Language Models (VLMs) offer rich scene understanding, they struggle with reliable spatial grounding - often producing hallucinated or geometrically invalid bounding boxes when asked to localize objects. We propose VANGUARD (Video Anomaly Understanding through Reasoning and Grounding), a framework that unifies anomaly classification, spatial grounding, and chain-of-thought reasoning within a single VLM. VANGUARD introduces a three-stage curriculum that progressively layers training objectives: (1) classifier warmup on frozen backbone features, (2) LoRA-adapted spatial grounding, and (3) chain-of-thought generation. To overcome the sparse annotation typical of VAD benchmarks, we employ a teacher-student annotation pipeline in which a VLM (Qwen3-VL-4B) generates structured per-subclip reasoning trajectories based on manual annotations available from the UCA Dataset. Further, GroundingDINO provides bounding box supervision. On UCF-Crime, VANGUARD achieves 94% ROC-AUC with 84% F1 while simultaneously producing interpretable chain-of-thought explanations and spatial grounding of anomalous objects - capabilities absent from prior VAD methods. Ablations confirm that staged training outperforms monolithic optimization, and that structured reasoning acts as an implicit regularizer yielding more balanced predictions than classification-only fine-tuning. Zero-shot transfer to XD-Violence and ShanghaiTech demonstrates cross-domain generalization without target-domain adaptation.

Abstract

Explore similar work

Reasoning-Guided Grounding: Elevating Video Anomaly Detection through Multimodal Large Language Models

Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning

Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understanding