QVAD: A Question-Centric Agentic Framework for Efficient and Training-Free Video Anomaly Detection

Lokman Bekit; Hamza Karim; Nghia T Nguyen; Yasin Yilmaz

cs.CVApr 3, 2026

QVAD: A Question-Centric Agentic Framework for Efficient and Training-Free Video Anomaly Detection

Authors: Lokman Bekit, Hamza Karim, Nghia T Nguyen, Yasin Yilmaz

Organizations: Department of Electrical Engineering, University of South Florida, Tampa, Florida, USA.

Abstract

Video Anomaly Detection (VAD) is a fundamental challenge in computer vision, particularly due to the open-set nature of anomalies. While recent training-free approaches utilizing Vision-Language Models (VLMs) have shown promise, they typically rely on massive, resource-intensive foundation models to compensate for the ambiguity of static prompts. We argue that the bottleneck in VAD is not necessarily model capacity, but rather the static nature of inquiry. We propose QVAD, a question-centric agentic framework that treats VLM-LLM interaction as a dynamic dialogue. By iteratively refining queries based on visual context, our LLM agent guides smaller VLMs to produce high-fidelity captions and precise semantic reasoning without parameter updates. This ``prompt-updating" mechanism effectively unlocks the latent capabilities of lightweight models, enabling state-of-the-art performance on UCF-Crime, XD-Violence, and UBNormal using a fraction of the parameters required by competing methods. We further demonstrate exceptional generalizability on the single-scene ComplexVAD dataset. Crucially, QVAD achieves high inference speeds with minimal memory footprints, making advanced VAD capabilities deployable on resource-constrained edge devices.

Figures & tables

Figure 1: Overview of the proposed QVAD framework. The VLM and LLM agents engage in an iterative dialogue, where the LLM generates a query $q_{t}$ conditioned on the captions $C_{t}$ produced by the VLM and directs it back to the VLM. This feedback loop progressively refines their shared understanding of the scene

Figure 2: Detailed architecture of the proposed QVAD framework.

Methods	Supervision	Expl.	UCF	XD
			(AUC)	(AP)
Specialized
HL-Net	Weak	$\times$	82.44	73.67
RTFM	Weak	$\times$	84.30	77.81
VadCLIP	Weak	$\times$	88.02	84.51
VERA	Weak	$\checkmark$	86.55	70.11

Table 1: Performance comparison on UCF-Crime and XD-Violence benchmarks. We report AUC (%) for UCF-Crime and AP (%) for XD-Violence. “Expl.” indicates explainability.

Methods	Supervision	Expl.	UB	CVAD
			(AUC)	(AUC)
Specialized
RTFM	Weak	$\times$	64.94	–
HyCoVAD	SSL	$\checkmark$	–	72.5
MLLM-EVAD	Semi	$\checkmark$	–	71.0
Holmes-VAU	Instru.	$\checkmark$	56.77	–

Table 2: Performance comparison on UBNormal (UB) and ComplexVAD (CVAD) benchmarks. We report AUC (%) for both datasets.

Panel of Figure 3 — Figure 3: Qualitative comparison of Static vs. Dynamic Prompting. In Turn 0 (standard VLM prompting), the model captures general scene dynamics but misses fine-grained semantic cues, leading to False Negatives. In Turn 1 or on Turn 2, the QVAD Agent hypothesizes a potential anomaly and generates a targeted query, correcting the prediction without parameter updates.

Figure 4: Examples of anomaly scores and reasoning for videos from UCF-Crime and XD-Violence test set.

Method	Total Params (B)	LPM (B)	$\approx$ Min. GPU
LAVAD	79.60	13.0	30 (GB)
Panda	N/A	N/A	48 (GB)
VAD Tree	22.20	14.0	32 (GB)
QVAD (Ours)	8.02	4.0	9 (GB)

Table 3: Comparison with state-of-the-art methods in terms of model size and estimated GPU memory requirements. LPM denotes the largest parameterized model used by each method.

Models	QWEN3-VL-4B	QWEN3-VL-2B
QWEN3-4B	79.60 (8.56 GB)	77.76 (8.36 GB)
QWEN3-1.7B	75.92 (8.56 GB)	73.88 (4.20 GB)

Table 4: Ablation study on UBNormal (UB) AUC with different models and their GPU memory usage observed with the Memory-Optimized Pipeline.

Models	QWEN3-VL-4B	QWEN3-VL-2B
QWEN3-4B	84.28	81.51
QWEN3-1.7B	83.12	83.75

Table 5: Ablation study on UCF-Crime (UCF) AUC with different model combinations under the Memory-Optimized Pipeline.

Turns	UCF	XD	UB	CVAD
	(AUC)	(AP)	(AUC)	(AUC)
0	77.23	68.23	77.95	49.81
1	82.18	68.73	78.20	49.67
2	84.28	68.53	79.6	68.02

Table 6: Ablation study for number of turns.

Configuration	Vec	Sel	UCF (AUC)
QVAD (Vec+Sel)	$\checkmark$	$\checkmark$	84.28
QVAD (Sel)	$\times$	$\checkmark$	82.35
QVAD (Base)	$\times$	$\times$	81.67

Table 7: Ablation study on UCF (AUC)

Configuration	Window / Stride	UCF (AUC)
QVAD	64 / 32	81.92
QVAD	128 / 64	84.28
QVAD	256 / 128	83.00

Table 8: Ablation study on window size and stride on UCF (AUC)

Configuration	# Frames	UCF (AUC)
QVAD	4	79.30
QVAD	8	84.28
QVAD	16	83.10

Table 9: Ablation study on number of frames on UCF (AUC)

Confidence Threshold $\theta$	AUC UBnormal (%)
0.6	66.30
0.7	79.60
0.8	65.56

Table 10: Ablation study on confidence threshold

\theta

(AUC UBnormal)

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	T1	T2	T3	Mean	STD
UCF	83.78	84.28	84.02	84.03	0.204
UB	78.98	79.60	79.02	79.20	0.283

Appendix

Table 11: Stability Analysis on UBnormal (AUC) and UCF (AUC)

Frame Selection
Parameter	Symbol	Value
Uniform pre-sample count	–	32
Motion-aware target count	$T$	8
Gaussian blur kernel size	$\sigma$	21
Motion threshold	$\delta$	25
Token Limits

Appendix

Table 12: QVAD Hyperparameters

	UCF-Crime	UB-Normal	XD-Violence	Complex-VAD
$\sigma_{1}$	280	145	142	420
$\sigma_{2}$	0.3	0.5	0.7	0.4
$\alpha$	0.05	0.2	0.4	0.21

Appendix

Table 13: Optimal values for hyper-parameters:

\sigma_{1}

\sigma_{2}

and

\alpha

Explore similar work

COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection
May 14, 2026Darryl Cherian Jacob, Xinyu Liu, Kai Wang +1VLM Adaptation Video Anomaly Detection
Auburn University · Tencent Hunyuan
Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understanding
Oct 1, 2026Mohd Ubaid Wani, Sara Atito, Josef Kittler +1CoT Reasoning Interpretable Anomaly Detection
Centre for Vision, Speech and Signal Processing (CVSSP) University of Surrey, UK · Surrey Institute for People-Centred AI (PAI) University of Surrey, UK
Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning
Aug 7, 2026Shibo Gao, Peipei Yang, Xu-Yao Zhang +1Anomaly Localization Video Anomaly Detection
School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China. · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.

May 14, 2026J/K move · Enter open · S save

COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection

Darryl Cherian Jacob, Xinyu Liu, Kai Wang, Pan He

Auburn University · Tencent Hunyuan

VLM Adaptation Video Anomaly Detection

Open paper AskPDF arXiv

Vision-language models (VLMs) have shown strong performance in video anomaly detection (VAD) while providing interpretable predictions. However, existing VLM-based VAD methods suffer from a fundamental mismatch between training and inference in both data distribution and model configuration. First, most approaches rely on static post-training adaptation, limiting generalization under distribution shifts such as unseen environments or anomaly types. Second, they train VLMs on sparse frames from long videos, but perform inference on densely sampled short segments, creating inconsistencies between training and testing. To address these limitations, we propose COPRA, a conditional parameter adaptation framework for VLM-based VAD. Instead of fixed prompts or shared parameter updates, COPRA generates input-specific parameter updates to dynamically adapt a frozen VLM for each video segment during both training and inference. Experiments show strong performance on standard VAD benchmarks, consistently outperforming static baselines in both in-domain and cross-domain settings. Moreover, COPRA generalizes beyond VAD to unseen tasks such as multiple-choice Video Question Answering and Dense Captioning. These results highlight COPRA as an effective weight-space generation framework for scalable, adaptive, and context-aware video understanding. The code will be released at https://github.com/THE-MALT-LAB/COPRA

Abstract

Explore similar work

COPRA: Conditional Parameter Adaptation with Reinforcement Learning for Video Anomaly Detection

Cog-VADU: A Training-Free Cognitive Reasoning Framework for Video Anomaly Detection and Understanding

Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning