Are Multimodal LLMs Ready for Surveillance? A Reality Check on Zero-Shot Anomaly Detection in the Wild

Shanle Yao; Armin Danesh Pazho; Narges Rashvand; Hamed Tabkhi

cs.CVMar 5, 2026

Are Multimodal LLMs Ready for Surveillance? A Reality Check on Zero-Shot Anomaly Detection in the Wild

Authors: Shanle Yao, Armin Danesh Pazho, Narges Rashvand, Hamed Tabkhi

Organizations: Electrical and Computer Engineering Department, University of North Carolina at Charlotte, Charlotte, USA

Abstract

Multimodal large language models (MLLMs) have demonstrated impressive general competence in video understanding, yet their reliability for real-world Video Anomaly Detection (VAD) remains largely unexplored. Unlike conventional pipelines relying on reconstruction or pose-based cues, MLLMs enable a paradigm shift: treating anomaly detection as a language-guided reasoning task. In this work, we systematically evaluate state-of-the-art MLLMs on the ShanghaiTech and CHAD benchmarks by reformulating VAD as a binary classification task under weak temporal supervision. We investigate how prompt specificity and temporal window lengths (1s--3s) influence performance, focusing on the precision--recall trade-off. Our findings reveal a pronounced conservative bias in zero-shot settings; while models exhibit high confidence, they disproportionately favor the 'normal' class, resulting in high precision but a recall collapse that limits practical utility. We demonstrate that class-specific instructions can significantly shift this decision boundary, improving the peak F1-score on ShanghaiTech from 0.09 to 0.64, yet recall remains a critical bottleneck. These results highlight a significant performance gap for MLLMs in noisy environments and provide a foundation for future work in recall-oriented prompting and model calibration for open-world surveillance, which demands complex video understanding and reasoning.

Figures & tables

Fig. 1 : Conceputal Overview

Fig. 2 : System architecture for prompt-based video anomaly detection. The workflow illustrates the transformation of raw input videos into segmented video clips, which are then processed by Multimodal Models guided by different prompts. This pipeline yields a classification prediction that is directly evaluated against the established ground truth at the Video level

Rank	Prompt Base		Peak F1 (with class)
1	GPT instant medium	Analyze the provided one-second video clip and decide whether it contains any anomalous human-related behavior. An anomalous behavior is any human action or event that is out of the ordinary, unexpected, unsafe, or inconsistent with normal real-world human activities and environments. Routine, everyday human actions should be considered non-anomalous. Based only on the visual content, output a binary decision. Output 1 if an anomaly is present. Output 0 if no anomaly is present. Do not include any other text.	0.64
2	Gemini fast medium	Analyze this one-second video clip to determine if it contains any anomalous human behavior. For the purposes of this evaluation, an anomaly includes any human action or interaction that is unusual, unsafe, or unexpected in a typical real-world environment. Ordinary daily activities should be considered normal. Examine the clip for any events that should not normally occur and decide if the behavior shown is anomalous or not anomalous. You must indicate your decision by outputting a binary label where 1 represents an anomaly and 0 represents a normal event. Do not include any descriptions or additional commentary in your response. The only acceptable output is the single digit 1 or 0.	0.63
3	Gemini pro short	Examine this one-second video for anomalous human behavior which is defined as unexpected or unsafe action distinct from normal daily activities. Output 1 if an anomaly is present and 0 if the behavior is normal. Provide only the binary digit as your answer with no other text.	0.59
4	Gemini pro long	You are an advanced video analysis system tasked with evaluating a one-second video clip to detect anomalous human behavior. For the purpose of this analysis, an anomaly is strictly defined as any human action, interaction, or event that is out of the ordinary, unexpected, unsafe, or deviates from standard real-world norms and environmental contexts. Normal, routine daily human activities are not considered anomalies and should be classified as negative. You must rigorously analyze the visual content to distinguish between standard behavior and irregular occurrences. Your output must be a strict binary classification. If the video contains an anomaly, output the digit 1. If the video contains only normal behavior, output the digit 0. You are prohibited from providing any text, explanation, reasoning, or justification.	0.58
5	GPT instant short	Watch the one-second video clip and determine whether any anomalous human-related behavior is present. Output only 1 for anomalous or 0 for not anomalous.	0.57

TABLE I : Summary of Top 3 Performing Prompt Configurations by Peak F1-Score

Prompt	F1-score			Precision			Recall
	1s	2s	3s	1s	2s	3s	1s	2s	3s
Human	0.04	0.09	0.10	100%	100%	94.12%	1.85%	4.91%	5.08%
Human + class	0.39	0.45	0.51	79.92%	88.39%	88.89%	26.08%	30.58%	35.67%
GPT instant long	0.11	0.04	0.06	84.48%	100%	100%	6.05%	2.23%	2.86%
GPT instant long + class	0.13	0.14	0.16	85.07%	100%	90.32%	7.06%	7.59%	8.89%
GPT instant medium	0.06	0.06	0.09	89.66%	100%	100%	3.23%	3.12%	4.46%

TABLE II : Performance evaluation of various configurations on the ShanghaiTech dataset [ 11 ] . The results compare F1-score, Precision, and Recall across three temporal clip lengths (1s, 2s, and 3s) and evaluate the impact of including class-specific instructions in the prompt list.

Prompt	F1-score			Precision			Recall
	1s	2s	3s	1s	2s	3s	1s	2s	3s
Human	0.13	0.14	0.14	96.82%	97.75%	98.46%	7.07%	7.51%	7.76%
Human + class	0.23	0.17	0.12	93.56%	93.16%	90.16%	12.92%	9.42%	6.66%
GPT instant long	0.15	0.13	0.11	92.31%	93.18%	92.59%	8.36%	7.09%	6.05%
GPT instant long + class	0.23	0.24	0.23	90.26%	90.06%	90.08%	13.45%	14.12%	13.21%
GPT instant medium	0.13	0.17	0.13	97.44%	94.59%	95.10%	7.06%	9.07%	7.18%

TABLE III : Performance evaluation of various configurations on the CHAD dataset [ 12 ] The results compare F1-score, Precision, and Recall across three temporal clip lengths (1s, 2s, and 3s) and evaluate the impact of including class-specific instructions in the prompt list.

Dataset	Prompt Base	1s $\Delta$ F1	2s $\Delta$ F1	3s $\Delta$ F1	Mean $\Delta$ F1
ShanghaiTech [ 11 ]	Human	+0.36	+0.36	+0.41	+0.38
	GPT instant long	+0.02	+0.10	+0.11	+0.07
	GPT instant medium	+0.43	+0.53	+0.56	+0.51
	GPT instant short	+0.40	+0.42	+0.41	+0.41
	GPT think long	+0.17	+0.19	+0.17	+0.18
	GPT think medium	+0.29	+0.36	+0.46	+0.37

TABLE IV : Comparative analysis of the change in F1-score (

\Delta F_{1}=F_{1,\text{class}}-F_{1,\text{no-class}}

) as adding anomalous class to the prompt bases. This table quantifies the performance gains (or losses) when transitioning from non-class to with-class across both the ShanghaiTech and CHAD datasets.

Dataset	Prompt	$\Delta(1s\rightarrow 2s)$	$\Delta(1s\rightarrow 3s)$
ShanghaiTech [ 11 ]	Human	+0.06	+0.09
	GPT instant long	-0.03	-0.01
	GPT instant medium	+0.05	+0.09
	GPT instant short	+0.07	+0.08
	GPT think long	+0.06	+0.09
	GPT think medium	+0.15	+0.13

TABLE V : Comparative analysis of the change in F1-score (

\Delta

) as temporal clip length increases. This table quantifies the performance gains (or losses) when transitioning from a 1s window to 2s and 3s windows across both the ShanghaiTech and CHAD datasets.

Explore similar work

Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models
May 12, 2026Furkan Mumcu, Michael J. Jones, Anoop Cherian +1Large Vision-Language Models Anomaly Localization
University of South Florida · Mitsubishi Electric Research Laboratories (MERL)
Steering and Rectifying Latent Representation Manifolds in Frozen Multi-modal LLMs for Video Anomaly Detection
Feb 27, 2026Zhaolin Cai, Fan Li, Huiyu Duan +2Video Anomaly Detection Multimodal Anomaly Detection
Xinjiang University · Xi’an Jiao Tong University · Shanghai Jiao Tong University +1
No Need For Real Anomaly: MLLM Empowered Zero-Shot Video Anomaly Detection
Feb 22, 2026Zunkai Dai, Ke Li, Jiajia Liu +2Video Anomaly Detection Zero-Shot Anomaly Detection
Beijing University of Posts and Telecommunications · Northwestern Polytechnical University

May 12, 2026J/K move · Enter open · S save

Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models

Furkan Mumcu, Michael J. Jones, Anoop Cherian, Yasin Yilmaz

University of South Florida · Mitsubishi Electric Research Laboratories (MERL)

Large Vision-Language Models Anomaly Localization Video Anomaly Detection Weakly Supervised Video Anomaly Detection

Open paper AskPDF arXiv

Recent video anomaly detection research has expanded rapidly with an emphasis on general models of normality intended to work across many different scenes. While this focus has led to improvements in scalability and multi-scene generalization, it has also shifted the field away from modeling the scene-specific and context-dependent nature of normal behavior. Contemporary approaches frequently rely on video-level weak supervision and opaque pretrained representations from multi-modal large language models (MLLMs), which encourage models to respond to familiar semantic anomaly categories rather than to deviations from the normal patterns of a particular environment. This trend suppresses spatial localization, introduces semantic bias, and reduces anomaly detection to a form of action recognition. In this paper, we examine whether these prevailing formulations align with the core requirements of real-world VAD, which is typically performed within a single scene where normality is determined by local geometry, semantics, and activity patterns. Through targeted visual analyses and empirical evaluations, we demonstrate the practical consequences of these limitations and show that meaningful progress in VAD requires renewed focus on single-scene, spatially-aware, and explainable formulations that capture the nuanced structure of normality within individual environments.

Abstract

Explore similar work

Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models

Steering and Rectifying Latent Representation Manifolds in Frozen Multi-modal LLMs for Video Anomaly Detection

No Need For Real Anomaly: MLLM Empowered Zero-Shot Video Anomaly Detection