Anomaly-Guided Observation and Two-Level Semantic Feedback for Training-Free Video Anomaly Analysis

Hangil Park; Seungheon Kim; Tae-Kyun Kim

cs.CVOct 12, 2026

Anomaly-Guided Observation and Two-Level Semantic Feedback for Training-Free Video Anomaly Analysis

Authors: Hangil Park, Seungheon Kim, Tae-Kyun Kim

Organizations: Korea Advanced Institute of Science and Technology (KAIST)

Abstract

Training-free language-based video anomaly detection (VAD) uses frozen vision--language models (VLMs) and large language models (LLMs) to localize abnormal events without task-specific model-weight optimization. Existing approaches increasingly exploit broader temporal and semantic context, yet generic context does not necessarily foreground abnormal events or specify how video-level semantics should modify local anomaly scores. We introduce a training-free semantic feedback framework that uses VAD-guided active observation for video anomaly understanding (VAU) and returns the resulting anomaly-focused description to refine temporal detection. Starting from an upstream anomaly-score curve, we adapt reasoning-based iterative frame selection: high-score regions seed the observations, and anomaly scores together with VLM relevance assessments guide subsequent temporal exploration. A frozen video-language model generates a single open-ended anomaly description from the selected frames. The description is returned through two complementary feedback levels: video-level feedback determines whether the score distribution should be suppressed, preserved, or sharpened, with selective visual rechecking, while local feedback retrieves event-related windows and promotes scores only after target-local visual verification. All model weights remain frozen, and feedback is applied once without regenerating the description. On the HIVAU video-summary evaluations of UCF-Crime and XD-Violence, VAD-guided observation improves GPT-rated reasonability, detail, and consistency over the matched CLIP-guided variant. For VAD, semantic feedback improves the re-evaluated upstream detector from 84.36% to 86.25% AUC on UCF-Crime and from 68.07% to 72.58% AP on XD-Violence, where the final AUC is 91.74%.

Figures & tables

Panel of Figure 1 — Figure 1: Changes from initial (RAW) to URF-HVAA-refined scores on UCF-Crime. (a) Per-video changes in GT-normal and GT-anomalous mean scores; the dashed line denotes equal changes, and 120 unchanged videos overlap at the origin. (b) Mean changes averaged equally across all 140 test videos containing both GT region types.

Figure 2: Anomaly-guided observation and two-level semantic feedback. (a) Initial scores $b^{(0)}$ guide seeding and interval priorities within the inherited A.I.R. observation loop; the frames indexed by $F$ yield one description $Y$ . (b) The video interpretation selects one score modulation, with selective visual rechecking of anomaly hypotheses. (c) For anomaly judgments, event hypotheses from $Y$ , captions $C$ , and $b^{(0)}$ retrieve windows; target-clip evidence authorizes promotion, and current-score quantiles select the anchors. The score path is $b^{(0)}\!\rightarrow b^{(v)}\!\rightarrow b^{(1)}$ . All visual models use frames derived from $V$ and remain frozen. Curves and tokens are schematic; the single return pass precedes final smoothing and expansion.

Method	Zero-shot	Training-free	UCF-Crime	XD-Violence		UBNormal	MSAD
Method	Zero-shot	Training-free	AUC(%)	AUC(%)	AP(%)	AUC(%)	AUC(%)	AP(%)
Additional training or reference data
RTFM ( Tian et al., 2021 )	$\times$	$\times$	83.31	–	77.81	60.94	86.7	66.3
MGFN ( Chen et al., 2023 )	$\times$	$\times$	86.98	–	80.11	–	85.0	63.5
MULDE ( Micorek et al., 2024 )	$\times$	$\times$	78.50	–	–	72.80	–	–
EGO ( Ding et al., 2025 )	$\times$	$\times$	81.71	–	65.77	–	87.3	64.4

Table 1: Comparison with representative video anomaly detection methods. The upper block includes methods using additional training or reference data. The lower block reports selected zero-shot, training-free methods. Legacy comparison values follow the published VAD comparison in URF-HVAA ( Lin et al., 2025 ) ; MSAD results for RTFM and MGFN follow the official MSAD benchmark ( Zhu et al., 2024 ) . Bold denotes the highest reported value within the lower block. All values are percentages; “–” denotes an unreported metric.

Method	UCF-Crime							XD-Violence
Method	BLEU	CIDEr	METEOR	ROUGE-L	GPT-R	GPT-D	GPT-C	BLEU	CIDEr	METEOR	ROUGE-L	GPT-R	GPT-D	GPT-C
Supervised Fine-tuning
Hawk ( Tang et al., 2024 )	0.379	0.008	0.217	0.187	0.255	0.580	0.214	0.375	0.016	0.176	0.188	0.408	0.586	0.365
HolmesVAU ( Zhang et al., 2025b )	0.435	0.021	0.194	0.257	0.448	0.356	0.391	0.376	0.011	0.182	0.253	0.715	0.581	0.673
Zero-shot, training-free
InternVideo2.5-8B ( Wang et al., 2025 )	0.159	0.011	0.088	0.103	0.240	0.266	0.205	0.209	0.013	0.119	0.130	0.456	0.447	0.433

Table 2: Video anomaly understanding on the HIVAU video-summary setting. GPT-R, GPT-D, and GPT-C denote reasonability, detail, and consistency, respectively. Published baseline values follow URF-HVAA ( Lin et al., 2025 ) ; the final row reports our VAD-guided active observation pipeline. URF-HVAA (re-eval.) reports the released URF-HVAA descriptions under the same evaluator used for our generated descriptions.

Figure 3: Motorcycle-fire example. VAD-guided observations support the VAU description, which guides video-level sharpening and visually authorized local promotion. The displayed scores track both feedback stages at the illustrated location.

Method	Guidance	Active	UCF-Crime					XD-Violence
Method	Guidance	Active	BLEU	METEOR	ROUGE-L	CIDEr	GPT-Mean	BLEU	METEOR	ROUGE-L	CIDEr	GPT-Mean
ATS	VAD	$\times$	0.3854	0.1829	0.1970	0.0357	0.4812	0.4344	0.1977	0.2089	0.0488	0.6724
AIR–CLIP	CLIP	$\checkmark$	0.3541	0.1735	0.1897	0.0230	0.4327	0.4334	0.2035	0.2068	0.0320	0.6897
Ours	VAD	$\checkmark$	0.3856	0.1833	0.1959	0.0232	0.4884	0.4647	0.2111	0.2117	0.0468	0.7144

Table 3: VAU observation ablation. All rows share the 16-frame answerer input budget, answerer, prompting, spatial conditioning, and generation settings. ATS and Ours share the VAD prior; AIR–CLIP and Ours share the active-observation pipeline. Active observation additionally uses candidate-analysis calls. GPT-Mean averages GPT-R, GPT-D, and GPT-C. Higher is better.

	UCF-Crime		XD-Violence
Configuration	AUC(%)	AP(%)	AUC(%)	AP(%)
URF-refined base	84.36	36.16	91.32	68.07
+ Video-level feedback	86.23	37.57	91.64	71.37
+ Local feedback ( Ours )	86.25	38.36	91.74	72.58

Table 4: Cumulative ablation of the two semantic-feedback levels. All rows use the URF-refined score base; feedback rows use the VAU description generated by our VAD-guided active observation. Video-level feedback includes selective visual rechecking, and the final row additionally applies evidence-grounded local promotion.

Score sequence	GT-normal ( $\Delta$ )		GT-anomaly ( $\Delta$ )		Difference
RAW	0.2217	(–)	0.3013	(–)	0.0796
+URF-HVAA ( Lin et al., 2025 )	0.2445	(+0.02286)	0.3223	(+0.02105)	0.0778
+Ours	0.2449	(+0.00036)	0.3366	(+0.01428)	0.0917

Table 5: Mean scores and incremental changes under sequential refinement on UCF-Crime. GT-normal and GT-anomaly denote normal and anomalous temporal regions according to the ground-truth annotations. Parentheses report changes from the preceding row.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Refinement base	Description / feedback	UCF-Crime		XD-Violence
Refinement base	Description / feedback	AUC	AP	AUC	AP
RAW	URF-HVAA / Video	84.33	35.73	88.78	64.26
	URF-HVAA / Video + local	84.31	35.92	89.23	66.17
	Ours / Video	84.33	36.36	88.95	64.69
	Ours / Video + local	84.35	37.10	89.29	66.61
URF-refined	URF-HVAA / Video	86.21	37.61	91.55	70.63

Appendix

Table 6: Additional analysis of description sources, refinement bases, and feedback stages. “Video” denotes video-level semantic feedback including selective visual rechecking; “Video + local” additionally applies evidence-grounded local promotion. The same shared feedback configuration is used across datasets, description sources, and refinement bases.

$\alpha$	$\beta$	$f$	$m$	Global quantile	$\lambda$	$\tau_{\ell}$
0.40	0.80	0.85	0.45	0.70	0.95	0.85

Appendix

Table 7: Feedback calibration parameters shared across all evaluated datasets.

$\alpha$	UCF-Crime				XD-Violence
	AUC (%)		AP (%)		AUC (%)		AP (%)
Baseline ( Lin et al., 2025 )	84.356		36.164		91.323		68.071
Main config.	86.246	(+1.890)	38.363	(+2.199)	91.738	(+0.414)	72.584	(+4.514)
0.05	86.551	(+2.195)	38.437	(+2.273)	91.656	(+0.332)	72.557	(+4.486)
0.1	86.572	(+2.216)	38.439	(+2.274)	91.667	(+0.343)	72.559	(+4.489)
0.15	86.587	(+2.231)	38.439	(+2.275)	91.680	(+0.357)	72.563	(+4.492)

Appendix

Table 8: Sensitivity to

\alpha

with all other calibration parameters fixed to the shared main configuration. Bold marks the value used in the main configuration. Parentheses report changes from the re-evaluated URF-refined base in percentage points.

$\beta$	UCF-Crime				XD-Violence
	AUC (%)		AP (%)		AUC (%)		AP (%)
Baseline ( Lin et al., 2025 )	84.356		36.164		91.323		68.071
Main config.	86.246	(+1.890)	38.363	(+2.199)	91.738	(+0.414)	72.584	(+4.514)
0.1	82.938	(-1.418)	39.858	(+3.694)	90.113	(-1.211)	72.556	(+4.485)
0.25	84.323	(-0.033)	39.973	(+3.809)	90.880	(-0.443)	73.199	(+5.128)
0.4	85.251	(+0.895)	39.863	(+3.699)	91.354	(+0.030)	73.514	(+5.444)

Appendix

Table 9: Sensitivity to

\beta

$f$	UCF-Crime				XD-Violence
	AUC (%)		AP (%)		AUC (%)		AP (%)
Baseline ( Lin et al., 2025 )	84.356		36.164		91.323		68.071
Main config.	86.246	(+1.890)	38.363	(+2.199)	91.738	(+0.414)	72.584	(+4.514)
0.5	86.197	(+1.840)	39.276	(+3.111)	91.522	(+0.199)	70.878	(+2.807)
0.55	86.219	(+1.863)	39.319	(+3.155)	91.529	(+0.206)	70.882	(+2.811)
0.65	86.267	(+1.910)	39.417	(+3.252)	91.575	(+0.252)	71.036	(+2.965)

Appendix

Table 10: Sensitivity to

f

$m$	UCF-Crime				XD-Violence
	AUC (%)		AP (%)		AUC (%)		AP (%)
Baseline ( Lin et al., 2025 )	84.356		36.164		91.323		68.071
Main config.	86.246	(+1.890)	38.363	(+2.199)	91.738	(+0.414)	72.584	(+4.514)
0.05	82.655	(-1.701)	26.925	(-9.240)	89.623	(-1.700)	68.973	(+0.902)
0.1	82.655	(-1.701)	26.925	(-9.240)	89.623	(-1.700)	68.973	(+0.902)
0.15	82.787	(-1.569)	27.005	(-9.159)	89.727	(-1.596)	69.090	(+1.020)

Appendix

Table 11: Sensitivity to

m

Global quantile	UCF-Crime				XD-Violence
	AUC (%)		AP (%)		AUC (%)		AP (%)
Baseline ( Lin et al., 2025 )	84.356		36.164		91.323		68.071
Main config.	86.246	(+1.890)	38.363	(+2.199)	91.738	(+0.414)	72.584	(+4.514)
0.1	86.117	(+1.761)	34.583	(-1.581)	91.432	(+0.108)	70.496	(+2.425)
0.25	86.117	(+1.761)	34.583	(-1.581)	91.553	(+0.229)	71.716	(+3.645)
0.4	86.138	(+1.782)	35.209	(-0.956)	91.733	(+0.410)	72.589	(+4.518)

Appendix

Table 12: Sensitivity to the global quantile with all other calibration parameters fixed to the shared main configuration. Bold marks the value used in the main configuration. Parentheses report changes from the re-evaluated URF-refined base in percentage points.

$\lambda$	UCF-Crime				XD-Violence
	AUC (%)		AP (%)		AUC (%)		AP (%)
Baseline ( Lin et al., 2025 )	84.356		36.164		91.323		68.071
Main config.	86.246	(+1.890)	38.363	(+2.199)	91.738	(+0.414)	72.584	(+4.514)
0.05	86.234	(+1.877)	37.654	(+1.489)	91.655	(+0.332)	71.460	(+3.389)
0.1	86.241	(+1.885)	37.738	(+1.573)	91.667	(+0.344)	71.553	(+3.482)
0.2	86.251	(+1.895)	37.862	(+1.698)	91.688	(+0.365)	71.734	(+3.663)

Appendix

Table 13: Sensitivity to

\lambda

$\tau_{\ell}$	UCF-Crime				XD-Violence
	AUC (%)		AP (%)		AUC (%)		AP (%)
Baseline ( Lin et al., 2025 )	84.356		36.164		91.323		68.071
Main config.	86.246	(+1.890)	38.363	(+2.199)	91.738	(+0.414)	72.584	(+4.514)
0.1	85.889	(+1.533)	34.835	(-1.330)	89.747	(-1.576)	61.320	(-6.751)
0.25	85.884	(+1.527)	33.707	(-2.457)	90.214	(-1.109)	63.830	(-4.241)
0.4	86.018	(+1.662)	34.179	(-1.985)	90.962	(-0.361)	67.271	(-0.800)

Appendix

Table 14: Sensitivity to

\tau_{\ell}

Explore similar work

Geometry-Aware Semantic Reasoning for Training Free Video Anomaly Detection
Mar 10, 2026Ali Zia, Usman Ali, Muhammad Umer Ramzan +3Video Anomaly Detection Multimodal Anomaly Detection
School of Computing, Engineering & Mathematical Sciences, La Trobe University, Melbourne, Australia · School of Engineering and Applied Sciences, GIFT University, Gujranwala 52250, Pakistan
CoReVAD: A Contextual Reasoning Framework for Training-Free Video Anomaly Detection
May 22, 2026Hyeongmuk Lim, Youngbum HurInterpretable Anomaly Detection VLM Reasoning
Department of Industrial Engineering, Inha University, Incheon, Republic of Korea
Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning
Aug 7, 2026Shibo Gao, Peipei Yang, Xu-Yao Zhang +1Anomaly Localization Video Anomaly Detection
School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China. · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China. · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.

Mar 10, 2026J/K move · Enter open · S save

Geometry-Aware Semantic Reasoning for Training Free Video Anomaly Detection

Ali Zia, Usman Ali, Muhammad Umer Ramzan, Hamza Abid+2

School of Computing, Engineering & Mathematical Sciences, La Trobe University, Melbourne, Australia · School of Engineering and Applied Sciences, GIFT University, Gujranwala 52250, Pakistan

Video Anomaly Detection Multimodal Anomaly Detection Test-Time Prompt Tuning

Open paper AskPDF arXiv

Training-free video anomaly detection (VAD) has recently emerged as a scalable alternative to supervised approaches, yet existing methods largely rely on static prompting and geometry-agnostic feature fusion. As a result, anomaly inference is often reduced to shallow similarity matching over Euclidean embeddings, leading to unstable predictions and limited interpretability, especially in complex or hierarchically structured scenes. We introduce MM-VAD, a geometry-aware semantic reasoning framework for training free VAD that reframes anomaly detection as adaptive test-time inference rather than fixed feature comparison. Our approach projects caption-derived scene representations into hyperbolic space to better preserve hierarchical structure and performs anomaly assessment through an adaptive question answering process over a frozen large language model. A lightweight, learnable prompt is optimised at test time using an unsupervised confidence-sparsity objective, enabling context-specific calibration without updating any backbone parameters. To further ground semantic predictions in visual evidence, we incorporate a covariance-aware Mahalanobis refinement that stabilises cross-modal alignment. Across four benchmarks, MM-VAD consistently improves over prior training-free methods, achieving 90.03% AUC on XD-Violence and 83.24%, 96.95%, and 98.81% on UCF-Crime, ShanghaiTech, and UCSD Ped2, respectively. Our results demonstrate that geometry-aware representation and adaptive semantic calibration provide a principled and effective alternative to static Euclidean matching in training-free VAD.

Abstract

Explore similar work

Geometry-Aware Semantic Reasoning for Training Free Video Anomaly Detection

CoReVAD: A Contextual Reasoning Framework for Training-Free Video Anomaly Detection

Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning