SEEK-VAU: Towards Evidence-Faithful Video Anomaly Understanding via Agentic Search

Minghe Wang; Weijun Zeng; Dapeng Luo; Xinyue Wang; Nong Sang

cs.CVNov 3, 2026

SEEK-VAU: Towards Evidence-Faithful Video Anomaly Understanding via Agentic Search

Authors: Minghe Wang, Weijun Zeng, Dapeng Luo, Xinyue Wang, Nong Sang

Organizations: School of Software Engineering, Huazhong University of Science and Technology · School of Software & Microelectronics, Peking University · School of Mechanical Engineering and Electronic Information, China University of Geosciences · Key Laboratory of Image Processing and Intelligent Control, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology

Abstract

Most video anomaly understanding (VAU) systems still infer anomaly judgments from evidence collected in a feed-forward way under predefined rules, typically sampled frames, or events clustered from such frames. This protocol fails when the collected evidence is incomplete under imperfect predefined rules, because once reasoning begins the model can no longer search for the missing evidence. We address this limitation with SEEK-VAU, which reframes VAU as an agentic search-and-verify interaction protocol under a bounded search budget: instead of passively reasoning over pre-collected evidence, the policy actively gathers evidence across the three stages of an anomaly event chain (precursor, trigger, and aftermath) and verifies its sufficiency before finalization, turning event-chain completeness into an explicit design rather than an outcome of predefined rules. A bounded budget caps the cost of this search, but does not by itself teach the policy where to look or when to stop. To make this behavior learnable, we introduce evidence-faithful counterfactual verification (EFCV), which rewards selected evidence that remains supportive, compact, and necessary under counterfactual verification. We further introduce SEEK-Bench, featuring video-level episodes with temporal interval annotations, semantic QA and event-chain stage labels. Together, SEEK-VAU and SEEK-Bench establish a strong foundation for evidence-faithful and actively verifiable VAU.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Paper term	Implementation label	Role
$r_{\mathrm{faith}}$	evidence faithfulness	Scores the committed claim-evidence pair using committed selected evidence only
$r_{\mathrm{min}}$	minimal_subset	Tests whether a compact sufficient core exists without excessive redundant selected evidence
$r_{\mathrm{drop}}$	drop_trigger	Tests whether trigger-stage evidence is decision-critical

Appendix

Table 1

Symbol	Value	Role
$w_{\mathrm{acc}}$	0.5	Weight on the answer-correctness reward $R_{\mathrm{acc}}$
$w_{\mathrm{efcv}}$	0.5	Weight on the evidence-faithfulness reward $R_{\mathrm{efcv}}$
$\theta$	0.5	Verifier-deems-sufficient threshold; gates $r_{\mathrm{faith}}$ (via the indicator) and the compact-sufficient core $E_{T}^{\mathrm{min}}$
$B_{\mathrm{red}}$	2	Redundancy budget allowing up to two extra evidence windows beyond the minimal sufficient core

Appendix

Table 2

Field	Source status	SEEK contribution	Production and use
Anomaly status/category	MSAD/ECVA base labels	Canonical mapping	Source label plus human check; QA/classification
Temporal interval	Partial or absent upstream	Added	Human annotation; Temporal mIoU
Event-chain stage	Not upstream	Added	Human annotation; protocol supervision
Semantic QA	Not upstream	Added	Human authored and independently verified; QA Accuracy
Evidence window	Not upstream	Added	Human selection and audit; Evidence F1
Provenance metadata	Upstream identifiers	SEEK manifest	Deterministic; split and leakage audits

Appendix

Table 8: Source and field provenance for SEEK-Bench.

Category	Count	Category	Count
Normal	488	Natural hazard	127
Agricultural disaster	102	Pedestrian incident	75
Animals hunting people	7	Protests	23
Assault	15	Public facility incident	108
Building incident	88	Public safety incident	79
Environmental disaster	175	Robbery	232

Appendix

Table 9: Canonical category vocabulary used by SEEK-Bench release files and evaluator. Counts are episode-level totals across the train and val splits of the current release candidate (2,878 episodes; 2,390 anomaly + 488 normal).

Subset	Anomaly	Normal	Total	Ratio
Train	1,912	390	2,302	80.0%
Val	478	98	576	20.0%
All	2,390	488	2,878	100.0%

Appendix

Table 10: Episode-level train/val split for the SEEK-Bench release candidate. The split is video-disjoint and stratified by anomaly category; ratios are reported relative to the 2,878-episode total.

Hyperparam	Operating	Perturbed	$\Delta$ mIoU	$\Delta$ QA	$\Delta$ E.F1
$w_{\mathrm{efcv}}$	0.5	0.25	$-0.5$	$-0.7$	$-0.4$
$w_{\mathrm{efcv}}$	0.5	0.75	$+0.3$	$+0.4$	$+0.2$
$\theta$	0.5	0.25	$-0.4$	$-0.6$	$-0.5$
$\theta$	0.5	0.75	$+0.2$	$+0.5$	$+0.3$
$B_{\mathrm{red}}$	2	1	$-0.1$	$-0.2$	$-0.1$
$B_{\mathrm{red}}$	2	3	$+0.0$	$+0.1$	$+0.0$

Appendix

Table 11: Single-parameter sensitivity around the reported operating point.

Method	T. mIoU	QA Acc.	E.F1
VAD-R1-style (matched)	39.5	36.2	28.7
SEEK-VAU	55.6	45.5	43.7

Appendix

Table 12: Adapted VAD-R1-style fixed-observation control on SEEK-Bench. The control uses Qwen3-VL-8B with VAD-R1’s Perception-to-Cognition Chain-of-Thought (P2C-CoT) and AVA-GRPO-style self-verification reward under 18 fixed frames and matched training, but cannot seek; it is an adapted control, not an official VAD-R1 reproduction.

Method	UCF-Crime frame-AUC $\uparrow$	XD-Violence frame-AP $\uparrow$
Fixed matched	70.4	65.1
Two-stage matched	75.4	67.3
Closed-loop w/o verify	78.6	71.3
SEEK-VAU w/o EFCV	81.1	79.3
SEEK-VAU	87.7	85.4

Appendix

Table 13: Frozen zero-shot transfer of the matched acquisition ladder (Table 4 ) to UCF-Crime and XD-Violence under the same maximum 18-frame cap.

Metric	Self	Gemini
Sufficiency accuracy (%)	78.4	81.1
AUROC	77.1	76.6
ECE (%)	4.6	4.1
Stage F1	79.3	83.0
Trigger drop / control drop	0.22/0.11	0.24/0.15
Swap FPR (%)	9.8	7.6

Appendix

Table 14: Verifier audit on 100 stratified anomaly episodes: self-verifier vs. frozen Gemini visual auditor against blinded human majority labels.

Quantity	Value
Mean scan/seek/verify/finalize calls per episode	1.0/2.4/2.4/1.0
Seek-count distribution (0/1/2/3 calls)	2%/8%/38% / 52%
Median/P95 scan latency	0.80/1.20 s
Median/P95 seek latency	1.10/1.60 s
Median/P95 verify latency	1.25/1.85 s
Median/P95 finalize latency	1.65/2.40 s

Appendix

Table 15: Per-action efficiency statistics over the 576 validation episodes.

Add-one-at-a-time (over baseline)
EFCV variant	T. mIoU	QA Acc.	E.F1
RL w/o EFCV (baseline)	54.4	42.7	41.6
$+\,r_{\mathrm{faith}}$ only	54.7	44.5	42.0
$+\,r_{\mathrm{min}}$ only	54.6	43.1	42.8
$+\,r_{\mathrm{drop}}$ only	55.0	43.4	42.3
Leave-one-out (from full EFCV)

Appendix

Table 16: EFCV reward component ablation on SEEK-Bench. All variants share the same Qwen3-VL-8B backbone, SFT warm-start, and answer-correctness reward

r_{\mathrm{acc}}

; they differ only in which EFCV terms are active. T. mIoU is temporal-grounding mean IoU, QA Acc. is QA accuracy, and E.F1 is top-3 evidence-window F1.

Video Anomaly Detection

Video anomaly detection identifies unusual events or behaviors in video, often when examples of the anomalies are scarce or unknown.

See all 114 papers

AgenticVAU: Multi-Agent Explore-Verify Reasoning for Video Anomaly Understanding
Aug 4, 2026Yuxiang Duan, Huining Li, Ao Li +6Multi-Agent Systems Video Anomaly Detection
C-FAIR & School of software, Shandong University · Renmin University of China · Nanjing Agricultural University +1
Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization
Jul 1, 2026Mengjingcheng Mo, Jiaxu Leng, Xinbo GaoVideo Understanding Direct Preference Optimization
School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China · Chongqing College of Artificial Intelligence, Chongqing, China
TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understanding
Aug 6, 2026Kepeng Yang, Dongxuan Liu, Rongxin Gao +8VLM Evaluation Anomaly Localization
XMU · THU · CUHK +1

Aug 4, 2026J/K move · Enter open · S save

AgenticVAU: Multi-Agent Explore-Verify Reasoning for Video Anomaly Understanding

Yuxiang Duan, Huining Li, Ao Li, Shuai Feng+5

C-FAIR & School of software, Shandong University · Renmin University of China · Nanjing Agricultural University+1

Multi-Agent Systems Video Anomaly Detection Temporal Video Understanding

Open paper AskPDF arXiv

Video anomaly understanding (VAU) focuses on comprehensively interpreting abnormal events in videos, requiring models to identify anomalous occurrences, discover their supporting evidence, and explain the underlying causes beyond simple anomaly detection. Existing VAU methods often rely on specialized training or limited observations, restricting generalization or evidence coverage. Although single-agent alternatives support adaptive video observation, they still integrate exploration, observation, and decision-making within a unified reasoning process, offering limited role specialization and structured evidence coordination. To address these limitations, we present AgenticVAU, a training-free multi-agent framework that casts VAU as an explore--verify process, where the system first discovers potential anomalies and then verifies them through targeted observations. To achieve this, four specialized agents are introduced to handle visual-rule construction, search planning, video observation, and final decision, respectively. These agents communicate through an anchor registry, a shared evidence memory that binds each observation. Guided by this agent framework, AgenticVAU interleaves broad temporal exploration, dense local verification, and cross-interval comparison until sufficient evidence is collected. We conduct extensive experiments on the ECVA, UCF-Crime, and MSAD subsets of VAU-Bench, the results show that AgenticVAU outperforms zero-shot inference and reinforcement learning-based baselines, demonstrating the value of multi-agent collaboration for video anomaly understanding.

Abstract

Video Anomaly Detection

Explore similar work

AgenticVAU: Multi-Agent Explore-Verify Reasoning for Video Anomaly Understanding

Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization

TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understanding