Test-time Reinforcement Learning for Anomalous Video Understanding

Huining Li; Yuxiang Duan; Jiyang Tan; Qian Li; MingCai Chen; Jian Zhang; Xingdong Sheng; Yuntao Du

cs.CVAug 18, 2026

Test-time Reinforcement Learning for Anomalous Video Understanding

Authors: Huining Li, Yuxiang Duan, Jiyang Tan, Qian Li, MingCai Chen, Jian Zhang, Xingdong Sheng, Yuntao Du

Organizations: C-FAIR & School of software, Shandong University · Nanjing University of Posts and Telecommunications · Lenovo Research

Abstract

Anomalous video understanding aims to identify abnormal events in videos and interpret their semantic meanings beyond simple anomaly detection. Recent video large language models (Video-LLMs) have demonstrated promising zero-shot capabilities for this task, yet their performance remains limited due to insufficient adaptation to diverse anomaly patterns and evolving environments. Test-time reinforcement learning offers a promising solution by enabling models to improve through self-generated feedback signals without requiring additional human annotations. However, applying it to anomalous video understanding remains challenging due to three issues: (1) generated pseudo-labels can be unreliable when consensus is weak; (2) binary reward designs fail to capture uncertainty in model generations, resulting in ineffective optimization signals; and (3) unanimous rollout groups receive identical rewards, causing group-relative advantages to collapse and eliminating effective policy-gradient signals. To address these challenges, we present a novel test-time reinforcement learning framework for anomalous video understanding by introducing dual-query consistency filtering, an entropy-aware consensus reward, and a virtual negative anchor mechanism. The framework retains reliable samples through consistency across semantically equivalent queries, combines answer agreement with generation uncertainty for reward estimation, and introduces a virtual negative anchor to create reward variation in unanimous rollout groups, thereby preserving effective group-relative optimization signals. Experiments on VAU-Bench show that our method outperforms the compared frozen and supervised baselines. The gains are most pronounced on the ECVA subset of VAU-Bench with thinking, where accuracy improves from 75.81% to 90.00% relative to the frozen backbone.

Figures & tables

Figure 1: Comparison between conventional supervised learning and test-time adaptation.

Figure 2: Overview of the proposed online test-time reinforcement learning framework. Original and reformulated questions are used to identify reliable test samples. Retained samples are optimized using an entropy-aware consensus reward, and virtual-anchor-enhanced advantage estimation if needed.

Figure 3: An example of the entropy-aware consensus reward computation with five rollouts retained from the first stage.

Dataset	Model	w/o Think	w/ Think
MSAD	Qwen2.5-VL-3B	85.83	82.50
	VAU-R1	88.33	87.08
	Ours	92.50 (+6.67)	90.00 (+7.50)
UCF-Crime	Qwen2.5-VL-3B	91.63	83.27
	VAU-R1	92.03	91.63
	Ours	92.83 (+1.20)	92.43 (+9.16)

Table 1: Multiple-choice QA accuracy (%) on VAU-Bench. Best results are highlighted in bold.

Dataset	Model	w/o Think	w/ Think
Dataset	Model	Bin. Acc./Multi. Acc.	Bin. Acc./Multi. Acc.
MSAD	Qwen2.5-VL-3B	79.17 / 69.58	73.33 / 56.67
	VAU-R1	82.08 / 71.25	74.58 / 60.83
	Ours	87.50 / 73.75	78.33 / 62.61
UCF-Crime	Qwen2.5-VL-3B	64.54 / 58.57	62.55 / 52.19
	VAU-R1	62.55 / 57.77	62.15 / 56.97

Table 2: Binary and multi-class anomaly classification accuracy (%). Bin.Acc. and Multi.Acc. denote binary and multi-class accuracy, respectively.

Dataset	Reward	w/o Think	w/ Think
MSAD	Qwen2.5-VL-3B	85.83	82.50
	Binary $0/1$ reward	89.58	88.75
	Entropy-Aware Reward	92.50	90.00
UCF-Crime	Qwen2.5-VL-3B	91.63	83.27
	Binary $0/1$ reward	92.03	90.04
	Entropy-Aware Reward	92.83	92.43

Table 3: Controlled comparison of reward designs.

Configuration	MSAD	UCF-Crime	ECVA
Configuration	w/o T / w/ T	w/o T / w/ T	w/o T / w/ T
Qwen2.5-VL-3B	85.83 / 82.50	91.63 / 83.27	85.58 / 75.81
+ ER	91.67 / 89.17	92.03 / 90.44	88.54 / 88.54
+ ER + VA	92.08 / 89.58	92.43 / 90.84	90.98 / 88.78
+ ER + VA + DQ	92.50 / 90.00	92.83 / 92.43	91.46 / 90.00

Table 4: Cumulative component analysis. Here, w/o T and w/ T abbreviate w/o Think and w/ Think, respectively. ER, VA, and DQ denote the entropy-aware reward, virtual negative anchor, and dual-query filtering.

Table 5: Hyperparameter analysis of batch size and sampled numbers on MSAD subset of VAU-Bench under the w/o Think setting.

Figure 4: Qualitative case study of multiple-choice QA. The base model Qwen2.5-VL-3B misinterprets lighting changes as a malfunction, whereas our method uses dual-query consistency and entropy-aware rewards to correct the prediction from D to A.

Figure 5: Distribution of model failure modes across MSAD, UCF-Crime, and ECVA. The bars report the number of errors attributed to each failure mode.

Test-Time RL

Test-Time Reinforcement Learning adapts policies at deployment using interaction or self-generated feedback, without a separate task-specific training phase.

See all 48 papers

Aug 4, 2026

AgenticVAU: Multi-Agent Explore-Verify Reasoning for Video Anomaly Understanding

Video anomaly understanding (VAU) focuses on comprehensively interpreting abnormal events in videos, requiring models to identify anomalous occurrences, discover their supporting evidence, and explain the underlying causes beyond simple anomaly detection. Existing VAU methods often rely on specialized training or limited observations, restricting generalization or evidence coverage. Although single-agent alternatives support adaptive video observation, they still integrate exploration, observation, and decision-making within a unified reasoning process, offering limited role specialization and structured evidence coordination. To address these limitations, we present AgenticVAU, a training-free multi-agent framework that casts VAU as an explore--verify process, where the system first discovers potential anomalies and then verifies them through targeted observations. To achieve this, four specialized agents are introduced to handle visual-rule construction, search planning, video observation, and final decision, respectively. These agents communicate through an anchor registry, a shared evidence memory that binds each observation. Guided by this agent framework, AgenticVAU interleaves broad temporal exploration, dense local verification, and cross-interval comparison until sufficient evidence is collected. We conduct extensive experiments on the ECVA, UCF-Crime, and MSAD subsets of VAU-Bench, the results show that AgenticVAU outperforms zero-shot inference and reinforcement learning-based baselines, demonstrating the value of multi-agent collaboration for video anomaly understanding.

Yuxiang Duan, Huining Li, Ao Li +6

C-FAIR & School of software, Shandong University · Renmin University of China · Nanjing Agricultural University +1

Multi-Agent Systems Video Anomaly Detection Temporal Video Understanding

Ask PDF

Jul 1, 2026

Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization

Video anomaly understanding (VAU) relies on sparse, context-dependent cues. However, existing passive paradigms suffer from observational aliasing, where static sampling fails to disambiguate semantically distinct events. To overcome this, we propose

Anom\text{-}π

, a closed-loop framework that reconceptualizes video understanding as an active sequential decision-making process within a dynamic environment. Inspired by human video-reviewing behavior, this framework unifies internal cognitive reasoning and strategic evidence acquisition into an interleaved policy, utilizing temporal atomic operators such as local backtracking, temporal expansion, and fine-grained sampling to endow the model with perceptual proactivity. To learn such complex interaction strategies under video-level weak supervision, we design Interactive Direct Preference Optimization (iDPO) to achieve trajectory-level policy alignment, guided by an Active Evidence Inquiry (AEI) utility that balances task success, informative evidence acquisition, and interaction cost. This approach enables the agent to learn to actively disambiguate hypotheses while suppressing redundant exploration. Extensive experiments demonstrate that our framework, with only 2B parameters, achieves highly competitive performance, significantly outperforming state-of-the-art large-scale VAU models in complex scenarios.

Mengjingcheng Mo, Jiaxu Leng, Xinbo Gao

School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China · Chongqing College of Artificial Intelligence, Chongqing, China

Video Understanding Direct Preference Optimization Video Anomaly Detection Weakly Supervised Video Anomaly Detection

Ask PDF

Aug 6, 2026

TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understanding

Humans understand anomalous events through a coherent perceptual process in which they identify the focal instance, follow its behavior as the event unfolds, and interpret why it violates the expectations of the surrounding scene. Video anomaly understanding (VAU) seeks to endow models with a similar capability, moving beyond deciding whether a video is anomalous toward explaining how the event develops and why it matters. Although recent vision--language models (VLMs) can generate detailed and plausible anomaly descriptions, their semantic fluency does not ensure that these interpretations remain grounded in the correct anomaly instance over time. Existing benchmarks typically evaluate tracking and semantic understanding through separate protocols, leaving such instance--semantic inconsistency largely unmeasured. We therefore introduce TAU-Bench, a track-centric benchmark for jointly evaluating anomaly instance tracking and fine-grained anomaly understanding. TAU-Bench contains 1,118 videos, 1,454 tracks, and 202,438 pixel-level masks spanning 49 event and 45 scene categories, together with track-centric annotations that connect instance-level identification, event-level understanding, and scene-level reasoning. To build TAU-Bench at scale, we developed an automated data engine integrating anomaly suitability filtering, anomaly instance track construction, hierarchical caption annotation, and human quality control. Evaluations across representative VLM families show that models producing plausible anomaly interpretations may still fail to localize and track the correct instance reliably, revealing a persistent gap between semantic reasoning and visual grounding. These findings therefore highlight instance-grounded evaluation as an important step toward more faithful and reliable VAU systems.

Kepeng Yang, Dongxuan Liu, Rongxin Gao +8

XMU · THU · CUHK +1

VLM Evaluation Anomaly Localization Video Anomaly Detection

Ask PDF

Abstract

Test-Time RL

Explore similar work

AgenticVAU: Multi-Agent Explore-Verify Reasoning for Video Anomaly Understanding

Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization

TAU-Bench: From Anomaly Instance Tracking to Fine-Grained Video Anomaly Understanding