What Do Interaction Representations Actually Measure? Pre-Event Separability in Weakly-Supervised Violence Detection

Parishruthi Ganesh

cs.CVAug 28, 2026

What Do Interaction Representations Actually Measure? Pre-Event Separability in Weakly-Supervised Violence Detection

Authors: Parishruthi Ganesh

Organizations: Auburn University

Abstract

Articulated human pose provides detailed body-configuration information beyond coarse spatial relationships, but whether this detail yields greater discriminative information when the downstream pipeline is held fixed remains unclear. We examine this through early violence detection. Holding the tracker, temporal head, supervision, folds, and evaluation fixed, we compare five interaction representations spanning coarse bounding-box geometry, a matched handcrafted pose analogue, enriched pose descriptors, and a matched-capacity encoder learned from raw joints, under video-level evaluation with cluster-bootstrap intervals. No pose-based representation outperforms coarse geometry, though with fifteen anomalous videos this subset cannot rule out small effects. Extending the pipeline to frozen visual encoders, and repeating the comparison on XD-Violence (137 anomalous videos, nine times our UCF-Crime sample), person-crop appearance and whole-frame context both exceed geometry by a wide margin, yet context matches appearance on UCF-Crime and exceeds it on the larger split: cropping to the interacting people yields no advantage over encoding the whole frame. This prompts a direct test of what the benchmark measures. Scoring anomalous videos using only frames preceding the annotated onset, under a control removing sequence length as a cue, retains 39-91% of above-chance separation on both benchmarks, including for seven hand-designed geometric channels. Inspection of the tightest pre-onset windows identifies concrete provenance artifacts: editorial title cards and platform watermarks absent from the surveillance footage supplying the normal class. Video-level AUC here is thus a composite of event evidence and pre-event source cues, a shared source of discrimination that can obscure differences between representations. The diagnostic requires only annotations these benchmarks already ship.

Figures & tables

	AUC		Paired $\Delta$ AUC vs. G7 $[95\%$ CI $]$
Representation	Probe	MIL	Probe	MIL
G7 (geometry, 7)	$\mathbf{0.724}$	$\mathbf{0.684}$	—	—
P7 (tight pose, 7)	$0.580$	$0.604$	$-0.153\,[\text{-}0.317,\text{-}0.011]^{*}$	$-0.080\,[\text{-}0.201,0.043]$
P11 (tight+rich, 11)	$0.624$	$0.632$	$-0.104\,[\text{-}0.259,0.034]$	$-0.049\,[\text{-}0.209,0.109]$
PR4 (rich only, 4)	$0.643$	$0.597$	$-0.081\,[\text{-}0.222,0.039]$	$-0.084\,[\text{-}0.233,0.068]$
Learned pose (raw joints) †	$0.631$		—	—

Table 1: Video-level AUC for each interaction representation under a linear probe (representational) and MIL (operational) evaluation, with paired

\Delta

AUC and

95\%

cluster-bootstrap confidence intervals over videos. Folds are shared across all representations. ∗ marks the only paired difference whose interval excludes zero. † The learned encoder is trained end-to-end against the detection objective and evaluated as a single video-level AUC under the same repeated cross-validation; it does not decompose into a separate probe/MIL pair, so we do not compute a paired interval against G7 for it. Because of this asymmetry, statements that both evaluation paradigms agree refer to the four handcrafted representations; the learned encoder is reported separately and also falls below geometry.

Figure 1: Paired

\Delta

AUC (representation minus geometry) with

95\%

cluster-bootstrap confidence intervals, for the linear probe and MIL evaluations. Intervals crossing the dashed zero line indicate no difference from coarse geometry that survives video-level uncertainty; only the tight pose analogue under the probe (P7

-

G7) falls entirely below zero.

	UCF ( $n\!=\!165$ )		XD ( $n\!=\!436$ )
Rep.	AUC	vs. G7	AUC	vs. G7
G7	$.685_{\pm.024}$	—	$.829_{\pm.009}$	—
APP	$.892_{\pm.016}$	$+.207^{4/4}$	$.924_{\pm.007}$	$+.095^{4/4}$
CTX	$.907_{\pm.016}$	$+.219^{4/4}$	$.959_{\pm.002}$	$+.130^{4/4}$

Table 2: The MIL representation ladder on both benchmarks, each under its own folds and head. Mean

{}_{\pm\text{sd}}

over four seeds; the superscript gives the number of seeds whose paired cluster-bootstrap interval excludes zero. G7 = geometry, APP = person crops, CTX = full frame. Coarse geometry is worse than every frozen-encoder representation on both datasets, in every seed.

	UCF-Crime		XD-Violence
Representation	Probe	MIL	Probe	MIL
G7 (geometry)	$79\%$	$84\%$	$46\%$	$39\%$
APP (person crops)	$82\%$	$70\%$	$57\%$	$50\%$
CTX (full frame)	$91\%$	$83\%$	$70\%$	$71\%$

Table 3: Separation retained before the event, under the length-matched control (normal videos truncated to lengths drawn from the anomalous pre-onset distribution, so sequence length carries no class information). Entries are

(\mathrm{AUC}_{\text{pre}}-0.5)/(\mathrm{AUC}_{\text{full}}-0.5)

, the fraction of above-chance separation available from pre-onset frames only; means over four seeds. Every representation on both benchmarks retains a substantial fraction. Full-video AUCs and the unmatched arm are tabulated per seed in the supplement.

Figure 2: Two pre-onset frames from UCF-Crime anomalous videos—footage the annotation marks as preceding the event. Left: the Fighting018 window contains no footage, only an editorial title card naming the video’s content. Right: a platform watermark is composited into every frame of Assault006, whose $39$ -second window is free of violence. Neither occurs in the surveillance footage supplying the normal class: both are class-informative without being event-informative—concrete instances of a source-distribution mechanism, not a complete account of it.

Explore similar work

Sep 30, 2026

Can We Anticipate Violence? Multimodal Learning from Pre-Incident Behavioral Cues

Detecting violence after it begins is important from recognizing behavioral cues that appear immediately beforehand. This work studies short-horizon pre-incident risk recognition from multimodal video signals. We construct a binary Normal-versus-Risky setting from temporally annotated XD-Violence clips, using 443 samples with source-level separation across training, validation, and test sets. Each sample consists of a variable-length pre-incident clip, with its duration determined by the observable behavioral context preceding the incident. The inci- dent itself is excluded from all input clips. We evaluate three complementary information sources: facial-region appearance, temporally aligned audio, and body-motion features derived from tracked keypoints. Controlled ablations are performed with Swin-Tiny, ViT-Tiny, and DeiT-Tiny to measure the contribution of each modality under the same split. Results show that combining all modalities is more effective than using any other combination alone. The best configuration, Deit-Tiny with audio, facial appearance, and motion, achieves 91.21% accuracy, 88.96% balanced accuracy, 93.65% F1-score, and 96.38% ROC-AUC on the held-out test set. These results suggest that complementary appearance, acoustic, and kinematic cues provide useful evidence for recognizing elevated pre-incident risk.

Sindhuja Penchala, Mohammed Yusuf Mujawar, Noorbakhsh Amiri Golilarz +2

Department of Computer Science, The University of Alabama, Tuscaloosa, AL 35487, USA

Human Behavior Prediction Multimodal Learning Video Surveillance

Ask PDF

Aug 12, 2026

Auditing Frame-Level AUC in Weakly Supervised Video Anomaly Detection: Granularity, Resolution, and Scene Bias

Frame-level area under the ROC curve (AUC) is the dominant evaluation metric for weakly supervised video anomaly detection (WSVAD). Its standard form measures whether an anomalous frame outranks a normal frame drawn from anywhere in the test set. We refer to this comparison as pooled AUC, since it aggregates frame pairs across test videos regardless of source. Pooled AUC therefore credits both event localization and differences between video sources. We audit this protocol on UCF-Crime across recent state-of-the-art models spanning different backbone families. Holding each model's frame scores fixed, we read them under three pairing granularities: global, per anomaly category, and within each video, then repeat the same three-granularity readout on zero-shot scores computed from the models' internal representations. We assess ranking reliability with a paired video bootstrap. Three findings follow. First, pooled AUC does not reliably predict within-video anomaly localization: models with similar pooled scores exhibit large localization differences and rank reversals under stricter granularities. Second, at the benchmark's test-split size, pooled AUC lacks the resolution to support state-of-the-art margins reported in the field. Within each backbone family, it resolves no comparison at those margins, while within-video AUC resolves several over identical predictions. Learned representations further reveal that within-video anomaly structure and detector localization are decoupled. Third, on normal footage alone, every model we examine separates videos by recording properties, such as resolution and color encoding, indicating that scene sensitivity is shared across the setting rather than specific to any architecture. We publicly release a granularity-aware protocol computable from existing predictions and scene-factor annotations for UCF-Crime.

Sara Abdulaziz, Egor Bondarev

Eindhoven University of Technology, 5612 AE Eindhoven, Netherlands

Anomaly Localization Algorithmic Bias Video Anomaly Detection Weakly Supervised Video Anomaly Detection Video Surveillance

Ask PDF

Sep 23, 2026

Beyond Isolated Entities: Relation-Aware Multi-Entity Modeling for Unsupervised Video Anomaly Detection

Video anomaly detection for patrol robots and surveillance systems must recognize abnormal interactions among familiar entities. Existing pixel-reconstruction and isolated-entity methods may fail when individual entities appear normal but their spatial or motion relations are abnormal. This work presents Interaction-Centric Network for Temporal Entity-Relation Analysis and Consistency Testing (INTERACT), a framework for unsupervised video anomaly detection. Person-Object Appearance-Motion Interaction (POAMI) moves beyond isolated-entity encoding by jointly modeling the appearance, motion, and spatial configurations of persons and objects in each frame. The resulting representations capture both individual entity states and their cross-entity context. Based on these representations, Target Geometry-Guided Relational Interaction Prediction (TGRIP) predicts target entity interaction states from historical relational memory and target-frame geometry without explicit identity tracking. Motion-Interaction Reconstruction and Alignment (MIRA) then evaluates these predictions through conditional flow reconstruction and semantic consistency checking, providing complementary anomaly evidence beyond prediction error alone. INTERACT achieves state-of-the-art performance, obtaining a frame-level AUC of 84.5% on the ShanghaiTech benchmark. Ablation studies show that removing cross-entity attention causes the largest performance drop, demonstrating the necessity of relational modeling. INTERACT is particularly effective for anomalies caused by changes in relations among people and objects, while maintaining competitive performance in general scenarios. The source code will be made publicly available at https://github.com/ppworkhard/INTERACT.

Zhongpeng Pan, Xina Cheng, Kailun Yang

School of Artificial Intelligence, Xidian University, China · School of Artificial Intelligence and Robotics, Hunan University, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China

Video Anomaly Detection Reconstruction-Based Anomaly Detection

Ask PDF