Can We Anticipate Violence? Multimodal Learning from Pre-Incident Behavioral Cues

Sindhuja Penchala; Mohammed Yusuf Mujawar; Noorbakhsh Amiri Golilarz; Sudip Mittal; Shahram Rahimi

cs.CVSep 30, 2026

Can We Anticipate Violence? Multimodal Learning from Pre-Incident Behavioral Cues

Authors: Sindhuja Penchala, Mohammed Yusuf Mujawar, Noorbakhsh Amiri Golilarz, Sudip Mittal, Shahram Rahimi

Organizations: Department of Computer Science, The University of Alabama, Tuscaloosa, AL 35487, USA

Abstract

Detecting violence after it begins is important from recognizing behavioral cues that appear immediately beforehand. This work studies short-horizon pre-incident risk recognition from multimodal video signals. We construct a binary Normal-versus-Risky setting from temporally annotated XD-Violence clips, using 443 samples with source-level separation across training, validation, and test sets. Each sample consists of a variable-length pre-incident clip, with its duration determined by the observable behavioral context preceding the incident. The inci- dent itself is excluded from all input clips. We evaluate three complementary information sources: facial-region appearance, temporally aligned audio, and body-motion features derived from tracked keypoints. Controlled ablations are performed with Swin-Tiny, ViT-Tiny, and DeiT-Tiny to measure the contribution of each modality under the same split. Results show that combining all modalities is more effective than using any other combination alone. The best configuration, Deit-Tiny with audio, facial appearance, and motion, achieves 91.21% accuracy, 88.96% balanced accuracy, 93.65% F1-score, and 96.38% ROC-AUC on the held-out test set. These results suggest that complementary appearance, acoustic, and kinematic cues provide useful evidence for recognizing elevated pre-incident risk.

Figures & tables

Figure 1 : Overview of the proposed multimodal pre-incident risk classification framework. The preprocessing stage maps XD-Violence source videos into binary Normal and Risky clips and aligns facial appearance, audio, and pose-derived motion inputs. The facial stream uses a pretrained visual backbone followed by a temporal Transformer, the audio stream processes log-magnitude spectrograms using a three-stage CNN, and the motion stream models pose-derived kinematic features using a temporal Transformer. The resulting 256-dimensional visual, 128-dimensional audio, and 64-dimensional motion representations are concatenated and passed through a late-fusion classification head to predict Normal or Risky behavior.

Panel of Figure 2 — Figure 2 : Feature-depth activation maps from the DeiT-Tiny facial-appearance branch for representative No-Risk and Risky samples. The first column shows the original facial crop, followed by activation maps from layers 1, 4, 8, and 12. Red and yellow regions indicate relatively strong feature responses, green and cyan indicate moderate responses, and blue and purple indicate weaker responses within each layer. These colors show activation strength.

Figure 3 : Class-averaged audio representations for Normal and Risky pre-incident clips using the DeiT-Tiny three-modality configuration. From left to right, the figure shows the mean absolute waveform, mean log-magnitude spectrogram, and learned audio representations at three successive network depths (32, 64, and 128 channels). The visualization illustrates how differences in the input acoustic patterns are progressively transformed into higher-level feature representations by the audio branch.

Panel of Figure 4 — Figure 4 : Representative pose-based motion sequences for No-Risk and Risky pre-incident clips. Four temporally ordered frames are shown for each example. Red markers denote detected body joints, blue lines denote skeletal connections, and green boxes indicate person detections. These colors are visualization annotations and do not directly represent risk; classification is based on temporal changes in the detected keypoint positions and derived motion features.

Ablation	Model	Accuracy	Bal. Acc.	Precision	Recall	F1	ROC-AUC
Motion only	Temporal Transformer	82.42	76.08	82.86	93.55	87.88	86.21
Audio only	Audio CNN	67.03	73.05	92.11	56.45	70.00	78.75
Facial appearance only	ViT-Tiny	87.91	85.62	90.48	91.94	91.20	91.94
Facial appearance only	Swin-Tiny	89.01	87.35	91.94	91.94	91.94	95.22
Facial appearance only	DeiT-Tiny	87.91	85.62	90.48	91.94	91.20	94.22
Audio + Motion	Audio CNN + Temporal Transformer	78.02	75.61	85.00	82.26	83.61	88.71

Table 1 : Final ablation results for individual and combined pre-incident modalities across different model configurations. We report Accuracy, Balanced Accuracy, Precision, Recall, F1-score, and ROC-AUC on the held-out test set. Bold values indicate the best performance obtained for each metric across all evaluated configurations.

Figure 5 : Mean predicted Risky scores across the original risk levels for the three-modality DeiT-Tiny model. Although the model is trained for binary Normal-versus-Risky classification, the average predicted risk score generally increases from No Risk to Low, Medium, and High Risk across the training, validation, and test partitions. The dashed line indicates the fixed decision threshold of 0.50.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6 : Distribution of predicted Risky probabilities for Normal and Risky clips in the held-out test set. Most Normal clips receive scores near zero, whereas most Risky clips receive scores near one, showing clear separation between the two classes. The dashed line indicates the fixed classification threshold of $0.5$ . Blue observations to the right of the threshold correspond to false positives, while orange observations to the left correspond to false negatives.

Label	Vision	Motion	Audio	Risk Interpretation
No Risk	Individuals appear calm and non-aggressive. No threatening posture, weapon visibility, or confrontation is observed. People may be walking, standing, sitting, or interacting normally.	Minimal body movement and stable trajectories. Normal interpersonal distance with smooth and predictable motion patterns. No sudden acceleration or aggressive gestures.	Background ambience or normal conversation only. No shouting, screaming, impact sounds, or sudden loud audio spikes.	The environment appears safe and stable with no visible indicators of aggression or escalation.
Low Risk	Mild suspicious or tense visual behavior may appear, such as prolonged staring, following behavior, defensive posture, or unusual attention toward another individual. No direct aggression occurs.	Slight increase in movement intensity or interpersonal focus. Gradual approach between individuals, mild trajectory changes, or increased body orientation toward another person.	Slightly elevated audio activity such as louder speech, crowd noise, or environmental tension without explicit aggressive sounds.	Early behavioral indicators suggest potential tension or discomfort, but no immediate threat is observed.
Medium Risk	Clear behavioral escalation becomes visible. Aggressive body posture, arm raising, threatening gestures, chasing behavior, or visible confrontation may appear.	Noticeable increase in motion intensity, abrupt trajectory changes, reduced interpersonal distance, and rapid body movements indicating possible conflict escalation.	Raised voices, arguments, aggressive speech tone, sudden audio spikes, or increased environmental disturbance may be present.	Behavioral patterns indicate a significant likelihood of an aggressive event occurring soon. The situation is unstable and escalating.
High Risk	Immediate pre-incident behavior is strongly visible. Direct confrontation, attack preparation, weapon handling, striking posture, or physically aggressive interaction is clearly observed before the incident starts.	Highly erratic and rapid motion patterns, aggressive acceleration, physical engagement attempts, collision-like movement, or intense approach dynamics are observed.	Screaming, impact sounds, weapon-related sounds, explosions, or highly elevated audio intensity may occur immediately before the event.	The scene shows strong pre-incident aggression cues indicating an imminent violent or dangerous event.

Appendix

Table 2 : Pre-Incident Risk Annotation Rubric

Explore similar work

May 4, 2026cs.CV

Human Activity Recognition Method for Moderate Violence Detection

Physical violence in public spaces is a significant public health concern, with minor incidents such as pushing often serving as precursors to more severe escalations. This research develops an automated system for the real-time detection of moderate physical violence, specifically pushing, in surveillance camera footage. The proposed solution integrates state-of-the-art computer vision models, utilizing YOLO11 and YOLO11-Pose for human detection and skeletal keypoint extraction. By calculating body inclination and joint angles between shoulders and hips, a Random Forest classifier was trained to distinguish between normal behavior and aggressive physical contact. The system's performance was evaluated through three progressive case studies representing increasing levels of difficulty. In controlled environments with frontal views, the model achieved a precision of 0.98. In the most challenging scenario, featuring high-altitude, steep-angle recordings from real-world surveillance infrastructure, the system maintained a precision of 0.72 despite significant perspective distortion and visual noise. These results demonstrate the feasibility of using skeletal analysis for early violence intervention in urban security contexts.

Luis Angel Aparicio Borjas, Victor Elias Nieto, Juan Irving Vasquez +2

ESCOM-IPN, Mexico City, México · Centro de Innovación y Desarrollo Tecnológico en Cómputo, Instituto Politécnico Nacional, Mexico City, México

Video Surveillance Violence Detection Computer Vision Human Activity Recognition Keypoint Detection Urban Environments

Ask PDF

Nov 14, 2025cs.CV

Short-Window Sliding Learning for Real-Time Violence Detection via LLM-based Auto-Labeling

This paper proposes a Short-Window Sliding Learning framework for real-time violence detection in CCTV footages. Unlike conventional long-video training approaches, the proposed method divides videos into 1-2 second clips and applies Large Language Model (LLM)-based auto-caption labeling to construct fine-grained datasets. Each short clip fully utilizes all frames to preserve temporal continuity, enabling precise recognition of rapid violent events. Experiments demonstrate that the proposed method achieves 95.25% accuracy on RWF-2000 and significantly improves performance on long videos (UCF-Crime: 83.25%), confirming its strong generalization and real-time applicability in intelligent surveillance systems.

Seoik Jung, Taekyung Song, Yangro Lee +1

PIA-SPACE Inc.

Video Surveillance Ai-Generated Video Detection Automatic Labelling Captions Bidirectional Long Short-Term Memory

Ask PDF

Jun 1, 2026cs.CL

PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warning

Between the first visible sign of danger and the moment an accident occurs, there is often a window where intervention remains possible. Video-capable multimodal large language models (MLLMs) could serve as always-on safety monitors that issue warnings during this window. Yet current benchmarks do not test this ability: they rely on static inputs, ignore timing precision, and omit false-positive measurement on safe scenes. We present PaSBench-Video, a 740-video benchmark with 481 risk and 259 no-risk videos across four domains: driving, healthcare, daily life, and industrial production. Risk videos are annotated with frame-level risk onset and accident boundaries. A model must observe the video causally and produce a warning that is both temporally calibrated and content-correct. Testing 13 MLLMs, we find that no model exceeds 20.0% on our strictest metric, and recall is tightly coupled with false-positive rate, with Pearson correlation 0.64: higher detection comes only at the cost of triggering warnings on the majority of safe clips. Performance splits sharply by domain: models achieve moderate recall at low false-positive rates in daily life, where risks are inherently anomalous, yet fire indiscriminately in driving, where routine and hazardous scenes look alike. These results indicate that current models rely on scene-level activity cues rather than reasoning about emerging harm.

Yusong Zhao, Yuejin Xie, Youliang Yuan +4

The Chinese University of Hong Kong, Shenzhen · Tsinghua University

Accident Source Term Evaluation Code Multimodal Large Language Models

Ask PDF