HarassGuard: Detecting Harassment Behaviors in Social Virtual Reality with Vision-Language Models

Junhee Lee; Minseok Kim; Hwanjo Heo; Seungwon Woo; Jinwoo Kim

cs.CVApr 1, 2026

HarassGuard: Detecting Harassment Behaviors in Social Virtual Reality with Vision-Language Models

Authors: Junhee Lee, Minseok Kim, Hwanjo Heo, Seungwon Woo, Jinwoo Kim

Organizations: Kwangwoon University · ETRI · Chungbuk National University

Abstract

Social Virtual Reality (VR) platforms provide immersive social experiences but also expose users to serious risks of online harassment. Existing safety measures are largely reactive, while proactive solutions that detect harassment behavior during an incident often depend on sensitive biometric data, raising privacy concerns. In this paper, we present HarassGuard, a vision-language model (VLM) based system that detects physical harassment in social VR using only visual input. We construct an IRB-approved harassment vision dataset, apply prompt engineering, and fine-tune VLMs to detect harassment behavior by considering contextual information in social VR. Experimental results demonstrate that HarassGuard achieves competitive performance compared to state-of-the-art baselines (i.e., LSTM/CNN, Transformer), reaching an accuracy of up to 88.09% in binary classification and 68.85% in multi-class classification. Notably, HarassGuard matches these baselines while using significantly fewer fine-tuning samples (200 vs. 1,115), offering unique advantages in contextual reasoning and privacy-preserving detection.

Figures & tables

Category	Subcategory	Definition
Aggressive Behavior	Punching	Attacking another avatar with a fist
	Slapping	Attacking another avatar with an open hand
	Hitting with object	Attacking anothe avatar using an object
Personal Space Violation	Looming	Bringing one’s avatar face close to another user’s avatar
	Following / Stalking	Tracking a user repeatedly over time
Disruptive Behavior	Blocking	Obstructing the movement of another avatar

Table 1: Categorization of harassment and benign behaviors in social VR for our user study.

Panel of Figure 1 — Figure 1: Examples of actions in Aggressive Behavior, Personal Space Violation, and Disruptive Behavior.

Panel of Figure 2 — Figure 2: Participants’ VR usage time and awareness of harassment responses in social platforms.

Figure 3: Participants’ Likert scale responses from the Scenario 1 (S1): Communication Room (AB: Aggressive Behavior, PSV: Personal Space Violation).

Figure 4: Participants’ Likert scale responses from the Scenario 2 (S2): Whack-a-pig Room and Scenario 3 (S3): Sling Shot Room (AB: Aggressive Behavior, PSV: Personal Space Violation).

Stage 1 Labels	Count	Ratio
Anomaly	2592	0.7606
Benign	816	0.2394
Stage 2 Labels
Aggressive Behavior	1408	0.4131
Personal Space Violation	880	0.2582
Disruptive Behavior	304	0.0892

Table 2: Dataset distribution by Stage 1 labels, Stage 2 labels, and room categories.

Model / Setting	Accuracy	Precision	Recall	F1-Score
LSTM/CNN-based (Baseline)	0.7704	0.6354	0.5013	0.4389
Transformer-based (Baseline)	0.8804	0.8518	0.8343	0.8424
\ourtool ( GPT-4o )
Baseline Prompt	0.6677	0.6888	0.7576	0.6505
+ Context	0.8090	0.7567	0.8257	0.7730
+ CoT	0.8151	0.7653	0.8394	0.7818

Table 3: Performance of binary classification (Baselines vs. \ourtool under different prompting and fine-tuning settings).

Model / Setting	Accuracy	Precision	Recall	F1-Score
LSTM/CNN-based (Baseline)	0.4304	0.2327	0.2504	0.1524
Transformer-based (Baseline)	0.7057	0.6758	0.6537	0.6571
\ourtool ( GPT-4o )
Baseline Prompt	0.5226	0.5058	0.4817	0.4207
+ Context	0.4827	0.6431	0.4633	0.3800
+ CoT	0.5015	0.5423	0.4671	0.4075

Table 4: Performance of multi-class classification (Baselines vs. \ourtool under different prompting and fine-tuning settings).

Virtual Reality

Virtual reality studies immersive, interactive environments and the technologies and human experiences that make them useful.

See all 40 papers

Video-Based Sexual Harassment Detection Using Fine-Tuned Video Transformers
Oct 11, 2026Jubayer- Al- Mahmud, Md Shamimul Islam, Md Nur HossainVideo Content Moderation Video Surveillance
Department of CSE, Manarat International University, Dhaka, Bangladesh · Department of Computer Science, Florida Polytechnic University, Lakeland, Florida, USA
Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality
Apr 7, 2026Yanming Xiu, Zhengyuan Jiang, Neil Zhenqiang Gong +1Augmented Reality VLM Robustness
Duke University
SafeGuard: A Multi-Agent Perception-Reasoning Framework for Social-Risk AI-Generated Video Detection
Jul 3, 2026Wenlin Wu, Sheng Zhou, Peipei Song +3Deepfake Detection Multi-Agent Reasoning
University of Science and Technology of China · King Abdullah University of Science and Technology · Vast Intelligence Lab

Oct 11, 2026J/K move · Enter open · S save

Video-Based Sexual Harassment Detection Using Fine-Tuned Video Transformers

Jubayer- Al- Mahmud, Md Shamimul Islam, Md Nur Hossain

Department of CSE, Manarat International University, Dhaka, Bangladesh · Department of Computer Science, Florida Polytechnic University, Lakeland, Florida, USA

Video Content Moderation Video Surveillance Transformer

Open paper AskPDF arXiv

Sexual harassment represents a pervasive societal challenge, with a substantial proportion of incidents remaining unreported due to insufficient evidence and delayed intervention mechanisms. Smart surveillance systems capable of detecting such incidents in real-time could assist law enforcement and serve as a deterrent. However, video-based harassment detection poses significant challenges, including variations in clothing, illumina- tion, partial occlusions, low resolution, and diverse camera view- points. Despite its social importance, this problem has received limited research attention, primarily due to the scarcity of large- scale annotated video datasets. To address this gap, we extend the Sexual Harassment Video (SHV) dataset to 600 short video clips sourced from YouTube and annotated into harassment and non-harassment categories using consistent labeling guidelines. We fine-tune three state-of-the-art video transformer models, VideoMAE, ViViT, and TimeSformer, pretrained on Kinetics- 400, for binary harassment classification. Experimental results show that all three models achieve competitive classification performance, with TimeSformer attaining the highest test accu- racy of 92.27%, outperforming VideoMAE 91.71% and ViViT 91.16% on the extended SHV test set. Our analysis reveals that performance is notably influenced by dataset size, video quality, frame sampling strategy, and model architecture. This work presents a comprehensive evaluation of modern video transformers for this sensitive real-world task.

Abstract

Virtual Reality

Explore similar work

Video-Based Sexual Harassment Detection Using Fine-Tuned Video Transformers

Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality

SafeGuard: A Multi-Agent Perception-Reasoning Framework for Social-Risk AI-Generated Video Detection