See No Evil: Semantic Context-Aware Privacy Risk Detection for AR

Jialu Liu; Yao Li; Zhuoheng Li; Huining Li; Ying Chen

cs.CVApr 14, 2026

See No Evil: Semantic Context-Aware Privacy Risk Detection for AR

Authors: Jialu Liu, Yao Li, Zhuoheng Li, Huining Li, Ying Chen

Organizations: Pennsylvania State University, State College, PA · North Carolina State University, Raleigh, NC

Abstract

Augmented reality (AR) systems pose unique privacy risks due to their continuous capture of visual data. Existing AR privacy frameworks lack semantic understanding of visual content, limiting their effectiveness in detecting context-dependent privacy risks. We propose PrivAR, which leverages vision language models (VLMs) with chain-of-thought prompting for contextual privacy risk detection in AR environments. PrivAR uses visual scene cues to infer potential sensitive information types, such as identifying password notes in office environments through contextual reasoning. PrivAR detects and obfuscates textual content, preventing exposure of sensitive information while preserving contextual cues necessary for VLM inference. Additionally, we investigate contextually-informed warning interfaces to enhance user privacy awareness. Experiments on a real-world AR dataset show that PrivAR achieves superior accuracy (81.48%) and F1-score (84.62%) compared to baselines, while reducing privacy leakage rate to 17.58%. User studies evaluating contextually-informed warning interfaces provide insights into effective privacy-aware AR design.

Figures & tables

Figure 1: System architecture of PrivAR.

Figure 2: Different warning modes when a risk of privacy leakage is identified: (a) center-screen warning, (b) top-screen warning, and (c) region overlay warning.

Detection method	Acc.(%) $\uparrow$	Prec.(%) $\uparrow$	Rec.(%) $\uparrow$	F1(%) $\uparrow$
Rule-based	39.58	44.00	8.63	14.43
Sensitive object recognition	55.79	50.00	83.77	62.62
Scene captioning-based	67.36	82.02	57.25	67.44
PrivAR (ours)	81.48	83.02	86.27	84.62

Table 1: Privacy risk detection performance of PrivAR and baseline approaches.

VLM	Acc.(%) $\uparrow$	Prec.(%) $\uparrow$	Rec.(%) $\uparrow$	F1(%) $\uparrow$
GPT-4o mini	81.48	83.02	86.27	84.62
GPT-4o	82.87	84.41	87.06	85.71
LLaMA-4-Mav.	78.47	81.15	82.75	81.94

Table 2: Privacy risk detection performance of PrivAR with different VLMs.

Protection method	CER (%) $\uparrow$	PLR (%) $\downarrow$
Oracle-guided obfuscation	91.11±14.57	2.29
PriAR w/o obfuscation	74.30±26.70	82.18
PrivAR (ours)	95.71±6.88	17.58

Table 3: Privacy protection effectiveness of 3 methods.

Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality
Apr 7, 2026Yanming Xiu, Zhengyuan Jiang, Neil Zhenqiang Gong +1Augmented Reality VLM Robustness
Duke University
UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attacks
Apr 25, 2026Tianlong Yu, Yang Yang, Xiao Luo +6Privacy-Preserving Language Models LLM Guardrails
Hubei University, Wuhan, China · University of Southern California, Los Angeles, USA · Huazhong University of Science and Technology, Wuhan, China
How Far Are VLMs from Privacy Awareness in the Physical World? An Empirical Study
May 6, 2026Junran Wang, Xinjie Shen, Zehao Jin +1VLM Evaluation Vision-Language Models
Georgia Institute of Technology

Apr 7, 2026J/K move · Enter open · S save

Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality

Yanming Xiu, Zhengyuan Jiang, Neil Zhenqiang Gong, Maria Gorlatova

Duke University

Augmented Reality VLM Robustness VLM Evaluation Adversarial Attacks on VLMs Adversarial Attacks

Open paper AskPDF arXiv

Augmented reality (AR) has rapidly expanded over the past decade. As AR becomes increasingly integrated into daily life, its security and reliability emerge as critical challenges. Among various threats, contradictory virtual content attacks, where malicious or inconsistent virtual elements are introduced into the user's view, pose a unique risk by misleading users, creating semantic confusion, or delivering harmful information. In this work, we systematically model such attacks and present ContrAR, a novel benchmark for evaluating the robustness of vision-language models (VLMs) against virtual content manipulation and contradiction in AR. ContrAR contains 312 real-world AR videos validated by 10 human participants. We further benchmark 11 VLMs, including both commercial and open-source models. Experimental results reveal that while current VLMs exhibit reasonable understanding of contradictory virtual content, room still remains for improvement in detecting and reasoning about adversarial content manipulations in AR environments. Moreover, balancing detection accuracy and latency remains challenging.

Abstract

Explore similar work

Benchmarking Vision-Language Models under Contradictory Virtual Content Attacks in Augmented Reality

UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attacks

How Far Are VLMs from Privacy Awareness in the Physical World? An Empirical Study