Privacy Protection in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 18

All topics

SIEVE: Selective attention-value Suppression for Vision-Language Models Unlearning
Oct 1, 2026Si Qi Goh, Cap Dang Xuan Kiet, Tat-Jen Cham +1VLM Unlearning Privacy-Preserving Language Models
Digital Trust Centre, Nanyang Technological University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore
Anti-Persona: Disrupting Unauthorized Identity Binding and Recognition in Personalized Vision--Language Models
Oct 1, 2026Abhishek Basu, Fahad Shamshad, Karthik NandakumarVLM Robustness Vision-Language Models
MBZUAI, UAE · Michigan State University, USA
Hiding in Plain Sight: A Diffusion-based Mitigation of Geolocation Privacy Leakage in Vision-Language Models
Sep 18, 2026Yining Wang, Xi Li, Mi Zhang +4Privacy Protection in VLMs Adversarial Attacks on VLMs
Fudan University · East China University of Science and Technology · Shanghai University of Electric Power
Don't Send What You Don't Need: Question-Guided Token Pruning as a Privacy Defense for Vision-Language Models
Sep 14, 2026Md Khalid Syfullah, Alvi Ataur KhalilVisual Question Answering Privacy Protection in VLMs
Transformative Innovation for Trustworthy AI and Network Security (TITANS) Lab, Computer Science, Southern Illinois University Carbondale, USA
Where Identity Lives: Localized, Retain-Free Identity Unlearning in Multimodal Large Language Models
Aug 31, 2026Kangwook Ko, Jaehyuk Jang, Wonjun Lee +2MLLM Unlearning Privacy Protection in VLMs
KAIST
Imperceptible and Reversible Adversarial Examples against Vision-Language Models for Privacy Protection
Jul 11, 2026Qi Lu, Ziqi Zhou, Yufei Song +5VLM Robustness Adversarial Attacks on VLMs
School of Cyber Science and Engineering, Huazhong University of Science and Technology · College of Computer Science, Chongqing University · School of Software and engineering, Huazhong University of Science and Technology +2
TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction
Jul 5, 2026Nikos Athanasiou, Ilya A. Petrov, Angela Yao +7Gradient Inversion Attacks VLM Robustness
Meta, Zürich, Switzerland · Max Planck Institute for Intelligent Systems, Tübingen, Germany · University of Tübingen, Germany +1
The Vision Encoder as a Privacy Boundary: Visual-Token Side Channels in Encoder-Free Vision-Language Models
Jun 10, 2026Chenyu Zhou, Qiliang Jiang, Shuning Wu +1Vision-Language Models Adversarial Attacks
School of Engineering, Institute of Science Tokyo Tokyo, Japan · College of Control Science and Engineering, Zhejiang University Hangzhou, China · Department of Electrical and Computer Engineering, National University of Singapore Singapore, Singapore
Vision Language Model Helps Private Information De-Identification in Vision Data
Jun 8, 2026Tiejin Chen, Pingzhi Li, Kaixiong Zhou +2VLM Adaptation Privacy Protection in VLMs
Arizona State University · University of North Carolina at Chapel Hill · North Carolina State University
Seeing Without Exposing: Adaptive Privacy Control for Open-World, Context-Hungry MLLMs
Jun 5, 2026Siyuan Xu, Yibing Liu, Peilin Chen +3Privacy-Preserving Language Models Multimodal Large Language Models
City University of Hong Kong · Hon Hai Research Institute · Lingnan University
Cross-modal linkage risk in clinical vision-language models
Jun 1, 2026Soroosh Tayebi Arasteh, Mahshad Lotfinia, Sven Nebelung +1Medical VLMs Privacy Protection in VLMs
Lab for AI in Medicine, RWTH Aachen University, Aachen, Germany · Department of Diagnostic and Interventional Radiology, University Hospital RWTH Aachen, Aachen, Germany
Single-Sample Black-Box Membership Inference Attack against Vision-Language Models via Cross-modal Semantic Alignment
May 17, 2026Jiaqing Li, Yajuan Lu, Xiaochuan Shi +3Membership Inference Attacks Privacy Protection in VLMs
1Wuhan University · 2Tarim University
PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models
May 9, 2026Jiahui Guang, Zexun Zhan, Zhenlin Xu +5VLM Evaluation VLM Unlearning
Harbin Institute of Technology, Shenzhen, China · Sichuan University, Chengdu, China · Harbin Institute of Technology, Weihai, China +3
How Far Are VLMs from Privacy Awareness in the Physical World? An Empirical Study
May 6, 2026Junran Wang, Xinjie Shen, Zehao Jin +1VLM Evaluation Vision-Language Models
Georgia Institute of Technology
Before Forgetting, Learn to Remember: Revisiting Foundational Learning Failures in LVLM Unlearning Benchmarks
May 5, 2026JuneHyoung Kwon, MiHyeon Kim, Eunju Lee +3VLM Evaluation VLM Unlearning
Department of Artificial Intelligence, Chung-Ang University · KT Corporation · Graduate School of Advanced Imaging Sciences, Multimedia and Film, Chung-Ang University
Rethinking Visual Privacy: A Compositional Privacy Risk Framework for Severity Assessment with VLMs
Mar 23, 2026Efthymios Tsaprazlis, Tiantian Feng, Anil Ramakrishna +3VLM Evaluation Privacy Auditing
University of Southern California, Los Angeles CA 90089, USA · Meta Superintelligence Labs · Amazon AGI
Neural Gate: Mitigating Privacy Risks in LVLMs via Neuron-Level Gradient Gating
Mar 13, 2026Xiangkui Cao, Jie Zhang, Meina Kan +2Large Vision-Language Models Privacy Protection in VLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences University of Chinese Academy of Sciences · University of Chinese Academy of Sciences
Privacy-Aware Visual Language Models
May 27, 2024Laurens Samson, Nimrod Barazani, Sennay Ghebreab +1VLM Evaluation VLM Adaptation
Socially-Intelligent Artificial Systems Group, University of Amsterdam · University of Amsterdam · Fundamental AI Lab, University of Technology Nuremberg

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

SIEVE: Selective attention-value Suppression for Vision-Language Models Unlearning

Si Qi Goh, Cap Dang Xuan Kiet, Tat-Jen Cham, Kwok-Yan Lam

Digital Trust Centre, Nanyang Technological University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore

VLM Unlearning Privacy-Preserving Language Models MLLM Unlearning Privacy Protection in VLMs Machine Unlearning

Open paper AskPDF arXiv

The ability of vision-language models (VLMs) to associate visual identities with biographical information creates a need for selective unlearning of personally identifiable information (PII) while preserving permitted knowledge about the same individual. This setting is challenging because both sensitive and retained information can share the same visual inputs and intermediate representations. We introduce SIEVE, a simple and effective framework for selective VLM unlearning. SIEVE directly regularizes attention-value representations while also controlling model outputs. SIEVE suppresses attention values for forget examples toward a constant zero, while preserving retain-example representations by matching them to a frozen reference model. These objectives are combined with sequence-level forget and retain supervision, enabling targeted forgetting without largely affecting retained knowledge. Extensive experiments show that SIEVE achieves state-of-the-art performance on unlearning with multiple model-modality settings, while maintaining competitive retained utility. Ablation studies further show that value suppression and negative cross-entropy contribute complementary forgetting signals, while reference-based value matching substantially reduces utility degradation. These results demonstrate that attention values provide an effective intervention point for selective multimodal unlearning when sensitive and retained knowledge are closely related.

Momentum

Related topics

Latest papers 18