cs.CRJul 2, 2026

Vision Token Manipulation Attacks on Cloud-Edge Inference of Large Vision-Language Models

Authors: Zikai ZhangRui HuOlivera KotevskaJiahao Xu

Organizations: Department of Computer Science and Engineering, University of Nevada, Reno, Reno, USA · Oak Ridge National Laboratory, Oak Ridge, USA

Abstract

Cloud-edge Large Vision-Language Model (LVLM) inference enables efficient deployment by splitting computation between edge devices and cloud servers. In this process, intermediate vision tokens are transmitted from the edge to the cloud over a communication link, thereby exposing a new attack surface. We study vision token manipulation attack (VTM-Attack) under a black-box man-in-the-middle setting, where an adversary intercepts and manipulates a subset of transmitted vision tokens under a budget constraint. We propose four naïve attack strategies and an optimization-based token selection method. Experiments on 6 state-of-the-art LVLMs (3B-72B) across 4 benchmarks show that manipulating only 10% of vision tokens can reduce accuracy by up to 88.31%. These results reveal a critical vulnerability in cloud-edge LVLM inference.

Explore similar work

CardsList