With the advancement of powerful AI chips, on-device machine learning is gaining traction for its low latency and privacy protection. However, when edge devices are compromised, attackers can obtain white-box access to the deployed models by reversing the apps directly to obtain the knowledge of peer devices, which is known as Cross-Node Attacks, like performing efficient adversarial attacks. Most existing defenses target known vulnerabilities, yet white-box access still enables attackers to uncover more unknown weaknesses. This creates a critical risk for edge-device deployments, where one compromised device can endanger all models distributed from the same source. Motivated by findings that the model robustness is primarily influenced by the curvature of the decision boundary, we investigate how model parameters shape its curvature. In this paper, we propose a novel method, BMO, that obfuscates model parameters to diversify the decision boundary further resisting cross-node attack while preserving its benign accuracy. Specifically, we explore decomposing model parameters into boundary-position and boundary-curvature components based on null-space constraints, then models with differentiated decision boundaries are instantiated by injecting noise into the boundary-curvature component. Across all four major types of adversarial attacks with four widely-deployed on-device models, BMO achieves an average success rate of 68.68% in defense, far exceeding the most competitive baseline, which attains only 22.32% in defending against cross-node attacks. Moreover, BMO achieves comparable performance in benign accuracy compared with baselines.
Figures & tables
Figure 1: An Illustration of Cross-node Attack . End users install applications with integrated on-device models from app stores to enjoy the provided services. Meanwhile, an adversary can exploit these applications by unpacking them to access the model in white-box settings. Then, the peer devices installed the same model are totally exposed to adversary leading a large-scale compromising.
Figure 2: An Overview of BMO. BMO decomposes model weights into Magnitude components related to the prediction of the normal sample, and Direction components related to the shape of the decision boundary. It then obfuscates the model decision boundary by altering the direction components to reduce the effectiveness of cross-node adversarial samples, while maintaining the magnitude components to preserve predictive performance.
Figure 3: Main Results of BMO. We comprehensively evaluate the effectiveness of BMO under defense conditions by thoroughly moderating the three elements that determine the effectiveness of an attack, i.e . , the intensities, algorithms, knowledge, and the attack targets.
Figure 4
Figure 6: Ablation Studies of BMO. Lower C-ASR and higher Anti-ASR indicate better defense performance against cross-node attacks, while higher Performance denotes better model performance on the target dataset.
Figure 7: Mean Anti-ASR under increasing perturbation budgets across six representative edge-device models.
Anti-ASR
Model
( ϵ=0.05 )
MLP
ResNet
DeiT_tiny
DeiT_small
MobileViT_s
MobileNet_v2
MNIST
86.70/2.43
69.21/2.96
72.20/2.08
76.39/3.04
61.71/2.80
89.76/2.99
CIFAR10
-
40.06/2.78
35.16/2.06
47.04/2.85
61.22/2.75
61.01/2.86
CIFAR100
-
52.72/2.66
46.34/3.15
56.23/3.19
45.52/2.78
58.73/2.93
ImageNet
-
40.55/3.13
50.53/3.25
62.04/2.99
38.72/2.85
53.49/2.99
Table 1: Cross-Dataset and Cross-Architecture Robustness (Anti-ASR) Across Six Edge-Device Models
Figure 8: Effectiveness Against Different Attack Families.
Neural networks (NNs) are increasingly deployed on edge devices such as mobile and IoT platforms, offering low latency and improved privacy compared to cloud-based deployment. However, on-device models embedded in mobile applications can be unpacked and exposed to adversaries, enabling cross-node attacks in centrally distributed ecosystems, where adversarial examples crafted on one compromised device transfer effectively to other nodes sharing the same models. To address this threat, we propose VeilNN, a defense framework that reduces adversarial transferability across on-device models via model obfuscation. The core idea of VeilNN is to introduce controlled model diversity while preserving original task functionality. Specifically, VeilNN applies stealthy and irreversible obfuscation at both the architectural and parameter levels through network cell obfuscation and layer assembling, and is applicable to diverse NN architectures, including MLPs, CNNs, and Vision Transformers. Extensive experiments demonstrate that VeilNN effectively defends against cross-node attacks, achieving an average defense success of 49.19% and up to 81.19% across four adversarial attack types, with more than 33.16% improvement over existing defenses.
The rise of edge-based machine learning has enabled distributed adaptation of language models across mobile and IoT devices, offering privacy preservation and real-time responsiveness. However, distributed fine-tuning of language models on untrusted or heterogeneous edge nodes introduces new vulnerabilities. Compromised or unreliable devices can inject poisoned updates, leading to stealthy model manipulation or convergence degradation. Classical defenses such as robust aggregation or temporal anomaly detection operate on a single global model and are therefore limited in detecting coordinated or persistent poisoning. This work proposes a new system-level defense based on model multiplicity. Instead of maintaining one global model, the system rotates or concurrently trains multiple small language models (e.g., DistilGPT-2), each updated by independently sampled subsets of edge nodes. These models evolve under distinct training trajectories, creating multiple independent views of the same distributed population. Divergence between models quantified through gradient similarity, loss evolution, or parameter variance serves as a signal of anomalous or adversarial behavior. When one model deviates significantly from the ensemble mean, the system flags its contributing nodes for isolation or re-weighting. We implement this framework and evaluate it on edge-scale simulations of Small Language Model (SLM) training under varying heterogeneity and attack conditions. Results show that model multiplicity enables earlier and more reliable detection of poisoning compared to classical single-model defenses such as Flanders and Robust methods. Our findings demonstrate that diversity in model evolution can serve as a practical and effective defense mechanism for secure distributed learning on resource-constrained edge devices.
Stefan Behfar, Richard Mortier
Computer Lab, University of Cambridge, Cambridge, United Kingdom
Mobile on-device AI (MoAI) systems that integrate locally deployed AI models with conventional mobile software components are emerging as a key paradigm for delivering intelligent functionality directly on end-user devices. By moving inference from remote cloud services to the local mobile environment, such systems enable privacy-preserving, low-latency, and offline-capable AI functionality, yet introduce new security risks arising from the local storage of AI models. This paper presents the first comprehensive systematization of knowledge on MoAI security, covering security pillars, attack landscape, and defense landscape of MoAI systems. We further identify unresolved gaps in current attack and defense research and point to promising directions for future research in this emerging area. Our work establishes the first systematic framework for understanding the attack and defense landscapes of MoAI systems, serving as a foundation for building secure MoAI systems and advancing research in this critical domain. Companion resources are available at https://github.com/Jinxhy/Awesome-MoAI-Security.
Yujin Huang, Xin Zheng, Xingliang Yuan +1
The University of Melbourne · RMIT University · Nanyang Technological University