Could Mobile AI be Exploited?: Understanding the Exploitations of Trust and Ethics Gap in On-Device Small Language Models
Organizations: SPIES Research Lab Department of Computer Science & Engineering Texas A&M University
Abstract
In this paper, we present a very first study to investigate trust and ethical implications of on-device artificial intelligence (AI), focusing on small language models (SLMs) amenable for personal devices like smartphones. While on-device SLMs promise enhanced privacy, reduced latency, and improved user experience compared to cloud-based services, we posit that they might also introduce significant risks and vulnerabilities compared to their on-server counterparts. As part of our study, we conduct a systematic evaluation of the state-of-the-art on-devices SLMs, compared to their on-server counterparts, using well-established trust and ethics frameworks. Our results illustrate that on-device SLMs are significantly less trustworthy and lacks ethical safeguards, specifically demonstrating more stereotypical, unfair, privacy-breaching and harmful content generation behavior. Further, these risks of on-device SLMs could be exploited through Open Knowledge (OkNow) Attacks, and we demonstrated them using potentially unethical vanilla prompts, to which the on-device SLMs answer with valid responses without any filters and without the need for any jailbreaking or prompt engineering. These responses can be abused for various harmful and unethical scenarios like: societal harm, illegal activities, hate, self-harm, exploitable phishing content, disinformation and many others.
Figures & tables
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Environment | Average | ||||
|---|---|---|---|---|---|---|
| 0-Shot | 1-Shot | 3-Shot | 1-Shot | 3-Shot | ||
| (P) | (P) | (A) | (A) | |||
| Gemma-2 | On-Server | 0.25 | 27.38 | 0.00 | 89.00 | 92.75 |
| On-Device | 42.00 | 11.00 | 0.38 | 45.63 | 27.75 | |
| Phi-2 | On-Server | 64.00 | 0.50 | 0.00 | 91.00 | 97.13 |
| On-Device | 82.88 | 57.88 | 25.63 | 75.25 | 71.00 | |
| Perspective | SLM | Statistical Diff. ( p ) | Effect size ( r ) |
|---|---|---|---|
| Stereotype | Gemma-2B | < 0.005 | Large (0.70) |
| Phi-2 | < 0.005 | Medium (0.48) | |
| RedPajama-3B | < 0.005 | Large (0.87) | |
| Fairness | Gemma-2B | = 0.043 | Large (0.81) |
| Phi-2 | = 0.042 | Large (0.81) | |
| RedPajama-3B | = 0.031 | Large (0.90) |
Language Model Safety Evaluation
Language model safety evaluation measures harmful behavior, refusal, robustness, and over-refusal across realistic prompts, users, and deployment settings.
See all 576 papers