PocketTSLM: Deploying an On-Device Soft-Prompt Time-Series Language Model for Privacy-Preserving Smartphone Reasoning
Authors: Max Rosenblattl, Nikhil Krishnaswamy, Leon Nissen, Manuel Milling, Nicolas Zumarraga, Ning Wang, Thomas Kaar, Robert Jakob, Björn W. Schuller, Elgar Fleisch, Patrick Langer, Paul Schmiedmayer
Organizations: Stanford University, USA · CHI, Technical University of Munich University Hospital, Germany · ETH Zurich, Switzerland · Aionic Labs, Switzerland · GLAM, Imperial College London, UK · Centre for Digital Health Interventions (CDHI), University of St. Gallen, Switzerland
Consumer phones and wearables increasingly collect physiological time-series data, yet most mobile systems treat these streams as summaries or text rather than as temporal evidence. This representation is poorly matched to personal health systems that need to retrieve, route, or summarize sensor evidence while preserving temporal structure. Cloud-hosted time-series language models avoid smartphone limits, but they repeatedly move sensitive physiology off the device and add latency and per-query compute cost. Mobile model alternatives that aggregate time-series data into statistical features or serialize raw samples as text either compress temporal structure or exhaust the context and memory budgets of small on-device models. We present PocketTSLM, a mobile-native deployment of OpenTSLM SoftPrompt, the soft-prompt variant of the OpenTSLM time-series language model. PocketTSLM runs fully on a consumer smartphone by injecting encoded time-series windows as embeddings instead of tokenizing time series as text. A numerical parity protocol verifies identical greedy-decoded token sequences between the Swift port and a Python reference using Apple's MLX machine-learning framework. Answers also match the PyTorch reference, although rationale wording can differ. An on-device context-economics benchmark compares the soft-prompt path with a text-serialized baseline on the same backbone. In our benchmark format, soft-prompting uses approximately 12 times fewer time-series tokens and reduces prompt-prefill cost. PocketTSLM processes 30-second Sleep-EDF electroencephalography clips for sleep staging and a 1,000-sample (10 s) 12-lead electrocardiogram clip from ECG-QA. In comparison, text-serialized baselines exceed the phone's context or memory budget at longer time-series windows. All inference runs entirely on the device, with no health data transmitted during inference.
Figures & tables
Figure 1: The OpenTSLM-SPpipeline running fully on-device. Health time-series data are split into patches and encoded by a CNNpatch encoder and transformer (the time-series encoder), projected into the LLMembedding space, and interleaved with the embedded text prompt. The combined sequence is decoded by a frozen pre-trained LLMwith a LoRAadapter into a natural-language model rationale. The entire pipeline executes on the device through the MLX-Swiftruntime on Apple silicon, with no health data leaving the phone. Figure adapted from Langer et al. [11] .
Figure 2: On-device EEGsleep staging in the demo app , running on an iPhone Air in airplane mode. The app’s task menu (left), the Sleep-EDF prompt with its EEGinput summary (center), and the model-generated rationale and answer (right).
Figure 3: On-device context economics (iPhone, Llama-3.2-1B, greedy decoding), for single-channel EEG(Sleep-EDF [ 34 , 35 ] ) and 12-lead ECG(ECG-QA [ 28 ] ). (a) Time-series tokens vs. series length: text serialization costs approximately 12× as many tokens as soft prompts across evaluated lengths and both tasks (soft-prompt tokens are injected embeddings, not vocabulary tokens). (b) TTFT: soft-prompt TTFTis lower across the evaluated sweep, while the text arm grows steeply and drops out once its full prompt exceeds the 4,300-token on-device budget ( L≥250 for ECG, L≥1500 for EEG). Error bars show ± SD over five sweeps. Peak memory (Tables 1 , 2 ) tracks the same token count.
time-series tokens
TTFT(s)
peak mem (MiB)
L
text
soft
ratio
text
soft
text
soft
100
300
25
12.0 ×
2.15±0.69
1.29±0.40
3878
3621
250
750
63
11.9 ×
3.88±1.05
1.46±0.49
3890
3668
500
1500
125
12.0 ×
6.80±1.00
1.68±0.53
4023
3722
1000
3000
250
12.0 ×
14.84±0.76
2.44±0.11
4232
3836
1500
4500
375
12.0 ×
n/a
3.02±0.10
n/a
3885
Table 1: On-device context economics, EEGsleep staging (Sleep-EDF [ 34 , 35 ] , fixed raw sample, greedy, max 80 new tokens). The text and soft columns report only the time-series tokens. The full prompt adds a fixed ∼ 211-token pre/post scaffold shared by both arms. “n/a” marks a series whose full prompt exceeds the 4,300-token on-device budget and is skipped to avoid an uncatchable OSmemory termination. TTFTis mean ± SD over five sweeps; peak memory varied by ≤3 MiB across runs and is reported as the mean; soft-prompt tokens are injected embeddings, not vocabulary tokens.
time-series tokens
TTFT(s)
peak mem (MiB)
L
text
soft
ratio
text
soft
text
soft
100
3600
300
12.0 ×
13.64±2.58
2.85±0.77
4372
3885
250
9000
756
11.9 ×
n/a
4.27±0.98
n/a
3959
500
18000
1500
12.0 ×
n/a
6.08±1.14
n/a
4087
1000
36000
3000
12.0 ×
n/a
11.62±2.16
n/a
4256
Table 2: On-device context economics, 12-lead ECG(ECG-QACoT [ 28 ] , fixed bundled sample, greedy, max 80 new tokens). Columns as in Table 1 (time-series tokens; full prompt adds a fixed ∼ 533-token scaffold).
Stress is a pervasive determinant of mental health and a key target for mobile health interventions. On-device language models (ODLMs) offer privacy-preserving inference without cloud dependency, yet their feasibility for health prediction under mobile resource constraints remains underexplored. We evaluate ODLMs for multi-modal stress prediction using zero-shot prompting, measuring predictive accuracy alongside latency and throughput. Our results show that objective sensor features marginally outperform subjective self-reports on average, and that lightweight sub-2B models achieve low latency with predictable resource usage. Our findings highlight both the promise and the practical constraints of ODLMs for mobile mental health.
Ming Hsieh Department of Electrical & Computer Engineering, University of Southern California · Thomas Lord Department of Computer Science, University of Southern California
The success of Large Audio Language Models has driven the development of massive multimodal networks exceeding billions of parameters. However, the demand for privacy-preserving, low-latency processing has shifted focus toward Small Audio Language Models (SALMs) capable of on-device execution. In this paper, we introduce Samsone, a family of SALMs designed for edge computing. Our core model, Samsone-134M, establishes a new state-of-the-art for its size class across multiple benchmarks. We further explore the scaling laws of SALMs by introducing Samsone-99M and Samsone-356M. Despite their compact footprint, the Samsone family delivers performance competitive with models orders of magnitude larger. To foster open research and reproducibility, we train Samsone on publicly available data. We release the training code, model weights, mobile-optimized checkpoints and provide an open-source Android application to demonstrate real-time on-device inference of Samsone.
Piotr Masztalski, Michał K. Grzeszczyk, Olaf Sikorski
Samsung R&D Institute Poland · AGH University of Krak´ow, Poland
On-device Small Language Models (SLMs) promise fully offline, private AI experiences for mobile users (no cloud dependency, no data leaving the device). But is this promise achievable in practice? This paper presents a longitudinal practitioner case study documenting the engineering challenges of integrating SLMs (Gemma 4 E2B, 2.6B parameters; Qwen3 0.6B, 600M parameters) into Palabrita, a production Android word-guessing game. Over a 5-day development sprint comprising 204 commits (~90 directly AI-related), the system underwent a radical transformation: from an ambitious design where the LLM generated complete structured puzzles (word, category, difficulty, and five hints as JSON) to a pragmatic architecture where curated word lists provide the words and the LLM generates only three short hints, with a deterministic fallback if it fails. We identify five categories of failures specific to on-device SLM integration: output format violations, constraint violations, context quality degradation, latency incompatibility, and model selection instability. For each failure category, we document the observed symptoms, root causes, and the prompt engineering and architectural strategies that effectively mitigated them, including multi-layer defensive parsing, contextual retry with failure feedback, session rotation, progressive prompt hardening, and systematic responsibility reduction. Our findings demonstrate that on-device SLMs are viable for production mobile applications, but only when the developer accepts a fundamental constraint: the most reliable on-device LLM feature is one where the LLM does the least. We distill our experience into eight actionable design heuristics for practitioners integrating SLMs into mobile apps.