Tandem: Serving Hybrid Linear-Attention GUI Agents on Mobile NPUs
Authors: Zhuomin Chen, Jingchao Ni, Xu Zheng, Yong Chen, Mo Sha, Wei Cheng, Dongsheng Luo
Organizations: Florida International University, Miami, FL, USA · University of Houston, Houston, TX, USA · Independent Researcher · NEC Laboratories America, Princeton, NJ, USA · Singapore Management University, Singapore
Smartphone GUI agents complete tasks in mobile apps by repeatedly reading the screen and asking a model for the next action. Mobile neural processing units (NPUs) enable local execution, keeping screen contents on the phone, but model latency accumulates with every action. Hybrid linear attention models reduce memory requirements, yet reusing their recurrent state requires checkpoints at the appropriate input positions. Agent requests retain instructions but rewrite recent history, so checkpoint policies designed for chat continuation can miss reusable prefixes. Retaining rollback states for speculative decoding can also move prompt processing off the NPU's matrix unit, adding overhead that outweighs decoding savings on short action outputs. A loaded model can further hold the NPU awake between tasks. We present Tandem, a runtime for hybrid GUI agents on mobile NPUs. Tandem saves recurrent state where consecutive prompts diverge, allowing later requests to reuse previously processed context. It keeps most prefill on the matrix unit and produces rollback states through a short per-token tail. Its verification kernel reads each weight block once for all proposed tokens. Between request bursts, Tandem releases NPU power votes without unloading the model. We implement Tandem in llama.cpp and evaluate it on recorded agent interactions across two Snapdragon generations. On a Snapdragon 8 Elite, Tandem reduces per-step latency by 30% for text-based screen descriptions and 16% for screenshots relative to llama.cpp's fastest NPU configuration, reduces energy per text step by 33%, and lowers standby power with the model loaded from 0.76 to 0.22 W.
Figures & tables
Figure 1: Workload-driven serving challenges for mobile GUI agents. Each step observes the screen and generates an action. (a) Consecutive prompts share a prefix but diverge in recent history or screen content, making chat-oriented checkpoints poorly placed. (b) Short outputs provide few opportunities to amortize the prefill overhead of speculative rollback states. Tandem retains matrix-based prefill and uses a short recurrent tail. (c) Between tasks, active NPU power requests can waste energy even when the model remains idle. Tandem releases these requests without unloading the model.
Figure 2: Server time by phase on the app set, with stock checkpoint and batching policies, Tandem ’s speculative NPU kernels and fan cooling. Phase times cover text requests 2–21 and screenshot requests 4–21; totals are 1.31 and 5.39 s.
Content
Text
Screenshot
Rules
324
325
Task and history
49
49
On-screen controls
275
275
Image and markers
0
1,129
Total
648
1,778
Table 1: Mean prompt token composition over all 21 screens. The image count includes two boundary markers.
Figure 3: New prompt tokens per request of the replay trace: the app set, then five recorded episodes (dotted lines mark episode starts). The full-attention Qwen3-VL-4B runs on the same engine. Divergence-point checkpoints (§ 4.2 ) close most of the checkpoint gap.
Figure 4: Tandem ’s serving architecture, showing cross-request recurrent checkpoints, temporary rollback states for speculative decoding, and idle NPU session control.
Figure 5: Producing four rollback states for a 448-token batch. The stock path runs the entire recurrence on HVX. Tandem obtains the first required state from HMX and uses HVX for the three remaining updates. Widths are schematic.
Figure 6: Step latency on a Snapdragon 8 Elite with the fine-tuned Qwen3.5-4B in Q4_K_M: text uses the 61-request replay trace; screenshots use the 21-screen app set. MNN screenshot bars average 4 of the 21 steps.
Workload
Steps
llama.cpp
Marconi
Tandem
Gain
Replay trace, text
61
2.01
1.82
1.44
28%
App set, screenshots
21
5.76
5.25
4.87
15%
Held-out set, text
510
2.02
1.86
1.45
28%
AndroidControl, text
300
1.61
1.51
1.16
28%
AndroidControl, screenshots
300
3.95
3.86
3.44
13%
Held-out set, screenshots
510
6.32
–
5.35
15%
Table 2: Server time per step (s) on the main and larger workloads. Each block of rows was measured in one session with alternating runs. Marconi denotes llama.cpp with Marconi’s checkpoint policy. Gain is Tandem ’s latency reduction relative to llama.cpp.
Figure 7: Cumulative ablation on the replay trace after three text warm-up requests, using runtime switches in one executable. Idle power release is disabled. The last two additions enable batch consolidation and divergence-point checkpoints.
Step
Model
llama.cpp
Tandem
Gain
Text
full attention
1.33
1.29
3%
hybrid
1.90
1.31
31%
hybrid / attention
1.43 ×
1.01 ×
Screenshot
full attention
5.09
5.00
2%
hybrid
5.56
4.61
17%
hybrid / attention
1.09 ×
0.92 ×
Table 3: Server time per step (s) of the hybrid policy and of the full-attention Qwen3-VL-4B, both with Q4_0 weights, on llama.cpp (fastest configuration) and on Tandem , each step type measured in one session. Text steps: the replay trace; screenshot steps: the app set, at each model’s native resolution. Gain denotes the relative latency reduction.
Figure 8: Sensitivity on the replay trace, one setting varied at a time. (a)–(c) server time per step; the labels in (b) give the fraction of draft tokens accepted. (d) Share of the replay in which the NPU is power-collapsed when the agent pauses after each response, for three idle-release thresholds; without release it is 0.
Engine
Step (s)
Energy (J)
Power (W)
Tandem (NPU)
1.24
10.1
9.0
llama.cpp (NPU)
1.91
15.1
9.5
llama.cpp (NPU, MTP)
2.40
17.1
8.9
MNN (GPU, OpenCL)
5.14
35.6
8.1
llama.cpp (GPU)
5.21
33.0
7.3
llama.cpp (CPU)
21.2
86.2
5.1
Table 5: Server time, energy above idle power and the phone’s mean power while serving, per text step of the app set, on battery without any cooling, for the engines that run the hybrid model on the NPU, GPU and CPU; means over the runs of one session. The GPU rows come from a separate block, in which the Tandem reference run used 11.2 J.
Text step
Screenshot step
Model
llama.cpp
Tandem
Gain
llama.cpp
Tandem
2B
0.99
0.73
27%
3.46
3.11
4B
1.79
1.26
30%
5.94
4.84
9B
2.99
1.99
34%
20.3
18.1
Table 6: Server time per step (s) of the stock Qwen3.5 models on the NPU, with the cooler: text steps of the replay trace and screenshot steps of the app set. llama.cpp runs in its fastest configuration, without speculation. Gain denotes the relative text-step latency reduction.
LFM2
Granite
Recurrent + attention layers
22 conv. + 8
36 Mamba2 + 4
New prompt tokens per step
stock checkpoints
581
610
Tandem checkpoint policy
518
534
Server time per step (s)
llama.cpp
0.95
–
Table 7: LFM2-2.6B and Granite-4.0-H-Micro on the NPU, with the cooler: new prompt tokens and server time per text step of the replay trace, and power-collapse time during a 60-second idle window. Neither model has a draft head or a Gated DeltaNet layer. Granite’s Mamba2 scan runs on the CPU in both engines.
Foreground app
llama.cpp
Tandem
Gain
Slowdown (llama.cpp / Tandem )
Home screen
1.99
1.44
28%
–
Video playback
2.03
1.44
29%
1.9% / 0.4%
Feed scrolling
2.09
1.45
31%
5.3% / 1.1%
Camera preview
2.41
1.68
30%
21% / 17%
Table 8: Server time per step (s) on the replay trace while another app runs in the foreground, and the slowdown relative to the home screen; each scenario was measured in one session with alternating runs.
With the advancement of multimodal large language models (MLLMs), building GUI agent systems has become an increasingly promising direction--especially for mobile platforms, given their rich app ecosystems and intuitive touch interactions. Yet mobile GUI agents face a critical dilemma: truly on-device models (4B or smaller) lack sufficient performance, while capable models (starting from 7B) are either too large for mobile deployment or prohibitively costly (e.g., cloud-only closed-source MLLMs). To resolve this, we propose OpenPhone, a mobile GUI agent system that leverages device-cloud collaboration to tap the cost-efficiency of on device models and the high capability of cloud models, while avoiding their drawbacks. Specifically, OpenPhone enhances Qwen2.5-VL-3B via two-stage SFT->GRPO training on synthetic GUI data for strong decision-making, integrates an efficient long-reasoning and memory management mechanism to utilize historical interactions under tight resources, and defaults to on-device execution--only escalating challenging subtasks to the cloud via real-time complexity assessment. Experiments on the online AndroidLab benchmark and diverse apps show OpenPhone matches or nears larger models, with a significant reduction in cloud costs.
Running GUI agents on edge devices can keep sensitive screens and interaction histories local, but the computational cost of inference at every action step makes deployment challenging. Existing GUI agent systems either perform full vision-language model (VLM) inference at each action step or reuse coarse-grained knowledge matched to prior tasks. However, dynamic mobile environments and user tasks make it difficult to fully utilize prior task executions without additional fine-tuning or task-specific offline exploration. To address this challenge, we present PastForward, a system that accelerates GUI agents through validated, fine-grained reuse of computational experience accumulated during ordinary task execution. During decoding, PastForward retrieves prior output sequences as device-adaptive multi-token proposals and verifies them in a single VLM forward pass. Across action steps, it uses prior GUI transitions to begin next-step inference while the device executes the current action, retains the early computation only when the predicted screen matches the observed screen, and carries reusable KV states forward. We evaluate PastForward on AndroidWorld workloads derived from real mobile usage patterns using multiple VLM backbones across server and edge platforms. On device, PastForward achieves action-step latency speedups of 1.63-2.36× while maintaining task success rates.
Deploying Large Language Models (LLMs) on mobile devices enhances privacy and reduces latency, but is severely bottlenecked by hardware inefficiency. We present the first comprehensive, cross-layer measurement study of mobile LLM inference, uniquely spanning five mainstream frameworks (e.g., llama.cpp, GENIE) and three hardware backends (CPU, GPU, NPU). To enable this analysis, we develop PowerBench, a fine-grained profiling tool that provides the first backend-specific energy attribution, moving beyond traditional device-level measurements. Our study yields three critical insights: (1) Framework-induced performance gaps are substantially amplified on NPUs, reaching up to 10x using custom operators due to divergent offloading and quantization strategies. (2) We identify a distinct phase split where NPUs excel at compute-bound prefilling, while CPUs outperform all other backends in memory-bound decoding. This is driven by the NPU's preference for large, fixed-shape workloads, which conflicts with the small-kernel, dynamic nature of decoding. (3) Backend-specific profiling uncovers substantial scheduling headroom missed by prior work. Suboptimal thread configurations, uncoordinated NPU sleep latencies, and CPU polling intervals result in up to 40% energy waste. Leveraging these findings, we present an energy-oriented best-practice configuration for mobile LLM inference. We estimate that this configuration could reduce energy consumption by up to 54.8% on the NPU backend across three datasets.