Tandem: Serving Hybrid Linear-Attention GUI Agents on Mobile NPUs
Authors: Zhuomin Chen, Jingchao Ni, Xu Zheng, Yong Chen, Mo Sha, Wei Cheng, Dongsheng Luo
Organizations: Florida International University, Miami, FL, USA · University of Houston, Houston, TX, USA · Independent Researcher · NEC Laboratories America, Princeton, NJ, USA · Singapore Management University, Singapore
Smartphone GUI agents complete tasks in mobile apps by repeatedly reading the screen and asking a model for the next action. Mobile neural processing units (NPUs) enable local execution, keeping screen contents on the phone, but model latency accumulates with every action. Hybrid linear attention models reduce memory requirements, yet reusing their recurrent state requires checkpoints at the appropriate input positions. Agent requests retain instructions but rewrite recent history, so checkpoint policies designed for chat continuation can miss reusable prefixes. Retaining rollback states for speculative decoding can also move prompt processing off the NPU's matrix unit, adding overhead that outweighs decoding savings on short action outputs. A loaded model can further hold the NPU awake between tasks. We present Tandem, a runtime for hybrid GUI agents on mobile NPUs. Tandem saves recurrent state where consecutive prompts diverge, allowing later requests to reuse previously processed context. It keeps most prefill on the matrix unit and produces rollback states through a short per-token tail. Its verification kernel reads each weight block once for all proposed tokens. Between request bursts, Tandem releases NPU power votes without unloading the model. We implement Tandem in llama.cpp and evaluate it on recorded agent interactions across two Snapdragon generations. On a Snapdragon 8 Elite, Tandem reduces per-step latency by 30% for text-based screen descriptions and 16% for screenshots relative to llama.cpp's fastest NPU configuration, reduces energy per text step by 33%, and lowers standby power with the model loaded from 0.76 to 0.22 W.
Figures & tables
Figure 1: Workload-driven serving challenges for mobile GUI agents. Each step observes the screen and generates an action. (a) Consecutive prompts share a prefix but diverge in recent history or screen content, making chat-oriented checkpoints poorly placed. (b) Short outputs provide few opportunities to amortize the prefill overhead of speculative rollback states. Tandem retains matrix-based prefill and uses a short recurrent tail. (c) Between tasks, active NPU power requests can waste energy even when the model remains idle. Tandem releases these requests without unloading the model.
Figure 2: Server time by phase on the app set, with stock checkpoint and batching policies, Tandem ’s speculative NPU kernels and fan cooling. Phase times cover text requests 2–21 and screenshot requests 4–21; totals are 1.31 and 5.39 s.
Content
Text
Screenshot
Rules
324
325
Task and history
49
49
On-screen controls
275
275
Image and markers
0
1,129
Total
648
1,778
Table 1: Mean prompt token composition over all 21 screens. The image count includes two boundary markers.
Figure 3: New prompt tokens per request of the replay trace: the app set, then five recorded episodes (dotted lines mark episode starts). The full-attention Qwen3-VL-4B runs on the same engine. Divergence-point checkpoints (§ 4.2 ) close most of the checkpoint gap.
Figure 4: Tandem ’s serving architecture, showing cross-request recurrent checkpoints, temporary rollback states for speculative decoding, and idle NPU session control.
Figure 5: Producing four rollback states for a 448-token batch. The stock path runs the entire recurrence on HVX. Tandem obtains the first required state from HMX and uses HVX for the three remaining updates. Widths are schematic.
Figure 6: Step latency on a Snapdragon 8 Elite with the fine-tuned Qwen3.5-4B in Q4_K_M: text uses the 61-request replay trace; screenshots use the 21-screen app set. MNN screenshot bars average 4 of the 21 steps.
Workload
Steps
llama.cpp
Marconi
Tandem
Gain
Replay trace, text
61
2.01
1.82
1.44
28%
App set, screenshots
21
5.76
5.25
4.87
15%
Held-out set, text
510
2.02
1.86
1.45
28%
AndroidControl, text
300
1.61
1.51
1.16
28%
AndroidControl, screenshots
300
3.95
3.86
3.44
13%
Held-out set, screenshots
510
6.32
–
5.35
15%
Table 2: Server time per step (s) on the main and larger workloads. Each block of rows was measured in one session with alternating runs. Marconi denotes llama.cpp with Marconi’s checkpoint policy. Gain is Tandem ’s latency reduction relative to llama.cpp.
Figure 7: Cumulative ablation on the replay trace after three text warm-up requests, using runtime switches in one executable. Idle power release is disabled. The last two additions enable batch consolidation and divergence-point checkpoints.
Step
Model
llama.cpp
Tandem
Gain
Text
full attention
1.33
1.29
3%
hybrid
1.90
1.31
31%
hybrid / attention
1.43 ×
1.01 ×
Screenshot
full attention
5.09
5.00
2%
hybrid
5.56
4.61
17%
hybrid / attention
1.09 ×
0.92 ×
Table 3: Server time per step (s) of the hybrid policy and of the full-attention Qwen3-VL-4B, both with Q4_0 weights, on llama.cpp (fastest configuration) and on Tandem , each step type measured in one session. Text steps: the replay trace; screenshot steps: the app set, at each model’s native resolution. Gain denotes the relative latency reduction.
Figure 8: Sensitivity on the replay trace, one setting varied at a time. (a)–(c) server time per step; the labels in (b) give the fraction of draft tokens accepted. (d) Share of the replay in which the NPU is power-collapsed when the agent pauses after each response, for three idle-release thresholds; without release it is 0.
Engine
Step (s)
Energy (J)
Power (W)
Tandem (NPU)
1.24
10.1
9.0
llama.cpp (NPU)
1.91
15.1
9.5
llama.cpp (NPU, MTP)
2.40
17.1
8.9
MNN (GPU, OpenCL)
5.14
35.6
8.1
llama.cpp (GPU)
5.21
33.0
7.3
llama.cpp (CPU)
21.2
86.2
5.1
Table 5: Server time, energy above idle power and the phone’s mean power while serving, per text step of the app set, on battery without any cooling, for the engines that run the hybrid model on the NPU, GPU and CPU; means over the runs of one session. The GPU rows come from a separate block, in which the Tandem reference run used 11.2 J.
Text step
Screenshot step
Model
llama.cpp
Tandem
Gain
llama.cpp
Tandem
2B
0.99
0.73
27%
3.46
3.11
4B
1.79
1.26
30%
5.94
4.84
9B
2.99
1.99
34%
20.3
18.1
Table 6: Server time per step (s) of the stock Qwen3.5 models on the NPU, with the cooler: text steps of the replay trace and screenshot steps of the app set. llama.cpp runs in its fastest configuration, without speculation. Gain denotes the relative text-step latency reduction.
LFM2
Granite
Recurrent + attention layers
22 conv. + 8
36 Mamba2 + 4
New prompt tokens per step
stock checkpoints
581
610
Tandem checkpoint policy
518
534
Server time per step (s)
llama.cpp
0.95
–
Table 7: LFM2-2.6B and Granite-4.0-H-Micro on the NPU, with the cooler: new prompt tokens and server time per text step of the replay trace, and power-collapse time during a 60-second idle window. Neither model has a draft head or a Gated DeltaNet layer. Granite’s Mamba2 scan runs on the CPU in both engines.
Foreground app
llama.cpp
Tandem
Gain
Slowdown (llama.cpp / Tandem )
Home screen
1.99
1.44
28%
–
Video playback
2.03
1.44
29%
1.9% / 0.4%
Feed scrolling
2.09
1.45
31%
5.3% / 1.1%
Camera preview
2.41
1.68
30%
21% / 17%
Table 8: Server time per step (s) on the replay trace while another app runs in the foreground, and the slowdown relative to the home screen; each scenario was measured in one session with alternating runs.