PastForward: Faster On-Device GUI Agents via Computational Experience Reuse

Taehwan Park; Changmin Lee; Hayeon Lee; Taesik Gong

cs.AISep 26, 2026

PastForward: Faster On-Device GUI Agents via Computational Experience Reuse

Authors: Taehwan Park, Changmin Lee, Hayeon Lee, Taesik Gong

Organizations: Meta Superintelligence Labs

Abstract

Running GUI agents on edge devices can keep sensitive screens and interaction histories local, but the computational cost of inference at every action step makes deployment challenging. Existing GUI agent systems either perform full vision-language model (VLM) inference at each action step or reuse coarse-grained knowledge matched to prior tasks. However, dynamic mobile environments and user tasks make it difficult to fully utilize prior task executions without additional fine-tuning or task-specific offline exploration. To address this challenge, we present PastForward, a system that accelerates GUI agents through validated, fine-grained reuse of computational experience accumulated during ordinary task execution. During decoding, PastForward retrieves prior output sequences as device-adaptive multi-token proposals and verifies them in a single VLM forward pass. Across action steps, it uses prior GUI transitions to begin next-step inference while the device executes the current action, retains the early computation only when the predicted screen matches the observed screen, and carries reusable KV states forward. We evaluate PastForward on AndroidWorld workloads derived from real mobile usage patterns using multiple VLM backbones across server and edge platforms. On device, PastForward achieves action-step latency speedups of 1.63-2.36 $\times$ while maintaining task success rates.

Figures & tables

Panel of Figure 1 — Figure 1. On-device measurements of GUI agent latency using Qwen3-VL-2B on a Galaxy S26 Ultra.

Panel of Figure 2 — Figure 2. Recurrence across AndroidWorld workloads. (a) Fraction of generated tokens covered by prior exact sequences at each minimum match length $L_{\min}$ . (b) Fraction of action steps matching a prior $(S_{t},a_{t})$ pair and the subset that also reproduces $S_{t+1}$ .

Figure 4. Workflow of Experience-Guided Adaptive Decoding.

Figure 5. Workflow of Early Inference via Next-Screen Prediction.

Figure 6. Number of executions for each task under the two workloads (

M=348

task instances).

Platform	CPU/SoC	GPU/NPU	Memory
Server	2 $\times$ EPYC 9554	RTX PRO 6000 / –	96 GB GDDR7
Jetson Orin Nano	6-core Cortex-A78AE	Ampere / –	8 GB LPDDR5
Galaxy S26 Ultra	SD8 Elite Gen 5	Adreno/Hexagon	16 GB LPDDR5X
Xiaomi 15	SD8 Elite	Adreno/Hexagon	12 GB LPDDR5X

Table 1. Evaluation platforms. SD8 denotes Snapdragon 8; memory denotes server GPU memory or edge-device system memory.

Model(s)	Method	Task-uniform				User-calibrated
Model(s)	Method	SR (%)	Prefill (s)	Decoding (s)	Action step (s)	SR (%)	Prefill (s)	Decoding (s)	Action step (s)
Qwen3.5-122B-A10B	MobileGPT	26.34 $\pm$ 1.78	2.851 $\pm$ 0.416	– †	5.15 $\pm$ 0.41	23.90 $\pm$ 2.28	2.700 $\pm$ 0.515	– †	5.02 $\pm$ 0.79
Qwen3.5-122B-A10B + Llama-3.1-8B verifier	V-Droid	43.58 $\pm$ 1.76	1.692 $\pm$ 0.190	0.321 $\pm$ 0.007	4.02 $\pm$ 0.16	37.21 $\pm$ 8.75	1.957 $\pm$ 0.693	0.329 $\pm$ 0.020	4.21 $\pm$ 0.77
Qwen3-VL-8B	Vanilla	46.12 $\pm$ 1.01	0.447 $\pm$ 0.004	2.443 $\pm$ 0.017	5.62 $\pm$ 0.20	42.91 $\pm$ 6.33	0.452 $\pm$ 0.007	2.465 $\pm$ 0.121	5.52 $\pm$ 0.14
	FastV	41.62 $\pm$ 1.30	0.420 $\pm$ 0.004	2.421 $\pm$ 0.016	5.47 $\pm$ 0.16 (1.03 $\times$ )	40.71 $\pm$ 4.81	0.425 $\pm$ 0.006	2.463 $\pm$ 0.090	5.49 $\pm$ 0.12 (1.01 $\times$ )
	Agent-X	45.98 $\pm$ 2.24	0.454 $\pm$ 0.003	1.252 $\pm$ 0.002	4.44 $\pm$ 0.08 (1.26 $\times$ )	44.78 $\pm$ 4.97	0.458 $\pm$ 0.008	1.253 $\pm$ 0.025	4.30 $\pm$ 0.14 (1.28 $\times$ )

Table 2. Server-side performance on AndroidWorld under the task-uniform and user-calibrated workloads. SR and latency values are reported as mean

\pm

s.d. across three random seeds. Values in parentheses report action-step-latency speedup over Vanilla. † For MobileGPT, prefill includes the entire LLM call due to its non-streaming API.

Method	SR (%)	Prefill (s)	Decoding (s)	Action step (s)
Jetson Orin Nano
Vanilla	21.84	1.964	14.388	19.71
FastV	17.82	1.877	14.018	19.33 (1.02 $\times$ )
Agent-X	24.14	1.969	7.344	12.67 (1.56 $\times$ )
PastForward	21.70	1.597	4.227	8.36 (2.36 $\times$ )
Galaxy S26 Ultra

Table 3. Performance across three edge devices with Qwen3-VL-2B and the user-calibrated workload.

Figure 7. Action-step latency for three VLM backbones as PastForward components are added cumulatively to Vanilla under the user-calibrated workload.

Figure 8. Number of tokens processed during prefill per action step with and without Cross-Step State Carryover.

Panel of Figure 9 — Figure 9. Cumulative experience-reuse rates and action-step latency as computational experience accumulates under the user-calibrated workload with MAI-UI-8B.

Panel of Figure 10 — Figure 10. Comparison of fixed proposal lengths and our proposal-length controller on two edge devices. $\star$ marks the best value for each metric.

Panel of Figure 11 — Figure 11. Proposal-length adaptation under changing power modes on a Jetson Orin Nano. (a) Proposal length selected online by PastForward. Dashed lines mark mode transitions. (b) Corresponding results compared with fixed proposal lengths. $\star$ marks the best value.

Figure 12. Next-screen lookup-key ablation under the user-calibrated workload. Gray bars show the launch rate; blue bars show the matched-screen rate.

Panel of Figure 13 — Figure 13. Computational overhead in memory use and decoding-memory budget tradeoff on the Jetson Orin Nano. (a) Peak RAM usage comparison with Vanilla. (b) Retrieval and decoding latency under different decoding memory budgets.

Mobile GUI Agents

Mobile GUI agents use visual and structured interface information to carry out tasks across smartphone apps.

See all 81 papers

MobileExplorer: Accelerating On-Device Inference for Mobile GUI Agents via Online Exploration
May 26, 2026Runxi Huang, Liyu Zhang, Shengzhong Liu +1Mobile GUI Automation Efficient VLM Inference
Hong Kong University of Science and Technology · Shanghai Jiao Tong University
Jev-Mobile: Jev as an Executor for Mobile GUI Agents
Sep 24, 2026Linghua ZhangEfficient VLM Inference Mobile GUI Agents
Executable Agentic Memory for GUI Agent
May 12, 2026Zerui Qin, Sheng Yue, Xingyuan Hua +2GUI Agents LLM Agent Memory
1Tsinghua University, China · 2Sun Yat-sen University, China

May 26, 2026J/K move · Enter open · S save

MobileExplorer: Accelerating On-Device Inference for Mobile GUI Agents via Online Exploration

Runxi Huang, Liyu Zhang, Shengzhong Liu, Xiaomin Ouyang

Hong Kong University of Science and Technology · Shanghai Jiao Tong University

Mobile GUI Automation Efficient VLM Inference GUI Agents Mobile GUI Agents On-Device Inference

Open paper AskPDF arXiv

Mobile graphical user interface (GUI) agents enable AI models to autonomously operate smartphones on behalf of users. However, most existing systems focus primarily on optimizing task accuracy and rely on cloud-hosted models for inference, which introduces privacy concerns and network-dependent latency. As a result, fully on-device deployment of mobile GUI agents remains underexplored. We propose MobileExplorer, a new framework that accelerates on-device inference for vision-based mobile GUI agents via online exploration. The key idea is to exploit the long per-step reasoning time of vision-language models (VLMs) by performing lightweight, parallel exploration of UI elements. During model inference, the agent proactively probes semantically relevant UI elements and records these exploration traces as structured memory. To ensure reliable execution in live mobile environments, we design a two-level rollback mechanism that robustly restores the initial UI state when a fast but naive backtracking strategy fails. The collected exploration traces are then summarized into concise contextual hints and injected into the prompt to enhance the subsequent reasoning step. We evaluate MobileExplorer on multiple off-the-shelf devices using the AndroidWorld benchmark, as well as newly designed, more complex tasks and dynamic on-device environments. MobileExplorer reduces the average number of reasoning steps and end-to-end latency by 23%, while maintaining or improving task success rates by up to 5%. A video demonstration of MobileExplorer performance in the real world is available at https://youtu.be/thK7MJmdlvM .

Abstract

Mobile GUI Agents

Explore similar work

MobileExplorer: Accelerating On-Device Inference for Mobile GUI Agents via Online Exploration

Jev-Mobile: Jev as an Executor for Mobile GUI Agents

Executable Agentic Memory for GUI Agent