Agent+P: Guiding UI Agents via Symbolic Planning

Shang Ma; Xusheng Xiao; Yanfang Ye

cs.MAOct 7, 2025

Agent+P: Guiding UI Agents via Symbolic Planning

Authors: Shang Ma, Xusheng Xiao, Yanfang Ye

Organizations: University of Notre Dame · Arizona State University

Abstract

Large Language Model (LLM)-based UI agents show great promise for UI automation but often hallucinate in long-horizon tasks due to their lack of understanding of the global UI transition structure. To address this, we introduce AGENT+P, a novel framework that leverages symbolic planning to guide LLM-based UI agents. Specifically, we model an app's UI transition structure as a UI Transition Graph (UTG), which allows us to reformulate the UI automation task as a pathfinding problem on the UTG. This further enables an off-the-shelf symbolic planner to generate a provably correct and optimal high-level plan, preventing the agent from redundant exploration and guiding the agent to achieve the automation goals. AGENT+P is designed as a plug-and-play framework to enhance existing UI agents. Evaluation on the AndroidWorld benchmark demonstrates that AGENT+P improves the success rates of state-of-the-art UI agents by up to 14.32% and reduces the action steps by 37.70%. Our code is available at: https://github.com/KiteFlyKid/agentp.

Figures & tables

	Nodes		Edges
Agent	$>$	$\leq$	$>$	$\leq$
DroidRun	27.0	71.2	62.7	168.0
LX-GUIAgent	29.3	53.8	65.9	129.2
AutoGLM	39.4	42.0	90.6	100.3
Finalrun	23.8	55.0	59.5	125.6
UI-Venus	18.0	50.7	66.0	108.0

Table 1: Average UTG node and edge counts. Headers compare app-specific success rate (

\text{SR}_{\text{app}}

) with the overall average (

\overline{\text{SR}}

UI Automation Notation	Classical Planning Equivalent
A UI state (UI) $u\in\mathcal{U}$	A state $s\in\mathcal{S}$ where the predicate $at(u)$ is true
The set of all UIs $\mathcal{U}$	The state space $\mathcal{S}$
The initial UI $u_{init}$	The initial state $s_{init}$ , defined by $at(u_{init})$
The target UI $u_{target}$	The goal $\mathcal{G}$ , specified by the condition $at(u_{target})$
A UI transition via the edge $(u_{i},u_{j})$ with label $a$	A planning action
Precondition: the app is on UI $u_{i}$	$pre(a):at(u_{i})$

Table 2: Mapping of UI automation notation to classical planning equivalent.

User Task Execution
Approach	SR (%)	Steps	Time (s)
DroidRun	15.91	5.18	33.38
+ Agent+P	28.41	17.09	99.50
MobileUse	9.09	20.45	256.02
+ Agent+P	11.36	19.61	219.28
T3A	24.32	17.89	114.39

Table 3: Agent+P compared with existing UI agents and traditional UI Automation approaches. We highlight the rows where Agent+P is integrated. SR is short for Success Rate.

Component	Variant	SR (%)
UTG Builder	w/o Dynamic Update	34.09
Node Selector	w/o Embedding	36.36
Node Selector	w/o MLLM	27.27
Plan Generator	DFS-based	22.73
Plan Generator	BFS-based	25.00
Agent+P (Full Method)		38.64

Table 4: Ablation results for Agent+P .

Panel of Figure 2 — Figure 2: Node Selector accuracy.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Step-by-step case study.

App	Tasks	Nodes	Edges
VLC	3	85	190
Simple Calendar Pro	17	24	25
Tasks	5	86	127
Markor	16	14	17
OsmAnd	3	152	508

Appendix

Table 5: Statistics of the AndroidWorld subset used in evaluation.

Method	SR (%)	Improvement
DroidRun	77.59	–
+ Agent+P	78.45	+0.86
MobileUse	51.72	–
+ Agent+P	62.93	+11.21
T3A	62.07	–
+ Agent+P	63.79	+1.72

Appendix

Table 6: Performance of Agent+P and baselines on the full AndroidWorld benchmark.

Figure 4: Graphviz visualization of the UTG of Simple Calendar Pro in AndroidWorld. An activity is a unit of Android UI Google Developers (2025) . Edge labels are removed for visual clarity.

Category	Content
Natural Language Instructions	— UTG Navigation Guide — Current UI: MainActivity — NAVIGATION PLAN FOR YOUR GOAL — Goal Analysis: Based on your goal, the system identified these target destinations: • ManageEventTypesActivity Confidence: 95% OPTIMAL PATH (Follow these steps in order): Step 1: Navigate from MainActivity to SettingsActivity Step 2: Navigate from SettingsActivity to ManageEventTypesActivity IMMEDIATE NEXT ACTION: $\rightarrow$ on click the ImageView widget with content-description “more options” via API call “virtualinvoke r0.<android.content.Context: void startActivity(android.content.Intent)>(r1) ()” This will take you to: SettingsActivity Total steps in optimal path: 2
Usage Tips	This path was computed using PDDL planning for guaranteed optimality — USAGE TIPS — • If a NAVIGATION PLAN is shown above, follow it step by step for optimal path • The plan was computed using formal PDDL planning algorithms • If no plan exists, the target is unreachable from current location • Some UI elements may not be visible - scroll if needed
Fallback	— ALL AVAILABLE NAVIGATION OPTIONS FROM HERE — • TO REACH: EventActivity $\rightarrow$ on click ImageButton widget with content-description “New Event” via API call “virtualinvoke r0.<android.content.Context: void startActivity(android.content.Intent)>(r1) ()" • TO REACH: SettingsActivity $\rightarrow$ on click Button widget with content-description “Settings” via API call “virtualinvoke r0.<android.content.Context: void startActivity(android.content.Intent)>(r1) ()" • TO REACH: TaskActivity $\rightarrow$ on click ImageButton widget via api call “virtualinvoke r2.<android.content.Context: void startActivity(android.content.Intent)>( r3) ()” — End Navigation Guide —

Appendix

Table 7: Example natural language prompt for the Simple Calendar Pro app with the task Add a new event type named ‘1-on-1 meeting’ .

Explore similar work

Executable Agentic Memory for GUI Agent
May 12, 2026Zerui Qin, Sheng Yue, Xingyuan Hua +2Graphical User Interface Agents Agentic Memory
1Tsinghua University, China · 2Sun Yat-sen University, China
SEE: Structure-aware Exploring & Exploiting for Long-horizon GUI Agent Trajectory Synthesis
Jul 20, 2026Zhuohang Fan, Beichen Zhang, Yuanfa Li +4Graphical User Interface Agents Graphical User Interface
Harbin Institute of Technology, Weihai, China · Harbin Institute of Technology (Weihai) Qingdao Research Institute, Qingdao, China
VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation
Apr 23, 2026Qijun Han, Haoqin Tu, Zijun Wang +11Graphical User Interface Agents Agent Loop
UC Santa Cruz · CMU · UNC-Chapel Hill +2

May 12, 2026, cs.AIJ/K move · Enter open · S save

Executable Agentic Memory for GUI Agent

Zerui Qin, Sheng Yue, Xingyuan Hua, Yongjian Fu+1

1Tsinghua University, China · 2Sun Yat-sen University, China

Graphical User Interface Agents Agentic Memory Knowledge Graphs Efficient Planning Value Functions

Open paper AskPDF arXiv

Modern GUI agents typically rely on a model-centric and step-wise interaction paradigm, where LLMs must re-interpret the UI and re-decide actions at every screen, which is fragile in long-horizon tasks. In this paper, we propose Executable Agentic Memory (EAM), a structured Knowledge Graph (KG) that shifts GUI planning from free-form generation to a robust retrieval-and-execution process. Our approach includes a sample-efficient memory construction pipeline using state-aware DFS and action-group mining to compress multi-step routines. To ensure efficient planning, we introduce a value-guided graph search where a lightweight Q-function model steers Monte Carlo Tree Search (MCTS) over the KG. We theoretically establish bias-consistency for the Q-model and derive sample complexity bounds for path recovery. Empirically, EAM outperforms state-of-the-art baselines like UI-TARS-7B by up to

19.6\%

on AndroidWorld, while reducing token costs

6\times

relative to GPT-4o. With a

2.8

s average latency, EAM enables reliable, quick, and long-horizon GUI automation.

Abstract

Explore similar work

Executable Agentic Memory for GUI Agent

SEE: Structure-aware Exploring & Exploiting for Long-horizon GUI Agent Trajectory Synthesis

VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation