K^2-Agent: Co-Evolving Know-What and Know-How for Hierarchical Mobile Device Control
Authors: Zhe Wu, Donglin Mo, Hongjin Lu, Junliang Xing, Jianheng Liu, Yuheng Jing, Kai Li, Kun Shao, Jianye Hao, Yuanchun Shi
Organizations: Department of Computer Science and Technology, Tsinghua University · Huawei Noah’s Ark Lab · Institute of Automation, Chinese Academy of Sciences
Existing mobile device control agents often perform poorly when solving complex tasks requiring long-horizon planning and precise operations, typically due to a lack of relevant task experience or unfamiliarity with skill execution. We propose K2-Agent, a hierarchical framework that models human-like cognition by separating and co-evolving declarative (knowing what) and procedural (knowing how) knowledge for planning and execution. K2-Agent's high level reasoner is bootstrapped from a single demonstration per task and runs a Summarize-Reflect-Locate-Revise (SRLR) loop to distill and iteratively refine task-level declarative knowledge through self-evolution. The low-level executor is trained with our curriculum-guided Group Relative Policy Optimization (C-GRPO), which (i) constructs a balanced sample pool using decoupled reward signals and (ii) employs dynamic demonstration injection to guide the model in autonomously generating successful trajectories for training. On the challenging AndroidWorld benchmark, K2-Agent achieves a 76.1% success rate using only raw screenshots and open-source backbones. Furthermore, K2-Agent shows powerful dual generalization: its high-level declarative knowledge transfers across diverse base models, while its low-level procedural skills achieve competitive performance on unseen tasks in ScreenSpot-v2 and Android-in-the-Wild (AitW).
Figures & tables
Figure 1: K²-Agent’s co-evolutionary learning curve on AndroidWorld. The main curve shows the agent’s success rate steadily improving. Ablations (lower curves) confirm the contribution of key components, and subplots below illustrate the expanding mastery over new apps and tasks.
Figure 2: Overview of the K²-Agent . Top : The SRLR loop where declarative knowledge (knowing what) is iteratively improved using feedback. Bottom : The skill acquisition process where procedural knowledge (knowing how) is learned via C-GRPO, bootstrapped from a single demonstration.
Figure 3: An illustration of the SRLR self-evolution loop. (1) Summarize: An initial knowledge base is automatically distilled from a demonstration. (2) Reflect: The agent analyzes its execution trace to identify deviations. (3) Locate: The failure’s root cause is pinpointed. (4) Revise: Atomic operators repair the knowledge base for the next iteration.
Figure 4: Our C-GRPO framework, featuring its two main curriculum components: Error-Decoupled Replay Balancing (left) to construct balanced mini-batches, and Dynamic Demonstration Injection (center) to provide adaptive guidance for the GRPO update (right).
Type
Agent
Base Model
Input
SR
Training free
M3A ( Rawles et al., 2024 )
GPT-4 Turbo
Screenshot + A11y
30.6
AndroidGen ( Lai et al., 2025 )
GPT-o
A11y Tree
46.8
Agent S2 ( Agashe et al., 2025 )
Claude-3.5-Sonnet
Screenshot
54.3
MobileUse ( Li et al., 2025 )
Qwen2.5-VL-72B
Screenshot
62.9
DroidRun ( DroidRun, 2025 )
Gemini-2.5-Pro
Screenshot + A11y
63.0
FinalRun ( FinalRun, 2025 )
GPT-5
Screenshot + A11y
76.7
Table 1: Comparison of K 2 -Agent and baselines on AndroidWorld. Our method reports the mean ± std over 3 independent runs.
Agent
AitW-General (SR %)
AitW-WebShopping (SR %)
SoM ( Zheng et al., 2024 )
16.7
11.5
AppAgent ( Zhang et al., 2025 )
17.7
8.3
CogAgent ( Hong et al., 2024 )
25.0
38.5
AutoUI ( Zhang and Zhang, 2023 )
22.9
25.0
DigiRL ( Bai et al., 2024 )
71.9
67.2
K 2 -Agent
86.5
68.3
Table 2: Zero-shot transfer to AitW using the low-level executor trained on AndroidWorld.
Mobile
Desktop
Web
Agent Model
Text
Icon
Text
Icon
Text
Icon
Overall
Operator ( OpenAI, 2025 )
47.3
41.5
90.2
80.3
92.8
84.3
70.5
Claude 3.7 Sonnet ( Anthropic, 2025a )
-
-
-
-
-
-
87.6
UI-TARS-72B ( Qin et al., 2025 )
94.8
86.3
91.2
87.9
91.5
87.7
90.3
JEDI-7B ( Xie et al., 2025 )
96.9
87.2
95.9
87.9
94.4
84.2
91.7
GUI-Owl-32B ( Ye et al., 2025 )
98.6
90.0
97.9
87.8
94.4
86.7
93.2
Table 3: Zero-shot performance of the K²-Agent executor on the ScreenSpot-v2 benchmark.
Figure 5: Ablation and Component Analysis. (a) : The declarative knowledge from our SRLR loop provides a substantial performance boost across four different powerful VLM backbones. (b) : Training reward curves for the low-level executor comparing full C-GRPO (blue), C-GRPO without replay balancing (red), and C-GRPO without demonstration injection (green).
Configuration
SR (%)
No Hierarchy
35.3
No Hierarchy + SRLR
58.6
SRLR + SFT-Low
62.0
SRLR + GRPO-Low
68.9
K²-Agent (Full)
76.1
Table 4: Ablation study of K²-Agent components on AndroidWorld benchmark.
Figure 6: Parameter sensitivity of C-GRPO. (a) : Varying the conventional pool ratio βcon . (b) : Effect of the injection temperature T in the difficulty-gating function.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Cognitive inspiration for the K²-Agent framework. The design maps the human distinction between declarative (“knowing what”) and procedural (“knowing how”) knowledge onto a hierarchical agent architecture.
Figure 8: Distribution of the induced declarative knowledge
Figure 9: An example of a single training data sample in JSON format.
Component
Hyperparameter
Value
High-Level Planner ( πH )
MAX_ITER
Max revision cycles per task
10
SUCCESS_THRESH
Consecutive successes to stop
3
Low-Level Executor ( πL )
Base Learning Rate
Learning rate for policy
1×10−6
Epoch Count
Number of training epochs
2
Appendix
Table 5: Comprehensive list of hyperparameters for K²-Agent.
Application
Description
# Tasks
Simple Calendar Pro
A calendar app for creating, deleting, and managing events and appointments.
17
Settings
The Android system settings app for managing device settings such as Bluetooth, Wi-Fi, and brightness.
15
Markor
A note-taking app for creating, editing, deleting, and managing notes and folders.
14
Broccoli - Recipe App
A recipe management app for adding, deleting, and organizing recipes.
13
Pro Expense
An expense tracking app for adding, deleting, and managing expenses.
9
Simple SMS Messenger
An SMS app for sending, replying to, and resending text messages.
7
Appendix
Table 6: Overview of the 20 applications and the number of associated tasks in the AndroidWorld ( Rawles et al., 2024 ) . The description for each app highlights its core functionality.
Action
Description
Arguments
click
Performs a standard, short tap on a specific screen location. Used for activating buttons, selecting items, or placing cursor focus.
coordinate
long_press
Executes a sustained press at a given coordinate. Essential for tasks like selecting text, opening context menus, or revealing hidden options.
coordinate
swipe
Drags from a starting point to an ending point. Used for scrolling, navigating pages, or adjusting sliders.
coordinate, coordinate2
type
Inputs a character sequence into the currently focused text field. This action directly injects text, bypassing the on-screen keyboard.
text
system_button
Triggers a system-level hardware button command, such as navigating back or returning to the home screen.
button
terminate
Ends the current task episode, reporting the final outcome. This signals to the high-level planner whether the goal was achieved.
status
Appendix
Table 7: The structured action space of K²-Agent. Each action is defined as a function call with specific arguments to control the mobile device.
Figure 10: Public leaderboard of AndroidWorld as of August 2025, showing K²-Agent ranked 1st among all methods using only raw screenshots.
Task
K²-Agent
mobile-use
K²-Agent(Gemini)
UI-Venus-72B
DroidRun
K²-Agent(Gpt-4o)
AudioRecorderRecordAudio
✓
✓
✓
✓
✓
✗
AudioRecorder-FileName
✓
✓
✓
✗
✓
✓
BrowserDraw
✗
✗
✗
✗
✗
✗
BrowserMaze
✓
✓
✗
✗
✓
✗
BrowserMultiply
✓
✓
✓
✗
✓
✓
CameraTakePhoto
✓
✓
✓
✓
✓
✓
Appendix
Table 8: Detailed performance comparison on all 116 tasks in AndroidWorld.
Mobile
Desktop
Web
Agent Model
Text
Icon
Text
Icon
Text
Icon
Overall
Operator ( OpenAI, 2025 )
47.3
41.5
90.2
80.3
92.8
84.3
70.5
Claude 3.7 Sonnet ( Anthropic, 2025a )
-
-
-
-
-
-
87.6
UI-TARS-1.5 ( Qin et al., 2025 )
-
-
-
-
-
-
94.2
Seed-1.5-VL ( Guo et al., 2025 )
-
-
-
-
-
-
95.2
SeeClick ( Cheng et al., 2024 )
78.4
50.7
70.1
29.3
55.2
32.5
55.1
Appendix
Table 9: Zero-shot performance of the K²-Agent executor on the ScreenSpot-v2 benchmark, compared to other state-of-the-art GUI grounding models. Our executor demonstrates competitive performance, highlighting the strong generalization of its learned procedural skills.
Figure 11: Successful execution of the complex, multi-app task MarkorCreateNoteAndSms .
Figure 12: A representative failure case from the RetroCreatePlaylist task.
Figure 13: Declarative knowledge generalization on Android-in-the-Wild (AitW).
Figure 14: Zero-shot procedural skill transfer to the ScreenSpot-v2 benchmark.
Mobile agents powered by vision-language models have demonstrated impressive capabilities in automating mobile tasks, with recent leading models achieving a marked performance leap, e.g., nearly 70% success on AndroidWorld. However, these systems keep their training data closed and remain opaque about their task and trajectory synthesis recipes. We present OpenMobile, an open-source framework that synthesizes high-quality task instructions and agent trajectories, with two key components: (1) The first is a scalable task synthesis pipeline that constructs a global environment memory from exploration, then leverages it to generate diverse and grounded instructions. and (2) a policy-switching strategy for trajectory rollout. By alternating between learner and expert models, it captures essential error-recovery data often missing in standard imitation learning. Agents trained on our data achieve competitive results across three dynamic mobile agent benchmarks: notably, our fine-tuned Qwen2.5-VL and Qwen3-VL reach 51.7% and 64.7% on AndroidWorld, far surpassing existing open-data approaches. Furthermore, we conduct transparent analyses on the overlap between our synthetic instructions and benchmark test sets, and verify that performance gains stem from broad functionality coverage rather than benchmark overfitting. We release data and code at https://njucckevin.github.io/openmobile/ to bridge the data gap and facilitate broader mobile agent research.
Kanzhi Cheng, Zehao Li, Zheng Ma +11
1Nanjing University · 4Shanghai AI Laboratory · 2SenseTime +3
Long-horizon mobile manipulation presents significant challenges due to compounding execution errors and capacity interference between locomotion and arm control. While recent Vision-Language-Action models excel at short-horizon tasks, they lack the hierarchical reasoning required for multi-stage objectives. Furthermore, existing hierarchical agents suffer from rigid sub-task mapping, inflexible replanning, and a lack of continuous learning. To address these limitations, we introduce MobiAgent, a dual-loop agentic framework that bridges robust deployment execution and recursive policy self-improvement. During deployment, the Inner Loop decouples high-level reasoning from low-level control through highly composable atomic skills. It employs Vision-Language models for receding-horizon planning and visual reflection, dynamically composing skills to ensure robust error recovery. These skills are executed by specialized flow-matching experts that share a unified VLM backbone, maximizing reusability while mitigating capacity interference. Concurrently, the Outer Loop drives automated lifelong learning by autonomously segmenting and verifying deployment rollouts, clustering them to discover atomic skills, and continuously fine-tuning the skill library without human annotations. Evaluations on RoboCasa, BEHAVIOR-1K, and real-world tasks demonstrate the effectiveness of MobiAgent. It outperforms π0.5-TA by 22.5 percentage points on BEHAVIOR-1K and enables robust recovery from execution failures. Through autonomous data recycling, success improves from 7.50% to 27.50% on RoboCasa and from 32.5% to 57.5% on Astribot S1.
Chenzhi Liu, Yue Zhang, Jiehong Lin +4
The University of Hong Kong · Astribot · Southern University of Science and Technology
We introduce OpenMobile-2, a near-frontier mobile agent with fully open training environments and recipes. We make three key advances: (1) Diverse environments with simulated commercial apps: We build MobileGym++, featuring 35 realistic, functionally rich commercial-style apps with cross-app workflows, while preserving full controllability for reset and verification. Together with newly configured apps in Android emulators, this yields a diverse playground spanning over 110 apps. (2) Open training data at scale: Building on this foundation, we curate nearly 12K mobile interaction trajectories for supervised fine-tuning and the largest open collection of verifiable RL training data for mobile agents, comprising over 2K executable tasks with automatic rewards. (3) Hybrid GUI and app-native tool use: We explore an experimental mobile-use setting where apps expose selected functionalities as app-native tools alongside their GUIs, allowing agents to interleave GUI actions and tool calls within a task. We implement this setting in MobileGym++ with over 300 carefully scoped tools across 50+ apps. Additionally, we introduce MobileGym++ Bench for complex, long-horizon commercial mobile scenarios, supporting both GUI-only and hybrid GUI-tool evaluation on a shared task suite. OpenMobile-2 performs competitively across established benchmarks, including AndroidWorld (79.9) and MobileWorld (50.4), while showing promising transfer to real-device mobile use, nearly doubling SPA-Bench performance from 31.9 to 59.6. All resources are available at https://os-copilot.github.io/OpenMobile2-Home/.
Chenyang Yan, Yingying Zhang, Kanzhi Cheng +21
Nanjing University · SenseTime · The University of Hong Kong +3