OpenPhone: Mobile Agentic Foundation Models
Organizations: The University of Hong Kong
Abstract
With the advancement of multimodal large language models (MLLMs), building GUI agent systems has become an increasingly promising direction--especially for mobile platforms, given their rich app ecosystems and intuitive touch interactions. Yet mobile GUI agents face a critical dilemma: truly on-device models (4B or smaller) lack sufficient performance, while capable models (starting from 7B) are either too large for mobile deployment or prohibitively costly (e.g., cloud-only closed-source MLLMs). To resolve this, we propose OpenPhone, a mobile GUI agent system that leverages device-cloud collaboration to tap the cost-efficiency of on device models and the high capability of cloud models, while avoiding their drawbacks. Specifically, OpenPhone enhances Qwen2.5-VL-3B via two-stage SFT->GRPO training on synthetic GUI data for strong decision-making, integrates an efficient long-reasoning and memory management mechanism to utilize historical interactions under tight resources, and defaults to on-device execution--only escalating challenging subtasks to the cloud via real-time complexity assessment. Experiments on the online AndroidLab benchmark and diverse apps show OpenPhone matches or nears larger models, with a significant reduction in cloud costs.
Figures & tables
| Model | Agent Mode | Input | Size | SR |
|---|---|---|---|---|
| General Models | ||||
| Gemini-2.5-Pro | Reason | Screen | - | 56.5 |
| GLM-4.5-V | Reason | Screen | - | 49.3 |
| Claude-Sonnet-4 | Simple | Screen | - | 40.6 |
| Gemini-2.5-Flash | Reason | Screen | - | 36.2 |
| Qwen2.5-VL-32B | Reason | Screen | 32B | 31.2 |
| Variant | Agent Mode | SN | SR |
|---|---|---|---|
| OpenPhone w/o Tuning | Reason | 3 | 2.2 |
| OpenPhone w/o SFT | Reason | 12 | 8.7 |
| OpenPhone w/o GRPO | Reason | 10 | 7.2 |
| OpenPhone w/o Reasoning | Simple | 12 | 8.7 |
| OpenPhone w/o History | Reason | 6 | 4.3 |
| OpenPhone | Reason | 21 | 15.2 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Agent Mode | Input | Size | SR |
|---|---|---|---|---|
| Baseline Methods | ||||
| GLM-4.5-V | Reason | Screen | - | 60.0 |
| Gemini-2.5-Flash | Reason | Screen | - | 56.0 |
| GPT-5-mini | Reason | Screen | - | 40.0 |
| UI-Tars-7B | - | Screen | 7B | 32.0 |
| Claude-3.5-Haiku | Reason | Screen | - | 24.0 |
| App | Task ID | Task Description | Evaluation Type |
|---|---|---|---|
| Chrome Browser | chrome_1 | Find the address and founding date of the University of *** | query_detect |
| Chrome Browser | chrome_2 | Set to dark mode | operation |
| Chrome Browser | chrome_3 | Enter bookmarks and find the website you saved in Mobile Bookmarks | query_detect |
| TikTok | tiktok_1 | Go to the homepage of "IShowSpeed" | operation |
| TikTok | tiktok_2 | Go to the homepage of "IShowSpeed" and check whether you follow this creator | query_detect |
| TikTok | tiktok_3 | Search for videos about "iphone 17" | operation |
| Action | Parameters | Description |
|---|---|---|
| Tap | index:int | Tap the element with the given index number. |
| Type | input_str: str | Enter text into the currently focused input field. |
| Swipe | index: int, direction: str, dist: str | Swipe on the element with given index, direction and distance. |
| Long Press | index: int | Long press the element with the given index number. |
| Home() | none | Simulate the home button. |
| Wait | interval: int | Pauses execution for the specified number of seconds (default is 5 seconds). |