Do Multi-Agents Dream of Electric Screens? Achieving 100% on AndroidWorld Through Task Decomposition
Organizations: minitap · Thoughtworks · Columbia University · New York University
Abstract
We present Minitap's mobile-use, a multi-agent system for Android UI control, and evaluate it on the 116 tasks of AndroidWorld, on Android emulators in Minitap's cloud device infrastructure, real Pixel 7 phones from our device farm and the stock emulator. On fresh, held-out task instances (three pre-registered seeds never used during development), the final system reaches a pass@3 of 100.0% (95% CI 96.8-100.0); pass^3 is 94.0% (95% CI 89.7-97.4) and the mean success rate 98.0% (95% CI 96.0-99.7). Minitap decomposes each task across five LLM agents (planning, subgoal tracking, state capture, decision, execution) and deterministic nodes, with mechanisms that read typed text back from the device, verify the goal on device evidence and detect repeated failures. A study of the initial system measures what each part contributes: removing only the decomposition changes success by -26.1 [-35.6, -16.7] points, the mechanisms together add +11.7 [+2.2, +21.1] points over a single agent, and the decision agent is the one role that needs a frontier model. We then improve reliability with general changes, a few of them triggered by the goal's wording. On MobileMiniWoB++, a second benchmark never used during development, the final system improves on the initial one by 7.1 points (McNemar p = 0.035; 95% CI -0.5 to +14.7). Minitap is open source: https://github.com/minitap-ai/mobile-use
Figures & tables
| Method | Type | Src. | SR (%) |
|---|---|---|---|
| Agentic Frameworks | |||
| Minitap (Ours) | Agent | R | 100.0 |
| FluizAI† | Agent | C | 100.0 |
| Kirk Engine† | Agent | C | 100.0 |
| MobileRun | Agent | C | 100.0 |
| AGI-0† | Agent | C | 100.0 a |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Configuration | SR, 62 tasks | SR, 92 tasks | $/task | Median s |
|---|---|---|---|---|
| Full system | 70.2 [58.9, 80.6] | 64.1 [54.9, 73.4] | 0.54 | 96 |
| w/o decomposition | 78.2 [67.7, 87.1] | 70.7 [61.4, 79.3] | 0.34 | 38 |
| Single-agent baseline | 50.0 [37.9, 62.1] | 46.7 [37.0, 56.5] | 0.20 | 26 |
| Config | Seed | Date | Solved | SR (%) | Infra re-queued |
|---|---|---|---|---|---|
| Full system | 30 | 2026-11-10 | 91/116 | 78.4 | 5 |
| Full system | 31 | 2026-11-10 | 102/116 | 87.9 | 5 |
| Single-agent baseline | 30 | 2026-11-10 | 51/116 | 44.0 | 3 |
| Single-agent baseline | 31 | 2026-11-10 | 51/116 | 44.0 | 3 |
| Task | seed 30 | seed 31 |
| AudioRecorderRecordAudioWithFileName | ✓ | |
| BrowserMultiply | ✓ | |
| ExpenseAddMultiple | ✓ | |
| ExpenseAddMultipleFromGallery | ✓ | |
| ExpenseDeleteDuplicates | ✓ | |
| ExpenseDeleteMultiple2 | ✓ |
| Task | seed 71 | seed 72 | seed 73 |
| MarkorCreateNoteAndSms | ✓ | ✓ | |
| NotesTodoItemCount | ✓ | ✓ | |
| RecipeDeleteMultipleRecipesWithNoise | ✓ | ✓ | |
| SimpleSmsSendReceivedAddress | ✓ | ✓ | |
| SportsTrackerActivitiesCountForWeek | ✓ | ✓ | |
| SportsTrackerLongestDistanceActivity | ✓ | ✓ |
| Model | Role(s) | Precision | Features used |
|---|---|---|---|
| google/gemini-3.1-pro-preview | Cortex; every role in All Frontier ( ) | proprietary API | tools, structured output |
| google/gemini-3-flash-preview | video analyzer | proprietary API | neither (plain completion) |
| meta-llama/llama-4-scout | Planner | fp8 | structured output |
| openai/gpt-oss-120b ∗ | Orchestrator | bf16 | structured output |
| meta-llama/llama-3.3-70b-instruct | Executor | fp16 | tools |
| meta-llama/llama-3.1-8b-instruct | Contextor | bf16 | structured output |
| Benchmark | Configuration | Cells | Attempts | Busy h | Instance h |
|---|---|---|---|---|---|
| Cloud-90 | Full system | 180 | 248 | 27.9 | 48.6 |
| Cloud-90 | Single-agent baseline | 180 | 194 | 22.0 | 37.8 |
| Cloud-90 | w/o decomposition (single agent + mechanisms) | 180 | 188 | 21.4 | 32.6 |
| Cloud-90 | - Loop detector | 180 | 198 | 24.2 | 26.0 |
| Cloud-90 | - Text comparison | 180 | 212 | 25.4 | 29.6 |
| Cloud-90 | - Goal verification | 180 | 204 | 23.3 | 25.6 |
| Model | $/M in | $/M out | Calls |
|---|---|---|---|
| google/gemini-3-flash-preview | 0.500 | 3.000 | 19 |
| google/gemini-3.1-pro-preview | 2.000 | 12.000 | 50234 |
| meta-llama/llama-3.3-70b-instruct | 0.710 | 0.710 | 30514 |
| meta-llama/llama-4-scout | 0.107 | 0.232 | 2245 |
| openai/gpt-5-nano | 0.050 | 0.400 | 3395 |
| openai/gpt-oss-120b | 0.050 | 0.250 | 8026 |
| Configuration | Cortex | Planner | Executor | Orch. | Context. | Hopper | Output. |
|---|---|---|---|---|---|---|---|
| All Frontier | |||||||
| Platform Default | Llama 4 Scout | Llama 3.3 70B a | GPT-OSS 120B | Llama 3.1 8B | GPT-5 Nano | GPT-5 Nano | |
| Degrade Cortex | Llama 4 Scout | Llama 3.3 70B a | GPT-OSS 120B | Llama 3.1 8B | GPT-5 Nano | GPT-5 Nano | |
| Degrade all non-Cortex |
| Configuration | SR (%) | SR | $/task | Calls |
|---|---|---|---|---|
| All Frontier | 78.9 [70.6, 86.7] | — | 1.26 | — |
| Platform Default | 79.4 [71.1, 87.2] | [ , ] | 1.32 | 26.0 |
| Degrade Cortex | 10.0 [4.4, 16.1] | [ , ] | 0.32 | 77.8 |
| Degrade all non-Cortex | 79.4 [71.1, 87.2] | [ , ] | 1.12 | 23.5 |
| Tasks | pass@3 | pass ˆ 3 | Mean SR | Never solved |
|---|---|---|---|---|
| AndroidWorld, 116 | 100.0 % (95 % CI 96.8–100.0) | 94.0 % (95 % CI 89.7–97.4) | 98.0 % (95 % CI 96.0–99.7) | none |
| Cloud emulators, 87 | 100.0 % (95 % CI 95.8–100.0) | 97.7 % (95 % CI 94.3–100.0) | 99.2 % (95 % CI 97.7–100.0) | none |
| Phones, 16 | 100.0 % (95 % CI 80.6–100.0) | 81.2 % (95 % CI 62.5–100.0) | 93.8 % (95 % CI 83.3–100.0) | none |
| Stock emulator, 13 | 100.0 % (95 % CI 77.2–100.0) | 84.6 % (95 % CI 61.5–100.0) | 94.9 % (95 % CI 84.6–100.0) | none |
| Fresh instances, 92 | 100.0 % (95 % CI 96.0–100.0) | 92.4 % (95 % CI 87.0–97.8) | 97.5 % (95 % CI 94.9–99.6) | none |
Mobile GUI Agents
Mobile GUI agents use visual and structured interface information to carry out tasks across smartphone apps.
See all 92 papers