Do Multi-Agents Dream of Electric Screens? Achieving 100% on AndroidWorld Through Task Decomposition
Authors: Pierre-Louis Favreau, Jean-Pierre Lo, Clement Guiguet, Charles Simon-Meunier, Nicolas Dehandschoewercker, Allen G. Roush, Judah Goldfeder, Ravid Shwartz-Ziv
Organizations: minitap · Thoughtworks · Columbia University · New York University
We present Minitap's mobile-use, a multi-agent system for Android UI control, and evaluate it on the 116 tasks of AndroidWorld, on Android emulators in Minitap's cloud device infrastructure, real Pixel 7 phones from our device farm and the stock emulator. On fresh, held-out task instances (three pre-registered seeds never used during development), the final system reaches a pass@3 of 100.0% (95% CI 96.8-100.0); pass^3 is 94.0% (95% CI 89.7-97.4) and the mean success rate 98.0% (95% CI 96.0-99.7). Minitap decomposes each task across five LLM agents (planning, subgoal tracking, state capture, decision, execution) and deterministic nodes, with mechanisms that read typed text back from the device, verify the goal on device evidence and detect repeated failures. A study of the initial system measures what each part contributes: removing only the decomposition changes success by -26.1 [-35.6, -16.7] points, the mechanisms together add +11.7 [+2.2, +21.1] points over a single agent, and the decision agent is the one role that needs a frontier model. We then improve reliability with general changes, a few of them triggered by the goal's wording. On MobileMiniWoB++, a second benchmark never used during development, the final system improves on the initial one by 7.1 points (McNemar p = 0.035; 95% CI -0.5 to +14.7). Minitap is open source: https://github.com/minitap-ai/mobile-use
Figures & tables
Figure 1: Minitap’s execution graph (LangGraph nodes and edges of the released code), with the Cortex as the central decision agent. (1) Plan: the Planner decomposes the goal into ordered subgoals and the Orchestrator sets their statuses. (2) Act: the Contextor captures device state, the Cortex decides the next actions and/or marks subgoals for review, the Executor turns decisions into tool calls, and the ExecutorToolNode runs them in order on the device, aborting after a failure. The Summarizer trims the message history. Convergence waits for both branches, then routes from the subgoal statuses: continue, replan if a subgoal failed, or end when all are completed. Filled boxes call an LLM; dashed boxes are deterministic. Dashed arrows are taken only in some cycles (replan, subgoal review, no tool call).
Table 2
Method
Type
Src.
SR (%)
Agentic Frameworks
Minitap (Ours)
Agent
R
100.0
FluizAI†
Agent
C
100.0
Kirk Engine†
Agent
C
100.0
MobileRun
Agent
C
100.0
AGI-0†
Agent
C
100.0 a
Table 3: AndroidWorld results (116 tasks). SR (%): for other systems, the success rate as reported by each source; for ours, pass@3 over 3 held-out seeds (Section 7 ). Agent : multi-component frameworks; Model : end-to-end VLM (one model call per step). † closed-source. Src. : R = run by us under the protocol of Section 5.1 ; C = cited from the public AndroidWorld leaderboard [ 23 ] , snapshot of 2026-11-09 (the system’s own page or paper: Appendix C ). Cited rows are self-reported and not independently verified; every one that states a trial count, including all five at 100%, is a single trial. Rows shown: all agentic entries ≥ 90% and all end-to-end models ≥ 60% in the snapshot. a The 100 appears only in a leaderboard note, which also states that its models were trained on AndroidWorld tasks; AGI’s own report of 2025-10-08 gives 97.4 [ 2 ] . d Leaderboard only; no report found, and the Seed1.8 model card gives 70.7 for the model [ 8 ] . e Reported as MobileRL-9B [ 74 ] .
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Configuration
SR, 62 tasks
SR, 92 tasks
$/task
Median s
Full system
70.2 [58.9, 80.6]
64.1 [54.9, 73.4]
0.54
96
w/o decomposition
78.2 [67.7, 87.1]
70.7 [61.4, 79.3]
0.34
38
Single-agent baseline
50.0 [37.9, 62.1]
46.7 [37.0, 56.5]
0.20
26
Appendix
Table 4: MobileMiniWoB++ (cloud emulators, Android 14, 2 seeds per configuration). SR: mean with 95% CI (bootstrap over tasks and runs); cost and median wall-clock per task on all 92 tasks.
Config
Seed
Date
Solved
SR (%)
Infra re-queued
Full system
30
2026-11-10
91/116
78.4
5
Full system
31
2026-11-10
102/116
87.9
5
Single-agent baseline
30
2026-11-10
51/116
44.0
3
Single-agent baseline
31
2026-11-10
51/116
44.0
3
Appendix
Table 5: Runs per configuration and seed (solved tasks, success rate, infrastructure failures re-queued).
Task
seed 30
seed 31
AudioRecorderRecordAudioWithFileName
×
✓
BrowserMultiply
×
✓
ExpenseAddMultiple
×
✓
ExpenseAddMultipleFromGallery
×
✓
ExpenseDeleteDuplicates
×
✓
ExpenseDeleteMultiple2
×
✓
Appendix
Table 6: Initial system: tasks solved in exactly one of the two runs (full configuration, 116 tasks, seeds 30 and 31; ✓ solved, × failed).
Task
seed 71
seed 72
seed 73
MarkorCreateNoteAndSms
✓
×
✓
NotesTodoItemCount
×
✓
✓
RecipeDeleteMultipleRecipesWithNoise
✓
✓
×
SimpleSmsSendReceivedAddress
×
✓
✓
SportsTrackerActivitiesCountForWeek
✓
×
✓
SportsTrackerLongestDistanceActivity
✓
×
✓
Appendix
Table 7: Final system: tasks solved in some but not all of the 3 runs (116 tasks, held-out seeds 71, 72 and 73; ✓ solved, × failed).
Figure 2: Time to success of solved tasks, full configuration of the initial system (30-minute cap per task).
Figure 3: Success rate per application, full configuration of the initial system.
Model
Role(s)
Precision
Features used
google/gemini-3.1-pro-preview
Cortex; every role in All Frontier ( F )
proprietary API
tools, structured output
google/gemini-3-flash-preview
video analyzer
proprietary API
neither (plain completion)
meta-llama/llama-4-scout
Planner
fp8
structured output
openai/gpt-oss-120b ∗
Orchestrator
bf16
structured output
meta-llama/llama-3.3-70b-instruct
Executor
fp16
tools
meta-llama/llama-3.1-8b-instruct
Contextor
bf16
structured output
Appendix
Table 8: Serving conditions per model. All runs used strict, fixed serving conditions: each open-weight model was served at a fixed precision (bf16/fp16 unquantized, or fp8 where no unquantized endpoint supports the required features) by one pinned inference endpoint, with provider fallback disabled. The one exception is gpt-oss-120b ∗ , which had two endpoints with the same precision, ordered, and fallback restricted to them (1.0 % of its calls were served by the second endpoint). Proprietary models were called on their vendor’s API. Features used: what the model’s roles send (tool calling; structured output as a JSON schema). The endpoint that served each call is logged, and no call was served outside the allowed set.
Benchmark
Configuration
Cells
Attempts
Busy h
Instance h
Cloud-90
Full system
180
248
27.9
48.6
Cloud-90
Single-agent baseline
180
194
22.0
37.8
Cloud-90
w/o decomposition (single agent + mechanisms)
180
188
21.4
32.6
Cloud-90
- Loop detector
180
198
24.2
26.0
Cloud-90
- Text comparison
180
212
25.4
29.6
Cloud-90
- Goal verification
180
204
23.3
25.6
Appendix
Table 9: Cloud device time per configuration.
Model
$/M in
$/M out
Calls
google/gemini-3-flash-preview
0.500
3.000
19
google/gemini-3.1-pro-preview
2.000
12.000
50234
meta-llama/llama-3.3-70b-instruct
0.710
0.710
30514
meta-llama/llama-4-scout
0.107
0.232
2245
openai/gpt-5-nano
0.050
0.400
3395
openai/gpt-oss-120b
0.050
0.250
8026
Appendix
Table 10: Per-model API gateway prices recovered from the logged per-call costs, and number of calls.
Figure 4: Cost, latency and LLM calls per task by agent role, full configuration of the initial system.
Configuration
Cortex
Planner
Executor
Orch.
Context.
Hopper
Output.
All Frontier
F
F
F
F
F
F
F
Platform Default
F
Llama 4 Scout
Llama 3.3 70B a
GPT-OSS 120B
Llama 3.1 8B
GPT-5 Nano
GPT-5 Nano
Degrade Cortex
B
Llama 4 Scout
Llama 3.3 70B a
GPT-OSS 120B
Llama 3.1 8B
GPT-5 Nano
GPT-5 Nano
Degrade all non-Cortex
F
B
B
B
B
B
B
Appendix
Table 11: Model of each LLM role per routing configuration. Frontier model in bold . a Llama 3.3 70B replaces the original Llama 3.1 70B, whose only remaining endpoint has no tool calling.
Figure 5: Cost–success trade-off across the routing configurations of Table 2 (90 cloud-emulator tasks); error bars: 95% CI over runs. The Pareto frontier is drawn on means: it joins Degrade Cortex and Degrade all non-Cortex, while Platform Default and All Frontier lie off it (the success differences among the three upper points are not significant); budget configurations, Degrade Cortex: mean over 2 runs; Degrade all non-Cortex: mean over 2 runs.
Figure 6: Leave-one-out deltas versus the full system on the 90 cloud-emulator tasks, with 95% CIs (Table 2 ).
Configuration
SR (%)
Δ SR
$/task
Calls
All Frontier
78.9 [70.6, 86.7]
—
1.26
—
Platform Default
79.4 [71.1, 87.2]
+0.6 [ −6.1 , +7.2 ]
1.32
26.0
Degrade Cortex
10.0 [4.4, 16.1]
−68.9 [ −77.8 , −60.0 ]
0.32
77.8
Degrade all non-Cortex
79.4 [71.1, 87.2]
+0.6 [ −6.1 , +7.2 ]
1.12
23.5
Appendix
Table 12: Routing results on the AndroidWorld Cloud-90 subset (90 tasks), mean ± 95% CI over two runs; budget rows, Degrade Cortex: mean over 2 runs; Degrade all non-Cortex: mean over 2 runs. Δ SR relative to All Frontier. Calls: mean LLM calls per task of the downgraded role(s).
Tasks
pass@3
pass ˆ 3
Mean SR
Never solved
AndroidWorld, 116
100.0 % (95 % CI 96.8–100.0)
94.0 % (95 % CI 89.7–97.4)
98.0 % (95 % CI 96.0–99.7)
none
Cloud emulators, 87
100.0 % (95 % CI 95.8–100.0)
97.7 % (95 % CI 94.3–100.0)
99.2 % (95 % CI 97.7–100.0)
none
Phones, 16
100.0 % (95 % CI 80.6–100.0)
81.2 % (95 % CI 62.5–100.0)
93.8 % (95 % CI 83.3–100.0)
none
Stock emulator, 13
100.0 % (95 % CI 77.2–100.0)
84.6 % (95 % CI 61.5–100.0)
94.9 % (95 % CI 84.6–100.0)
none
Fresh instances, 92
100.0 % (95 % CI 96.0–100.0)
92.4 % (95 % CI 87.0–97.8)
97.5 % (95 % CI 94.9–99.6)
none
Appendix
Table 13: Final system on the held-out seeds 71, 72 and 73 (3 runs per task): pass@3, passˆ3 and mean success rate with 95% CI, and the number of tasks never solved.
Mobile GUI agents can carry out tasks across apps, yet most run on a separate computer that drives a phone over the Android Debug Bridge (ADB), out of reach of ordinary users and of the apps through which most people reach Web services. We present Coati, a source-available agent that users install as an Android app. Its runtime on the phone, the host, records what a model alone cannot see: the effect of each action at its target, result screens across apps, and the progress of repeated steps. VerifyLoop checks action outcomes and completion claims against host evidence and reports a completion it cannot audit as unverified. AppMemory builds per-app procedural memory from observations that the interface supports and reuses it on later visits. A routed fast executor lets a lightweight selection model take simple, confident steps and lets the host repeat loops of simple steps that the large model plans once. On all 116 AndroidWorld tasks, with Gemini 3.8 Flash as every agent's backbone, Coati succeeds on 86.5% of the tasks over three trials, ahead of the strongest open-source agent Artemis (77.6% with the same accessibility-only access, 76.5% even when allowed ADB) and far ahead of FinalRun and Midscene (about 59%), and VerifyLoop cuts false completion claims by about a third at comparable success. In everyday use, the fast executor cuts the median time of phone tasks by 39% at comparable success and runs three of four loop tasks 2.8-5.4x faster with 6-9x fewer model tokens, while AppMemory shortens a repeated deep route in an unfamiliar app from 18.5 steps to 6-7. On an Android 16 phone, routed execution completes all 27 benchmark runs with 76% fewer Reasoning calls on the critical path. Host evidence, not model self-assessment, makes an installable agent reliable and fast.
Mobile agents on personal smartphones can affect private data, payments, settings, and the foreground screen. We present MATE (Mobile Agent Trustworthy Execution), an execution-layer system that controls the boundary between agent action proposals and device effects without modifying the underlying model. MATE represents each UI primitive as a structured action record that combines the instruction, target app, screen context, primitive, and action fields. Runtime Guardrail (RG) maps this record to an allow, block, or user-approval decision; Predefined Task Routing (PTR) serves recurring requests through verified routes with post-execution screen checks; and a floating interface keeps execution in a compact overlay. On MobileSafetyBench, MATE raises high-risk blocking from 29.7% to 97.0% under GPT-4o. On AndroidWorld, it reduces LLM calls by up to 6x and raises recurring-use task success to 95.7%.
Minsuk Jang, Hee-Seon Kim, Hui-Cheol Yu +1
Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea
Real applications provide the training setting closest to phone-agent deployment, but are difficult to reset, scale safely, and verify programmatically. Static screenshots and interaction trajectories preserve realistic evidence but cannot generate new experience. We introduce PhoneWorld, a trace-grounded framework that converts such evidence into runnable, resettable, and verifiable Android environments. PhoneWorld induces a usage-weighted interaction skeleton from observed pages, transitions, and state-changing operations; translates it into a behavior-grounded app specification; realizes the specification through an autonomous build--inspect--repair loop; and synthesizes executable tasks with programmatic verifiers. The resulting suite spans 34 consumer-facing apps across 16 domains and supports an audited online benchmark, verified trajectory generation, and online RL through common reset and verification interfaces. Evaluations with diverse general and open-source GUI agents show that PhoneWorld supports reliable end-to-end online interaction and exposes capabilities complementary to AndroidWorld. Controlled SFT experiments further show that PhoneWorld trajectories complement AndroidWorld supervision, transfer across online and offline benchmarks, and become more effective as data volume and app coverage increase. Under a matched RL budget, combining PhoneWorld mock-app rollouts with real-app rollouts improves performance over real-app RL alone on both real-phone tasks and AndroidWorld. Together, these results demonstrate that trace-grounded executable abstraction can bridge realistic mobile behavior and scalable agent learning, turning limited real-app evidence into a growing supply of controllable and verifiable environments for training and evaluation.
Yuxuan Liu, Xin Lai, Junyi Li +21
Gaoling School of Artificial Intelligence, Renmin University of China · Tencent Hunyuan · The Chinese University of Hong Kong, Shenzhen +1