Computer-Use Agents Gain Little Benefit from a World Model When Conditioned on Tutorials

Anh Nguyen; Tianyi Xu; Phat Tran; Harry Yu; Huazheng Wang; Stefan Lee

cs.LGNov 2, 2026

Computer-Use Agents Gain Little Benefit from a World Model When Conditioned on Tutorials

Authors: Anh Nguyen, Tianyi Xu, Phat Tran, Harry Yu, Huazheng Wang, Stefan Lee

Organizations: Oregon State University

Abstract

Recent work has improved computer-use agents (CUAs) by combining retrieved task-relevant tutorials and look-ahead planning using world models with MLLM judges. However, the contribution of each component -- tutorials, the judge, and the world model -- to overall improvement has not been evaluated in isolation. To fill this gap, we first construct OSWorld-Tutorial and ScienceBoard-Tutorial: two datasets for CUAs with curated multimodal tutorials, covering everyday and scientific domains. We then propose RAG-PolicyJudge (RAG-PJ), a policy-judge framework for CUAs conditioned on task-specific tutorials: the policy proposes candidate actions, and the judge selects among them. To isolate the effect of world models given tutorials, we compare RAG-PJ with and without common world models: using either zero-shot text-based descriptions of next states or an oracle that returns ground-truth visual observations. We find that RAG-PJ gains little benefit from these rollouts when tutorials are present, even with perfect visual futures. This suggests prior world-model gains in tutorial-conditioned methods come mainly from tutorial understanding and the judge's ability to select the proposed actions, rather than from predicted rollouts. Error analysis reveals that the bottleneck is not rollout fidelity, but the judge's ability to reason over future states and recognize task completion. This highlights the need to improve not just next-state prediction in world models, but also reasoning over their predicted rollouts. Datasets and code are available at https://github.com/realjoenguyen/RAGPJ-official.

Figures & tables

Figure 1: Success rate on OSWorld-Tutorial (Qwen3-VL-32B) of RAG-PJ, a policy–judge CUA, whose judge sees no rollout, a predicted text rollout (R-WoM Mei et al. (2026) ), or an oracle visual rollout (real future screenshots). Rollouts help RAG-PJ without a tutorial but add almost nothing with one. Full results in § 6 and Appendix C .

Figure 2: Overview of RAG-PJ. Given the instruction, current screenshot, and trajectory history, a tutorial-conditioned policy proposes candidate actions, a tutorial-conditioned judge selects one, and a grounding model maps it to keyboard/mouse inputs.

#	Method	Qwen3-VL 32B	Qwen3.8 27B
Policy only
1	Policy Only	14.8 ${\pm}$ 1.0	29.0 ${\pm}$ 0.6
2	+ RAG	36.1 ${\pm}$ 0.6	44.2 ${\pm}$ 1.4
Policy + Judge
3	Policy + Judge (GTA-1)	18.3 ${\pm}$ 2.4	30.4 ${\pm}$ 1.2
4	RAG-PJ (Ours)	40.2 ${\pm}$ 0.6	48.3 ${\pm}$ 0.7

Table 1: Success rate (%) on ScienceBoard-Tutorial. Mean

\pm

sample std. over 3 completed runs. “RAG” indicates tutorial use. Shaded cells (ground-truth rollouts, rows 11--13 ) are not significantly different from RAG-PJ ( row 4 ). 9 9 9 Hierarchical bootstrap, sign-flip permutation, and Wilcoxon signed-rank tests over tasks;

p

-values in Appendix B .

Figure 3: Error analysis of 2,116 failed trajectories with ground-truth visual rollouts in OSWorld-Tutorial. Given a failed task, we use Gemini 3.8 Flash to identify a primary failure reason and additional secondary reasons. We categorize the failure reasons into 9 categories and show their frequency. For a detailed breakdown, see Appendix R .

#	Method (no tutorial)	15 steps		50 steps
1	Policy + Judge (GTA-1)	43.8 ${\pm}$ 0.1		47.4 ${\pm}$ 1.3
+ Text rollout, WebDreamer ( Gu et al., 2025 )
2	$H{=}1$	41.4 ${\pm}$ 1.4	( $-2.4$ )	47.0 ${\pm}$ 2.0	( $-0.4$ )
3	$H{=}2$	40.1 ${\pm}$ 1.0	( $-3.7$ )	47.5 ${\pm}$ 1.1	( $+0.1$ )
+ Text rollout, R-WoM ( Mei et al., 2026 )
4	$H{=}1$	44.2 ${\pm}$ 1.2	( $+0.4$ )	48.2 ${\pm}$ 3.5	( $+0.8$ )

Table 2: Success rate (%) on OSWorld-Tutorial without tutorials (Qwen3-VL-32B policy and judge). Mean

\pm

sample std. over 3 completed runs. Parentheses: change (pp) over Policy + Judge ( row 1 ).

	Method	15 steps		50 steps
#		Human	MLLM	Human	MLLM
Policy only
1	Policy Only	60.2 ${\pm}$ 3.4		63.8 ${\pm}$ 1.4
2	+ RAG	66.6 ${\pm}$ 1.4	70.4 ${\pm}$ 1.5	68.7 ${\pm}$ 2.3	74.7 ${\pm}$ 1.9
Policy + Judge
3	Policy + Judge (GTA-1)	61.7 ${\pm}$ 0.9		69.6 ${\pm}$ 3.7

Table 3: Success rate (%) on 110 OSWorld-Tutorial tasks with human-verified (Human) vs. MLLM-generated (MLLM) tutorials (Qwen3-VL-32B policy/judge, 15/50 steps). Tutorial-free baselines shown once. Mean

\pm

sample std. over 3 completed runs.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

	Method	Qwen3-VL-8B		Qwen3-VL-32B		Qwen3.8-27B
#	Method	15 steps	50 steps	15 steps	50 steps	15 steps	50 steps
Policy only
1	Policy Only	47.2% ${\pm}$ 0.9%	55.9% ${\pm}$ 0.5%	60.2% ${\pm}$ 3.4%	63.8% ${\pm}$ 1.4%	68.9% ${\pm}$ 3.1%	75.3% ${\pm}$ 1.0%
2	+ RAG	59.3% ${\pm}$ 2.3%	61.5% ${\pm}$ 3.2%	66.6% ${\pm}$ 1.4%	68.7% ${\pm}$ 2.3%	73.2% ${\pm}$ 2.6%	76.2% ${\pm}$ 4.0%
Policy + Judge
3	Policy + Judge (GTA-1)	44.7% ${\pm}$ 3.7%	50.8% ${\pm}$ 5.7%	61.7% ${\pm}$ 0.9%	69.6% ${\pm}$ 3.7%	67.1% ${\pm}$ 3.1%	76.5% ${\pm}$ 1.9%

Appendix

Table 4: Success rate of RAG-PJ and other baselines on the 110 OSWorld-Tutorial tasks with human-verified online tutorials , over maximum runtimes of 15 and 50 steps. Results are mean

\pm

sample std. across 3 completed runs. “RAG” indicates tutorial use. Shaded cells are not significantly different from RAG-PJ (row 4) in the same column.

	Method	Qwen3-VL-8B		Qwen3-VL-32B		Qwen3.8-27B
#	Method	15 steps	50 steps	15 steps	50 steps	15 steps	50 steps
Policy only
1	Policy Only	20.9% ${\pm}$ 0.9%	25.1% ${\pm}$ 1.4%	34.7% ${\pm}$ 1.8%	36.6% ${\pm}$ 2.4%	51.7% ${\pm}$ 3.1%	61.0% ${\pm}$ 0.2%
2	+ RAG	32.4% ${\pm}$ 2.9%	38.6% ${\pm}$ 1.7%	39.5% ${\pm}$ 1.2%	44.3% ${\pm}$ 2.5%	54.5% ${\pm}$ 2.0%	64.3% ${\pm}$ 1.6%
Policy + Judge
3	Policy + Judge (GTA-1)	23.3% ${\pm}$ 1.6%	24.5% ${\pm}$ 1.4%	35.8% ${\pm}$ 0.3%	37.7% ${\pm}$ 1.5%	51.6% ${\pm}$ 0.6%	59.5% ${\pm}$ 0.3%

Appendix

Table 5: Success rate of RAG-PJ and other baselines on the 250 OSWorld-Tutorial tasks with AI-generated tutorials from tutorial_osworld over maximum runtimes of 15 and 50 steps. Results are mean

\pm

sample std. across 3 completed runs. “RAG” indicates tutorial use. Shaded cells are not significantly different from RAG-PJ (row 4) in the same column.

	Method	Qwen3-VL-32B
#	Method	Overall	OS	Office	Daily	Professional	Workflow
15 steps
1	Policy only baseline	42.5% ${\pm}$ 0.7%	65.3% ${\pm}$ 4.8%	33.5% ${\pm}$ 1.8%	63.5% ${\pm}$ 4.7%	60.5% ${\pm}$ 1.2%	21.0% ${\pm}$ 3.3%
2	Policy only + RAG policy	47.8% ${\pm}$ 0.7%	88.9% ${\pm}$ 2.4%	35.6% ${\pm}$ 3.5%	68.8% ${\pm}$ 3.4%	72.8% ${\pm}$ 1.2%	22.0% ${\pm}$ 2.1%
3	RAG policy + RAG judge (RAG-PJ)	51.7% ${\pm}$ 1.4%	91.7% ${\pm}$ 4.2%	41.5% ${\pm}$ 2.0%	66.6% ${\pm}$ 2.7%	83.7% ${\pm}$ 2.0%	25.0% ${\pm}$ 2.2%
4	GT rollout $H{=}1$	49.4% ${\pm}$ 0.2%	87.5% ${\pm}$ 7.2%	39.9% ${\pm}$ 0.9%	66.2% ${\pm}$ 2.7%	72.1% ${\pm}$ 5.9%	25.7% ${\pm}$ 2.1%

Appendix

Table 6: Success rate per category on 360 OSWorld-Tutorial tasks over 15 and 50 steps, using Qwen3-VL-32B as the policy and judge. Categories are OS, Office, Daily, Professional, and Workflow. Mean

\pm

sample std. across 3 completed runs.

#	Method	Celestia	ChimeraX	GrassGIS	KAlgebra	Lean	TeXstudio	Overall
		(33)	(29)	(34)	(31)	(26)	(16)	(169)
Policy only
1	Policy Only	20.2% ${\pm}$ 1.7%	26.4% ${\pm}$ 2.0%	8.8% ${\pm}$ 2.9%	18.3% ${\pm}$ 1.9%	0.0% ${\pm}$ 0.0%	12.5% ${\pm}$ 0.0%	14.8% ${\pm}$ 1.0%
2	+ RAG	31.3% ${\pm}$ 3.5%	74.7% ${\pm}$ 2.0%	22.5% ${\pm}$ 1.7%	62.4% ${\pm}$ 3.7%	0.0% ${\pm}$ 0.0%	12.5% ${\pm}$ 0.0%	36.1% ${\pm}$ 0.6%
Policy + Judge
3	Policy + Judge (GTA-1)	25.3% ${\pm}$ 1.7%	31.0% ${\pm}$ 6.0%	10.8% ${\pm}$ 4.5%	26.9% ${\pm}$ 10.4%	0.0% ${\pm}$ 0.0%	10.4% ${\pm}$ 3.6%	18.3% ${\pm}$ 2.4%

Appendix

Table 7: Per-application success rate on the 169 ScienceBoard tasks with the Qwen3-VL-32B policy (15 steps). Mean

\pm

sample std. over 3 completed runs; task counts in parentheses. “RAG” marks tutorial use.

#	Method	Celestia	ChimeraX	GrassGIS	KAlgebra	Lean	TeXstudio	Overall
		(33)	(29)	(34)	(31)	(26)	(16)	(169)
Policy only
1	Policy Only	19.2% ${\pm}$ 4.6%	56.3% ${\pm}$ 2.0%	27.5% ${\pm}$ 4.5%	46.2% ${\pm}$ 1.9%	2.6% ${\pm}$ 2.2%	12.5% ${\pm}$ 0.0%	29.0% ${\pm}$ 0.6%
2	+ RAG	34.3% ${\pm}$ 3.5%	79.3% ${\pm}$ 3.4%	29.4% ${\pm}$ 2.9%	79.6% ${\pm}$ 1.9%	14.1% ${\pm}$ 2.2%	12.5% ${\pm}$ 0.0%	44.2% ${\pm}$ 1.4%
Policy + Judge
3	Policy + Judge (GTA-1)	25.3% ${\pm}$ 1.7%	62.1% ${\pm}$ 3.4%	27.5% ${\pm}$ 6.1%	40.9% ${\pm}$ 4.9%	3.8% ${\pm}$ 3.8%	12.5% ${\pm}$ 0.0%	30.4% ${\pm}$ 1.2%

Appendix

Table 8: Per-application success rate on the 169 ScienceBoard tasks with the Qwen3.8-27B policy (15 steps). Mean

\pm

sample std. over 3 completed runs; task counts in parentheses. “RAG” marks tutorial use.

Panel of Figure 4 — Figure 4: Celestia task B-02 (5-step budget): the task instruction, the initial screenshot, and the retrieved tutorial. Method 1 omits the step that gives the 3D viewport keyboard focus, so after Neptune is clicked in the list, pressing G or double-clicking does not travel to Neptune.

Panel of Figure 5 — Figure 5: The three alternatives the tutorial gives for one step of “Go to Neptune” (Qwen3.8-27B), each executed by GT rollout on a copy of the VM after Neptune is selected in the list. RAG-PJ executed (a) for real.

Figure 6: Success rate with vs. without tutorials (Qwen3-VL-32B). Green arrows: gap from the judge-only reference (dashed) to the best rollout method. Without tutorials, rollout beats the judge by up to

4.2

%; with tutorials, the gap nearly vanishes. Numbers from Tables 2 and .

	Human-verified online	AI-generated
Correct	15	18
Partially correct	4	2
Wrong	0	0
Preferred by the reviewer	3	17

Appendix

Table 9: Verdicts on 20 random tutorial pairs. A tutorial is correct if following it as written completes the task.

Panel of Figure 7 — Figure 7: Impress task ac9bb6cb-1888-43ab-81e4-a98a547918cd . The online tutorial never enters Master View, where the slide number is set.

Panel of Figure 8 — Figure 8: VLC task 9195653c-f4aa-453d-aa95-787f6ccfaae9 . Only the online tutorial restarts VLC, which the new maximum needs.

Panel of Figure 9 — Figure 9: Writer task 88fe4b2d-3040-4c70-9a70-546a47764b48 . The online tutorial only sets double spacing. The AI-generated tutorial is correct, but its generator searched for the task’s files.

Panel of Figure 10 — Figure 10: Parallel warm ground-truth rollout with horizon $H$ . All candidate VMs restore the same pre-action checkpoint and generate real $H$ -step trajectories concurrently. The judge compares these trajectories, while only the first action of the selected branch is committed to the canonical main VM. Vertically stacked insets separate rollout depth from branch-level parallelism: the upper row shows State 1; each branch then re-invokes the same policy model, prompted for a single action, on the history, $o_{t}$ , and that branch’s own actions and observations so far to produce the next action, repeating this process through State $H$ ; this within-branch policy call occurs only when $H>1$ , marked by the amber region. Both the $K$ -action policy and JUDGE additionally condition on the interaction history $\mathcal{H}_{<t}$ alongside the current state. Warm-pool VM processes are refreshed from the newest checkpoint and reused rather than relaunched. Insets come from an actual rollout with $K=3$ and $H=2$ ; the construction supports arbitrary $K$ and $H$ . The dashed return path indicates process reuse after state refresh at the next decision step.

Panel of Figure 11 — Figure 11: Parallel fan-out of the current state $s_{t}$ . A single consistent checkpoint $C_{t}$ is captured once and then treated as an immutable source (navy region). Its machine state $M_{t}$ and its disk image $D_{t}$ , which carries the internal snapshot, are both copied privately into each of the $K$ branch packages; branches therefore share no mutable state. The dispatch band applies one warm-restore protocol independently to every branch $k$ ; the teal region draws that protocol once rather than repeating it $K$ times: the warm VM process is reused, its active disk is switched to the branch copy, the vmstate copy is attached and loaded via a QMP snapshot-load ( loadvm ), the temporary source is detached, and guest services recover into State 0, which is compared across branches by perceptual hash. A readiness barrier ensures candidate execution begins only after every one of the $K$ branches has been restored; if any branch fails, the pool is discarded and the $K$ VMs are cold-launched.

Panel of Figure 12 — Figure 12: A Chrome task example with a retrieved web tutorial. The tutorial provides updated step-by-step browser instructions with inline UI icons.

Program	Version
Ubuntu	22.04.3 LTS
Google Chrome	130.0.6723.58
GIMP	2.10.30
LibreOffice (Calc, Impress, Writer)	7.3.7.2
Mozilla Thunderbird	115.6.0
VLC	3.0.16

Appendix

Table 10: Software versions supplied to the tutorial-generation prompt for OSWorld tasks.

Program	Version
Ubuntu	22.04 LTS
UCSF ChimeraX	1.9 with the chimerax-states 0.5 bundle
GRASS GIS	8.4.1 (wxPython GUI)
Celestia	1.7-dev (Qt6 GUI)
KAlgebra	KDE Education, Qt 6 build
TeXstudio	Ubuntu 22.04 PPA build

Appendix

Table 11: Software versions supplied to the tutorial-generation prompt for ScienceBoard tasks.

Figure 14: Chrome task example with a retrieved web tutorial. The top panel shows the initial screenshot; the lower panel shows the retrieved markdown tutorial with inline images preserved. Task ID: 030eeff7-b492-4218-b312-701ec99ee0cc . Source: Google Chrome Help .

Panel of Figure 15 — Figure 15: GIMP task example with an MLLM-synthesized tutorial. The top panel shows the initial screenshot; the lower panel shows the synthesized tutorial for exporting the image as export.jpg on the desktop. Task ID: 77b8ab4d-994f-43ac-8930-8ca087d7c4b4 .

Panel of Figure 16 — Figure 16: Excluded task IDs for websites that block automated access through bot detection, CAPTCHAs, or access-denial systems.

Panel of Figure 17 — Figure 17: Celestia task example with an MLLM-generated tutorial. The top panel shows the initial screenshot; the lower panel shows the tutorial generated by Google Gemini 3.8 Flash. Task ID: Celestia/B-02 .

Panel of Figure 18 — Figure 18: ChimeraX task example with an MLLM-generated tutorial. The top panel shows the initial screenshot; the lower panel shows the tutorial generated by Google Gemini 3.8 Flash. The task instruction is quoted verbatim from ScienceBoard, including its typo. Task ID: ChimeraX/B-06 .

MLLM base	15 steps
MLLM base	w/ thought	w/o thought
Qwen3-VL-8B	28.5% $\pm$ 1.6%	28.1% $\pm$ 1.2%
Qwen3-VL-32B	39.0% $\pm$ 1.2%	38.8% $\pm$ 0.4%

Appendix

Table 12: Ablation of including intermediate thought tokens in the execution history. We use the setting vanilla policy without tutorial retrieval. Each result is mean and sample std. over 3 completed runs. We find that including intermediate thought tokens does not significantly improve performance.

Hyperparameter	Value
temperature	0.7
top_p	0.8
top_k	20
repetition_penalty	1.0
presence_penalty	1.5
max_output_length	16384

Appendix

Table 13: Default decoding hyperparameters used for the RAG-PJ policy.

Hyperparameter	Value
temperature	0
top_p	1

Appendix

Table 14: Default decoding hyperparameters used for the RAG-PJ judge.

Computer-Use Agents

Computer-use agents perceive and operate graphical interfaces and software tools to complete tasks across websites, apps, and desktops.

See all 171 papers

Oct 19, 2026

CUA-Lite: An Open Platform for Computer Use Agents

Open resources for computer use agents (CUAs) -- environments, datasets, and models -- remain scattered: without a standard, these resources cannot be integrated and scaled; additionally, evaluation, RL, and SFT cannot share one infrastructure. We present CUA-Lite, an open platform for developing CUAs, built around three standardized abstractions: (1) Unified environment interface, under which we integrate 12 benchmarks across desktop, browser, and mobile, plus lightweight VM-free sandboxes that replicate OSWorld's desktop at much lower cost, providing over 30,000 verifiable tasks. (2) Unified supervised data format, into which we convert the major public CUA datasets, from GUI grounding and understanding to full rollouts, and record our fresh rollouts from frontier CUAs on these verifiable environments. (3) Model harnesses shared across evaluation, RL, and SFT, one per model and implemented for 14 model families. Each harness runs its model in these computer environments, producing the rollouts that evaluation and RL consume, or renders stored supervised data into the model's own training format for SFT. We release reference scores across the integrated benchmarks, and show that RL and SFT through the same harness improve open-weight models. We release our dataset, code, and models, and the platform is open to contributions.

Zhanhui Zhou, Weichen Zhang, Haoran Liu +6

Computer-Use Agents Computer-Use Agent Benchmarks Computer-Use Agent Evaluation

Ask PDF

Jun 23, 2026

Qwen-AgentWorld: Language World Models for General Agents

A world model predicts environment dynamics based on current observations and actions, serving as a core cognitive mechanism for reasoning and planning. In this work, we investigate how world modeling based on language models can further push the boundaries of general agents. (i) We first focus on building foundation models for agentic environment simulation. We introduce Qwen-AgentWorld-35B-A3B and Qwen-AgentWorld-397B-A17B, the first language world models capable of simulating agentic environments covering 7 domains via long chain-of-thought reasoning. Leveraging more than 10M environment interaction trajectories of 7 domains in real-world environments, we develop Qwen-AgentWorld through a three-stage training pipeline: CPT injects general-purpose world modeling capabilities from the state transition dynamics and augmented professional corpora, SFT activates next-state-prediction reasoning, and RL sharpens simulation fidelity through a tailored framework with hybrid rubric-and-rule rewards. To evaluate language world models, we present AgentWorldBench, a comprehensive benchmark constructed from real-world interactions of 5 frontier models on 9 established benchmarks. Empirical results demonstrate that Qwen-AgentWorld significantly outperforms existing frontier models. (ii) Beyond foundation models, we further investigate two complementary paradigms through which world modeling enhances general agents. First, as a decoupled environment simulator, Qwen-AgentWorld supports scalable and controllable simulation of thousands of real-world environments for agentic RL, yielding gains that surpass real-environment training alone. Second, as a unified agent foundation model, world-model training acts as a highly effective warm-up that improves downstream performance across 7 agentic benchmarks. Code: https://github.com/QwenLM/Qwen-AgentWorld

Yuxin Zuo, Zikai Xiao, Li Sheng +30

Qwen Team

LLM World Models Agentic RL World Model Learning World Model Evaluation Large Language Model-Based Simulation

Ask PDF

Aug 30, 2026

How do World Models and Policies Compose in LLM Agents? A Joint Spectral and Behavioral Account

How do LLM agents come to both understand environments they act in and master tasks set within them? Through controlled experiments combining world-model training (next-state prediction) and policy training (reward maximization), we investigate this question. We dissect the resulting models through their additive parameter updates. Geometrically, we find effective world-model updates are low-rank and share an input-feature subspace with policy updates while writing to nearly orthogonal output directions, whether trained separately or sequentially. However, we find that, in projection interventions, the sequential update induces more robustness than separate policy RL when removing the world model's leading input directions, suggesting that it has learned alternative input pathways. Behaviorally, we find the sequentially trained agent explores a wider range of states and actions. Based on this, we ask: does policy training preserve world knowledge as well as it could? We probe this with training-free merging built on the geometrically motivated input basis plus an online world-model loss during policy RL, and show both improve over the untreated baseline. Our findings suggest world knowledge and task-directed ability can be learned in geometrically complementary forms, and that future post-training pipelines should consider how best to engineer the interface between them.

Ruize Xu, Xiao Yu, Yujin Tang +2

Dartmouth College · Columbia University

World Model Learning World Models Model Merging Policy Learning RL for LLM Agents

Ask PDF

Abstract

Computer-Use Agents

Explore similar work

CUA-Lite: An Open Platform for Computer Use Agents

Qwen-AgentWorld: Language World Models for General Agents

How do World Models and Policies Compose in LLM Agents? A Joint Spectral and Behavioral Account