CUA-Lite: An Open Platform for Computer Use Agents

Zhanhui Zhou; Weichen Zhang; Haoran Liu; Lingjie Chen; Tianneng Shi; Kevin Lin; Zhengyuan Yang; Lijuan Wang

cs.AIOct 19, 2026

CUA-Lite: An Open Platform for Computer Use Agents

Authors: Zhanhui Zhou, Weichen Zhang, Haoran Liu, Lingjie Chen, Tianneng Shi, Kevin Lin, Zhengyuan Yang, Lijuan Wang,

Abstract

Open resources for computer use agents (CUAs) -- environments, datasets, and models -- remain scattered: without a standard, these resources cannot be integrated and scaled; additionally, evaluation, RL, and SFT cannot share one infrastructure. We present CUA-Lite, an open platform for developing CUAs, built around three standardized abstractions: (1) Unified environment interface, under which we integrate 12 benchmarks across desktop, browser, and mobile, plus lightweight VM-free sandboxes that replicate OSWorld's desktop at much lower cost, providing over 30,000 verifiable tasks. (2) Unified supervised data format, into which we convert the major public CUA datasets, from GUI grounding and understanding to full rollouts, and record our fresh rollouts from frontier CUAs on these verifiable environments. (3) Model harnesses shared across evaluation, RL, and SFT, one per model and implemented for 14 model families. Each harness runs its model in these computer environments, producing the rollouts that evaluation and RL consume, or renders stored supervised data into the model's own training format for SFT. We release reference scores across the integrated benchmarks, and show that RL and SFT through the same harness improve open-weight models. We release our dataset, code, and models, and the platform is open to contributions.

Figures & tables

Figure 1: CUA-Lite overview. One environment interface, Lite.Gym, serves every integrated environment, regardless of its runtime. One supervised data format, Lite.Sample, stores all supervised data, converted from public datasets or recorded through rollouts. One harness per model connects it to both, so evaluation, RL, and SFT share the same code (§ 2.3 ).

Panel of Figure 2 — Figure 2: One step and one rollout through the environment interface. Three GUI actions in one computer call return one result per call_id , so action batching costs one round trip.

GUI actions
Platform	Tool and actions
Desktop, Browser	computer : click , mouse_move , mouse_down , mouse_up , drag , scroll , type , key , key_down , key_up , hold_key , wait , screenshot , cursor_position
Mobile	mobile : tap , long_press , swipe , drag , pinch , type , system_button
Grounding	point , bbox : return a coordinate or a box
Extra tools
All platforms	response , terminate : finish the task

Table 1: GUI actions and extra tools per platform. See Appendix B for details.

Benchmark	#Eval tasks	Benchmark	#Eval tasks
Grounding		Browser
ScreenSpot-Pro ( 2025 )	1,581	WebVoyager ( 2024 )	643
OSWorld-G ( 2025 )	564	WebArena ( 2024 )	812
Desktop		MiniWoB++ ( 2018 )	125
OSWorld ( 2024 )	369	Mobile
OSWorld 2.1 ( 2026 )	108	AndroidWorld ( 2025 )	116

Table 2: The benchmarks and training environments in CUA-Lite. Each Lite.* environment runs the tasks of its source (OSWorld ( Xie et al., 2024 ) , SCALECUA ( Lv et al., 2026 ) , CUA-Gym ( Wang et al., 2026 ) , or Gym-Anything ( Aggarwal et al., 2026 ) ) on our VM-free desktop sandboxes; every other environment runs on its own runtime.

Figure 3: Lite.OSWorld against OSWorld. Left: cost of one instance, median over

20

cold starts with both runtimes allocated

4

GB and

4

cores; memory (resident usage within that allocation) and CPU are cgroup counters over a

300

s window with no action sent (Appendix D ). Right: evaluation scores (%) of

13

models. The dashed line marks equal scores. See Table D.1 for per-model scores.

Figure 4: The stored Lite.Samples. Left: the

9

converted datasets, by platform (inner ring) and by task type within each platform (outer ring). Right: GPT-5.5’s rollouts with verified reward on our training environments, by platform; Appendix E.3 adds those of Qwen3.5-27B and Qwen3.8-27B.

Specialist CUAs
Model	ScreenSpot-Pro	OSWorld-G	OSWorld	OSWorld 2.1	WAA	WebVoyager	WebArena	MiniWoB++	AndroidWorld	AndroidLab	MobileWorld	MobileGym
	Grounding		Desktop			Browser			Mobile
UI-TARS-1.5-7B	49.8	60.6	27.5	0.5	16.7	46.2	9.1	18.4	19.8	36.2	11.1	13.3
EvoCUA-8B	43.5	57.3	35.6	0.7	16.6	5.8	10.8	56.0	–	–	–	–
MAI-UI-8B	62.2	70.0	–	–	–	–	–	–	61.2	49.3	29.9	25.8
UI-Venus-2-9B	71.9	78.2	50.5	1.4	41.3	72.9	30.7	59.2	62.1	47.8	59.0	34.4

Table 3: Reference evaluation scores (%) on all

12

benchmarks. Bold: best per benchmark; underline: best open-weight model. WAA: WindowsAgentArena. –: not applicable. Qwen models run with reasoning off and API models at medium effort. See Appendix F for details.

Figure 5: Effect of reasoning on score and rollout length. Top: the change in score. Bottom: the ratio of rollout lengths. For Qwen models, each bar compares reasoning on with reasoning off; for GPT models, medium (wide bar) or xhigh (narrow, darker bar) reasoning effort with none . Each platform averages its benchmarks. The Qwen3 and Qwen3.5 rows average Qwen3-VL (4B–32B) and Qwen3.5 (4B–27B), shown per model in Figure F.1 .

Without SFT
	Desktop	Browser	Mobile
	Lite.OSWorld	WebVoyager	MobileGym
w/o reasoning	22.6	4.4	14.6
w/ o reasoning	27.1	14.8	13.9
SFT on GPT-5.5 rollouts
w/o reasoning	37.2	48.7	21.4

Table 4: Evaluation scores (%) of Qwen3.5-4B before and after SFT on GPT-5.5 rollouts. The rows without SFT differ by whether Qwen3.5-4B’s own reasoning is on, the SFT rows by whether the GPT-5.5 rollouts used for SFT contain reasoning. The rows without SFT use the configuration of SFT (one screenshot) for fair comparison rather than the official configuration of Table 3 (up to four screenshots), so their scores differ from Table 3 .

Figure 6: SFT ablations of Qwen3.5-4B. Left: desktop evaluation scores (%) after SFT on each teacher’s rollouts, with and without reasoning. Right: evaluation scores (%) with the one or four most recent screenshots in context, without SFT and after SFT on GPT-5.5 rollouts, both with reasoning. Desktop screenshots are

1920\times 1080

on the left and

1280\times 720

on the right, as in Table G.1 .

Figure 7: GRPO from the SFTed Qwen3.5-4B. Top: training reward per GRPO step, smoothed over five GRPO steps. Bottom: held-out evaluation score. Both rows show the mean and standard error over five seeds. Mobile evaluates on a subset of Table 4 ’s held-out tasks, and desktop on OSWorld’s LibreOffice tasks.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Action	Arguments
computer (desktop, browser)
click , mouse_down , mouse_up	coordinate , button , clicks $\in\{1,2,3\}$
mouse_move	coordinate
drag	start_coordinate , coordinate , button
scroll	direction , amount , coordinate
type	text , press_enter

Appendix

Table B.1 : GUI actions and their arguments. Coordinates are normalized to

[0,1000]

(§ 2.1 ).

Tool	Arguments	Environments
All platforms
response	text	every non-grounding environment
terminate	status , reason	every non-grounding environment
Desktop
bash	command	Lite.OSWorld, Lite.ScaleCUA, Lite.CUAGym, Lite.CUAWorld
Browser

Appendix

Table B.2 : Extra tools per platform.

Figure D.1 : Four tasks, step by step, on both runtimes. Each block is one task, the OSWorld VM on the upper row and the Lite.OSWorld container on the lower. A row gives a rollout’s first four steps and its last, with $\cdots$ for the steps between and, under each panel, the action GPT-5.5 took from that screen.

Model	OSWorld	Lite.OSWorld	$\bm{\Delta}$
GPT-5.5	72.3	67.9	$-4.4$
Qwen3.5-27B	46.2	44.9	$-1.3$
EvoCUA-8B	42.8	37.8	$-5.0$
Qwen3.5-9B	38.1	37.1	$-1.0$
Qwen3-VL-32B	35.8	33.0	$-2.8$
Qwen3-VL-8B	34.2	29.6	$-4.6$

Appendix

Table D.1 : Evaluation scores (%) on OSWorld and Lite.OSWorld. Both runtimes run in one parity campaign with the same settings, separate from the reference run of Table 3 .

Application	Synthesized	Perturbed	Evaluation
	Training
Chrome	132	91	46
GIMP	73	42	26
LibreOffice Calc	278	106	47
LibreOffice Impress	287	108	47
LibreOffice Writer	249	43	23

Appendix

Table D.2 : Lite.OSWorld tasks per application.

Panel of Figure 17 — Figure E.1 : The Lite.Sample types.

Panel of Figure 18 — Figure E.2 : One Lite.Sample per task type.

Dataset	Platform	Grounding	Understanding	Rollout	Total
Aguvis ( 2025b )	Desktop	$260$	$0$	$0$	$260$
	Browser	$634{,}314$	$0$	$3{,}729$	$638{,}043$
	Mobile	$57{,}498$	$0$	$17{,}820$	$75{,}318$
ScaleCUA ( 2026a )	Browser	$69{,}246$	$32{,}445$	$80{,}800$	$182{,}491$
GUI-360 ( 2025 )	Desktop	$72{,}415$	$86{,}730$	$10{,}482$	$169{,}627$
GUIOdyssey ( 2025 )	Mobile	$0$	$125{,}197$	$8{,}330$	$133{,}527$

Appendix

Table E.1 : The

\bm{9}

converted datasets, in Lite.Samples by platform. Grounding sums the action , point , and bbox sub-types.

Training environment	Platform	GPT-5.5	Qwen3.5-27B	Qwen3.8-27B	Total
Lite.ScaleCUA	Desktop	$17{,}493$	$17{,}539$	$17{,}807$	$52{,}839$
Lite.CUAGym	Desktop, browser	$7{,}793$	–	$10{,}052$	$17{,}845$
Lite.OSWorld	Desktop	$2{,}374$	–	$2{,}410$	$4{,}784$
WebGym	Browser	$3{,}143$	$967$	–	$4{,}110$
MobileGym	Mobile	$1{,}279$	–	–	$1{,}279$
Lite.CUAWorld	Desktop	$728$	–	–	$728$

Appendix

Table E.2 : Rollouts with verified reward per teacher, in Lite.Samples. –: not recorded.

Model	Screenshots	Text history
Specialist CUAs
UI-TARS-1.5	5	full
EvoCUA	4	action summary
MAI-UI	3	full
UI-Venus-2	3	full
General VLMs

Appendix

Table F.1 : Evaluation settings of Table 3 . Top: per model, the most recent screenshots and the text history kept in context, the same on every platform. An action summary (e.g., ‘Previous Actions: ’) keeps earlier action descriptions in the context. When a fifth screenshot arrives, Qwen3.5 and Qwen3.8 drop the four they keep and start again from it, so they keep one to four. Bottom: per benchmark, the screenshot resolution.

Figure F.1 : Effect of reasoning on score and rollout length, per model. Figure 5 with Qwen3 (Qwen3-VL) and Qwen3.5 split into their sizes.

Model	ScreenSpot-Pro	OSWorld-G	OSWorld	OSWorld 2.1	WAA	WebVoyager	WebArena	MiniWoB++	AndroidWorld	AndroidLab	MobileWorld	MobileGym
	Grounding		Desktop			Browser			Mobile
Specialist CUAs
UI-TARS-1.5-7B	49.8	60.6	27.5	0.5	16.7	46.2	9.1	18.4	19.8	36.2	11.1	13.3
EvoCUA-8B	43.5	57.3	35.6	0.7	16.6	5.8	10.8	56.0	–	–	–	–
MAI-UI-8B	62.2	70.0	–	–	–	–	–	–	61.2	49.3	29.9	25.8
UI-Venus-2-9B	71.9	78.2	50.5	1.4	41.3	72.9	30.7	59.2	62.1	47.8	59.0	34.4

Appendix

Table F.2 : Reference evaluation scores (%) of the

15

models at each reasoning setting we evaluate. A row marked

\hookrightarrow

reruns the model above it at another reasoning setting: reasoning on for an open-weight model, and the named reasoning effort for an API model. Qwen3-VL’s reasoning rows run Qwen3-VL-Thinking, a separate checkpoint, instead of Qwen3-VL-Instruct. Qwen3.5 and Qwen3.8 switch reasoning within one checkpoint. WAA: WindowsAgentArena. –: not applicable.

\circ

: not run, because WebArena rollouts are slow. Rows marked (default), which Table 3 reports, run open-weight models with reasoning off and API models at medium effort.

Without SFT
		Desktop		Browser	Mobile
		Lite.OSWorld		WebVoyager	MobileGym
	Screenshots	$1920\times 1080$	$1280\times 720$	$1280\times 720$	$720\times 1600$
w/o reasoning	1	22.6	14.9	4.4	14.6
	4	–	25.0	8.3	15.2
w/ o reasoning	1	27.1	22.6	14.8	13.9

Appendix

Table G.1 : Evaluation scores (%) of Qwen3.5-4B in every setting of § 4.1 . The table extends Table 4 with the data of Figure 6 . Screenshots: the number kept in context. –: not run.

Computer-Use Agents

Computer-use agents perceive and operate graphical interfaces and software tools to complete tasks across websites, apps, and desktops.

See all 155 papers

ProCUA-SFT Technical Report
Jun 15, 2026Jaehun Jung, Ximing Lu, Brandon Cui +11Supervised Fine-Tuning Computer-Use Agents
1NVIDIA · University of Washington
OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models
Jul 30, 2026Qiushi Sun, Kanzhi Cheng, Yian Wang +20VLM Evaluation Computer-Use Agent Benchmarks
The University of Hong Kong · Nanjing University · National University of Singapore +4
SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL
Jul 13, 2026Bowen Lv, Xiao Liu, Yanyu Ren +7Computer-Use Agents Reinforcement Learning with Verifiable Rewards
Tsinghua University

Jun 15, 2026J/K move · Enter open · S save

ProCUA-SFT Technical Report

Jaehun Jung, Ximing Lu, Brandon Cui, Muhammad Khalifa+10

1NVIDIA · University of Washington

Supervised Fine-Tuning Computer-Use Agents LLM Agent Trajectory Synthesis LLM Agent Training

Open paper AskPDF arXiv

Training computer-use agents (CUAs) -- models that interact with graphical desktops through screenshots and keyboard/mouse actions -- requires large-scale, diverse trajectory data collected in full desktop environments. The largest public resource, AgentNet (22.5K human trajectories), leads to negative transfer when used for supervised fine-tuning (SFT): continuing training UI-TARS 7B on AgentNet causes OSWorld success rate to fall from 26.3% to 8-10%. We present ProCUA-SFT, a dataset of 3.1M step-level SFT samples distilled from 93K synthetic trajectories across 2,484 application combinations. The dataset is produced by a fully automated pipeline that (i) synthesizes grounded tasks on live desktops seeded with real-world content -- 912 spreadsheets from SpreadsheetBench, approximately 10K permissively-licensed presentations from Zenodo10K, and multi-application OSWorld configs -- and (ii) verifies each task's feasibility through binary precondition checking before rollout. A single VLM (Kimi-K2.5) serves as goal generator, precondition judge, and trajectory executor, eliminating planner-actor capability gaps. Each trajectory is expanded into step-prefix samples that exactly reproduce the context layout seen at inference time. Fine-tuning UI-TARS 7B on ProCUA-SFT for one epoch yields 45.0% on OSWorld -- an 18.7 percentage-point improvement over the base model and over 35% above AgentNet-trained counterparts. A subset of ProCUA was incorporated into the training data for the Nemotron 3 Nano Omni model, contributing to its computer-use capabilities.

Abstract

Computer-Use Agents

Explore similar work

ProCUA-SFT Technical Report

OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward Models

SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL