Computer-use agents have advanced on benchmarks like OSWorld and AndroidWorld, but still act mostly through the GUI, often producing inefficient trajectories. Real-world computer work is hybrid, combining visual-state inspection with precise, high-throughput command-line operations, so capable agents must coordinate both modalities over shared application state. Yet scalable hybrid environments remain scarce because supporting both GUI and CLI over real applications typically requires substantial manual engineering for each application. Existing agents also struggle to use the two interfaces complementarily: CLI-native agents lack visual perception for tasks involving interface state or layout, while GUI-native agents are inefficient for operations better executed through commands. We introduce CUA-Universe, a scalable environment-to-data pipeline that turns real desktop software into hybrid GUI+CLI environments. App-Forge adapts applications into reproducible VMs and command-line surfaces it discovers, wraps, or generates, scaling to 16 applications; Task-Weave synthesizes diverse hybrid tasks of controllable difficulty from reusable operations over seed files; and Path-Steer steers rollouts along efficient hybrid paths and harvests verified trajectories for post-training. Training on this data shifts behavior from inefficient GUI interaction and brittle CLI scripting toward effective GUI+CLI orchestration. Our 9B model improves both success and efficiency on CUA-Verse (Score +39.3 pts; -37% steps, -60% tokens), OSWorld (SR +16.8 pts; -57% steps, -44% tokens), and OSWorld-MCP (Score +7.84 pts; -27% steps, -30% tokens). CUA-Universe provides a scalable path toward more capable and efficient computer-use agents.
Figures & tables
Figure 1: Overview of CUA-Universe. (1) Scale: 16 real desktop applications spanning creative & interactive, office & knowledge, media & playback, and technical & web domains, each exposed through both a GUI and application-specific CLI interfaces. (2) Closed-loop environment-to-data pipeline: App-Forge adapts an application into a reproducible VM and the command-line surface it discovers, wraps, or generates (installer agent → reproducible VM + metadata → tool construction); Task-Weave synthesizes diverse hybrid GUI+CLI tasks at scale over seed files (operation abstraction → task instantiation → agentic refinement); and Path-Steer rolls agents out onto efficient hybrid paths and harvests high-quality, directly reusable trajectories for post-training. (3) Impact: training on this data shifts interaction behavior from inefficient GUI interaction and brittle CLI scripting toward effective GUI+CLI orchestration, producing more capable and efficient agents with consistent improvements across diverse benchmarks, including CUA-Verse (Score +39.3 pts; −37% steps, −60% tokens), OSWorld (SR +16.8 pts; −57% steps, −44% tokens), and OSWorld-MCP (Score +7.84 pts; −27% steps, −30% tokens).
Figure 2: Training-data composition. Our pipeline scales to diverse applications and produces substantial training data, with 4,923 episodes and approximately 235K training steps across all 16 applications. The balanced coverage of CUA-Universe extensions and OSWorld applications further demonstrates the diversity and scalability of the generated data.
Model
Mode
CUA-Verse
OSWorld
Score
Steps
Token (K)
SR (%)
Steps ↓
Step Gain ↑
Token(K) ↓
Token Gain ↑
GUI
53.7
27.2
224.8
Kimi K2.5
GUI+CLI
0.522
21.1
275
54.5
28.9
1.15
277.3
1.09
GUI
55.7
28.4
188.0
Seed2.1 Pro
GUI+CLI
0.599
40.1
449
59.0
27.7
1.10
215.6
0.94
GUI
66.8
14.7
595.0
Table 1: Combined performance on CUA-Verse and OSWorld. CUA-Verse : 8 apps × 20 tasks; Score is the average VLM judge score, and Token is the average number of tokens per episode in thousands. OSWorld : 244-task controlled scope excluding os and multi-app tasks, comparing GUI and GUI+CLI execution. Token/task is reported in thousands. Step Gain and Token Gain are the mean per-task GUI-to-GUI+CLI cost ratios over jointly solved tasks. A gain of k× indicates that GUI+CLI uses k times fewer steps or tokens on the same successfully solved tasks.
Figure 3: Application-level performance on the eight CUA-Verse applications. Each cell is the score for one model–application pair; darker shading is better. The horizontal rule separates proprietary (top) from open-source (bottom) models. Ours surpasses its Qwen3.5-9B on all eight applications, and is strongest on audio/video apps (Audacity, OBS).
Figure 5
Group
Accept Rate ( ≥0.75 )
Mean score
Avg steps
Avg tokens
Avg cost
Kimi K2.5 w/ Path-Steer
0.51
0.71
22.75
331,988
$0.26
Kimi K2.5 w/o Path-Steer
0.44
0.63
26.68
385,107
$0.31
Seed2.1 Pro w/ Path-Steer
0.54
0.75
25.53
263,500
$0.29
Seed2.1 Pro w/o Path-Steer
0.45
0.67
28.78
303,653
$0.33
Table 3: Rollout efficiency of Path-Steer (enabled vs. disabled) , for the Kimi K2.5 data-generation backbone and Seed2.1 Pro (a cross-backbone check). Accept Rate is the fraction of the 320 tasks scoring ≥0.75 ; tokens are averaged per task.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Steps
CLI %
Score
Kimi K2.5
21.1
0 30.3
0.522
Seed2.1 Pro
40.1
0 49.2
0.599
GPT-5.5
23.1
100.0
0.768
Qwen3.5-9B
56.2
00 0.0
0.189
EvoCUA-8B
41.7
00 3.0
0.330
Ours
35.2
0 25.3
0.582
Appendix
Table 4: Action-modality behavior on CUA-Verse , averaged over the eight applications. Steps is the mean executed action count, CLI % is the CLI share of executed GUI + CLI actions, and Score is the average VLM-judge score. Greater CLI use often accompanies higher efficiency and performance, but CLI usage alone is insufficient: even GPT-5.5, which executes entirely through CLI in these trajectories, does not fully solve the benchmark, reflecting the visually grounded demands of CUA-Verse.
App
Base
8-app LoRA (OOD-only)
16-app LoRA (+in-domain)
Chrome
23.9
26.1
37.0
GIMP
26.9
30.8
42.3
Calc
8.5
10.6
18.8
Impress
27.7
25.8
31.8
Writer
34.8
39.1
52.2
Thunderbird
33.3
46.7
73.3
Appendix
Table 5: Out-of-domain training transfer on OSWorld under the controlled 244-task GUI+CLI scope. Values are success rates in percent. The 8-app OOD-only LoRA is trained on cua-universe extensions applications disjoint from the OSWorld applications, whereas the full 16-app LoRA is trained on both. Parenthesized values in the final row denote absolute percentage-point gains over the untuned Qwen3.5-9B.
Application
Domain
Seed
OSWorld applications
Chrome
Web browsing
Profile
GIMP
Image editing
.xcf
LibreOffice Calc
Spreadsheets
.ods
LibreOffice Impress
Presentations
.odp
LibreOffice Writer
Word processing
.odt
Appendix
Table 6: Desktop applications supported by CUA-Universe. Blue rows indicate applications inherited from OSWorld, whereas orange rows indicate applications added by CUA-Universe.
Figure 5: Agent-visible command count per application, grouped by source: the eight applications reused from OSWorld and the eight CUA-Universe extensions ( ∼ 404 commands in total).
Figure 6: Six opened seed examples used to ground task synthesis across diverse GUI applications. Each seed is a concrete project or file with metadata and verifiable app state, rather than a generic natural-language prompt.
Figure 7: VLC task trajectories w/ Path-Steer and w/o Path-Steer . The Path-Steer trajectory shows optimal solution using CLI tools and GUI actions, whereas the trajectory without sterr-path guidance becomes stuck during filter configuration and fails the task.
App
p
FP
FN
Prec. (%)
FNR (%)
Chrome
0.53
0
3
100.0
10.0
GIMP
0.52
1
3
96.7
10.0
LibreOffice Calc
0.56
0
1
100.0
3.3
LibreOffice Impress
0.51
0
2
100.0
6.7
LibreOffice Writer
0.53
0
0
100.0
0.0
Thunderbird
0.59
0
0
100.0
0.0
Appendix
Table 7: VLM judge vs. human labels, all 16 applications. Three annotators, blind to the judge score, labeled 30 judge-accepted and 30 judge-rejected trajectories per application. p is the true acceptance rate; FP counts accepted trajectories that were not a human “fully success” (out of 30, all of them partial successes—zero hard failures); FN counts rejected trajectories that were in fact fully successful (out of 30). Precision is on the acceptance set and FNR on the rejection set. Reweighting each application by p gives an overall agreement of 97.0% and Cohen’s κ=0.94 . Blue rows are OSWorld applications; orange rows are CUA-Universe extensions.
Split
Episodes
Step records
Total (GB)
CUA-Verse
2,526
140,088
172.97
OSWorld
2,397
95,320
113.67
Total
4,923
235,408
286.64
Appendix
Table 8: Final training splits. “Total (GB)” is the on-disk size including all screenshots; step records reference images by path.
Parameter
Value
Tuner type
LoRA
LoRA rank
8
LoRA α
32
LoRA dropout
0.05
LoRA bias
none
Target modules
all-linear
Appendix
Table 9: LoRA configuration.
Parameter
Value
Optimizer
ms-swift default
Learning rate
1×10−4
Warmup ratio
0.05
Per-device train batch size
2
Gradient accumulation
1
Effective batch size (8 GPUs)
16
Appendix
Table 10: Optimization and sequence settings.
Figure 8: An OSWorld VS Code task under GUI+CLI vs. GUI-only interfaces. GUI+CLI mode combines GUI navigation with direct settings-file editing: it locates the target setting, writes debug.focusEditorOnBreak: false to settings.json , and verifies persistence, completing the task in 5 CLI calls and 12 GUI actions (reward 1 ). GUI-only mode repeatedly searches and clicks through the settings UI but fails to commit the correct configuration, exhausting the 60-action budget (reward 0 ).
Figure 9: CUA-Verse task composition. Mean number of abstract tools per task for each application, split into CLI (orange) and GUI (blue) operations. Every application requires both modalities.