Effective computer-use agents (CUAs) should seamlessly combine visual GUI interaction with code and terminal execution within a single policy. We present Mage-VL-Cowork, a 4B hybrid CUA trained from a general multimodally aligned checkpoint through an end-to-end, progressive five-stage curriculum across midtraining, SFT, and RL. To power this curriculum at scale, Mage-CUA-Forge replaces per-task agent construction with reusable, skill-conditioned generation workflows and application-specific preparers, synthesizing 293,806 desktop and web tasks across 6,915 skills while reducing recurring per-task expenditure by 82.6%. Rather than directly distilling a default teacher, we evolve the teacher harness and instantiate specialized CUA harnesses to yield high-quality trajectories alongside stage-specific supervision. Mage-VL-Cowork-4B finally climbs to 66.0% on OSWorld-Verified through different training stages with more efficient reasoning cost, achieving state-of-the-art performance at its parameter size and pushing forward the frontier of on-device intelligence.
Figures & tables
Figure 1: OSWorld-Verified overall score versus model size. Mage-VL-Cowork-4B is compared with compact and larger open-source agents; frontier models without reported parameter counts are shown in the right strip. The dashed line marks human performance (72.4%).
Figure 2: Schematic overview of the data generation pipeline. (1) An authoring agent evolves the hybrid computer-use harness by proposing scaffold, tool-interface, prompt, and terminal-environment variants and selecting the harness H with the best success–cost trade-off on development tasks. (2) An authoring agent builds one reusable generation workflow W per skill. A shared specification coordinates source-file and setup-script branches, whose preparers and gates produce a task package of instruction, environment setup script, and rubric. Bottom: skills sampled from the taxonomy S are instantiated through W , the teacher agent with H executes the tasks in a sandbox through GUI and terminal actions, and rubric-judged, quality-filtered trajectories form D for SFT. Student performance feedback guides adaptive expansion and refinement of S across generation rounds.
Figure 3: Teacher–harness evolution across channel, scaffold, model, and prompt refinements. Left: task success and execution length for selected configurations. Right: the refinement dimension at each transition. Execution length is measured in model turns for α and β2 , and in tool calls for γ2 . Channel, scaffold, and prompt refinements modify H ; model transitions update MT .
Figure 4: Application coverage and task–file reference families. Application counts measure multi-label application–task incidences. The 17,542 InfiniteWeb [ 19 ] tasks are reported separately. File-family shares are computed over 877,608 references.
Stage
Trajectories
Tokens
Compute (ZFLOPs)
1. GUI-Oriented Midtraining
—
24.6B
1.18
2. GUI Agent SFT
156.7k
32.2B
1.55
3. Coding & Terminal SFT
563.7k
5.7B
0.27
4. Hybrid CUA SFT
318.8k
45.1B
2.17
5. Online Hybrid RL
—
—
2.62
Total
—
—
7.79
Table 1: Compute shares, training data, and nominal compute across the five stages. The pie chart (left) shows stage-wise compute shares, with colors corresponding to the stage rows. Tokens are reported in billions per epoch ( B=109 / epoch) of packed capacity, including padding and image tokens; midtraining and SFT use two epochs ( ZFLOPs=1021 FLOPs). Stage 1 packs multi-source examples and Stage 5 uses online RL rollouts, so their fixed trajectory or token entries are shown as dashes. Appendix B details the compute-estimation scope.
Figure 5: Rollout-as-a-Service architecture for asynchronous desktop RL. The resource manager coordinates task dispatch, sandbox scheduling, and compact episode storage. Runners handle environment interaction, harness execution, trajectory construction, and evaluation. The relay separates control, inference, and trajectory traffic; model weights synchronize within the GPU cluster. The diagram shows a 24-training/8-inference-GPU deployment.
Model
Steps
OS
Office
Daily
Professional
Workflow
Overall
Human
—
—
—
—
—
—
72.4
Frontier models
GPT-5.5
—
—
—
—
—
—
78.7
Claude Opus 4.7
—
—
—
—
—
—
78.0
GPT-5.4
—
—
—
—
—
—
75.0
Kimi-K2.6
—
—
—
—
—
—
73.1
Table 2: OSWorld-Verified results across model scales. Mage-VL-Cowork achieves 66.0% with 4B parameters under a 50-step budget. Baseline scores and step budgets are shown as reported. † Evaluated by us in one run on 361 tasks, excluding Google Drive. Blue shading marks the two highest overall scores in each model group.
Env Synth. Method
Tokens per task
Expenditure per task (USD)
Agent-first
1,276K
0.906
Mage-CUA-Forge
85K
0.158
Table 3: Recurring task-generation expenditure per task. Mage-CUA-Forge reuses workflows and application preparers; one-time workflow authoring and debugging are excluded from the per-task comparison.
Figure 6: Grounding progression during Stage 1 GUI-oriented midtraining on OSWorld-G (left) and ScreenSpot-v2 (right). Dots show individual checkpoint evaluations, solid curves show five-checkpoint moving averages, and horizontal lines mark the Qwen3-VL-4B and Qwen3.5-4B reference baselines.
Figure 7: OSWorld-Verified scores during reinforcement learning, using all 33 recorded evaluations. The final model scores 66.0%.
Figure 8: Stage-wise task performance, execution length, and token usage. Execution length counts model-call traces per task. Think/reasoning tokens are a subset of output tokens. Detailed mean, median, and P90 distributions are reported in Appendix B (Table 6 ).
Model
WebVoyager-Static (%)
Live reported (%)
Δ (pp)
Open-source models
Fara1.5-4B
68.0
80.8
−12.8
Qwen3-VL-8B-Thinking [ 34 ]
53.7
—
—
Qwen3.5-9B
53.3
—
—
Qwen3-VL-4B-Thinking [ 34 ]
49.6
—
—
OpenWebRL-4B [ 35 ]
47.1
74.1
−27.0
Table 4: WebVoyager-Static results across all 643 tasks under identical offline evaluation settings, alongside self-reported live WebVoyager scores where available [ 36 , 35 , 21 ] . Blue shading marks the best and second-best WebVoyager-Static success rates.
Teacher
Student
Metric
β1
β2
β1
β2
Success rate (%) ↑
92.0
89.6
37.6
25.1
Steps ↓
25.0
35.3
30.91
39.4
Table 5: Teacher–student comparison for two hybrid harness variants. All four agents are evaluated on the same 672 held-out multi-application tasks. Each student is trained on trajectories generated by the corresponding teacher variant. Higher success rates and fewer steps are preferred; bold marks the better student result for each metric.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Construction and reset of a web replica. Reference structure, assets, and records are combined with generated content to assemble the local site. Each episode operates on a working database restored from the read-only seed. The website and records are illustrative.
Figure 10: Structured task synthesis and contract compilation. The illustrative example links database predicates, a navigation path, and a compositional skill graph to a shared task specification. The instruction renderer receives only public query constraints; the verifier compiler uses the private target to check page visits and final database state.
Figure 11: Three design choices in Stage 1 GUI-oriented midtraining: (1) decoupling the click-only grounding prompt from the full agent schema to prevent action-space collapse, (2) deduplicating repeated target elements within packed single-screenshot sequences to eliminate in-context coordinate copying, and (3) selectively masking interior <think> tokens to balance reasoning format retention with grounding and tool-syntax learning.
Figure 12: Sliding-window context design and step-level rationale synthesis: (1) each turn conditions on folded imperative Action: summaries of steps 1…t−4 together with the three most recent visual turns ( t−3,t−2,t−1 ), (2) retaining three screenshots exposes whether prior actions altered the interface and prevents repetitive no-op click loops, and (3) a reasoning synthesis pass grounds first-person <think> traces and Action: summaries on before/after screenshots and the executed tool call.
Table 6: Execution and token statistics by training stage. Triples report mean, median, and P90; token-counting conventions are given in Section 6.5 .
Computer-use agents have advanced on benchmarks like OSWorld and AndroidWorld, but still act mostly through the GUI, often producing inefficient trajectories. Real-world computer work is hybrid, combining visual-state inspection with precise, high-throughput command-line operations, so capable agents must coordinate both modalities over shared application state. Yet scalable hybrid environments remain scarce because supporting both GUI and CLI over real applications typically requires substantial manual engineering for each application. Existing agents also struggle to use the two interfaces complementarily: CLI-native agents lack visual perception for tasks involving interface state or layout, while GUI-native agents are inefficient for operations better executed through commands. We introduce CUA-Universe, a scalable environment-to-data pipeline that turns real desktop software into hybrid GUI+CLI environments. App-Forge adapts applications into reproducible VMs and command-line surfaces it discovers, wraps, or generates, scaling to 16 applications; Task-Weave synthesizes diverse hybrid tasks of controllable difficulty from reusable operations over seed files; and Path-Steer steers rollouts along efficient hybrid paths and harvests verified trajectories for post-training. Training on this data shifts behavior from inefficient GUI interaction and brittle CLI scripting toward effective GUI+CLI orchestration. Our 9B model improves both success and efficiency on CUA-Verse (Score +39.3 pts; -37% steps, -60% tokens), OSWorld (SR +16.8 pts; -57% steps, -44% tokens), and OSWorld-MCP (Score +7.84 pts; -27% steps, -30% tokens). CUA-Universe provides a scalable path toward more capable and efficient computer-use agents.
Haoting Shi, Wenhao Wang, Weicheng Fang +6
Shanghai Jiao Tong University · Zhejiang University
Computer use agents (CUAs) have demonstrated strong capabilities in completing digital tasks. However, existing CUAs either rely solely on graphical user interface (GUI) interactions, which are often inefficient and error prone, or augment GUI interactions with application specific APIs or tools, which require substantial engineering effort and are difficult to scale across applications. We argue that the next generation of CUAs should combine GUI interactions with the command line interface (CLI), leveraging the generality of the GUI and the efficiency of shell commands. A critical challenge, however, is that current models do not know when or how to use the CLI during task execution. To address this challenge, we develop a data construction pipeline that produces three types of trajectories: GUI only, CLI only, and interleaved GUI and CLI trajectories. This pipeline results in HybridCUA-8K, containing 5K hybrid trajectories and 3K verified RLVR tasks. Building on these data, we propose a training framework with two stages: supervised fine tuning on the constructed trajectories, followed by reinforcement learning with our CLI aware rewards that encourages agents to use the CLI selectively and reliably. Experiments show that HybridCUA-9B achieves 53.6% accuracy on OSWorld, improving over the base model by 14.8 percentage points, and improves performance on WindowsAgentArena by 4.0 percentage points. These results demonstrate the effectiveness and cross platform generalizability of the hybrid GUI and CLI paradigm for computer use agents.
Tongbo Chen, Junbo Niu, Zhengxi Lu +8
Zhejiang University · Peking University · Tsinghua University
Native computer use offers a general interface for agents to operate almost any software available to people, but requires long-horizon state tracking, large-scale interactive experience, and learning from sparse yet verifiable outcomes. We introduce Qwen-CUA, a native computer-use agent with a 397B-A17B Qwen mixture-of-experts backbone. It observes only screenshots and acts through keyboard and mouse events, without DOM trees, accessibility metadata, or task-specific APIs. Its scaffold maintains up to 20 active screenshots and folds older visual history in fixed-size blocks to retain recent evidence while preserving reusable prompt prefixes. For training, we build a cloud rollout fleet with access to nearly 100,000 vCPUs and tens of thousands of concurrent environments, construct approximately 40,000 verifiable tasks, and collect personalized long-horizon workflows across everyday and professional software. We optimize complete trajectories with verifiable rewards and trajectory slicing, while iterative training runs refresh supervised data and recalibrate reinforcement-learning tasks. Across eight benchmarks, Qwen-CUA outperforms Qwen3.7 and remains competitive with leading proprietary systems, reaching 86.2 on OSWorld-Verified and 18.5/48.4 binary/partial completion on OSWorld 2.0. Scaling the same recipe to a model with over one trillion parameters yields Qwen-CUA-Max, improving these scores to 87.6 and 21.2/53.3. Qwen-CUA also reduces RedTeamCUA attack success from 36.6 to 16.4 relative to Qwen3.7. Efficiency analyses, a browser deployment, and Bash-augmented experiments further characterize practical behavior. These results establish native computer use as a broadly capable agent foundation and highlight scalable verifiable interaction and hybrid tool use as key directions.