NorthCUA Technical Report: Turbocharging Computer-Use Agents through Experience at Scale
Organizations: Shanghai Qiji Zhifeng Co., Ltd.
Abstract
Scaling computer-use agents from individual actions to complex software work requires experience that connects decisions to complete outcomes. We present NorthCUA, an integrated system for turbocharging computer-use agents through experience at scale. Its Experience Flywheel supplies diverse executable tasks to the NorthCUA Harness, which sustains long interactions, records decisions, and checks task outcomes. The Flywheel uses these records to curate verified trajectories. For unsolved tasks, recursive self-improvement evolves reusable skills and preserves verified milestones so later attempts build on earlier progress. Staged Learning combines this experience with grounding data from the Flywheel to develop GUI competence through mid-training and long-horizon supervised fine-tuning. The policy then learns from fresh attempts in the same Harness through joint desktop-web reinforcement learning, closing the loop between experience production and policy improvement. The resulting Nano, Mini, and Pro family achieves state-of-the-art results among open-source models across GUI grounding, desktop execution, and Web interaction under the reported evaluation settings. For more responsive Web workflows, NorthCUA-Ultrafast pairs direct action selection with open-ended generation on pre-RL Nano, offering a measured quality-latency tradeoff. Together, these advances establish NorthCUA as a high-performing model family and a reusable foundation for agents that carry complex software work from intent to completion.
Figures & tables
| Interface | Desktop agent | Web agent |
| Interaction surface | Applications, windows, and system interfaces | Pages, tabs, and browser state |
| Policy observations | Desktop screenshots | Page screenshots and DOM-derived information |
| Environment-facing actions | Mouse-and-keyboard GUI operations | Navigation, element-reference or coordinate actions, keyboard input, forms, and tab operations |
| Execution context | Active application, foreground window, and input focus | Active tab, page state, and element references |
| Retained evidence | Screenshots, action results, and application artifacts | Screenshots, page snapshots, action results, and browser artifacts |
| Dimension | Assessment | Analysis and selection use |
| Task completion | Criterion fulfillment, overall verdict, and completion rating | Identify supported completion examples and unresolved requirements |
| Process quality | Suitability of actions, tool use, and avoidable mistakes | Distinguish reliable demonstrations from fragile paths to the same result |
| Efficiency | Repetition, detours, unnecessary steps, and recovery behavior | Compare successful paths and apply cohort-specific efficiency criteria |
| Safety and side effects | Sensitive operations, unsolicited changes, and observed risks | Review unsafe or undesired behavior separately from completion |
| Failure diagnosis | Failure mechanism, controllability, and supported divergence location | Analyze capability gaps and distinguish them from environment blockers |
| Judgment uncertainty | Confidence, missing information, and uncertain verdicts | Route unresolved cases for further checking |
| Stage | Learning focus | Data and acceptance | Reported scale |
| Broad GUI mid-training | Localization and broad software use | Grounding, judge-selected task trajectories, and auxiliary evaluator demonstrations | trajectories GUI steps |
| Verified long-horizon SFT | Sustained completion of dependent workflows | Complete trajectories selected for script acceptance, dependencies, and execution quality | trajectories GUI steps |
| Joint desktop–web agentic RL | Improvement through the policy’s own execution | Human-reviewed tasks with outcome variation, followed by fresh verifier-scored task groups | desktop tasks Web tasks |
| Source | Representative coverage |
| Desktop base tasks | Office operations, visual/media work, and workspace/information management |
| Desktop long workflows | Cross-source reasoning, conflicting-information resolution, and dependent work toward a shared deliverable |
| Web tasks | Information retrieval, form filling, human-verification flows, and frontend testing |
| Single-step grounding | Locating requested controls in desktop and web screenshots |
| Evaluator trajectories | Tool-assisted inspection and evidence-based assessment of task outcomes |
| Benchmark | Coverage | Metric (%) |
| Grounding | ||
| ScreenSpot-Pro ( Li et al., 2025b ) | High-resolution professional software and small controls | Localization accuracy |
| OSWorld-G ( Xie et al., 2025a ) | Element semantics, layout, precise targeting, and absent targets | Grounding accuracy |
| Desktop | ||
| OSWorld-Verified ( Xie et al., 2025b ) | Application, file, and cross-application workflows | Task success |
| OSWorld 2.0 ( Yuan et al., 2026 ) | Longer workflows with dependent requirements | Full completion, partial score |
| Model | Parameters | SSP | OSWorld-G |
| General-purpose models | |||
| Qwen3.5-35B-A3B † ( Qwen Team, 2026a ) | 35B / 3B | 64.5 | 63.6 |
| Qwen3.5-27B † ( Qwen Team, 2026a ) | 27B | 68.4 | 67.9 |
| Qwen 3.7 Plus † ( Qwen Team, 2026b ) | 397B / 17B | 68.9 | 78.2 |
| Qwen3.8-Max ∗ ( Qwen Team, 2026c ) | 2.4T / 95B | 84.5 | 71.6 |
| Seed 2.1 Pro † ( ByteDance Seed, 2026 ) | Undisclosed | 65.3 | 78.0 |
| Benchmark | Target subset | Items | NorthCUA-Nano | NorthCUA-Mini | NorthCUA-Pro |
| SSP | px | 88 | 53.4 | 85.2 | 92.1 |
| – px | 1,079 | 72.4 | 92.1 | 92.4 | |
| px | 414 | 78.3 | 94.7 | 95.9 | |
| OSWorld-G | Visible target | 510 | 84.0 | 88.6 | 91.7 |
| Absent target | 54 | 67.9 | 74.1 | 75.9 |
| Model | Parameters | Score |
| General-purpose models | ||
| Qwen3.5-9B ( Qwen Team, 2026g ) | 9B | 41.8 |
| Qwen3.5-35B-A3B ( Qwen Team, 2026e ) | 35B / 3B | 54.5 |
| Qwen3.5-397B-A17B ( Qwen Team, 2026f ) | 397B / 17B | 62.2 |
| Qwen3.8-Max ( Qwen Team, 2026c ) | 2.4T / 95B | 86.1 |
| Seed 2.1 Pro ( ByteDance Seed, 2026 ) | Undisclosed | 78.8 |
| Model | Parameters | Binary | Partial |
| Reference models, 108 tasks | |||
| GPT-5.5 ( OpenAI, 2026a ) | Undisclosed | 13.0 | 49.5 |
| GPT-5.6 Sol ( OpenAI, 2026b ) | Undisclosed | 27.3 | 62.7 |
| Claude Opus 4.8 ( Anthropic, 2026c ) | Undisclosed | 20.6 | 54.8 |
| Claude Opus 5 ( Anthropic, 2026f ) | Undisclosed | 31.4 | 68.3 |
| Claude Fable 5 ( Anthropic, 2026b ) | Undisclosed | 36.1 | 72.9 |
| Model | Parameters | Success |
| General-purpose models | ||
| Qwen3.7-Plus ( Qwen Team, 2026b ) | 397B / 17B | 55.3 |
| Qwen3.8-Max ( Qwen Team, 2026c ) | 2.4T / 95B | 66.8 |
| Claude Opus 4.8 ( Anthropic, 2026c ) | Undisclosed | 67.9 |
| Claude Fable 5 ( Anthropic, 2026b ) | Undisclosed | 71.3 |
| Gemini 3.1 Pro ( Google DeepMind, 2026a ) | Undisclosed | 64.3 |
| Model | Parameters | Precision | Recall | F1 |
| General-purpose models | ||||
| Nex-N2.5-mini ( Nex-AGI, 2026a ) | 35B | 54.7 | 51.4 | 48.6 |
| Nex-N2.5-Pro ( Nex-AGI, 2026b ) | 397B | 59.5 | 55.8 | 52.8 |
| Kimi-K3 ( Moonshot AI, 2026 ) | 2.8T / 104B | 67.3 | 48.2 | 52.4 |
| GPT-5.6 Sol ( OpenAI, 2026b ) | Undisclosed | 54.8 | 70.3 | 56.1 |
| GPT-6 Astra ( OpenAI, 2026d ) | Undisclosed | 55.1 | 74.6 | 58.2 |
| Model | Stage | OSWorld- Verified | OSWorld 2.0 | WebArena- Verified | |
| Success | Binary | Partial | Success | ||
| NorthCUA-Nano | Mid | 68.7 | 0.9 | 24.2 | 54.9 |
| SFT | 76.1 | 12.0 | 40.9 | 64.2 | |
| NorthCUA-Mini | Mid | 68.3 | 2.8 | 25.6 | 65.4 |
| SFT | 83.3 | 18.5 | 46.3 | 68.0 | |
| NorthCUA-Pro | Mid | 75.8 | 9.3 | 30.6 | 69.1 |
| Model | Stage | Desktop | Web | ||
| OSWorld- Verified | OSWorld 2.0 | WebArena- Verified | |||
| Success | Binary | Partial | Success | ||
| Our NorthCUA | |||||
| NorthCUA-Nano | Before RL | 76.1 | 12.0 | 40.9 | 64.2 |
| After RL | 78.9 | 13.0 | 45.6 | 67.2 | |
| NorthCUA-Mini | Before RL | 83.3 | 18.5 | 46.3 | 68.0 |
| Method | Task | Mean task | Decoding |
| success | time | bypass | |
| (%) | (s) | (%) | |
| Full generation | 64.2 | 616.8 | 0.0 |
| Ultrafast, Head-only | 59.2 | 288.8 | 40.2 |
| Ultrafast, Head + LoRA | 60.6 | 266.8 | 31.1 |
Appendix figures & tables30 assets
Supplementary material from the paper’s appendix.
Appendix
| Record | Contents and alignment rule |
| GUI target | Original image dimensions, target point or region, and any crop/resize transform. Predictions and labels refer to the same coordinate system. |
| Documentation match | Application and available version evidence, manual source, candidate illustrations and function descriptions, and the context supporting the selected match. |
| Query variants | Function-, name-, appearance-, or position-based queries linked to the same visible control. Variants remain grouped with their source target. |
| Candidate disposition | Accepted, rejected, or unresolved matches, keeping semantic ambiguity separate from geometric invalidity. |
| Source | Queries | Steps | Mean steps/query |
| Broad GUI mid-training | |||
| Desktop | |||
| Web | |||
| Grounding | |||
| Verified long-horizon SFT | |||
| Desktop | |||
| Field | NorthCUA-Nano | NorthCUA-Mini | NorthCUA-Pro |
| Backbone variant | Qwen3.5-9B | Qwen3.5- 35B-A3B | Qwen3.5- 397B-A17B |
| Language-model parameters | 9B, dense | 35B total 3B activated | 397B total 17B activated |
| Route | Target choice | Completion |
| CLICK | Element | Construct a click call directly. |
| HOVER | Element | Construct a hover call directly. |
| CHECK | Checkbox | Construct a check call directly. |
| UNCHECK | Checkbox | Construct an uncheck call directly. |
| FILL | Input field | Generate the original fill action and require its field to agree with the selected target. |
| NATIVE | None | Generate a complete offered tool call and its arguments. |
| Setting | Head-only | Head + LoRA |
| Trainable parameters | 4.214M (0.0448%) | 47.492M (0.5047%) |
| Choice head | Width 512, two blocks | Same |
| LoRA | None | Rank 16, scale 2, dropout 0 |
| Adapter scope | None | All 248 language-layer linear modules |
| Optimizer | AdamW | AdamW |
| Learning rate | , constant | Same |
| Policy | Validation joint | Test joint | Test action exact |
| Full generation | – | 85.0 | 64.6 |
| Head-only | 86.1 | 85.4 | 64.0 |
| Head + LoRA | 88.2 | – | – |
| Tool | Parameter schema | Operation or constraint |
|---|---|---|
| click | x, y: number button: string = left | Click at the specified point. |
| double_click | x, y: number button: string = left | Double click at the specified point. |
| move | x, y: number | Move the pointer without clicking. |
| drag | path: Point[] button: string = left | Follow an ordered path containing at least two points. |
| scroll | x?, y?: number delta_x : number = 0 delta_y : number = 0 | Scroll at the pointer or a supplied location. Positive vertical deltas scroll up, negative deltas down. |
| type | text: string target_kind : enum = auto | Enter literal text into the focused control. Target kind is auto , editable , or terminal . |
| Tool | Parameter schema | Operation or constraint |
|---|---|---|
| web_open | url: string | Navigate after URL-policy validation. |
| web_snapshot | {} | Read the current page structure. |
| web_get_url | {} | Read the active tab’s URL. |
| web_list_tabs | {} | List open browser tabs. |
| web_click_ref | ref: Ref | Click the referenced element. |
| web_hover_ref | ref: Ref | Hover over the referenced element. |
| Tool | Parameter schema | Purpose or scope |
|---|---|---|
| Recorded evidence | ||
| read_round_bundle | section?: string detail_level : string = standard | Read a compact bundle or a selected section, such as trajectory or final state. |
| read_step | session_step : integer detail_level : string = standard | Retrieve one recorded step. |
| read_trace | session_step : integer detail_level : string = standard | Alias of read_step . |
| query_artifacts | kind?: string | List evidence, optionally by kind. |
| query_runtime_events | query?: string | Search runtime event summaries. |
| Field | Type and allowed values |
|---|---|
| overall_verdict | Enum success , partial , failure , or uncertain . |
| outcome_label | Enum success , failure , or uncertain . |
| process_rating | Integer from 1 to 5 rating execution quality. |
| completion_rating | Integer from 1 to 5 rating task completion. |
| confidence_rating | Integer from 1 to 5 rating confidence in the assessment. |
| controllability | Enum controllable , uncontrollable , mixed , or unknown . |
| Responsibility | Supervised collection | Online RL |
| Acting policy | Configured demonstration policy | Current policy being optimized |
| Scheduling | Standalone batch service | Training-side group orchestration |
| Environment lifetime | Owned sandboxes released after required checks and capture | External orchestrator retains release authority |
| Learning output | Selected trajectories and role-specific demonstrations | Attempt outcomes and captured calls for task groups |
| Attempt evidence | Completion-demonstration disposition |
| Verified complete with quality checks | Eligible. Retain source and outcome evidence. |
| Supported infeasibility or uncontrollable blocker | Excluded. Retain separately if used. |
| Partial or valid failed outcome | Excluded. Retain as failure/diagnostic experience if declared. |
| Uncertain verdict or conflicting evidence | Unresolved. Assessment is required. |
| Required evaluation error or missing outcome evidence | Not admitted as a verified completion |
| SSP group | Examples | NorthCUA-Nano | NorthCUA-Mini | NorthCUA-Pro |
| Office | 230 | 82.6 | 93.0 | 93.0 |
| Creative | 341 | 71.3 | 90.6 | 91.5 |
| Operating systems | 196 | 68.9 | 93.9 | 93.9 |
| Development | 299 | 72.9 | 93.7 | 92.6 |
| Scientific | 254 | 71.7 | 92.1 | 94.5 |
| CAD | 261 | 70.5 | 92.0 | 95.0 |
| Target size | Target type | Items | NorthCUA-Nano | NorthCUA-Mini | NorthCUA-Pro |
| px | Text | 31 | 58.1 | 87.1 | 90.3 |
| Icon | 57 | 50.9 | 84.2 | 93.0 | |
| – px | Text | 661 | 84.6 | 94.1 | 94.6 |
| Icon | 418 | 53.1 | 89.0 | 89.0 | |
| px | Text | 285 | 85.3 | 97.2 | 97.2 |
| Icon | 129 | 62.8 | 89.2 | 93.0 |
| Model | SSP | OSWorld-G | |||
| Accuracy | Pass@3 | Mean SD | Best run | Pass@3 | |
| NorthCUA-Nano | 72.9 | 74.6 | 83.0 | 86.0 | |
| NorthCUA-Mini | 92.4 | 93.6 | 87.9 | 90.6 | |
| NorthCUA-Pro | 93.3 | 95.6 | 91.0 | 94.0 | |
| Model | 0/3 correct | 1/3 correct | 2/3 correct | 3/3 correct |
| NorthCUA-Nano | 79 | 20 | 20 | 445 |
| NorthCUA-Mini | 53 | 15 | 27 | 469 |
| NorthCUA-Pro | 34 | 23 | 18 | 489 |
| Reference region | Items | NorthCUA-Nano | NorthCUA-Mini | NorthCUA-Pro |
| Rectangle | 470 | 83.1 | 87.9 | 91.1 |
| Polygon | 40 | 95.0 | 97.5 | 98.3 |
| Absent target | 54 | 67.9 | 74.1 | 75.9 |
| Task group | Tasks | NorthCUA-Nano passes | NorthCUA-Mini passes | NorthCUA-Pro passes |
| Website scope | ||||
| GitLab only | 180 | 115 | 126 | 131 |
| Map only | 109 | 67 | 71 | 77 |
| Forum only | 106 | 87 | 85 | 89 |
| Shopping only | 187 | 123 | 123 | 135 |
| Store administration only | 182 | 122 | 135 | 151 |
| Model | Outcome | Traces | Mean | Median | Q25–Q75 |
| NorthCUA-Nano | Passed | 521 | 13.4 | 9 | 7–16 |
| Failed | 290 | 33.9 | 19 | 10–44 | |
| NorthCUA-Mini | Passed | 552 | 13.6 | 10 | 7–16 |
| Failed | 260 | 29.6 | 18 | 10–36.3 | |
| NorthCUA-Pro | Passed | 600 | 15.0 | 10 | 7–17.3 |
| Failed | 212 | 26.5 | 15.5 | 10–33.3 |
| Reference | Comparison setting |
| Qwen-UI-Agent ( Zhou et al., 2026b ) | SSP uses the no-zoom row. OSWorld-G uses refined instructions. Desktop evaluation allows CLI and batched actions, with 361 OSWorld-Verified tasks. The source table labels 79.5 as success rate, while its text describes partial progress. |
| Qwen-CUA / Qwen-CUA-Max ( Lu et al., 2026 ) | The reported 86.2 is earned credit on 360 tasks with a 100-step limit. Max reports 87.6 under the same protocol. Their native interface uses screenshots and keyboard/mouse actions. Max discloses over 1T total parameters without an activated count. |
| MAI-Gym-9B ( Tong et al., 2026 ) | GUI and CLI actions. The paper lists 361 OSWorld-Verified tasks with 100 steps and 108 OSWorld 2.0 tasks with 500 steps. It evaluates three trials per task, without a complete aggregation rule in the main table. |
| ScaleCUA-Qwen3.5-9B ( Lv et al., 2026 ) | Four-run mean with a 50-step budget. The official reproduction setup uses the 361-task non-Google-Drive list. The original paper calls the benchmark OSWorld. |
| Nex-N2.5-mini / Pro ( Nex-AGI, 2026a ; Nex-AGI, 2026b ) | These general-purpose models have 35B and 397B total parameters, respectively. Provider-reported computer/browser scores use the NorthCUA harness family. The release reports OSWorld 2.0 partial credit of 30.5/56.4 and WebArena-Verified success of 63.4/67.6 for mini/Pro. Corresponding binary scores are unavailable. |
| MiniMax M3 ( MiniMax, 2026b ) | Benchmark-author OSWorld 2.0 evaluation on 108 tasks with 500 steps, single-action calls, and thinking enabled ( Yuan et al., 2026 ) . |
Computer-Use Agents
Computer-use agents perceive and operate graphical interfaces and software tools to complete tasks across websites, apps, and desktops.
See all 171 papers