Secure-CUA: Controlling Untrusted Influence in Computer-Use Agents
Organizations: University of Wisconsin–Madison · Google · Google DeepMind
Abstract
Computer-use agents (CUAs) perform tasks across applications (such as desktops, mobile apps, and web browsers) by observing graphical interfaces and issuing commands such as clicks and keystrokes. These interfaces combine trusted controls and content with untrusted content needed for legitimate tasks. An adversary controlling this untrusted content can embed instructions or misleading visual cues to change the agent's intended action or redirect its commands to the wrong interface target. We formalize security requirements for both the agent's decisions and their execution through GUI commands. In an ideal execution model, we show that enforcing both requirements at each step protects execution traces. We instantiate this model in Secure-CUA, our system for secure CUA execution. Its key idea is to commit to an explicit per-action program, called an , before accessing untrusted content. Each transaction fixes its queries to untrusted content and the permitted uses of their responses. The system masks untrusted regions and evaluates each transaction to produce the next action, using an isolated query model to answer its queries. It then locates the intended interface target using the masked interface. Under the model's assumptions, Secure-CUA is secure by design, while generating a new transaction at each step helps maintain high task utility by adapting to changing interfaces. We evaluate Secure-CUA under benign conditions on 400 WebArena tasks using three frontier models across seeds, yielding execution traces. Secure-CUA achieves an average task success rate of , compared with for Vanilla-CUA and for CaMeL-CUA.
Figures & tables
| Symbol | Meaning | Symbol | Meaning |
|---|---|---|---|
| Computer-use agent. | Action generation. | ||
| Controller; environment; transducer. | Visual grounding. | ||
| User instruction; history. | Extracts endorsed values. | ||
| Controller/environment states. | Endorsed values. | ||
| Observation function and output. | Permitted action computation. | ||
| Trusted/untrusted components. | Intended application object. |
| Shopping | GitLab | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | Vanilla | CaMeL-CUA | Secure-CUA | Act / Grd / Qry | Vanilla | CaMeL-CUA | Secure-CUA | Act / Grd / Qry |
| Opus | 3.57 / 102 | 1.50 / 290 | 4.41 / 429 | 3.43 / 0.40 / 0.58 | 3.70 / 121 | 2.36 / 423 | 4.43 / 415 | 3.29 / 0.58 / 0.56 |
| Gemini | 3.88 / 219 | 1.63 / 467 | 3.42 / 483 | 2.64 / 0.42 / 0.36 | 4.41 / 109 | 1.96 / 534 | 5.12 / 647 | 3.42 / 1.01 / 0.69 |
| GPT | 3.12 / 97 | 1.34 / 287 | 3.29 / 314 | 2.59 / 0.33 / 0.37 | 2.35 / 127 | 1.82 / 374 | 4.45 / 365 | 3.23 / 0.82 / 0.40 |
| Postmill | Shopping Admin | |||||||
| Model | Vanilla | CaMeL-CUA | Secure-CUA | Act / Grd / Qry | Vanilla | CaMeL-CUA | Secure-CUA | Act / Grd / Qry |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Application | Entries | Examples of untrusted content |
|---|---|---|
| Postmill | 18 | Post titles, bodies, images, authors, and linked hosts; comments, forum descriptions, and user biographies. |
| Shopping | 24 | Product names, images, descriptions, specifications, reviews, and marketing content; product information repeated in carts, orders, and confirmation messages. |
| Shopping Admin | 123 | Customer and address fields, order records, product names and descriptions, images, reviews, search terms, and newsletter subscriber information. |
| GitLab | 66 | Project and user information, issues, merge requests, commit messages, labels, branch and file names, rendered documents, diffs, and editor contents. |
| Constructor | Retrieved-value argument |
|---|---|
| Click(target, anchor=None) | None. |
| Type(target, value, anchor=None) | value. |
| Press(target, key, anchor=None) | None. |
| Scroll(direction="down", amount=600) | None. |
| GoBack() | None. |
| Finish(status="success", answer="") | answer. |
| Action | Runtime behavior |
|---|---|
| Click | Click the validated coordinates. |
| Type | Focus the validated field, select its contents, and insert the bound text. |
| Press | Focus the validated control and issue the key command fixed in the transaction. |
| Scroll | Scroll in the direction and by the amount fixed in the transaction. |
| GoBack | Request browser back navigation. |
| Finish | Resolve the final answer and terminate execution. |
| Model | Metric | Shopping | GitLab | Postmill | Shopping Admin | All |
|---|---|---|---|---|---|---|
| Opus | Transactions (%) | |||||
| Queries/task | ||||||
| Structured (%) | ||||||
| String (%) | ||||||
| Gemini | Transactions (%) | |||||
| Queries/task |