Overview

Computer-use agents study how AI systems can interpret screens or structured interface data and act through clicks, typing, gestures, and software tools. Common approaches combine vision-language models with action planning, feedback and verification, reusable skills or memory, and training through demonstrations or reinforcement learning in real or synthetic environments. Benchmarks increasingly test long-horizon, cross-application work and measure task outcomes alongside efficiency, reliability, and safety. Agents can handle many routine interactions, but complex workflows remain difficult, and performance depends strongly on the interface, task design, and safeguards against errors or misuse.

Latest papers

CardsList

171 papers25 papers in the last four weeks, level with the four weeks before. 0.2% of all new papers.

  1. TeleTune: Evolving Agent Skills From Offline Telemetry

    Oct 4, 2026Justin Chih-Yao Chen, Elias Stengel-Eskin, Yan Chen +6Computer-Use AgentsLLM Agent Skill Learning

  2. CUAWright: A Minimal Unified Interface for Digital Agents

    Oct 2, 2026Yadong Lu, Theodore Lee, Yifei Li +5Computer-Use AgentsLLM Agent Harnesses

  3. Securing Computer-Use Agents Against Branch Steering Attacks

    Oct 2, 2026Giulio Zingrillo, Hanna Foerster, Ilia Shumailov +2Computer-Use AgentsAI Agent Security

  4. DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents

    Oct 1, 2026A. Said Gurbuz, Ahmed Nassar, Sunghwan Hong +2Computer-Use AgentsGUI Grounding

  5. cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents

    Sep 30, 2026Pranjal Aggarwal, Lawrence Keunho Jang, Sean Welleck +3Computer-Use Agent BenchmarksBenchmark Design

  6. OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software

    Sep 30, 2026Dingyuan Dai, Heli Qi, Lei Liu +28Computer-Use Agent BenchmarksAI for Science

  7. Talk2Agent: Benchmarking Voice Interfaces for Text Agents

    Sep 30, 2026Terumi Chiba, Guangzhi Sun, Zheqi Yuan +1ASR EvaluationVoice Agent Evaluation

  8. HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents

    Sep 29, 2026Tongbo Chen, Junbo Niu, Zhengxi Lu +8Computer-Use AgentsGUI Agents

  9. Absorbed in Inertia: Activation Analysis for Computer-Use Agents

    Sep 29, 2026Giulio Segalini, Zhi Wen Soi, Jérémie Decouchant +1Computer-Use AgentsLLM Agent Reliability

  10. Neuro-Symbolic Computer Use: Learning Reusable Policies for Reliable and Efficient Execution

    Sep 29, 2026Hyewon Suh, Thanh Minh Nguyen, Chih-Lun Lee +5Computer-Use AgentsRuntime Enforcement for AI Agents

  11. CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering

    Sep 26, 2026Prince Zizhuang Wang, Chenhao Liang, Zelong Xu +6Computer-Use Agent BenchmarksSoftware Engineering

  12. CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs

    Sep 25, 2026Zhenhao Zhang, Zhaoyu Fan, Haohan Ying +5CAPTCHA SolvingComputer-Use Agents

  13. CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments

    Sep 23, 2026Yuxuan Li, Will Epperson, Wesley Deng +1Computer-Use AgentsAgent Harness Optimization

  14. Affora: A Design System for Agent-Friendly Interfaces

    Sep 16, 2026Jin GaoComputer-Use AgentsGUI Agents

  15. AgentHijack: Visual Patch Attacks on Multimodal Computer-Use Agents

    Sep 6, 2026Zhihao Liu, Hongyu Sun, Zhiyuan Fu +7Adversarial Patch AttacksVisual Prompt Injection Attacks

  16. CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents

    Sep 4, 2026Haoting Shi, Wenhao Wang, Weicheng Fang +6Computer-Use AgentsTool-Using Agents

  17. From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agents

    Sep 4, 2026Longtao Hu, Xiao Liang, Linchao ZhuComputer-Use AgentsLLM Agent Skill Learning

  18. OmegaUse-SOP: SOP Engineering for Professional Computer Use from Human Demonstrations

    Sep 2, 2026Yixiong Xiao, Lang An, Hucheng Yang +9Computer-Use AgentsGUI Automation

  19. Are We There Yet? Assessing Computer-Use Agents for Blind Users' Accessible Interaction with Desktop Applications

    Sep 1, 2026Satwik Ram Kodandaram, Monalika Padma Reddy, Xiaojun Bi +3Computer-Use AgentsHuman-AI Interaction

  20. SIR: Self-improving Red-teaming for Compute Use Agents

    Aug 31, 2026Chen Xiong, Zhiyuan He, Pin-Yu Chen +2Computer-Use AgentsIndirect Prompt Injection

  21. CURA: Certified Runtime Alarms for Computer-Use Agents

    Aug 28, 2026Divake Kumar, Sina Tayebati, Devashri Naik +5AI Agent MonitoringComputer-Use Agents

  22. ASIL: Replacing Screenshot-and-Click with Structured State and Semantic Actions

    Aug 27, 2026Rui Xie, Lu ChenComputer-Use AgentsGUI Agents

  23. Beyond Task Completion: Training Capable and Safe Computer-Use Agents

    Aug 27, 2026Zeyu Kang, Zhenyun Yin, Yang Zhang +5Computer-Use Agent SafetyComputer-Use Agents