Computer-Use Agents

Ask about this topic

Overview

Computer-use agents study how AI systems can interpret screens or structured interface data and act through clicks, typing, gestures, and software tools. Common approaches combine vision-language models with action planning, feedback and verification, reusable skills or memory, and training through demonstrations or reinforcement learning in real or synthetic environments. Benchmarks increasingly test long-horizon, cross-application work and measure task outcomes alongside efficiency, reliability, and safety. Agents can handle many routine interactions, but complex workflows remain difficult, and performance depends strongly on the interface, task design, and safeguards against errors or misuse.

Latest papers

CardsList

160 papers31 papers in the last four weeks, up 158% on the four weeks before. 0.2% of all new papers.

  1. IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems

    Oct 22, 2026A. Jian Yang, An Huang, Bo Dan +174LLM Agent TrainingAI Coding Agents

  2. Evolving Agent Interfaces

    Oct 20, 2026Saaket Agashe, Yuzhe Yang, Gonzalo Gonzalez-Pumariega +5Computer-Use AgentsSelf-Evolving Agents

    University of California, Santa Barbara · Cornell University · Language Technologies Institute, Carnegie Mellon University

  3. Lost in Recovery: Demystifying Recovery Behavior of Computer-Use Agents via Goal-Oriented Action Replay

    Oct 20, 2026Byeonghun Bang, Sangwoo Park, Jinheon Baek +3Computer-Use AgentsComputer-Use Agent Evaluation

    KAIST · ETRI

  4. CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents

    Oct 20, 2026Xu Hu, Haoming Li, Yuxiang Wang +11Computer-Use AgentsComputer-Use Agent Benchmarks

    University of Texas at Dallas · Independent Researcher · University of Manchester +3

  5. Agent World Model in the Wild: Computer-Use Agent Environment Synthesis

    Oct 19, 2026Zhaoyang Wang, Qianhui Wu, Haorui Wang +9Computer-Use AgentsSynthetic Data Generation for LLM Agents

    UNC-Chapel Hill · Microsoft Research · Georgia Tech +2

  6. CUA-Lite: An Open Platform for Computer Use Agents

    Oct 19, 2026Zhanhui Zhou, Weichen Zhang, Haoran Liu +6Computer-Use AgentsComputer-Use Agent Benchmarks

  7. Adaptation of Computer Use Agents to New Software with User Manuals

    Oct 16, 2026Chulhoon Yoon, Kyungri Park, Jinyeong Yun +3Computer-Use AgentsLLM Agent Training

    Korea University · Hanyang University

  8. PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse Personas

    Oct 16, 2026Haoming Li, Defu Cao, Xu Hu +23Computer-Use Agent EvaluationComputer-Use Agent Benchmarks

    The University of Texas at Dallas · University of Southern California · Carnegie Mellon University +14

  9. LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agents

    Oct 15, 2026Qingni Wang, Changdae Oh, Yuzhe Yang +4Computer-Use AgentsAI Agent Monitoring

    Simular · University of Wisconsin–Madison

  10. OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agents

    Oct 15, 2026Bowen Yang, Kaiming Jin, Zichen Ding +8Computer-Use AgentsGUI Automation

    University of Science and Technology of China · Shanghai AI Laboratory · National University of Singapore +5

  11. OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use

    Oct 15, 2026Hanzhang Zhou, Long Li, Quyu Kong +12Computer-Use AgentsLLM Agent Harnesses

    Alibaba Token Hub, Alibaba Group

  12. Probe-and-Resume: When and How to Evaluate Agent Execution

    Oct 15, 2026Jiakai Xu, Tiancheng Ge, Tujie Guan +4Computer-Use AgentsInference-Time Search

    Columbia University

  13. CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline

    Oct 13, 2026Yuxiang Lai, Jike Zhong, Chengzhi Liu +6Computer-Use AgentsAI Agent Benchmarks

    Google · Emory University · University of Southern California +4

  14. Interaction-as-a-Judge

    Oct 13, 2026Wayne Chi, I-Hung Hsu, Li Chen +6AI Agent EvaluationAI Coding Agents

    Carnegie Mellon University · Arena · University of California, Berkeley +1

  15. A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software Evolution

    Oct 11, 2026Alan Wu, Wei Tao, Yiwen GuoComputer-Use AgentsComputer-Use Agent Benchmarks

    Independent Researcher · LIGHTSPEED

  16. Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill Induction

    Oct 9, 2026Yuqi Huang, Hui Li, Yifei Cao +9GUI AgentsAgent Skill Learning

    Shanghai Jiao Tong University · Fudan University · Nanyang Technological University +2

  17. SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software Environments

    Oct 8, 2026Tian Jin, Wenhao Wang, Jing Kang +4Computer-Use AgentsLLM Agent Training

    Shanghai Jiao Tong University · Zhejiang University

  18. Secure-CUA: Controlling Untrusted Influence in Computer-Use Agents

    Oct 7, 2026Sarthak Choudhary, Mihai Christodorescu, Ashish Hooda +3Computer-Use AgentsAI Agent Security

    University of Wisconsin–Madison · Google · Google DeepMind

  19. nanoMuse: An Open-Source Personal Agent for Every Device You Own

    Oct 6, 2026Guangyi Liu, Yong Liu, Jiangning ZhangPersonal AI AgentsComputer-Use Agents

    Zhejiang University

  20. TeleTune: Evolving Agent Skills From Offline Telemetry

    Oct 4, 2026Justin Chih-Yao Chen, Elias Stengel-Eskin, Yan Chen +6Computer-Use AgentsLLM Agent Skill Learning

    UNC Chapel Hill · Microsoft

  21. CUAWright: A Minimal Unified Interface for Digital Agents

    Oct 2, 2026Yadong Lu, Theodore Lee, Yifei Li +5Computer-Use AgentsLLM Agent Harnesses

    Microsoft Research · National University of Singapore · The Ohio State University +1

  22. Securing Computer-Use Agents Against Branch Steering Attacks

    Oct 2, 2026Giulio Zingrillo, Hanna Foerster, Ilia Shumailov +2Computer-Use AgentsAI Agent Security

    ETH Zurich · University of Cambridge · AI Sequrity Company +1

  23. When to Compile a Computer-Use Agent? Measuring Payback and Making Compilation Decisions for Token Efficiency

    Oct 2, 2026Yulong Ming, Jie Xu, Zihan Wu +1Computer-Use AgentsLLM Agent Workflow Optimization

    City University of Hong Kong

  24. DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents

    Oct 1, 2026A. Said Gurbuz, Ahmed Nassar, Sunghwan Hong +2Computer-Use AgentsGUI Grounding

    ETH Zurich · IBM Research Zurich · Microsoft

  25. cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents

    Sep 30, 2026Pranjal Aggarwal, Lawrence Keunho Jang, Sean Welleck +3Computer-Use Agent BenchmarksBenchmark Design

    Carnegie Mellon University

  26. ComputerSD: Online Self-Distillation from Real-Time Feedback for Computer-Use Agents

    Sep 30, 2026Yong Du, Tongbo Chen, Zhengxi Lu +5On-Policy Self-DistillationComputer-Use Agents

    Zhejiang University · Ant Group

  27. OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software

    Sep 30, 2026Dingyuan Dai, Heli Qi, Lei Liu +28Computer-Use Agent BenchmarksAI for Science

    University of California, Los Angeles · RIKEN AIP · Yale University +12

  28. Talk2Agent: Benchmarking Voice Interfaces for Text Agents

    Sep 30, 2026Terumi Chiba, Guangzhi Sun, Zheqi Yuan +1ASR EvaluationVoice Agent Evaluation

    Department of Electronic Engineering, Tsinghua University · University of Cambridge

  29. HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents

    Sep 29, 2026Tongbo Chen, Junbo Niu, Zhengxi Lu +8Computer-Use AgentsGUI Agents

    Zhejiang University · Peking University · Tsinghua University

  30. Absorbed in Inertia: Activation Analysis for Computer-Use Agents

    Sep 29, 2026Giulio Segalini, Zhi Wen Soi, Jérémie Decouchant +1Computer-Use AgentsLLM Agent Reliability

    Université de Neuchâtel