Mobile GUI Automation
GUI: Graphical User Interface
Overview
Mobile GUI automation studies how software can understand smartphone screens and complete tasks or test workflows through taps, swipes, typing, and other actions. Common approaches use vision-language models with planning and memory, supervised or reinforcement learning, and tools such as accessibility data, app-specific skills, world models, or CLI commands. Benchmarks including AndroidWorld and MobileWorld, along with real-device tests and simulated apps, measure task success, efficiency, and recovery from errors. Agents are becoming more capable, but long workflows, changing interfaces, reliable verification, and protection against privacy leaks or prompt injection remain difficult.
Latest papers
37 papers1 paper in the last four weeks, down 80% on the four weeks before. 0.0% of all new papers.