Mobile GUI Automation

Mobile GUI automation studies how software can understand smartphone screens and complete tasks or test workflows through taps, swipes, typing, and other actions. Common approaches use vision-language models with planning and memory, supervised or reinforcement learning, and tools such as accessibility data, app-specific skills, world models, or CLI commands. Benchmarks including AndroidWorld and MobileWorld, along with real-device tests and simulated apps, measure task success, efficiency, and recovery from errors. Agents are becoming more capable, but long workflows, changing interfaces, reliable verification, and protection against privacy leaks or prompt injection remain difficult.