cs.CVJul 16, 2026

HyMobileAgent: Data-Environment Co-Scaling for Efficient GUI Agents

Authors: Hy Vision TeamHuawen ShenZhengyang TangShangpin PengLiang WuAnran ZhangWeinong WangYiduo Guo+15 more

Organizations: Hy Vision Team

Abstract

As large multimodal models move from understanding content to operating on digital environments, mobile GUI has emerged as a challenging and consequential testbed for digital embodied intelligence. Mobile agents operate under three coupled constraints: precise perception of complex interfaces, scalable acquisition of high-quality interaction data, and robust long-horizon decision making under compounding execution errors. This report presents HyMobileAgent, a mobile GUI agent built on Hy3.0-VL-A3B, a vision-native foundation model featuring native any-resolution input, an A3B-scale deployment budget, and a 32K context window to model extended interaction histories. Rather than relying solely on model scaling, we develop a joint data and environment centric scaling framework to address the key bottlenecks of mobile interaction. Our framework integrates a GUI perception flywheel combining mock-interface synthesis, rejection sampling, and icon-specific augmentation; a knowledge pipeline that transforms tutorial videos into structured interaction data; a million-scale action data pipeline deployed across more than 2000 sandbox and real-device instances with automated failure attribution; the PhoneWorld Mock App Factory, providing a resettable training environment with 34 mock applications and over 34000 tasks; and a structured Planning-and-Reflection mechanism with explicit dead-loop detection for reliable long-horizon execution. We also introduce a progressive training recipe consisting of mid-training, supervised fine-tuning, and reinforcement learning with task-specific reward designs.

Explore similar work

CardsList
  1. Xiaomi-GUI-0 Technical Report

    Jun 30, 2026Wanxia Cao, Chengzhen Duan, Pei Fu +29Graphical User Interface AgentsMultimodal Agents