cs.CVNov 5, 2026

BiGrounder: Bidirectional Training Unlocks the Potential of GUI Grounding

Authors: Dengyao Shi, Tianyu Fu, Fei Hu, Anyang Su, Hanning Wang, Shuo Wang, Minghui Wu, Chenxu Zhao, 

Organizations: MAIS, Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Mininglamp Technology

Abstract

Graphical User Interface (GUI) grounding aims to map natural language instructions to precise operational coordinates, serving as a critical task for autonomous agent interaction. Current approaches formalize GUI grounding as predicting action-coordinate pairs from instructions, with end-to-end training supervised solely by final coordinates. Training datasets typically provide only a small number of descriptions per target coordinate, whereas real-world scenarios involve a much larger set of varied descriptions for the same target, which limits robustness to linguistic variation. While recent GUI grounding paradigms have begun employing reinforcement learning to enhance localization accuracy, models with limited exploratory capability before reinforcement yield constrained improvements even with identical training data during the reinforcement phase. To overcome these limitations, we propose a bidirectional reward paradigm that enables model self-reinforcement. During SFT, bidirectional training equips the model to generate action descriptions from target coordinates. During RL, the model samples semantically consistent instruction variations on-policy and grounds each variation back to its target. A cycle-consistency gate couples semantic quality to grounding success, turning the model's own instruction policy into a source of continually refreshed reinforcement trajectories. Our 4B-parameter model achieves competitive accuracy across five grounding benchmarks in our evaluation, indicating the effectiveness of the proposed training paradigm. https://github.com/Spell-shi/BiGrounder

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. GUI-C2^2: Coarse-to-Fine GUI Grounding via Difficulty-Aware Reinforcement Learning

    May 29, 2026Junlong Li, Chao Hao, Lap-Pui Chau +1Multimodal GroundingAgentic RL

  2. Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding

    May 1, 2026Yan Zhang, Daiqing Wu, Huawen Shen +2GUI GroundingOn-Policy Distillation

  3. GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding

    Nov 2, 2025Shijie Zhou, Viet Dac Lai, Hao Tan +4Multimodal GroundingGUI Grounding