Graphical User Interface

Ask about this topic

Momentum

11 papers in the last four weeks, up 175% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 67

All topics
CardsList

  1. HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents

    Sep 29, 2026Tongbo Chen, Junbo Niu, Zhengxi Lu +8Computer-Use AgentsGraphical User Interface

    Zhejiang University · Peking University · Tsinghua University

  2. SCA: Spatial Credit Assignment for Reinforcement Learning of GUI Agents

    Sep 29, 2026Shengtian Yang, Ziyu Xiong, Kaibing Yang +6Credit AssignmentGroup-Based Reinforcement Learning

    Southeast University, Nanjing, China · Kuaishou Technology, Beijing, China · Unaffiliated

  3. PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience Retrieval

    Sep 29, 2026Bin Kang, Jiarui Ouyang, Li Jiang +2Graphical User Interface AgentsGraphical User Interface

    University of Chinese Academy of Sciences · Harbin Institute of Technology · Shenzhen Loop Area Institute +2

  4. GUITAR: Structured Failure Diagnosis of GUI Agents via State Transitions

    Sep 28, 2026Shaoqing Zhang, Kehai Chen, Xuefeng Bai +4Graphical User Interface AgentsGraphical User Interface

    Harbin Institute of Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China · Shanghai Jiao Tong University, China

  5. Relevance Does Not Imply Applicability: Experience Activation for Personal GUI Agents

    Sep 27, 2026Fuyao Zhang, Xuan Wang, Zherui Li +3Graphical User Interface AgentsGraphical User Interface

    NTU & Alibaba

  6. Towards Omni-dimensional GUI Agent Navigation with Masked Trajectory Prediction

    Sep 22, 2026Yan Zhang, Pei Fu, Daiqing Wu +8Graphical User Interface AgentsGraphical User Interface

    Institute of Information Engineering, Chinese Academy of Sciences · MiLM Plus, Xiaomi Inc. · School of Cyber Security, University of Chinese Academy of Sciences +1

  7. RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop Selection

    Sep 16, 2026Liyang Fan, Xinping Bi, Yitai Li +3Graphical User InterfaceRanking

    College of Computer Science and Software Engineering Shenzhen University Shenzhen, China · Shenzhen Key Laboratory for High Performance Data Mining Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences Shenzhen, China · Shenzhen University of Advanced Technology Shenzhen, China

  8. Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents

    Sep 15, 2026Bofan Chen, Boxuan Zhang, Fei Tang +7Graphical User Interface AgentsGraphical User Interface

    Zhejiang University · UESTC

  9. AnchorGUI: Asymmetric Memory for Dual-Scale Learning in GUI Navigation

    Sep 14, 2026Shengjie Jin, Zelong Sun, Hengbo Xu +2Graphical User InterfaceVision-Language Navigation

    Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China

  10. Learning from Reliable Negatives: Confidence-Anchored Test-Time Adaptation for GUI Grounding

    Sep 14, 2026Yizhou Liu, Fei Tang, Yuchen Yan +8Graphical User InterfaceVision-Language Model Grounding

    Zhejiang University · Ant Group

  11. JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition

    Sep 9, 2026Zixiang Chen, Yuheng Lu, Zihao Cheng +8Graphical User Interface AgentsGraphical User Interface

    School of Computer Science and Engineering, Beihang University, Beijing, China · Beijing Institute of Technology, Beijing, China · Baidu Inc., Beijing, China

  12. Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI Agents

    Sep 3, 2026Zhaoyuan Huang, Tianjie Ju, Pengzhou Cheng +7Graphical User Interface AgentsGraphical User Interface

    School of Computer Science, Shanghai Jiao Tong University · Ant Group

  13. Test-Time Self-Evolving GUI Visual Grounding via Reflection-Guided On-Policy Self-Distillation

    Aug 11, 2026Shiyu Xuan, Zechao LiGraphical User InterfaceGraphical User Interface Agents

    School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China, 210094

  14. LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflection

    Aug 10, 2026Renshan Zhang, Haoyang Meng, Yixiao He +3Graphical User InterfaceWeak Visual Grounding

    Harbin Institute of Technology, Shenzhen · Beijing University of Posts and Telecommunications · Shenzhen Loop Area Institute +1

  15. Hallucination-Free GUI Grounding via Regression-Free Layout-Aware Matching

    Aug 10, 2026Yuke Li, Xuehan HouGraphical User InterfaceLayouts

    School of Electronic and Computer Engineering, Peking University, Shenzhen, China

  16. MissClick: Execution-Aware Adversarial Attacks on Coordinate Generation in GUI Grounding Models

    Aug 4, 2026Yu Ran, Wentao ZhaoGraphical User InterfaceAttack-Success Rate

    National University of Defense Technology

  17. GUI-Lens: Coarse-to-Fine Cropping for GUI Grounding with General-Purpose VLMs

    Aug 4, 2026Zichuan Fu, Shirong Wang, Wenlin Zhang +10Graphical User InterfaceCoarse-To-Fine

    City University of Hong Kong · Westlake University · Tencent Jarvis Lab

  18. Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees

    Jul 30, 2026Zihan Dong, Rui Qian, Qishi Zhan +3Graphical User Interface AgentsGraphical User Interface

    Georgia Institute of Technology · Fudan University · Marquette University +3

  19. Prior Directions: Why GUI Grounding Gets Locked in the Past

    Jul 29, 2026Weile Gong, Zijian Lu, Mingcai Chen +3Weak Visual GroundingLock-In

    Nanjing University of Posts and Telecommunications, Nanjing, China

  20. Desktop-Delta Bench: Do Computer-Use Models Understand Desktop GUI Transitions?

    Jul 28, 2026Abhishek Pillai, Samir Kumar Nayak, Yuan ChenGraphical User InterfaceTask-Relevant Information

    NVIDIA Austin, TX, USA · NVIDIA Bangalore, Karnataka, IND

  21. Scaling GUI Agents with Visual State Transitions

    Jul 27, 2026Xiangyan Liu, Kaixin Li, Haonan Wang +6Graphical User Interface AgentsGraphical User Interface

    NUS · UTS · University of Liverpool +1

  22. SEE: Structure-aware Exploring & Exploiting for Long-horizon GUI Agent Trajectory Synthesis

    Jul 20, 2026Zhuohang Fan, Beichen Zhang, Yuanfa Li +4Graphical User Interface AgentsGraphical User Interface

    Harbin Institute of Technology, Weihai, China · Harbin Institute of Technology (Weihai) Qingdao Research Institute, Qingdao, China

  23. Beyond Sequential Interaction: Benchmarking Parallel Execution and Coordination for GUI Agents

    Jul 17, 2026Zedong Yu, Qianxing Li, Zhi Gao +8Graphical User Interface AgentsGraphical User Interface

    Beijing University of Posts and Telecommunications, Beijing, China · State Key Laboratory for General Artificial Intelligence, BIGAI, Beijing, China · China University of Geosciences (Beijing), Beijing, China +1

  24. Plover: Steering GUI Agents through Plan-Centric Interaction

    Jul 16, 2026Madhumitha Venkatesan, Shicheng Wen, Jiajing Guo +3Graphical User Interface AgentsGraphical User Interface

    University of California, Davis · Davis, California, USA · Bosch Research North America +1

  25. One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding

    Jun 29, 2026Chen Liu, Ling Chen, Hanzhang Zhou +5Graphical User InterfaceObject Localization

    University of Technology Sydney · Tongyi Lab, Alibaba Group · Adelaide University

  26. WinDOM: Self-Family Distillation for Small-Model GUI Grounding

    Jun 24, 2026Chengheng Li-Chen, Zhiqian Zhou, Hao Chen +1Supervised FinetuningSmall Reasoning Model

    Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland · Logitech · Universitat Politècnica de Catalunya (UPC), Barcelona, Spain.

  27. GUI vs. CLI: Execution Bottlenecks in Screen-Only and Skill-Mediated Computer-Use Agents

    Jun 22, 2026Xiao Zhou, Siyue Zhang, Yilun Zhao +4Graphical User Interface AgentsGraphical User Interface

    NNYU Shanghai · Y Yale NLP Lab

  28. Trust the Right Teacher: Quality-Aware Self-Distillation for GUI Grounding

    Jun 16, 2026Jingyuan Huang, Zuming Huang, Yucheng Shi +4Graphical User InterfaceSelf-Distillation Framework

    University of Georgia · 2INFLY Tech · *This work was done during an internship at INFLY Tech. +2

  29. VISTA: View-Consistent Self-Verified Training for GUI Grounding

    Jun 12, 2026Xinyu Qiu, Yunzhu Zhang, Heng Jia +3Graphical User Interface

    Zhejiang University · Venus Team, Ant Group

  30. PhoneHarness: Harnessing Phone-Use Agents through Mixed GUI, CLI, and Tool Actions

    Jun 12, 2026Chenxin Li, Zhengyao Fang, Zhengyang Tang +18Computer-Use AgentsDial

    Tencent Hunyuan · The Chinese University of Hong Kong · The Chinese University of Hong Kong, Shenzhen +1

  31. Naive Visual Memory is Not Enough: A Failure-Mode Study of GUI Agents

    Jun 12, 2026Seoyoung Choi, Minseok Ko, Hyunseok Lee +4Graphical User Interface AgentsGraphical User Interface

    1Seoul National University · 2POSTECH · 3KAIST +1

  32. GUITrans2Act: Understanding User Operational Behaviors from Mobile GUI Interactions with Vision-Language Models

    Jun 11, 2026Yudong Zhang, Lei Hu, Daoyang Liu +4Recent Vision-Language ModelsGraphical User Interface

    Honor Device Co., Ltd · The Chinese University of Hong Kong, Hong Kong, China

  33. MacArena: Benchmarking Computer Use Agents on an Online macOS Environment

    Jun 4, 2026Victor Muryn, Maksym Shamrai, Sofiia Mazepa +1Graphical User InterfaceEvaluation Benchmarks

    1MacPaw Research, Kyiv, Ukraine

  34. STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language Models

    Jun 1, 2026Yuhang Han, Wenzheng Yang, Yujie Chen +4Key-Value Cache CompressionGraphics Processing Unit Memory

    1EPIC Lab, SJTU · 2HKUST (GZ) · 3The University of Sydney +3

  35. Comparing LLM-Based Conversational and Graphical Interfaces for Industrial Decision Tasks: An Exploratory Mixed-Methods Study

    May 29, 2026Roberto Figliè, Simone Caputo, Alan Serrano +2Conversational Artificial IntelligenceInteraction Data

    Department of Computer Science, University of Pisa, Pisa, Italy · San Matteo Hospital, Pavia, Italy · Department of Computer Science, Brunel University of London, London, United Kingdom

  36. GUI-C2^2: Coarse-to-Fine GUI Grounding via Difficulty-Aware Reinforcement Learning

    May 29, 2026Junlong Li, Chao Hao, Lap-Pui Chau +1Graphical User InterfaceGrounding

    The Hong Kong Polytechnic University

  37. UI-KOBE: Knowledge-Oriented Behavior Exploration for Lightweight Graph-Guided GUI Agents

    May 28, 2026Yuxiang Chai, Han Xiao, Xinyu Fu +3Graphical User Interface AgentsGraphical User Interface

    1CUHK MMLab · 2Huawei Research · 3Shenzhen Loop Area Institute +1

  38. GUITestScape: Towards Open-set Evaluation on Exploratory GUI Testing

    May 28, 2026Xiaoyi Chen, Yifei Gao, Yang Xu +3Graphical User InterfaceTest Generation

    Beijing Jiaotong University · Independent Researcher

  39. GUI Agents for Continual Game Generation

    May 27, 2026Yixu Huang, Bo Li, Na Li +8Motion GenerationGraphical User Interface

    αFudan University · βXiaohongshu Inc. · γTongji University +2

  40. MIRAGE: Context-Aware Prompt Injection against Mobile GUI Agents via User-Generated Content

    May 27, 2026Ruoqi Guo, Yi Liu, Gelei Deng +7Graphical User Interface AgentsGraphical User Interface

    1Griffith University · 2Quantstamp · 3Nanyang Technological University +4

  41. Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation

    May 26, 2026Heng Qu, Yike Liu, Renren Jin +4Vision-Language NavigationGraphical User Interface

    1Wuhan University · 2Xiaomi

  42. ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis

    May 24, 2026Guohong Liu, Jialei Ye, Pengzhi Gao +4Synthetic EnvironmentsGraphical User Interface Agents

    Institute for AI Industry Research (AIR), Tsinghua University · University of Electronic Science and Technology of China · MiLM Plus, Xiaomi Inc.

  43. AutoRPA: Efficient GUI Automation through LLM-Driven Code Synthesis from Interactions

    May 20, 2026Minghao Chen, Xinyi Hu, Zhou Yu +1Graphical User Interface AgentsAutomated

    Zhejiang Key Laboratory of Space Information Sensing and Transmission, School of Computer Science, Hangzhou Dianzi University, China.

  44. MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents

    May 18, 2026Ziyun Zeng, Hang Hua, Bocheng Zou +3Agentic MemoryGraphical User Interface

    University of Rochester · MIT-IBM Watson AI Lab · University of Wisconsin-Madison

  45. TClone: Low-Latency Forking of Live GUI Environments for Computer-Use Agents

    May 17, 2026Yutong Huang, Vikranth Srivatsa, Alex Asch +2WorkspacesComputer-Use Agents

    University of California, San Diego · University of California, San Diego and GenseeAI

  46. SE-GA: Memory-Augmented Self-Evolution for GUI Agents

    May 16, 2026Shilong Jin, Lanjun Wang, Zhuosheng ZhangGraphical User Interface AgentsSelf-Evolving Agents

    College of Intelligence and Computing, Tianjin University, Tianjin, China · School of New Media and Communication, Tianjin University, Tianjin, China · School of Computer Science, Shanghai Jiao Tong University, Shanghai, China

  47. PAGER: Bridging the Semantic-Execution Gap in Point-Precise Geometric GUI Control

    May 15, 2026Jingxuan Wei, Xi Bai, Shan Liu +8Graphical User InterfaceObject Geometry

    University of Chinese Academy of Sciences · China University of Petroleum-Beijing · Shanghai Artificial Intelligence Laboratory

  48. DRS-GUI: Dynamic Region Search for Training-Free GUI Grounding

    May 15, 2026Yichao Liu, Huawen Shen, Liu Yu +3Graphical User InterfaceMultimodal Search Agents

    1Nankai University · Institute of Information Engineering, Chinese Academy of Sciences

  49. Beyond Binary: Reframing GUI Critique as Continuous Semantic Alignment

    May 14, 2026Yuchen Sun, Pei Fu, Shaojie Zhang +6Graphical User InterfaceFine-Grained Actions

    1Shanghai Jiao Tong University · 2Xiaomi Inc.

  50. WinDeskGround: A Benchmark for Robust GUI Grounding in Complex Multi-Window Desktop Environments

    May 13, 2026Haoren Zhao, Tianyi Chen, Zhen WangGraphical User InterfaceGraphical User Interface Agents

    School of Cyberspace, Hangzhou Dianzi University, Hangzhou, China · 2Microsoft

  51. What Happens Before Decoding? Prefill Determines GUI Grounding in VLMs

    May 10, 2026Jiaping Lin, Fei Shen, Junzhe Li +4Attention-Guided TrainingPrefill

    1Guangming Laboratory · 2National University of Singapore · 3Peking University +2

  52. BAMI: Training-Free Bias Mitigation in GUI Grounding

    May 7, 2026Borui Zhang, Bo Zhang, Bo Wang +6Graphical User InterfaceGraphical User Interface Agents

    Tsinghua University, China · Lenovo Research, China

  53. AutoFocus: Uncertainty-Aware Active Visual Search for GUI Grounding

    May 4, 2026Ruilin Yao, Shegnwu Xiong, Tianyu Zou +2Graphical User InterfaceMotion Generation

    Wuhan University of Technology

  54. Faithful Mobile GUI Agents with Guided Advantage Estimator

    May 2, 2026Haowen Hu, Pengzhou Cheng, Zheng Wu +3Graphical User Interface AgentsGraphical User Interface

    1Shanghai Jiao Tong University.

  55. Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding

    May 1, 2026Yan Zhang, Daiqing Wu, Huawen Shen +2Graphical User InterfaceGraphical User Interface Agents

    Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · VCIP & TMCC & DISSec, College of Computer Science, Nankai University

  56. A11y-Compressor: A Framework for Enhancing the Efficiency of GUI Agent Observations through Visual Context Reconstruction and Redundancy Reduction

    May 1, 2026Michito Takeshita, Takuro Kawada, Takumi Ohashi +2Graphical User Interface AgentsGraphical User Interface

    Hosei University, Tokyo, Japan

  57. FineState-Bench: Benchmarking State-Conditioned Grounding for Fine-grained GUI State Setting

    Apr 30, 2026Fengxian Ji, Jingpu Yang, Zirui Song +5Graphical User InterfaceWeak Visual Grounding

    MBZUAI · Northeastern University, China

  58. GUI Agents with Reinforcement Learning: Toward Digital Inhabitants

    Apr 30, 2026Junan Hu, Jian Liu, Jingxiang Lai +6Graphical User Interface AgentsGraphical User Interface

    1Shandong University · 2The Hong Kong University of Science and Technology · 3The Hong Kong University +2

  59. Training Computer Use Agents to Assess the Usability of Graphical User Interfaces

    Apr 28, 2026Alice Gao, Weixi Tong, Rishab Vempati +4Computer-Use AgentsGenerative User Interface

    University of Washington Seattle, WA, USA · Purdue University West Lafayette, IN, USA

  60. Benchmarking and Improving GUI Agents in High-Dynamic Environments

    Apr 28, 2026Enqi Liu, Liyuan Pan, Zhi Gao +5Graphical User Interface AgentsGraphical User Interface

    Beijing Institute of Technology, Beijing, China · Beijing Institute for General Artificial Intelligence, Beijing, China · Yangtze Delta Region Academy of Beijing Institute of Technology, Jiaxing, China +1