cs.AIOct 19, 2026

VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards

Authors: Zhizhou Wang, Yaxi Lu, Zhong Zhang, Zikun Hu, Xin Cong, Yankai Lin, Fangming Liu

Organizations: Southwest Jiaotong University · Tsinghua University · University of Electronic Science and Technology of China · National University of Singapore · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Reinforcement learning for computer-use agents increasingly relies on proactive reward agents that interact with the post-execution GUI environment to verify what recorded screenshots cannot reveal. The reliability of such verification hinges on two coupled decisions: what to check and when to stop. Existing methods leave these decisions to prompted foundation models. However, neither scaling up the model nor providing few-shot examples reliably improves verification: prompted Judges ranging from 4B to GPT-5.4 all plateau at 68%-70% macro-average agreement. This is because the value of a check depends on the environment rather than the task alone, and becomes apparent only through execution. Learning from execution is itself constrained, as the Actor's terminal state cannot be faithfully replicated during training, allowing only one verification attempt per state. We introduce VeriCUA, a proactive reward agent that learns its verification strategy from these single attempts. A trainable Judge decides what to check and when to stop, while a frozen Probe executes the checks in the live environment. Online reinforcement learning optimizes the Judge for both verdict correctness and efficient Probe use, with returns normalized across terminal states. Trained only on OSWorld and evaluated without further training on MobileWorld and WebArena-Verified, VeriCUA achieves the highest reward agreement across all three benchmarks and lies on the agreement-cost Pareto frontier among proactive verifiers. It surpasses few-shot GPT-5.4 by 5.36 points with 47.6%-59.8% fewer Probe tokens, and even a 1B VeriCUA exceeds every prompted Judge. As the reward for Actor RL, it raises task success by 3.21 points overall and 5.40 points on held-out multi-application tasks. These results identify the verification strategy, not verifier scale, as the key to reliable GUI rewards.

Figures & tables

Figure 1: Learned check selection and stopping improve verification reliability and efficiency. Given the same task and frozen Probe , (a) prompted GPT-5.4 requests an exhaustive comparison and remains inconclusive, whereas (b) VeriCUA searches for a slide with missing notes and stops once this counterexample establishes task failure. (c, d) VeriCUA achieves higher agreement with fewer Probe tokens than few-shot GPT-5.4 and Kimi-K2.6.
Figure 2: Overview of VeriCUA . The Judge formulates task-success checks and assesses whether the available evidence supports a terminal verdict. When further inspection is requested, the frozen Probe gathers evidence for unresolved checks in the live environment and returns status updates with supporting observations.
Figure 3: Training procedure of VeriCUA . Online optimization uses one verification trajectory per live Actor terminal state. The Judge is updated using terminal feedback on verdict correctness and whether probing was used, while the Probe remains frozen.
VerifierModeOSWorld desktopMobileWorld mobileWebArena-V webMacro Avg. all envs.
GPT-5.4proprietary reference
Prompted VeriCUAPrompted81.8864.0058.5068.13
+ few-shotPrompted81.4066.2558.5068.72
Kimi-K2.6proprietary reference
Prompted VeriCUAPrompted75.8574.0059.0069.62
+ few-shotPrompted77.0570.2558.5068.60
Table 1: Reward reliability across executable environments (%). Results use Avg@ 44 except ZeroGUI ∗ , which follows its official Majority@ 44 aggregation. VeriCUA leads on all three benchmarks. Bold and underlined mark the best and second-best results.
Figure 4: Reward agreement versus Probe tokens per judgment: trained VeriCUA occupies the agreement–cost Pareto frontier among proactive verifiers. VAGEN † is billed for its full loop.
Figure 5: Learning ablations on OSWorld. (a) SFT improves all judgments; online RL mainly lifts those that invoke the Probe . (b) Agreement saturates at K=2K{=}2 , with most gain from the first round.
Figure 6: Case study. The baseline check specifies only the condition, and the Probe fails after nine steps. The VeriCUA check also specifies the evidence location, enabling verification in two steps.
Reward sourceAllIDOODΔ\boldsymbol{\Delta}
None (initial policy)43.9353.4017.57–
ZeroGUI45.3654.3720.27+1.43+1.43
VeriCUA (prompted)45.7154.8520.27+1.78+1.78
VeriCUA (trained)47.1455.8322.97+3.21\mathbf{+3.21}
Table 2: Pass@ 44 (%) of the RL-trained actor under different reward sources. Δ\Delta denotes the absolute improvement over the initial actor on the All split.
Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Behavioral proxyGPT-5.4VeriCUA -4BΔ\Delta
Direct-evidence probes (%) ↑\uparrow47.661.9+14.3+14.3
Aimless scroll and wait (%) ↓\downarrow16.25.5−10.7-10.7
Repeated targets (%) ↓\downarrow19.813.6−6.2-6.2
Dead-end probes (%) ↓\downarrow25.421.7−3.7-3.7
Appendix
Table 3: VeriCUA -4B directs more verification toward task-relevant evidence and reduces all three reported forms of unproductive exploration relative to prompted GPT-5.4. Rates are percentages, and Δ\Delta is VeriCUA minus GPT-5.4 in percentage points.
ProbeAgreement (%)95% CI
Holo3.1-35B-A3B82.0082.00[80.08,83.78][80.08,83.78]
Holo3.1-9B82.7382.73[80.83,84.47][80.83,84.47]
Holo3.1-4B82.0782.07[80.14,83.84][80.14,83.84]
Appendix
Table 4: Reward agreement remains stable across Holo3.1 Probe scales. The trained Judge , evaluation task set, and active-verification protocol are fixed, and only the frozen Probe changes. Confidence intervals are 95% Wilson intervals.
BenchmarkProbe
OSWorldHolo3.1-35B-A3B
MobileWorldGUI-Owl-1.5-32B-Think
WebArena-VerifiedHolo3.1-35B-A3B
Appendix
Table 5: Default frozen Probe configurations used for reward evaluation.
SettingValue
Judge backbonesQwen3-VL-2B-Thinking; Qwen3-VL-4B-Thinking; MiniCPM-V-4.6-Thinking
Teacher supervisionKimi-K2.6 Judge traces
Training examplesApproximately 1,2701{,}270
Random seed19791979
Fine-tuning scopeFull SFT; frozen vision tower and multimodal projector; trainable language model
Target maskingCurrent assistant turn only; previous turns used as context
Appendix
Table 6: Shared cold-start SFT configuration across Judge backbones.
SettingValue
Trainable Judge backbonesQwen3-VL-2B-Thinking; Qwen3-VL-4B-Thinking; MiniCPM-V-4.6-Thinking
InitializationRejection-sampled SFT
Training setOSWorld ( 280280 tasks)
Fixed Actor poolEvoCUA-32B; Qwen3-VL-4B-Thinking; Holo3.1-35B-A3B; Qwen3.6-27B
Fixed ProbeHolo3.1-35B-A3B
Verification budgetAt most K=2K{=}2 Probe rounds and 1515 GUI steps per round
Appendix
Table 7: Shared hyperparameters for online VeriCUA training across Judge backbones.
SettingValue
Actor modelQwen3-VL-4B-Thinking
Training setOSWorld ( 206206 single-application tasks)
InteractionScreenshot observations, PyAutoGUI actions, 3030 -step horizon
GRPO batch88 tasks ×\times 88 rollouts
Advantage estimatorGRPO
PPO clipping(0.20,0.28)(0.20,0.28)
Appendix
Table 8: Shared hyperparameters for downstream GUI Actor RL across reward sources.
OutcomeProbe usedReward
CorrectNo+1.0+1.0
CorrectYes+0.7+0.7
IncorrectNo−0.7-0.7
IncorrectYes−0.3-0.3
Unresolved or invalid–−1.0-1.0
Appendix
Table 9: Terminal reward schedule used for online Judge RL.
Behavioral metricPromptedTrainedpp
Probe attempt rate (%) ↓\downarrow5.704.280.0130.013
Failed Probe dispatch (%) ↓\downarrow16.63.23×10−43\times 10^{-4}
Unresolved uncertain (%) ↓\downarrow1.90.4—
False-negative rate (%) ↓\downarrow10.04.61×10−71\times 10^{-7}
Agreement with executable evaluator (%) ↑\uparrow77.881.4<0.001<0.001
Appendix
Table 10: Training reduces verification failures and improves terminal reward accuracy during downstream Actor RL.
Figure 7: Learned verification behavior during downstream Actor RL. (a) Training reduces failed Probe dispatches, unresolved verdicts, and false-negative verdicts. (b) The learned verifier maintains higher agreement as the Actor evolves. Unresolved outputs count as incorrect in (b) . Shaded bands show 95% Wilson confidence intervals.
More in this topic

AI Agent Reliability

AI agent reliability studies whether tool-using systems act correctly, consistently, safely, and recoverably across changing tasks and environments.

See all 338 papers

Explore similar work

CardsList
Sep 27, 2025

GUI-PRA: Process Reward Agent for GUI Tasks

Long-horizon GUI automation remains challenging due to error accumulation over extended interaction sequences. Process Reward Models (PRMs) provide dense step-level supervision for mitigating error accumulation, yet standard PRMs are poorly suited to GUI verification. Standard PRM judgments often rely on superficial visual alignment rather than functional correctness, reflecting an evaluative knowledge gap caused by missing domain-specific adjudication logic. Standard PRMs also perform passive, single-pass visual assessment, which creates Visual Ambiguity when reliable judgment requires actively locating, parsing, or inspecting task-relevant UI evidence. We introduce GUI-PRA, a Process Reward Agent that transforms GUI process evaluation from passive scoring into active investigation. GUI-PRA couples Experience-Injected Criterion Synthesis, which distills generalized verification principles into state-specific criteria, with Criterion-Guided Autoregressive Perception, which uses these criteria to navigate multi-granularity visual tools and gather grounded evidence. On AndroidWorld and Mobile-MiniWoB++, GUI-PRA achieves improvements of 5.0% and 6.5% over standard PRMs on the Qwen-VL series, with Qwen3-VL attaining 54.74% success rate on AndroidWorld. On the offline OS-Critic Bench, GUI-PRA demonstrates strong competitiveness against fully trained critic models.

Tao Xiong, Xavier Hu, Yurun Chen +6

Zhejiang University · MiLM Plus, Xiaomi Inc.

Process Reward ModelsGUI AgentsMultimodal Reward ModelingGUI Automation
AskPDF
Jul 28, 2026

Interactive Reward Agent: GUI Task Evaluation via Environment-State Verification

Graphical user interface task evaluation aims to determine whether a GUI agent has successfully completed a user instruction. Automated GUI task evaluation has received increasing attention because the evaluation results can serve as reward signals for both test-time scaling and post-training. However, reliable GUI task evaluation remains challenging because the judgments often require access to environment states, such as system configurations, file data, and application settings, beyond the screenshots of execution trajectories. In this paper, we propose an interactive reward agent (IRA) based on a propose-then-verify framework to acquire and verify evidence from the post-execution environment. Given a task instruction and a GUI environment after the GUI agent execution, IRA first proposes the task completion conditions and then verifies them by invoking system tools, application tools, and GUI tools. This design combines evidence from both visible interfaces and the environment state in an interactive process. We further introduce GUI-RewardBench, a benchmark of 321 GUI task trajectories spanning 10 Ubuntu desktop application categories. Experiments show that IRA achieves 86.9% accuracy on GUI-RewardBench, outperforming existing evaluator baselines. We further apply IRA to reinforcement learning of GUI agents, achieving a 34.0% OSWorld success rate, which demonstrates that IRA can provide effective reward signals for training GUI agents.

Chenrui Shi, Yuwei Wu, Yang Liu +5

Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology, Beijing, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China · Guangdong Laboratory of Machine Perception and Intelligent Computing, Shenzhen MSU-BIT University, Shenzhen, China +1

RL for GUI AgentsAutomated Evaluation
AskPDF
Apr 7, 2026

Don't Act Blindly: Robust GUI Automation via Action-Effect Verification and Self-Correction

Autonomous GUI agents based on vision-language models (VLMs) often assume deterministic environment responses, generating actions without verifying whether previous operations succeeded. In real-world settings with network latency, rendering delays, and system interruptions, this assumption leads to undetected action failures, repetitive ineffective behaviors, and catastrophic error accumulation. Moreover, learning robust recovery strategies is challenging due to the high cost of online interaction and the lack of real-time feedback in offline datasets.We propose VeriGUI (Verification-driven GUI Agent), which explicitly models action outcomes and recovery under noisy environments. VeriGUI introduces a Thinking--Verification--Action--Expectation (TVAE) framework to detect failures and guide corrective reasoning, and a two-stage training pipeline that combines Robust SFT with synthetic failure trajectories and GRPO with asymmetric verification rewards. We further construct a Robustness Benchmark based on AndroidControl to evaluate failure recognition and correction. Experiments show that VeriGUI significantly reduces failure loops and improves recovery success while maintaining competitive standard task performance.

Yuzhe Zhang, Xianwei Xue, Xingyong Wu +8

Beijing University of Technology, Beijing, China · Baidu Inc., Beijing, China

GUI AgentsGUI AutomationMobile GUI AutomationReinforcement Learning with Verifiable Rewards
AskPDF