EVIDENT: Evaluating Evidence Grounding in GUI Agents
Organizations: Korea Advanced Institute of Science & Technology
Abstract
GUI agents are increasingly evaluated by whether they complete web tasks, but surface success can hide a critical failure: the final answer may not be supported by evidence observed during navigation. We introduce EVIDENT, an evidence-driven benchmark for evaluating whether GUI agents ground their decisions in observable web evidence. EVIDENT contains 220 real-world English and Korean tasks spanning semantic grounding, contextual inference, and visual grounding, each paired with human-curated evidence metadata specifying what must be verified. We further separate surface task completion from evidence-grounded success using three metrics: TSR, ESR, and GSR, which respectively measure outcome success, required evidence observation, and claim-level evidence support. Experiments on commercial and open-source GUI agents reveal substantial gaps between surface success and evidence-grounded success, especially in contextual and visual tasks. These results show that current evaluations can overestimate agent reliability and highlight the need to assess not only whether agents answer, but whether they verify.
Figures & tables
| Benchmark | Live Web | Multi-step | Trajectory | Observable |
| Env. | Eval | Grounding | ||
| Mind2Web Deng et al. (2023) | ✗ | ✓ | ✗ | ✗ |
| WebArena Zhou et al. (2024) | ✗ | ✓ | ✗ | ✗ |
| OSWorld Xie et al. (2024) | ✓ | ✓ | ✗ | ✗ |
| Mind2Web 2 Gou et al. (2025) | ✓ | ✓ | ✓ | ✗ |
| evident (Ours) | ✓ | ✓ | ✓ | ✓ |
| Task Type | Model | English | Korean | ||||
| TSR | ESR | GSR | TSR | ESR | GSR | ||
| Overall | Gemini CUA | 87.14 | 70.00 | 55.71 | 81.33 | 70.67 | 45.33 |
| Claude CUA | 80.00 | 57.14 | 35.71 | 74.00 | 42.00 | 30.00 | |
| GPT CUA | 70.00 | 61.43 | 51.43 | 56.67 | 44.00 | 27.33 | |
| MolmoWeb-8B-Native | 57.14 | 51.43 | 45.71 | 16.00 | 10.00 | 4.67 | |
| Fara-7B | 57.14 | 50.00 | 35.71 | 8.00 | 0.67 | 0.00 | |
| Judge setting | Acc. (%) | |
| Qwen3-VL-235B | 91.27 | 0.864 |
| Avg. over 3 judges | 90.48 | 0.851 |
| Trajectory length (steps) | |||
| Judge Input | 1–17 | 18–34 | 35–50 |
| Extracted evidence text (ours) | 89% | 97% | 89% |
| Full trajectory (downsampled) | 72% | 64% | 68% |
| Image windows (8) + prior OCR | 70% | 58% | 58% |
| Image windows (8) only | 63% | 58% | 58% |
| Dimension | Closed-source | OSS | ||
| En | Ko | En | Ko | |
| Intent Grounding | 91.2 | 80.8 | 65.5 | 32.5 |
| Evidence Grounding | 52.5 | 35.2 | 37.9 | 10.7 |
| Response Completion | 81.4 | 70.6 | 57.6 | 16.1 |
| UI label mapping | 87.7 | 75.2 | 67.5 | 36.8 |
| Contextual Inference | 85.2 | 78.0 | 72.5 | 37.5 |
| State | Description |
| Task success | |
| True Success | Observes the required evidence and grounds its final claims in it. |
| Claim-Ungrounded Success | Observes relevant evidence but the final answer is not sufficiently supported. |
| Hallucinated Success | Answer appears successful despite missing the required evidence. |
| No task success | |
| Task Failure | Fails to meet the task-success criterion. |
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | English | Korean | ||||
| TSR | ESR | GSR | TSR | ESR | GSR | |
| No constraint | 87.14 | 70.00 | 55.71 | 81.33 | 70.67 | 45.33 |
| No internal knowledge | 71.43 | 64.29 | 48.57 | 59.33 | 52.67 | 41.33 |
| + few-shot | 61.43 | 57.14 | 45.71 | 51.33 | 44.00 | 33.33 |
| Model | Lang. | All tasks | Task-specific | ||||
| Intent Grounding | Evidence Grounding | Response Completion | User Interface Label Mapping | Contextual Inference | Visual Grounding | ||
| Gemini | En | 94.1 | 55.9 | 89.7 | 100.0 | 84.1 | 80.0 |
| Ko | 90.5 | 44.2 | 82.3 | 79.6 | 92.0 | 82.3 | |
| Claude | En | 91.2 | 36.8 | 82.4 | 77.8 | 95.2 | 90.0 |
| Ko | 80.0 | 28.7 | 72.7 | 70.0 | 74.7 | 51.0 | |
| GPT | En | 88.2 | 64.7 | 72.1 | 85.2 | 76.2 | 87.5 |
| Lang. | Model | TSR | ESR | GSR |
| English | Gemini CUA | 13.89 | 14.06 | 13.38 |
| Claude CUA | 13.66 | 14.62 | 15.16 | |
| GPT CUA | 23.73 | 24.58 | 24.11 | |
| MolmoWeb-8B | 13.70 | 13.31 | 12.72 | |
| Fara-7B | 13.75 | 13.91 | 12.08 | |
| UI-TARS-1.5-7B | 12.97 | 13.32 | 13.31 |
| Validation | #Ann. | Agreement |
| Independent re-authoring | 2 | 48/50 (96%) |
| Metadata audit | 1 | 49/50 (98%) |
| Comparison | Match | Acc. (%) | Cohen’s |
| Human vs. judge | |||
| Human vs. Qwen | 115 | 91.27 | 0.864 |
| Human vs. GPT | 111 | 88.10 | 0.817 |
| Human vs. Gemini | 116 | 92.06 | 0.871 |
| Average | – | 90.48 | 0.851 |
| Judge vs. judge | |||
| (a) Agreement by task type | ||||
| Comparison | Semantic | Contextual | Visual | Overall |
| Human vs. Qwen | 95.24 | 92.86 | 85.71 | 91.27 |
| Human vs. GPT | 90.48 | 90.48 | 83.33 | 88.10 |
| Human vs. Gemini | 95.24 | 95.24 | 85.71 | 92.06 |
| (b) Agreement by language | ||||
| Comparison | KO | EN | Overall | |
| Task | Comparison | Fail | TSR | ESR | GSR |
| Semantic | vs. Qwen | 2.38 | 2.38 | 2.38 | 2.38 |
| vs. GPT | 4.76 | 4.76 | 7.14 | 2.38 | |
| vs. Gemini | 0.00 | 0.00 | 2.38 | 2.38 | |
| Contextual | vs. Qwen | 0.00 | 0.00 | 0.00 | 7.14 |
| vs. GPT | 4.76 | 4.76 | 4.76 | 4.76 | |
| vs. Gemini | 0.00 | 0.00 | 0.00 | 4.76 |
Agent Evaluation
Agent evaluation measures how reliably AI systems act, use tools, and complete tasks across interactive environments.
See all 160 papers