cs.CLAug 30, 2026
SaveGUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
Organizations: Zhejiang University · Tongji University · University of California, San Diego · Yale University · China University of Geosciences · DAMO Academy, Alibaba Group
Abstract
GUI world models are increasingly evaluated as one-step next-screen predictors, yet their intended use is often as multi-step environments for GUI agents. This mismatch leaves a key requirement under-tested: generated states must remain contextually consistent when they are repeatedly reused for future interaction. We introduce GUI-CC, a benchmark that evaluates contextual consistency of GUI world models as agent environments rather than isolated next-screen predictors. GUI-CC contains two complementary tracks: an offline reference-action track that rolls models along real mobile GUI trajectories, and an online agent-loop track that lets fixed probing agents interact with model-generated UIs. We construct 500 offline trajectory tasks from GUIOdyssey and 200 emulator-verified online tasks across 30 mobile apps. GUI-CC evaluates transition fidelity, transition plausibility, contextual consistency, and task progress. Experiments show that plausible single-step generation does not guarantee reliable environment simulation: current models often produce usable-looking screens while failing to preserve task-relevant context or support executable multi-step rollouts.
Figures & tables
Figure 1: Representative failure cases of GUI world models under long-horizon rollouts. The examples illustrate app-context drift, action-effect lag, state-update propagation failure, and launcher-state memory loss.
| Evaluation | Release | Domain | Output State | Multi-Step | Pred. Fed Back | Agent-in-Loop | Direct WM Eval | Consistency Eval |
| WMA ( Chae et al., 2025 ) | 2024-10 | Web | Text | ✓ | ✗ | ✗ | ✗ | ✗ |
| MobileWorldBench ( Li et al., 2025 ) | 2025-12 | Mobile | Text | ✗ | ✗ | ✗ | ✓ | ✗ |
| gWorld ( Koh et al., 2026 ) | 2026-02 | Mobile | HTML code | ✗ | ✗ | ✗ | ✓ | ✗ |
| GEBench ( Li et al., 2026 ) | 2026-02 | Mobile / desktop | Image | ✓ | ✗ | ✗ | ✓ | ✗ |
| Code2World ( Zheng et al., 2026 ) | 2026-02 | Mobile | HTML code | ✗ | ✗ | ✗ | ✓ | ✗ |
| CUWM ( Guan et al., 2026 ) | 2026-02 | Desktop | Text / image | ✗ | ✗ | ✗ | ✓ | ✗ |
Table 1: Comparison of GUI-CC with prior GUI world-model evaluations. Pred. fed back : predicted states are reused as the next environment state during the evaluation rollout. Agent-in-loop : the rollout is driven by an agent acting on generated states. Direct WM eval : the headline metric is computed on the world model’s output rather than on downstream agent or search outcomes. Consistency eval : the evaluation directly tests cross-step state-reuse consistency. GUI-CC measures reference-action executability and context persistence in the offline track, and milestone progress, loops, and stagnation in the agent-loop track. GEBench reports an image-level temporal coherence score but does not test state-reuse consistency.
Figure 2: (Left) Data construction pipeline of GUI-CC. Both tasks are designed to stress contextual consistency under multi-step rollout. The offline trajectory task selects high-quality real mobile trajectories from GUIOdyssey, while the online agent-loop task creates emulator-verified tasks with ordered milestones for closed-loop agent evaluation. (Right) Statistics of the two tasks in GUI-CC . Dashes indicate statistics that are not applicable to the corresponding track: the offline reference-action track does not use task templates or ordered milestones, while the online agent-loop track is evaluated with a step budget rather than a fixed set of reference transitions.
| # | World Model | Setting | Transition Fidelity | Transition Plausibility | CC | TP | Overall | ||||||
| Code / HTML-output world models | |||||||||||||
| 1 | Claude Opus 4.7 | w/ history | 16.8 | 15.6 | 73.4 | 47.1 | 69.5 | 79.0 | 99.6 | 65.0 | 93.8 | 16.4 | 57.6 |
| 2 | GPT-5.5 | w/ history | 17.8 | 16.6 | 75.0 | 49.8 | 63.4 | 68.2 | 99.4 | 59.5 | 80.9 | 15.3 | 54.6 |
| 3 | Claude Opus 4.7 | w/o history | 13.0 | 11.9 | 71.0 | 45.9 | 62.0 | 78.3 | 99.6 | 46.9 | 78.1 | 13.1 | 52.0 |
| 4 | GPT-5.5 | w/o history | 13.1 | 11.8 | 71.5 | 47.5 | 53.3 | 63.7 | 98.2 | 35.9 | 55.1 | 14.7 | 46.5 |
Table 2: Offline reference-action track evaluation of GUI world models. The Overall column is the unweighted mean across all ten metrics. Setting denotes the conditioning context: w/o history uses only the current UI and action, while w/ history uses multi-step historical states and actions. CC denotes Contextual Consistency . TP denotes Task Progress . The metrics evaluate UI element alignment ( ), layout integrity ( ), global similarity measured by SigLIP ( ) and DINO ( ), action adherence ( ), action identifiability ( ), GUI state usability ( ), state and context persistence ( ), action-controlled rollout dynamics ( ), and reference-action progress ( ). Within each model group, the best score per column is bold and the second best is underlined .
| # | World Model | Setting | Transition Plausibility | CC | TP | Overall | |||
| Code / HTML-output world models | |||||||||
| 1 | GPT-5.5 | w/ history | 89.5 | 88.9 | 99.9 | 93.3 | 99.2 | 70.9 | 90.3 |
| 2 | GPT-5.5 | w/o history | 84.2 | 84.6 | 100.0 | 80.0 | 95.0 | 51.2 | 82.5 |
| 3 | Claude Opus 4.7 | w/ history | 84.7 | 84.0 | 99.4 | 83.3 | 89.2 | 48.1 | 81.4 |
| 4 | Claude Opus 4.7 | w/o history | 82.0 | 88.0 | 99.4 | 57.5 | 78.3 | 41.4 | 74.4 |
Table 3: Online agent-loop track evaluation of GUI world models. The Overall column is the unweighted mean across all six metrics. Setting denotes the conditioning context: w/o history uses only the current UI and action, while w/ history uses multi-step historical states and actions. CC denotes Contextual Consistency . TP denotes Task Progress . The metrics evaluate action adherence ( ), action identifiability ( ), GUI state usability ( ), state and context persistence ( ), action-controlled rollout dynamics ( ), and ordered milestone progress ( ). Within each model group, the best score per column is bold and the second best is underlined .
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Prompt template for the single-stage probing GUI agent used in the online agent-loop track. The agent selects one action per step by calling the computer tool and emits absolute pixel coordinates on the observed screenshot.
Figure 4: Prompt template for Code2World without history.
Figure 5: Prompt template for Code2World with observation-action history.
Figure 6: Prompt template for gWorld without history.
Figure 7: Prompt template for gWorld with observation-action history.
Figure 8: Prompt templates for MobileWorld-html-8B with and without history.
Figure 9: HTML generation prompt template for GPT-5.5 and Claude Opus 4.7 without history.
Figure 10: HTML generation prompt template for GPT-5.5 and Claude Opus 4.7 with observation-action history.
Figure 11: Image-generation prompt template for GPT Image 2 and Gemini 3.1 Flash Image Preview.
Figure 12: Image-generation prompt template for FLUX.2-dev and MobileWorld-Diffusion.
Figure 13: Image-generation prompt template for Qwen-Image-Edit-2511.
Figure 14: VLM judge prompt for GUI state usability .
Figure 15: VLM judge prompt for state and context persistence .
Figure 16: VLM judge prompt for action-controlled rollout dynamics .
Figure 17: VLM judge prompt for reference action progress .
Figure 18: VLM judge prompt for ordered milestone progress .
| Model | Provider | Type | Size | Access / Checkpoint | Reference |
|---|---|---|---|---|---|
| Code / HTML-output world models | |||||
| Claude Opus 4.7 | Anthropic | Proprietary (API) | – | claude-opus-4-7 | ( Anthropic, 2026 ) |
| GPT-5.5 | OpenAI | Proprietary (API) | – | gpt-5.5 | ( OpenAI, 2026a ) |
| Code2World | – | Open-source | 8B | GD-ML/Code2World | ( Zheng et al., 2026 ) |
| MobileWorld-html-8B | – | Open-source | 8B | xwk123/MobileWorld-html-8B | ( Xu et al., 2026 ) |
| gWorld-32B | Trillion Labs | Open-source | 32B | trillionlabs/gWorld-32B | ( Koh et al., 2026 ) |
Table 4: World models evaluated in our benchmark.
Figure 19: Single-step qualitative comparison for the action Clear the search bar . The figure compares the predicted next UI states from different GUI world models against the ground-truth next state.
Figure 20: Single-step qualitative comparison for the action Open the GenZArt app . The figure compares the predicted next UI states from different GUI world models against the ground-truth next state.
Figure 21: Qualitative comparison between GUI world model (Code2World) rollouts with and without historical context. The history-conditioned rollout preserves the task state across multiple steps and successfully reaches the expected final UI, whereas the no-history rollout deviates from the intended trajectory and fails to complete the task. This example illustrates the importance of historical conditioning for long-horizon GUI state prediction and task-progress consistency.
Figure 22: Offline rollout comparison under the same long-horizon cross-app action sequence. Green and orange borders indicate successful and failed step-level transitions, respectively. Code2World fails in most steps, while GPT Image 2 preserves early states but drifts later.