Long-horizon GUI agents rely on visual history to maintain task states, but retaining historical screenshots increases visual-token usage and inference costs, while textual summaries can discard critical state information, undermining long-horizon reliability. We propose ECHOES, which leverages training-time privileged information to guide textual history compression and strengthen long-horizon reasoning. Specifically, ECHOES adopts a Compression-Think-Action (CTA) format to recursively update a textual history state and reason. The same model, given privileged multi-image inputs, serves as a teacher that provides dense token-level on-policy distillation on student-generated CTA rollouts. Our Perception-Memory Fidelity (PMF) Gap-guided OPD-RL framework dynamically balances privileged distillation and task-reward optimization, with periodic teacher updates supporting continued improvement. We further introduce ECHOESBMK, a fine-grained diagnostic benchmark derived from real GUI agent failure trajectories to jointly evaluate visual perception and cross-modal state retention. Experiments on four GUI benchmarks and ECHOESBMK demonstrate substantial gains in task performance and memory fidelity. Without retaining historical screenshots at deployment, ECHOES achieves competitive performance with a favorable performance-inference-cost trade-off and exhibits positive zero-shot transfer to unseen visual planning tasks.
Figures & tables
Figure 1: Motivation and method comparison. (a) History-independent versus history-dependent decisions. (b) ECHOES’s performance–cost advantage on GUI-Odyssey. (c) Existing methods versus ECHOES with recursive CTA and training-time multi-image supervision.
Figure 2: Overview of ECHOES. Given the task, current screenshot, and previous C , the policy generates a CTA sequence. Training combines OPD from the same model’s privileged multi-image view with GRPO under PMF-guided weighting, refreshing the teacher at round boundaries.
Figure 3: Task taxonomy and statistics of ECHOESBMK. Seven tasks span element-level perception (L1), whole-screen state understanding (L2), and cross-frame state changes (L3). Bars indicate evaluation-instance counts.
Model
GUI-Odyssey
AC-v2-High
AITW
AMEX-L3
TM ↑
EM ↑
Acc ↑
Type ↑
Ground ↑
Step ↑
Type ↑
Step ↑
Type ↑
Proprietary / Closed-source Models
GPT-5.5
73.84
48.76
46.92
69.85
64.38
48.65
63.72
68.84
80.92
Gemini 3.1 Pro
77.18
54.92
50.82
74.36
68.22
53.18
65.46
72.36
83.46
Open-source Models
Qwen3-VL-8B
76.14
48.27
49.62
63.28
64.43
44.16
56.40
62.28
80.10
Table 1: Main results on four GUI benchmarks. All scores are reported in percentage (%). Best and second-best results are marked in bold and underline , respectively.
Figure 5
Variant
TM ↑
EM ↑
SFT
77.00
56.24
SFT + RL
79.55
63.65
SFT + OPD
81.05
64.01
SFT + OPD + RL (Linear λ )
81.31
65.12
Full (PMF-guided)
82.25
68.58
Table 3: Ablation studies on GUI-Odyssey. Left: contributions of RL, privileged OPD, and PMF-guided joint optimization. Right: interventions on the recurrent textual state C .
Table 7
Steps / Round
TM ↑
EM ↑
3
81.43
67.92
6
82.25
68.58
12
80.79
65.84
Table 6: Hyperparameter sensitivity on GUI-Odyssey. We vary the teacher-refresh interval and the OPD–RL transition range while keeping the remaining training configuration unchanged.
Model
B → F
S → B
Tab
Fore.
Block.
Select.
Diff
Avg. PMF ↓
Qwen3-VL-8B
0.584
0.773
0.675
0.767
0.753
0.347
0.718
0.054
Qwen3-VL-32B
0.613
0.789
0.691
0.796
0.762
0.287
0.736
0.041
Qwen3.5-27B
0.620
0.838
0.707
0.801
0.808
0.415
0.754
0.089
GLM-4.6V-Flash
0.582
0.752
0.634
0.792
0.848
0.307
0.792
0.063
Step3-VL-10B
0.626
0.651
0.667
0.753
0.832
0.380
0.585
0.058
MAI-UI-8B
0.569
0.804
0.650
0.723
0.826
0.318
0.633
0.059
Table 7: Perception and memory fidelity on ECHOESBMK. We report direct-vision scores ( ↑ ) for seven tasks and Avg. PMF ( ↓ ), the mean nonnegative performance drop from vision to text-only memory replay across four L2 tasks. Bold indicates the best results.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
GUI-Odyssey
AC-v2-High
AITW
AMEX-L3
Variant
TM ↑
EM ↑
Acc ↑
Type ↑
Ground ↑
Step ↑
Type ↑
Step ↑
Type ↑
Qwen3-VL-8B
Base
76.14
48.27
49.62
63.28
64.43
44.16
56.40
62.28
80.10
ECHOES
82.25
68.58
53.60
76.96
65.56
59.35
66.30
73.90
84.60
Δ over Base
+6.11
+20.31
+3.98
+13.68
+1.13
+15.19
+9.90
+11.62
+4.50
InternVL3.5-8B
Appendix
Table 8: Additional-backbone comparison on four GUI benchmarks. All scores are percentages, and Δ denotes differences in percentage points. Qwen3-VL-8B results are reproduced from the main-results table. Base denotes the released checkpoint before our training.
Model
Tab PMF ↓
Fore. PMF ↓
Block. PMF ↓
Select. PMF ↓
Avg. PMF ↓
Qwen3-VL-8B
0.138
0.050
0.004
0.025
0.054
Qwen3-VL-32B
0.089
0.040
0.016
0.017
0.041
Qwen3.5-27B
0.179
0.076
0.077
0.025
0.089
GLM-4.6V-Flash
0.073
0.059
0.053
0.068
0.063
Step3-VL-10B
0.106
0.019
0.065
0.042
0.058
MAI-UI-8B
0.114
0.049
0.049
0.025
0.059
Appendix
Table 9: Detailed PMF results on the diagnostic GUI benchmark. We report the visual-to-text performance gap for each of the four L2 state categories. Lower PMF indicates less information loss during visual-to-text transformation.
Round
SrP
SrD
SrP−SrD
λr
0
0.6473
0.5624
+0.0849
0.80
3
0.6706
0.6086
+0.0620
0.68
6
0.6867
0.6415
+0.0452
0.63
9
0.6855
0.6515
+0.0340
0.42
12
0.6812
0.6655
+0.0157
0.34
Final
0.6760
0.6858
-0.0098
0.20
Appendix
Table 10: Round-wise privileged–deployable gap and OPD weight.
Comparison
Δ EM ↑
95% CI
p -value
Full vs. SFT
+12.34
[10.61, 14.07]
<0.001
Full vs. SFT + RL
+4.93
[3.24, 6.60]
<0.001
Full vs. SFT + OPD
+4.57
[2.90, 6.22]
<0.001
Full vs. OPD + RL (Linear λ )
+3.46
[1.77, 5.12]
0.002
Appendix
Table 11: Statistical significance of the ablation comparisons on GUI-Odyssey. We report paired EM improvements of the full model over each training variant, with 95% confidence intervals obtained using trajectory-level paired bootstrap with 10,000 resamples.
Component
OPD–RL
ECHOES
Additional time
Baseline training workload
100.00
100.00
0.00
Additional deployable-view validation
0.00
4.83
4.83
Additional privileged-view validation
0.00
9.12
9.12
Additional teacher refresh/synchronization
0.00
2.75
2.75
Total normalized runtime
100.00
116.70
16.70
Relative overhead
0.0%
16.7%
—
Appendix
Table 12: Training-time overhead. Runtime is normalized to the OPD–RL baseline.
Figure 6: Cross-app fact retention through recursive compression. An 11-step example involving information retrieval in Opera and reminder creation in TickTick. Each panel shows the current screenshot and the corresponding Compression, Think, and Action outputs. Dates recorded at step 05 remain available after leaving Opera, while similar Inbox screens at steps 00 and 07 correspond to different task stages and actions. Each decision uses only the task instruction, current screenshot, and previous compression, without historical screenshot inputs.
Action
Parameter
Description
CLICK
(x,y)
Click a target position
LONG_PRESS
(x,y)
Long-press a target position
TYPE
text
Enter textual content
SCROLL
direction
Scroll up, down, left, or right
PRESS_BACK
–
Return to the previous screen
PRESS_HOME
–
Return to the home screen
Appendix
Table 13: Unified GUI action space used for post-training.
Figure 7: Representative examples of the seven ECHOESBMK tasks. Panels (a)–(b) illustrate L1 element-level perception: BBox-to-Function and Semantic-to-BBox. Panels (c)–(f) illustrate L2 whole-screen state understanding: Active-Tab, Foreground-Layer, Blocking-State, and Screen-Selection. Panel (g) illustrates L3 cross-frame state changes through Screen-State Diff. Each example includes a task prompt, visual observation(s), and a corresponding answer. Full screenshots are accompanied by enlarged details for readability.