Vision-language agents that browse websites, operate phones and act in homes reason before every action, so every step waits for a large model, a latency and cost that stand in the way of deployment. Step-level speculation lets a small model draft each step and calls the large model only on rejected drafts, but its outcome rests on the judge, and a judge that reads only a step's text cannot tell whether the step fits the screen it acts on. We find that a few samples of a small model can contain a correct step about as often as the large model gets the step right, and not always on the same steps: checked against the screen, small drafts can be not only cheaper but better. We present Vanguard, a training-free judge that leaves the draft and target models unchanged: a general decision model, with the screenshot in view, gives each candidate step a probability of being right in one forward pass; the best candidate is kept if it clears a threshold, and otherwise the large model writes the step. On web, mobile and embodied agents, a 2B draft judged by Vanguard is at least as accurate as its 8B target and significantly more accurate on Mind2Web and EB-ALFRED, while the target writes only a minority of the steps; it completes far more closed-loop episodes, and with a 27B target from another family it is both faster and more accurate. Vanguard is the most accurate judge we compare on the agent tasks; it relies on the screenshot on web and mobile tasks, its calibrated threshold transfers to unseen websites, and across model families its gain over the target grows with the draft's pass@4 relative to the target's accuracy. Vanguard composes with token-level speculative decoding and matches trained process reward models on mathematics, turning speculation from a way to save the large model's work into one that can improve on it.
Figures & tables
Figure 1: Vanguard keeps a drafted step only when a decision model, looking at the screenshot, judges it right. The small draft proposes K=4 candidate steps; a general decision model scores each action against the screenshot in one forward pass; the best is kept if it clears τ , otherwise the target writes the step.
Figure 2: With Vanguard, the 2B draft is at least as accurate as its 8B target on every agent task, and far more accurate in closed loop. Step success on AndroidControl (525 steps) and EB-ALFRED (1,849); episode success on 100 test episodes per BabyAI task; higher is better. Brackets: comparison with the target alone (*: significant gain); inside: share of target-written steps.
test_task
test_website
Method
Success ↑
8B steps ↓
Success ↑
8B steps ↓
Draft model alone (2B)
42.4
0
43.8
0
Target model alone (8B)
64.4
100
63.3
100
Random of four
45.0
0
42.6
0
Vote gate
56.1
33
56.9
37
Qwen-VL-PRM-7B
63.0
36
61.0
39
Table 1: With Vanguard, the 2B draft leads every method in this table on both held-out Mind2Web test sets. Step success (%) with four draft candidates and 8B fallback on 820 / 975 held-out steps; 8B steps: share written by the 8B (23% on development at τ=0.5 ); thresholds fixed on development data, each baseline at the share closest to Vanguard’s, the PRM at its single development threshold; significance tests: Secs. D.1 and D.16 .
Figure 3: Vanguard lies above the target model alone on Mind2Web and above the target asked the same question on AndroidControl. Step success against the share of target-written steps on test steps (higher and further left is better): Mind2Web (both test sets, development thresholds) and AndroidControl (development sweep; Vanguard from 19% target-written steps).
Figure 4: Vanguard keeps a draft when the screen confirms it and otherwise hands the step to the target. (a) A held-out Mind2Web step, unseen website: D separates the right drafts from the wrong ones and the step is kept; the 8B alone types into a later field. (b) One closed-loop BabyAI episode: navy, steps kept from the 2B draft; peach, steps written by the 8B (for Vanguard, after a rejection); ×n : actions in that cell; the 8B alone fails after 32 actions, and Vanguard reaches the key in 11. Both cases are from Figs. 20 a and 21 a.
Figure 5: Vanguard’s gain over the target grows with the draft’s headroom. Each point is one draft/target pair on one agent task, all sixteen; x : headroom (draft pass@4 minus target accuracy), y : Vanguard minus target (higher is better); filled: significant gain; dashed: least-squares fit, Pearson r=0.95 ( Sec. D.8 ).
Draft → target
Task
Target ↑
Vanguard ↑
Qwen2.5-VL 3B → 7B
Mind2Web
50.5
58.6
EB-ALFRED
67.9
76.3
Qwen3-VL-2B →
Mind2Web
64.1
66.6
Gemma-3-27B
AndroidControl
34.1
45.9
EB-ALFRED
76.6
78.3
Gemma-3 12B → 27B
Mind2Web
64.1
69.1
Table 2: In these further pairs, also across families, Vanguard lifts the draft significantly above the target. Step success (%) on the test steps (Mind2Web: both test sets), at each pair’s development threshold; every gain is significant (largest p=0.025 ; tests and further pairs: Secs. D.5 and D.6 ).
Figure 6: Vanguard is ahead of every other judge on both tasks, except for a tie with the 32B asked the same question on Mind2Web. Higher is better. Every judge scores the same candidates at a development threshold sending about the same share of steps to the 8B (full table: Tab. 5 ).
Figure 7: Selection and fallback are both needed, the judge must see this step’s screenshot, and its probabilities are the best calibrated. Test steps; higher step success and lower calibration error are better; gated and image variants at about the same share of target-written steps as Vanguard; right: calibration error on Mind2Web. Full tables: Tabs. 6 , 7 and 8 .
Figure 8: Vanguard is faster and more accurate than its target; without EAGLE-3, it saves the most time with a 27B target from another family. Shaded: faster and more accurate than the target. Same steps per method and target (8B: 300 Mind2Web steps per test set, τ=0.41 on test_task; Gemma-3-27B: 300, 200 and 300 steps of Mind2Web, AndroidControl and EB-ALFRED, Vanguard 67.7, 45.0 and 83.7% against 65.0, 36.0 and 78.3%); hollow diamonds: D at its nominal latency; bottom: token-level methods without training (50 and 200 Mind2Web steps).
Appendix figures & tables47 assets
Supplementary material from the paper’s appendix.
Appendix
Judge
No fallback
With fallback
Draft log-probability
29.7
33.1 (39)
Vote gate
30.1
47.6 (93)
Qwen3-VL-8B, same prompt
39.6
46.9 (42)
Vanguard
44.4
51.0 (19)
Draft alone 32.4 Random of four 32.4 Target alone 48.8
Appendix
Table 3: On AndroidControl, Vanguard is the most accurate judge, with and without fallback. AndroidControl step success (%) on 525 test steps with four candidates per step, without fallback (the judge must pick a candidate) and with 8B fallback; the share of target-written steps (%) is in parentheses. Vanguard uses its development threshold; each baseline is shown at its most accurate test point, which favours the baselines. Without fallback, Vanguard beats the 8B asked the same question on 35 steps against 10 ( p=0.0002 ).
EB-ALFRED
GoToLocal
PickupLoc
Method
Succ.
Target
Succ.
Target
Succ.
Target
Draft alone (2B)
60.6
0
37
0
16
0
Target alone (8B)
74.2
100
61
100
36
100
8B, same prompt
74.7
66
63
90
53
83
Vanguard
77.4
28
86
25
65
25
Appendix
Table 4: On the embodied tasks, Vanguard surpasses the 8B significantly while the 8B writes about a quarter of the steps or decisions. EB-ALFRED step success (%) on 1,849 test steps, and closed-loop episode success (%) on 100 test episodes of two BabyAI tasks; “Target” is the share (%) of steps or decisions written by the 8B, all thresholds are chosen on development data (Vanguard: 0.451 on EB-ALFRED, 0.3 on BabyAI). Vanguard against the 8B alone: 132/73 steps on EB-ALFRED ( p=5×10−5 ), 27/2 episodes on GoToLocal and 34/5 on PickupLoc ( p=2×10−6 for both).
Judge
Mind2Web
AndroidControl
Draft model alone (2B)
43.2
32.4
Target model alone (8B)
63.8
48.8
Qwen3-VL-2B, self-check
55.0
41.7
Qwen3-VL-8B, 0–9 grade
62.8
44.2
Qwen3-VL-8B, same prompt
65.3
44.0
Qwen3-VL-32B, same prompt
66.6
49.0
Appendix
Table 5: Vanguard is ahead of every other judge on both tasks, except for a tie with the 32B asked the same question on Mind2Web. Step success (%) of the same pipeline with different judges, each scoring the same candidates, with thresholds chosen on development data so that every judge sends about the same share of steps to the 8B as Vanguard (Mind2Web: both test sets, 1,795 steps; AndroidControl: 525 test steps). Against Vanguard, the 2B self-check wins/loses 84/293 steps and the 0–9 grade 69/137 on Mind2Web ( p<10−5 for both), and the 8B asked the same question wins/loses 14/51 on AndroidControl ( p=4×10−6 ).
K
Selection
Gate
Mind2Web
AndroidControl
1
–
×
43.2
32.4
4
random
×
43.7
32.4
4
✓
×
57.4
44.4
1
–
✓
58.5
41.1
4
✓
random
58.9
45.1
4
✓
✓
66.6
51.0
Appendix
Table 6: Only selection and fallback together lift the draft above the target. Step success (%) on the test steps. “Selection” picks the candidate with the highest probability from D ; the fallback gate hands the step to the 8B when that probability is below τ . Rows with a gate send the same share of steps to the 8B as Vanguard.
Mind2Web
AndroidControl
Image given to the judge
Success ↑
AUROC ↑
Success ↑
AUROC ↑
No screenshot
63.1
0.764
45.1
0.529
Another task’s screenshot
60.8
0.673
35.0
0.544
This step’s screenshot
66.6
0.807
51.0
0.714
Appendix
Table 7: The judge must see the screenshot. Step success (%) and AUROC of the gate on the test steps. Each variant rescores the same candidates, with its threshold chosen on development data to send about the same share of steps to the 8B. The correct screenshot beats no screenshot on 108 steps against 45 on Mind2Web ( p=4×10−7 ) and on 57 against 26 on AndroidControl ( p=0.0009 ); with another task’s screenshot, the pipeline falls below the 8B alone, which wins 207 steps against 154 on Mind2Web ( p=0.006 ) and 99 against 27 on AndroidControl ( p=7×10−11 ).
Judge
Expected calibration error ↓
Qwen3-VL-8B, same prompt
0.282
Qwen-VL-PRM-7B
0.272
Vanguard
0.106
Appendix
Table 8: Vanguard’s scores are far better calibrated. Expected calibration error of the selected candidate’s score on Mind2Web (ten bins).
Method
M2W task
M2W web
Target model (8B)
65.7 / 1.00
58.0 / 1.00
8B + EAGLE-3
67.0 / 1.55
59.7 / 1.61
Vanguard
71.3 / 1.17
64.0 / 1.15
+ EAGLE-3 on both
70.0 / 1.50
62.0 / 1.52
projected, D at 0.16 s
70.0 / 1.86
62.0 / 1.84
Appendix
Table 9: Vanguard is the most accurate on both Mind2Web subsets, and with EAGLE-3 it is nearly as fast as EAGLE-3 alone. Step success (%) / mean speedup over the 8B, measured end to end at batch size 1 on the same test steps (300 each). EAGLE-3 uses the public Qwen3-VL heads with five speculative tokens and is exact only up to floating-point rounding, so the 8B’s step success with it differs slightly from that of the plain 8B. Bold: most accurate. The last row (italic) is projected, with the measured API latency of D (0.4–0.6 s per step) replaced by its nominal 0.16 s. Thresholds: 0.41 on test_task (an operating point of the development sweep), 0.5 on test_website. AndroidControl timings: Tabs. 20 and 21 .
Method
Element
Task
Target model alone (8B)
74.1
18.0
Vanguard
78.4
19.9
Appendix
Table 10: Vanguard selects the right element more often and completes more tasks than the 8B. Standard Mind2Web metrics (%) over both test sets: element accuracy and task success, which counts tasks whose every evaluated step is correct.
Figure 9: On all three agent test sets, Vanguard reaches the highest step success of any configuration shown, above the 8B alone, while the 8B writes at most a quarter of the steps. Higher and further left is better. Step success against the share of target-written steps for the four-candidate pipelines with fallback, one line per judge through every threshold of its sweep (points at 0%: no fallback), on Mind2Web test_task (820 steps), Mind2Web test_website (975 steps) and AndroidControl (525 steps). On AndroidControl, the dashed light-navy line is Vanguard with one draft per step. Dashed horizontal lines: the 8B target alone and the 2B draft alone. Full numbers: Tabs. 11 , 12 and 13 .
Configuration
τ
Step success (%) ↑
Target steps (%) ↓
Wins/losses vs. 8B ( p )
Draft model alone (2B)
–
42.4
0
48/228, < 0.001
Target model alone (8B)
–
64.4
100
–
Four candidates, random
none
45.0
0.0
60/219, < 0.001
Four candidates, vote gate
none
47.2
0.0
58/199, < 0.001
Four candidates, vote gate
0.75
56.1
32.6
37/105, < 0.001
Four candidates, log-probability
none
43.2
0.0
46/220, < 0.001
Appendix
Table 11: On test_task, Vanguard is the most accurate configuration, above the 8B alone at both of its thresholds. Mind2Web with reasoning, test_task test part (820 steps); thresholds fixed on the development part.
Configuration
τ
Step success (%) ↑
Target steps (%) ↓
Wins/losses vs. 8B ( p )
Draft model alone (2B)
–
43.8
0
73/263, < 0.001
Target model alone (8B)
–
63.3
100
–
Four candidates, random
none
42.6
0.0
88/290, < 0.001
Four candidates, vote gate
none
46.3
0.0
85/251, < 0.001
Four candidates, vote gate
0.75
56.9
36.6
56/118, < 0.001
Four candidates, log-probability
none
43.1
0.0
83/280, < 0.001
Appendix
Table 12: On the unseen websites, Vanguard is the most accurate configuration, significantly above the 8B alone at τ=0.5 . Mind2Web with reasoning, test_website (975 steps, unseen websites), with the thresholds of test_task.
Configuration
τ
Step success (%) ↑
Target steps (%) ↓
Speedup over 8B ↑
Wins/losses vs. 8B ( p )
Draft model alone (2B)
–
32.4
0
2.16 ×
22/108, < 0.01
Target model alone (8B)
–
48.8
100
1.00 ×
–
Four candidates, random
none
32.4
0
1.59 ×
38/124, < 0.01
Four candidates, vote gate
none
30.1
0
1.59 ×
29/127, < 0.01
Four candidates, vote gate
0.75
44.4
82
0.69 ×
5/28, < 0.01
Four candidates, vote gate
1.0
47.6
93
0.64 ×
3/9, 0.15
Appendix
Table 13: On AndroidControl, Vanguard is the most accurate configuration and stays above the 8B alone at every threshold that sends 19–36% of the steps to it. AndroidControl with reasoning, test part (525 steps), full threshold sweeps. Speedups in this table are estimated over all 525 steps from the per-component times of Tab. 20 ( cK+cD+(1−ατ)cp against cp ); the end-to-end measurement on the timed subset of these steps is 1.06 × at τ=0.5 ( Tab. 21 ). † : speedup with the decision model at its nominal latency of 0.16 s instead of the measured latency.
Qwen3-VL (2B, 8B)
Qwen2.5-VL (3B, 7B)
Method
Success
Target
Success
Target
Draft model alone
60.6
0
69.7
0
Target model alone
74.2
100
67.9
100
Oracle (pass@4)
76.3
0
83.2
0
Random of four
61.8
0
62.5
0
Vote gate
72.7
39
71.3
27
Appendix
Table 14: With both families, Vanguard beats the target on EB-ALFRED while the target writes about a quarter of the steps. Step success (%) on the 1,849 test steps with four draft candidates per step and fallback to the target model at thresholds fixed on development data; “Target” is the share of target-written steps (%), and each judge is shown at its development-chosen threshold. Vanguard against the target model alone: 132/73 steps ( p=5×10−5 ) with Qwen3-VL and 213/58 ( p=5×10−22 ) with Qwen2.5-VL. With Qwen3-VL, the gate AUROC of the selected candidate’s score is 0.859 for Vanguard, 0.631 for the draft’s self-check and 0.666 for the target asked the same question; with thresholds matched on development data to the share of Vanguard, those two judges reach 66.1 (22% target-written) and 70.8 (30%).
GoToLocal
PickupLoc
Method
Success
Steps
Success
Steps
Draft model alone (2B)
37
33.6
16
40.1
Target model alone (8B)
61
22.7
36
32.0
Target model, same prompt
63
19.4
53
25.6
Vanguard
86
13.1
65
20.7
Appendix
Table 15: In closed loop, Vanguard completes far more episodes than the 8B while the 8B writes a quarter of the decisions. Episode success (%) and mean number of environment steps on 100 test episodes (seeds 30–129) of two BabyAI tasks; thresholds chosen on 30 development episodes ( τ=0.3 for both judges on both tasks). Vanguard against the 8B alone: 27/2 episodes on GoToLocal ( p=2×10−6 ) and 34/5 on PickupLoc ( p=2×10−6 ), with the 8B writing 25% of the decisions on both. The 8B asked the same question as a judge hands 83–90% of the decisions back to the 8B.
Method
Success
Steps
8B steps
s / step
s / episode
TFLOP / step
Draft model alone (2B)
12.1
16.5
0%
1.57 / 1.20
25.8
5.6
Target model alone (8B)
28.0
13.9
100%
2.99 / 2.76
41.4
21.6
Vanguard
29.3
14.9
29%
2.79 / 1.83
41.7
13.7
Appendix
Table 16: On AndroidWorld, Vanguard matches the 8B while the 8B writes 29% of the steps. Task success (%) on all 116 tasks (one parameter draw each, identical across methods), mean steps per episode, and the share of steps written by the 8B. Time per step (mean / median) and per episode is model and judge time estimated from the logged tokens (see text); compute counts 2× parameters × tokens of the two models. Vanguard against the 8B alone: 12/11 tasks; the 2B alone against the 8B: 6/25 ( p=9×10−4 ).
Mind2Web
EB-ALFRED
Method
Success
Target
Success
Target
Qwen2.5-VL [ 1 ] , 3B draft, 7B target
Draft alone
39.6
0
69.7
0
Target alone
50.5
100
67.9
100
Oracle (pass@4)
59.4
0
83.2
0
Target, same prompt
53.2
43
70.3
78
Appendix
Table 17: The gain carries over to two more model families, most of all to Qwen2.5-VL, whose draft has the larger headroom. Step success (%) and share of target-written steps (%) with two other model families, same prompts, splits and protocol as the main text (Mind2Web: both test sets pooled, 1,795 steps; EB-ALFRED: 1,849); thresholds chosen on development data, the target asked the same question at its development-chosen threshold. Vanguard against the target alone, Qwen2.5-VL: 275/130 ( p=5×10−13 ) and 213/58 ( p=5×10−22 ).
Target
Alone
Oracle (2B pass@4)
Vanguard
W/L, p
Gemma-3-27B
Mind2Web
64.1
62.8
66.6 (24)
169/123, 0.008
AndroidControl
34.1
52.8
45.9 (6)
125/63, 7×10−6
EB-ALFRED
76.6
76.3
78.3 (32)
112/80, 0.025
Gemma-3-12B
Mind2Web test_task
59.6
62.3
64.0 (26)
78/42, 0.001
Appendix
Table 18: With a Gemma-3 target, Vanguard significantly surpasses the target on every benchmark. The Qwen3-VL-2B draft and Vanguard with a target from another family (step success, %; “Alone”: the target model alone; share of target-written steps, %, in parentheses; W/L and p against the target alone). The draft’s four samples and their scores are those of the main experiments; only the target changes, with the same prompts and splits and a threshold chosen on each pair’s development data (Mind2Web: both test sets, 1,795 steps, except Gemma-3-12B, which we ran on test_task only).
Figure 10: With four further draft–target pairs, including Gemma-3 targets from another family, Vanguard is above the target alone in every pair and on every benchmark shown. Higher is better. Step success (%) per (draft → target pair, benchmark): the draft alone (white square), the target alone (peach), the target asked the same question (pink), the draft’s pass@4 (grey ring) and Vanguard (navy, value printed); the bar joins the target alone and Vanguard. Mind2Web pools both test sets except for Gemma-3-12B (test_task). ∗ : Vanguard significantly above the target alone. Full numbers: Tabs. 17 and 18 ; Gemma-3-27B asked the same question: Sec. D.6 .
Family
Benchmark
Vanguard
PRM
Target, same q.
Qwen3-VL
Mind2Web
66.2 (31)
61.2
65.8
AndroidControl
50.7 (28)
30.9
45.9
EB-ALFRED
77.4 (28)
69.6
70.8
Qwen3.5
Mind2Web
66.5 (19)
57.8
62.8
AndroidControl
43.8 (30)
26.9
41.3
EB-ALFRED
73.2 (40)
63.7
70.5
Appendix
Table 19: Vanguard is the most accurate judge in every (family, benchmark) cell. Three judges of the same draft candidates in each (family, benchmark) cell we report (step success, %, on the test steps). Bold: the most accurate judge in the cell. Vanguard uses its threshold chosen on development data (share of target-written steps, %, in parentheses); the vision-language process reward model Qwen-VL-PRM-7B and the target model asked the same question use thresholds rate-matched on development data to that share. Vanguard is significantly more accurate than the PRM in all nine cells ( p≤4×10−7 ) and than the target asked the same question in seven ( p≤0.002 ). For Qwen3-VL, this table chooses every threshold from its own development sweep, so the operating points of Vanguard differ from those of the main text, which uses τ=0.5 on both tasks: 66.2% at 31% target-written steps instead of 66.6% at 24% on Mind2Web, and 50.7% at 28% instead of 51.0% at 19% on AndroidControl.
Figure 11: In all nine (family, benchmark) cells, Vanguard is the most accurate of the three judges. Higher is better. Step success (%) of the pipeline with Vanguard (navy, value printed), the process reward model Qwen-VL-PRM-7B (yellow square) or the target model asked the same question (pink triangle) as the judge. Full numbers: Tab. 19 .
Time per step (s)
Mind2Web
AndroidControl
Four draft candidates, cK
1.26
1.57
Decision model, cD
0.56
0.42
Target step, cp
2.98
2.50
cK+cD
1.82
1.99
ατcp
2.26
2.02
Appendix
Table 20: Drafting and judging cost less than the target steps that Vanguard saves. Measured components of the cost condition cK+cD<ατcp of the main text for Vanguard at τ=0.5 (batch size 1, decision model through its public API). ατ is the share of accepted steps (76% on Mind2Web, 81% on AndroidControl); cD is the time added by judging the candidates.
Setting (target 8B unless stated)
Predicted
Measured
Mind2Web test_task
1.21
1.17
Mind2Web test_website
1.19
1.15
AndroidControl
1.06
1.06
Mind2Web test_task, EAGLE-3 on the 8B
1.32
1.28
Mind2Web test_task, EAGLE-3 on both
1.54
1.50
Mind2Web test_website, EAGLE-3 on both
1.55
1.52
Appendix
Table 21: Three measured times and the fallback rate predict the measured speedup. Mean speedup over the plain target model, predicted from per-component times as cp/(cK+cD+(1−ατ)c~p) , where c~p is the time of a fallback step (equal to cp unless EAGLE-3 runs on the target), against the speedup measured end to end at batch size 1 on the same steps. The prediction uses the component times and fallback rate of each timed run itself, so it can differ slightly from the estimates of Tab. 20 and the full-sweep tables, which combine component times measured separately on other steps.
Figure 12: Only Vanguard reaches the corner that is both faster and more accurate than the 8B alone. Up and to the right is better. Step success against speedup over the 8B (log scale) on the AndroidControl test part (525 steps) for every configuration of the full threshold sweeps, coloured by judge. Circles and solid lines: four candidates; squares and dashed lines: one draft; each line joins the thresholds of one judge. Hollow navy markers on dotted lines: Vanguard with the decision model at its nominal latency of 0.16 s. Numbers next to Vanguard’s points: τ (none: no fallback). Shaded: faster and more accurate than the 8B alone. The step-success axis is broken, with a larger scale above the break. Speedups are the per-component estimates of Tab. 13 over all 525 steps; the end-to-end measurement at τ=0.5 is 1.06 × ( Tab. 21 ). Full numbers: Tab. 13 .
Figure 13: In all nine settings, the predicted speedup is within 0.04 × of the measured one. Speedup over the plain target model predicted from the component times and the fallback rate, against the measured speedup; dashed line: predicted = measured. Circles: Vanguard; squares: with EAGLE-3 on the 8B; diamonds: with EAGLE-3 on both the draft and the target. M2W task / web: Mind2Web test_task / test_website; AC: AndroidControl. Full numbers: Tab. 21 .
Figure 14: More candidates raise accuracy on Mind2Web while fewer steps go to the target. Higher is better. Step success of Vanguard with the first K of the four draft candidates. On Mind2Web, the share of target-written steps falls from 40% at K=1 to 31% and 24% at K=2 and 4 .
Vanguard, K=4
K=8
Test set
Target
Seed 0
Seed 1
Seed 2
Success (Target)
Mind2Web test_task
64.4
66.5
66.3
66.6
68.3 (15)
Mind2Web test_website
63.3
66.8
67.2
67.1
68.6 (18)
AndroidControl
48.8
51.0
49.1
49.3
50.7 (12)
Appendix
Table 22: Vanguard stays above the target with every draw of the draft’s samples, and eight candidates raise it further on Mind2Web. Step success (%) of Vanguard at τ=0.5 ( τ=0.41 on test_task, the threshold of the speed measurements; in this table a tie in D goes to the first candidate) when the draft model’s four samples are redrawn with another seed, and with eight candidates per step (the samples of seeds 0 and 1 pooled; share of target-written steps, %, in parentheses). With K=8 , Vanguard beats the 8B on 80/48 steps on test_task ( p=0.006 ) and 111/59 on test_website ( p=8×10−5 ); the oracle over the candidates rises from 62.3 to 69.8 and from 63.3 to 70.5.
Figure 15: With every draw of the draft’s samples and with eight candidates, Vanguard stays above the target alone on all three test sets. Higher is better. Step success (%) of the 8B target alone (peach tick and dashed line) and of Vanguard with K=4 candidates for sampling seeds 0, 1 and 2 (dots, left to right) and with K=8 (diamond, labelled with the share of target-written steps). On both Mind2Web test sets, eight candidates are above every seed of K=4 . Each panel has its own y-axis range. Full numbers: Tab. 22 .
Input of D
Success ↑
vs. Vanguard
AUROC ↑
Vanguard (screen, history, action)
66.6
–
0.807
No screenshot
63.1
45/108
0.764
Screenshot of another task
60.8
57/161
0.673
No previous actions
64.6
17/54
0.766
With the draft’s reasoning
62.5
45/119
0.756
Wording of the question
Appendix
Table 23: The judge needs the screenshot and the history, and only the action. Mind2Web, both test sets, 1,795 steps. Each variant rescores the same candidates with one input or the question changed; its threshold is chosen on development data to send the same share of steps to the 8B as Vanguard (22–24% on test). “vs. Vanguard” gives steps won/lost against Vanguard; removing or replacing any input, adding the reasoning and negating the question are all significantly worse ( p≤5×10−4 ). AUROC is that of the gate on the selected candidate. Negated question: “Is this action a mistake?”
Figure 16: The judge is most accurate and best at separating right from wrong candidates when it sees the screen, the history and only the action. Higher step success and higher AUROC are better. Mind2Web, both test sets (1,795 steps). Left: step success (%) when one input of D is removed, replaced or added, or the question is negated; dashed line: the 8B target alone. Middle: steps won/lost against Vanguard. Right: AUROC of the gate on the selected candidate. Full numbers: Tab. 23 .
Mind2Web
AndroidControl
EB-ALFRED
Steps accepted
76
81
72
Accepted: draft / target right
71.6 / 67.9
51.3 / 48.5
86.5 / 82.1
Rejected: draft / target right
12.7 / 50.9
15.0 / 50.0
30.2 / 53.7
Fallbacks
432
100
514
target fixes a wrong draft
189 (71)
40 (14)
169 (67)
target replaces a right draft
55 (248)
15 (45)
155 (364)
Appendix
Table 24: The gate keeps the drafts that beat the target and sends it the steps it can fix. Decisions of Vanguard on the test steps (Qwen3-VL 2B → 8B). Percentages except for the counts of fallbacks; “draft” is the candidate selected by Vanguard on that step, “target” the 8B’s own action on the same step. In parentheses: the expected counts for a gate that falls back at random at the same rate.
Figure 17: Accepted drafts that are right form the largest group on every task. Where each test step ends up under Vanguard: accepted (right or wrong), or handed to the 8B, which fixes a wrong draft, rejects a draft that was right, or is wrong as well.
Figure 18: Vanguard spreads its scores and separates right from wrong candidates, whereas the 8B’s scores pile up at 0 and 1. Scores of correct and wrong candidates on the test steps. Top: Vanguard; bottom: the 8B asked the same question. The 8B’s probabilities are almost always near 0 or 1, so a threshold on them either accepts or rejects nearly everything.
Steps written by the target model
Judge
≈ 2%
≈ 12%
VisualPRM-8B
63.2 (2.5)
63.4 (13.4)
Qwen3-VL-8B, same prompt
64.5 (2.1)
66.7 (12.0)
Qwen-VL-PRM-7B
67.6 (2.5)
69.5 (15.1)
Vanguard
67.9 (1.5)
69.4 (11.9)
Draft alone 58.5 Random of four 60.4
Appendix
Table 25: On MathVista, Vanguard is on par with Qwen-VL-PRM-7B and ahead of VisualPRM-8B and the 8B asked the same question. Accuracy (%) on 800 held-out testmini problems with four candidates per step and 8B fallback; the exact share of target-written steps (%) is given in parentheses. With Holm correction over the six comparisons of this table, Vanguard is significantly more accurate than VisualPRM at both shares ( p≤0.004 ) and than the 8B asked the same question at the lower share ( p=0.020 ).
ProcessBench
F1, held-out subsets
Qwen3-8B, same prompt
49.9
Qwen3-32B, same prompt
61.0
Qwen2.5-Math-PRM-7B
63.0
Vanguard
66.6
MATH-500 test, best of four
Accuracy
Qwen3-8B as judge
79.0
Appendix
Table 26: On text reasoning, Vanguard is ahead of the general models asked the same question and of the trained math process reward model. Top: ProcessBench F1 for locating the first erroneous step, averaged over MATH, OlympiadBench and Omni-MATH, with every threshold chosen on GSM8K and every general model asked the correct question. Bottom: accuracy (%) on the 400 test problems of MATH-500 when the judge picks one of four complete draft solutions (draft alone 73.8); Vanguard beats the math process reward model on 15 problems against 3 ( p=0.008 ).
Judge
τ
GSM8K
MATH
Olymp.
Omni
Held-out
Qwen3-8B, same (correct)
0.08
55.8
56.2
46.9
46.6
49.9
Qwen3-8B, same (compute)
0.51
56.2
54.2
40.8
35.8
43.6
Qwen3-8B, same (error)
0.98
52.4
37.1
36.4
37.5
37.0
Qwen3-32B, same (correct)
0.51
74.4
67.2
59.0
56.9
61.0
Qwen3-32B, same (compute)
0.57
73.9
64.0
54.0
53.5
57.2
Qwen3-32B, same (error)
0.05
71.8
52.9
34.1
29.7
38.9
Appendix
Table 27: With either text-reasoning question, Vanguard is ahead of every general model asked the same question on the held-out subsets. ProcessBench F1 per subset for every check question we tried ( correct , compute and a third question, error ); Tab. 26 reports the correct question for every general model and, for Vanguard, the better of the two text-reasoning questions of the main text ( correct and compute ) on GSM8K, where thresholds are chosen; with either of these two questions, Vanguard is ahead of every other judge whose threshold is chosen on GSM8K on the held-out average. “Held-out” averages MATH, OlympiadBench and Omni-MATH.
Method
Acc.
Median
Mean
8B alone
78
1.0 ×
1.0 ×
Vanguard ( τ=0.7 )
80
2.0 ×
1.2 ×
Appendix
Table 28: On MATH-500, Vanguard matches the accuracy of the 8B and is faster. Accuracy (%) and measured speedups on 50 MATH-500 development problems, one request at a time.
Task (main-text item)
Vanguard against
n
W/L
p
Mind2Web test_task ( Tab. 1 )
Qwen-VL-PRM-7B
820
70/42
0.010
Vote gate
820
117/32
1×10−12
Random of four
820
201/25
2×10−35
Draft model alone
820
224/27
8×10−40
Mind2Web test_website ( Tab. 1 )
Qwen-VL-PRM-7B
975
104/48
6.5×10−6
Vote gate
975
133/37
6.5×10−14
Appendix
Table 29: Every comparison of the main text tested here is significant. Exact two-sided McNemar tests for the remaining comparisons of the main text (W/L: steps or episodes on which Vanguard is right and the other method wrong, and the reverse). Vanguard at τ=0.5 on the agent tasks ( τ=0.451 on EB-ALFRED, 0.3 in closed loop) unless stated otherwise; every other method at the operating point drawn in the main text (in parentheses: its step success, %).
Figure 19: D keeps a right draft on steps where the 8B alone is wrong. (a) Mind2Web, unseen website: the step selects the processor; two drafts choose the right one (G, Intel Core i3) and two a wrong one (H), which D scores 0.97 and 0.02, while the 8B skips ahead to the memory field (B); letters: Set-of-Mark tags as shown to the models. (b) EB-ALFRED: the robot holds a knife and has found the tomato; the 8B tries to pick the tomato up, which is impossible while holding the knife, whereas all four drafts slice it ( D 0.92, τ=0.451 ).
Figure 21: In closed loop, Vanguard reaches the goal where the 8B alone gets stuck. Closed-loop episodes on the same seeds, with each method’s path drawn on the start frame (navy: steps kept from the 2B draft; peach: steps written by the 8B, for Vanguard after a rejection; ×n : number of actions taken in that cell). The 8B alone gets stuck and runs out of steps; Vanguard reaches the goal in 11 and 9 actions, with one decision each written by the 8B.
Figure 22: On unseen websites, D picks out the right draft even where the 8B target alone goes wrong. Mind2Web test_website, draft Qwen3-VL-2B, target Qwen3-VL-8B, τ=0.5 . A step is shown if the four drafts propose at least two different actions (drafts with the same element, operation and value count as one; unparseable drafts are not counted), Vanguard keeps a right draft, and the target alone is wrong; we show the first such step of each of the first 3 qualifying tasks in test order, out of 47 qualifying tasks (65 steps; the task of Fig. 20 is not counted). Left: the model’s input view at native resolution with its Set-of-Mark boxes, cropped around the chosen elements. Right: the four drafts with their D scores (tick: right, cross: wrong), the 8B target alone, the gold action and D ’s decision.
Figure 23: On held-out AndroidControl steps where the 8B alone is wrong, D singles out a right 2B draft among the drafts’ different proposals, and Vanguard keeps it. We select steps by a fixed rule: the four drafts propose at least two different actions (clicks within the success radius count as one), Vanguard keeps a right draft, and the target alone is wrong; we show the first such step of each of the first 3 qualifying episodes in test order, out of 20 qualifying episodes (30 steps; episode 16628 of Fig. 27 is not counted). Under each screen: the goal, the last previous step instructions (…: earlier ones), the annotator’s instruction for this step (not shown to the models), the four drafts with their scores under D (bold navy: kept; ties go to the higher mean token log-probability), the 8B alone and the gold action, with clicks on a 0–1000 scale; tick and cross marks: right and wrong. On the screens, rings mark the drafts’ clicks, a peach × the 8B alone and a dark grey + the gold click; in the middle column, drafts 1 and 4 click 9 units apart and the outer grey ring is draft 1. τ=0.5 .
Figure 24: On held-out EB-ALFRED steps, D finds the right action even when only one or two of the four 2B drafts propose it, so Vanguard takes the right step where the 8B target alone does not. A test step qualifies if the four drafts propose at least two different actions (drafts that cannot be parsed are not counted), Vanguard keeps a right draft, and the target alone is wrong; we show the first such step of each of the first 3 qualifying episodes in test order, out of 48 qualifying episodes (67 steps; the two episodes shown in other figures are not counted). Left: the egocentric frame the models see. Right: the actions executed so far, the eight options, each draft’s choice with its D score, the 8B alone and the gold action; ticks and crosses mark right and wrong actions, and τ=0.451 is chosen on the dev split.
Figure 25: With a target from another family, D still keeps a right Qwen3-VL-2B draft at steps where Gemma-3-27B alone is wrong. Cases follow a fixed rule: a test step qualifies when the four drafts propose at least two different actions (clicks within the success radius count as one), Vanguard keeps a right draft, and the target alone is wrong; in each cell we show the first such step of the first qualifying episode in test order that no other figure shows, out of 97 qualifying tasks (130 steps) on Mind2Web, 45 qualifying episodes (71 steps) on AndroidControl and 45 qualifying episodes (59 steps) on EB-ALFRED. Each row lists the four drafts with their D scores (✓ right, × wrong; the kept draft in bold navy; ties in D go to the draft with the higher mean log-probability), the action of Gemma-3-27B alone, the gold action, and D ’s decision against the pair’s dev-chosen threshold τ . Mind2Web screenshots carry the Set-of-Mark boxes and letters shown to the models. On AndroidControl, rings mark the drafts’ clicks, × the click of Gemma-3-27B alone, + the gold click and the dashed outline the success radius (normalised distance 0.14); coordinates are on a 0–1000 scale, and the gold is printed with its annotated step instruction.
Figure 26: Within other model families too, D keeps a right draft where the larger target alone is wrong, for Qwen2.5-VL-3B → 7B and Gemma-3-12B → 27B. Cases follow the same fixed rule: a test step qualifies when the four drafts propose at least two different actions (clicks within the success radius count as one), Vanguard keeps a right draft, and the target alone is wrong; in each cell we show the first such step of the first qualifying episode in test order that no other figure shows, out of (a) 127 qualifying tasks (215 steps), (b) 84 qualifying episodes (137 steps), (c) 94 qualifying tasks (126 steps) and (d) 28 qualifying episodes (36 steps). Rows as in Fig. 25 (✓ right, × wrong; kept draft in bold navy; ties in D go to the higher mean log-probability; τ is each pair’s dev-chosen threshold). In (d), rings are the drafts’ clicks, × the click of Gemma-3-27B alone, + the gold click and the dashed outline the success radius (normalised distance 0.14).
Figure 27: On a held-out AndroidControl episode, Vanguard matches the annotated action on 6 of the 7 steps and the 8B alone on 3. Steps 4 to 10 of test episode 16628 (goal: “I want to send my friend a destination file via Gmail at [email protected] in Files by Google App because he is in need of that file.”), each with the step’s instruction and the scores of its four drafts under D (bold navy: kept). On the screens, rings mark the drafts’ clicks, a peach ring the click the 8B writes after every draft scored below τ=0.5 (peach frame), a peach × the 8B alone and a dark grey + the annotated click. On steps 4, 5 and 8 the kept draft is right and the 8B alone is wrong; on step 9 D keeps a wrong draft (0.71) and the 8B alone is wrong as well; on step 10 every draft scores at most 0.41 and the 8B writes the step correctly. Selection: of the 98 held-out episodes, this one has the most steps on which Vanguard is right and the 8B alone is wrong (4, at steps 4, 5, 8 and 11; no other episode has more than 3), and we show the seven steps from the first of them on.
Figure 28: On a held-out EB-ALFRED episode, Vanguard matches the gold action on 12 of the 21 steps and the 8B alone on 6. All 21 steps of test episode 38 of the complex-instruction split (Qwen3-VL-2B → Qwen3-VL-8B, τ=0.451 ), steps 1 to 10 on this page and 11 to 21 on the next. Top: the task and, for every step, whether Vanguard and the 8B alone are right (✓) or wrong ( × ); shaded: Vanguard right and the 8B alone wrong. Each step shows the egocentric frame the agent saw, the gold action (from the dataset’s shortest successful trajectory, which our evaluation follows), the distinct actions of the four drafts with their scores under D and how many drafts propose each (bold navy: kept; –: a draft that cannot be parsed; ties in D go to the higher mean token log-probability), and the action of the 8B alone; the frame is navy when Vanguard keeps a draft and peach when every draft scores below τ and the 8B writes the step. Selection: of the 182 held-out episodes, 82 have at least one step on which Vanguard is right and the 8B alone is wrong (132 such steps in all); this one has the most (6, at steps 3, 5, 9, 11, 18 and 19; no other episode has more than 5).
Figure 29: (continued) Steps 11 to 21 of the EB-ALFRED episode of the previous page, shown in the same way.
Agentic multimodal large language models (MLLMs) that think with images interleave reasoning with zoom-in tool calls. However, this sequential loop is slow, and its answers carry no reliability score. Prior speculation of tokens, reasoning steps, or answers keeps the loop, and answer speculation also accepts confident errors. We find that the loop mostly adds one zoom-in step, with which a small model is ahead of the agent on most splits. Our key insight is to speculate the agent's next observation, which, unlike a drafted answer, can be verified, corrected, and used to test the answer. We propose SpecLook, a training-free speculate-and-verify framework with three levels. A small model drafts a glimpse and re-answers from it, and the question ends if the answer is unchanged and confident. Otherwise, the agent's large model verifies and corrects the glimpse without tools, and their answers are fused. Only questions that still lack evidence reach the agent, in one forward pass that SpecLook-Fast skips, so no question runs the loop. The trace scores every answer's reliability without labels. Extensive experiments on V*, HR-Bench, and POPE show that both are more accurate than every baseline on every split. SpecLook reaches 87.2% and 86.2% mean accuracy with 7.7x and 4.7x speedups over the DeepEyes-7B and Thyme-RL agents, and SpecLook-Fast reaches 9.9x and 6.3x.
Large language model agents solve tasks through long sequences of tool calls, and the prevailing design has a single strong model generate every step, although many steps are routine enough for a much smaller model. Existing ways of sharing this work keep the strong model as the generator: routers and collaborative agents decide which model handles each query or step, and speculative methods check a small model's drafts one step at a time, usually against the strong model's own action. We introduce SpecAgent, a collaboration protocol that turns the strong model from the author of each step into a reviewer of steps drafted by a weak model. The weak model executes read-only tool calls as it drafts, so that every step is grounded in real observations, and holds the first action that would modify the environment. The strong model then verifies the drafted segment in a single prefill pass, commits the longest prefix it judges semantically valid, and repairs the first rejected step. Because verification replaces autoregressive decoding with parallel prefill, the strong model decodes only short verdicts and occasional repairs rather than every step. On τ2-Bench, SpecAgent comes within one point of the strong model's success rate while roughly halving memory traffic and wall time, and on AutomationBench and VitaBench it recovers 44-75% of the quality gap while lowering both. In an edge-cloud deployment on MCP-Atlas with three cloud models, it closes 53-75% of the weak-strong quality gap while the strong model generates at most half as many tokens. These results suggest that a strong model need not author every step of an agent: reviewing grounded drafts retains most of its quality while cutting both latency and memory traffic.
Zilin Wang, Yinhong Liu, Chao Chen +2
LARK, HKUST (GZ) · Yinwang · University of Cambridge +1
Speculative decoding has been treated purely as a systems optimization that speeds up large language model decoding without changing the model behavior. Under this view, the growing effort to train better draft models has been focused solely on acceptance rate. We argue that this view does not hold true for long-horizon agent reasoning. Measuring frontier open-weight models on complex agentic tasks with different speculative decoding approaches, we found that numerical noise grows within verification windows, and the draft acceptance results of the thinking process can change the output distribution of the model after 30 steps, thus noticeably influencing the reasoning accuracy of long-horizon agents in random directions. In this work, we improve the reasoning capability of target models by controlling such noise. While we cannot change the overall token distribution verified by the target model, we can encode critical reasoning with highest precision by training the draft model to be rejected. We identify these points in agent traces and train the draft model to favor adversarial preference. At inference the trained draft and the target part ways precisely at these decision points, verification restarts, so that critical thinking tokens are sampled from the target model and encoded under its least noisy conditions. This approach makes critical reasoning better indexed and more durable over a long horizon. On complex agentic coding benchmarks, our approach raises end-to-end success rate with no change to the target model or its output distribution. Our results recast the speculative decoding mechanism from a pure accelerator into an active allocator for context precision that improves the accuracy of long-horizon agents by training 0.06% to 0.28% of hosted parameters.
Hongyin Luo, Wei Fang, Lijie Yang +5
Subconscious Systems Technologies, Inc. · Massachusetts Institute of Technology · Princeton University +1