Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness, typically relying on hand-crafted heuristics for task termination. This limitation is particularly severe in long-horizon tasks involving cascaded sub-goals. In this work, we investigate the estimation and integration of task progress, proposing a novel model named {\textbf \vla}. Our technical contributions are twofold: (1) robust progress estimation: We pre-train a progress estimator on large-scale, unsupervised video-text robotic datasets. This estimator achieves a low prediction residual (0.07 on a scale of [0,1]) in simulation and demonstrates zero-shot generalization to unseen real-world samples, and (2) differentiable progress guidance: We introduce an inverse dynamics world model that maps predicted action tokens into future latent visual states. These latents are then processed by the progress estimator; by applying a maximal progress regularization, we establish a differentiable pipeline that provides progress-piloted guidance to refine action tokens. Extensive experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates and generalization over strong baselines.
Figures & tables
Progress Guidance
Pearson r↑
Avg. steps ↓
Success ↑
0.722
90.4
92.7
✓
0.934
77.3
93.6
TABLE I: Key motivating observation. Progress-guided sampling improves progress alignment and reduces completion steps on CALVIN. Pearson r is the correlation between predicted progress {p^t} and a linear ramp {t/T} over a rollout. Avg. steps denotes steps-to-completion, and Success is the task success rate. The baseline here is a standard diffusion policy for robotic manipulation.
Fig. 2: Overview of ProgressVLA. Conditioned on a language instruction and current observation, the diffusion policy first generates a candidate chunk of latent actions. A action-oriented world model then rolls out these actions within a pre-trained visual feature space to project future states, while a progress estimator assigns a completion score to the predicted outcomes. Finally, progress gradients are backpropagated through the world model as classifier guidance, steering the diffusion process toward actions that maximize task advancement.
Fig. 3: Architecture of action dynamics oriented world model.
Fig. 4: Reinforcement learning framework of ProgressVLA.
Method
Input
Task completed in a Row(%) ↑
Avg.Len.
1
2
3
4
5
RoboFlamingo [ 27 ]
S-RGB,G-RGB
82.4
61.9
46.6
33.1
23.5
2.47
GR-1 [ 44 ]
S-RGB,G-RGB,P
85.4
71.2
59.6
49.7
40.1
3.06
3D Diffuser [ 22 ]
S-RGBD,G-RGBD,P,Cam
92.2
78.7
63.9
51.2
41.2
3.27
GR-MG [ 26 ]
S-RGBD,G-RGBD,P
96.8
89.3
81.5
72.7
64.4
4.04
SuSIE [ 3 ]
S-RGB
87.0
69.0
49.0
38.0
26.0
2.69
TABLE II: The comparisons with state-of-the-art approaches on CALVIN (ABC → D) with the metrics of success rate and average success length. The abbreviations denote different input modalities: S-RGB for Static RGB, G-RGB for Gripper RGB, S-RGBD for Static RGB-D, G-RGBD for Gripper RGB-D, P for proprioceptive arm position, and Cam for camera parameters.
Method
Spatial
Object
Goal
Long
Average
LAPA [ 47 ]
73.8
74.6
58.8
55.4
65.7
Diffusion Policy [ 13 ]
78.3
92.5
68.3
50.5
72.4
Octo [ 41 ]
78.9
85.7
84.6
51.1
75.1
MDT [ 37 ]
78.5
87.5
73.5
64.8
76.1
OpenVLA [ 25 ]
84.7
88.4
79.2
53.7
76.5
MaIL [ 20 ]
74.3
90.1
81.8
78.6
83.5
TABLE III: The experimental results on the LIBERO benchmark. See the main text for more explanation.
Fig. 5: Illustration of the five tasks in real-world model deployment on an ARX robotic dual-arm.
Octo [ 41 ]
ProgressVLA (w/o CG)
ProgressVLA (w/ CG)
Task
Success ↑
Dist ↓
Steps ↓
Success ↑
Dist ↓
Steps ↓
Success ↑
Dist ↓
Steps ↓
Pick up the peach
35
1.04
151.6
70
0.78
96.0
90
0.61
39.1
Open the drawer
20
1.45
196.8
80
0.92
90.5
90
0.78
42.0
Stack the bowls
30
0.91
145.8
65
0.71
94.1
70
0.59
37.8
Pick up the orange → plate
20
1.63
228.3
60
1.38
117.2
75
1.12
72.0
Pick up the toy → drawer
10
1.48
216.8
55
1.03
106.1
55
0.96
75.6
TABLE IV: Real-robot results on five tasks. We report average end-effector path length (Avg. distance, m), success rate (%), and average steps (Avg. steps).
Fig. 6: Real-world scenarios used for investigating the generalization of progress estimator.
Fig. 7: Visualization of progress estimation and effect of guidance. The second column from the left illustrates the predicted trajectories, contrasting the baseline diffusion policy (in black) with our ProgressVLA approach (in red). Our method consistently generates more plausible, goal-directed paths.
Setting
Method
Pearson ↑
Stop ↑
MAE ↓
Origin Setting
From Scratch
0.912
53.8
0.14
Finetuned
0.977
82.1
0.10
Lighting Shift
From Scratch
0.809
3.6
0.24
Finetuned
0.953
80.8
0.12
Novel Objects
From Scratch
0.810
37.5
0.15
Finetuned
0.972
81.2
0.11
TABLE V: “From Scratch” trains the progress predictor from random initialization on our real-robot data, while “Finetuned” starts from the pretrained checkpoint and is finetuned on the same data.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 8: Real-world scenes. All real-robot trials are executed using only the right arm for manipulation, while the left arm remains stationary throughout the rollout.
Fig. 9: Progress estimator traces with/without progress guidance. The rollout corresponds to the instruction “pick up the orange and put it on the plate”. Top: with progress guidance. Bottom: without progress guidance. The orange curve shows the predicted task progress over time.
Fig. 10: Progress traces under lighting shift. The rollout is an expert-collected trajectory for evaluating the progress estimator (instruction: “stack the bowls”). Left: Pretrained+Finetuned. Right: From Scratch.
Fig. 11: Progress traces under novel objects. The rollout is an expert-collected trajectory for evaluating the progress estimator (instruction: “pick up the apple and put it on the plate”). Left: Pretrained+Finetuned. Right: From Scratch.