ΔVLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation
Authors: Yijie Zhu, Jie He, Rui Shao, Kaishen Yuan, Tao Tan, Xiaochen Yuan, Zitong Yu
Organizations: Harbin Institute of Technology, Shenzhen, Shenzhen 518055, China · Great Bay University, Dongguan 523000, China · Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511400, China · Macao Polytechnic University, Macao 999078, China
Recent vision-language-action (VLA) models have significantly advanced robotic manipulation by unifying perception, reasoning, and control. To achieve such integration, recent studies adopt a predictive paradigm that models future visual states or world knowledge to guide action generation. However, these models emphasize forecasting outcomes rather than reasoning about the underlying process of change, which is essential for determining how to act. To address this, we propose ΔVLA, a prior-guided framework that models world-knowledge variations relative to an explicit current-world knowledge prior for action generation, rather than regressing absolute future world states. Specifically, 1) to construct the current world knowledge prior, we propose the Prior-Guided WorldKnowledge Extractor (PWKE). It extracts manipulable regions, spatial relations, and semantic cues from the visual input, guided by auxiliary heads and prior pseudo labels, thus reducing redundancy. 2) Building upon this, to represent how world knowledge evolves under actions, we introduce the Latent World Variation Quantization (LWVQ). It learns a discrete latent space via a VQ-VAE objective to encode world knowledge variations, shifting prediction from full modalities to compact latent. 3)Moreover, to mitigate interference during variation modeling, we design the Conditional Variation Attention (CV-Atten), whichpromotes disentangled learning and preserves the independence of knowledge representations. Extensive experiments on both simulated benchmarks and real-world robotic tasks demonstrate ΔVLA achieves state-of-the-art performance while improving efficiency. Code and real-world execution videos are available at https://github.com/JiuTian-VL/DeltaVLA.
Figures & tables
Fig. 1 : Comparison with Previous Paradigms. (a) Image-based paradigms predict subgoal images for action generation. (b) World knowledge-based paradigms replace pixel-level prediction with future world states. (c) Δ VLA constructs an explicit current-world knowledge prior via PWKE and models discrete world-knowledge variations via LWVQ , enabling prior-grounded reasoning about how the world should change under actions. (d) Comparison of performance and efficiency with previous methods.
Fig. 2 : Overview of Δ VLA Framework. It first extracts current world knowledge via the Prior-Guided World Knowledge Extractor , which encodes semantic, regional, and depth cues from SigLIP and DINOv2 guided by auxiliary heads and prior pseudo labels to form a unified world prior. These representations are concatenated with world variation tokens and modeled through the Conditional Variation Attention for structured perception–variation reasoning. Finally, the Latent World Variation Quantization module learns a discrete codebook representing world knowledge variations, providing causal guidance for variation token learning and consistent action generation.
Fig. 3 : Illustration of PWKE. World tokens have different meanings across encoders: they correspond to semantic tokens in SigLIP and depth tokens in DINOv2. The dashed box indicates auxiliary heads used only for training supervision.
Fig. 4 : Illustration of CV-Atten. In red dashed boxes, each variation token attends to its corresponding world prior, ensuring type-specific modeling and minimizing cross-modality leakage. For action generation, we employ parallel decoding.
Method
Spatial
Object
Goal
Long
Average
SR ↑
RK ↓
SR ↑
RK ↓
SR ↑
RK ↓
SR ↑
RK ↓
SR ↑
RK ↓
OpenVLA [CoRL’24] [ 52 ]
84.7
10
88.4
10
79.2
10
53.7
12
76.5
10
MDT [RSS’24] [ 69 ]
78.5
11
87.5
11
73.5
11
64.8
11
76.1
11
π0 [RSS’25] [ 3 ]
96.8
5
98.8
2
95.8
3
85.2
9
94.2
7
OpenVLA-OFT [RSS’25] [ 70 ]
97.6
3
98.4
3
97.9
1
94.5
2
97.1
2
UniVLA [RSS’25] [ 71 ]
96.5
6
96.8
6
95.6
4
92.0
3
95.2
4
TABLE I : Simulation Experimental Results on the LIBERO Benchmark. Comparison of task success rates (SR) and their ranks (RK). “†” indicates our reproduced results. Bold indicates the highest score.
Method
Click A.
Open L.
Place C. P.
Dump B. B.
Press S.
Click B.
Stack B. T.
Place B. S.
Average
RDT [ 75 ]
61.0
59.0
78.0
64.0
41.0
80.0
76.0
5.0
58.0
ACT [ 76 ]
32.0
56.0
72.0
68.0
31.0
58.0
82.0
7.0
50.8
Diffusion Policy [ 77 ]
61.0
49.0
41.0
49.0
6.0
54.0
61.0
11.0
41.5
3D Diffusion Policy [ 55 ]
77.0
82.0
86.0
85.0
69.0
90.0
83.0
19.0
73.9
π0 [ 3 ]
63.0
85.0
88.0
83.0
62.0
44.0
91.0
23.0
67.4
OpenVLA-OFT† [ 70 ]
82.0
73.0
89.0
86.0
68.0
84.0
83.0
13.0
72.3
TABLE II : Simulation Experimental Results on the RoboTwin 2.0 Benchmark. Comparison of success rates across eight tasks, along with the overall average performance. “†” indicates our reproduced results. Bold indicates the highest score.
Method
Drawer Manipulation
Align Shoes
T-shirt Folding
Plate Lemon Apple
Average
Pull
+Place
+Push
Retrieve
+Retrieve
+Align
Step 1
+Step 2
+Step 3
Open
+Retrieve
+Cover
Galaxea R1 Lite platform
OpenVLA† [ 52 ]
15/25
13/25
10/25
13/25
12/25
10/25
10/25
8/25
6/25
14/25
13/25
10/25
36%
OpenVLA-OFT† [ 70 ]
18/25
17/25
16/25
16/25
16/25
15/25
12/25
10/25
8/25
13/25
12/25
12/25
51%
UniVLA† [ 71 ]
16/25
14/25
13/25
15/25
14/25
14/25
15/25
14/25
12/25
11/25
10/25
9/25
48%
Seer† [ 18 ]
14/25
14/25
12/25
16/25
13/25
10/25
11/25
11/25
11/25
15/25
13/25
12/25
45%
TABLE III : Real-World Results. “†” indicates results reproduced using the same settings as Δ VLA. The table shows success rates for the progressive stage across four long-horizon tasks, with the average success rate representing only the completion of the entire task.
Method
Latency ↓
Throughput ↑
Cost ↓
LIBERO SR ↑
OpenVLA† [ 52 ]
0.254 s
3.9 Hz
11.7 h
76.5%
OpenVLA-OFT† [ 70 ]
0.132 s
60.6 Hz
12.5 h
97.1%
PD-VLA† [ 72 ]
0.143 s
55.9 Hz
11.7 h
94.7%
Seer† [ 18 ]
0.125 s
8.0 Hz
14.3 h
-
DreamVLA† [ 25 ]
0.130 s
7.7 Hz
14.5 h
92.6%
Δ VLA
0.105 s
76.2 Hz
4.9 h
97.8%
TABLE IV : Efficiency Comparison Results. Latency denotes inference delay; Throughput denotes the number of predicted actions per second; Cost is training time per 10k steps. “†” denotes results reproduced under identical settings as Δ VLA. All results are reported on A800 GPUs for a fair comparison.
PWKE
LWVQ
CV-Atten
Spatial
Object
Goal
Long
95.2
96.4
95.4
91.8
✓
96.0
97.4
96.6
92.6
✓
96.4
97.0
96.2
93.0
✓
✓
98.0
98.6
96.6
94.8
✓
✓
✓
98.6
99.4
97.4
95.6
TABLE V : Ablation Study on Model Components. LWVQ denotes the use of latent world variation prediction.
Region
Semantic
Depth
Spatial
Object
Goal
Long
w/ FiLM
w/o FiLM
✓
95.6
97.2
96.4
93.4
✓
✓
96.8
98.0
96.8
93.8
✓
✓
97.0
98.2
96.6
94.4
✓
✓
✓
98.6
99.4
97.4
95.6
✓
✓
✓
97.4
98.2
96.2
94.0
TABLE VI : Ablation Study on PWKE. w/ FiLM indicates the use of textual guidance in extracting manipulable regions.
Method
Spatial
Object
Goal
Long
The Number of Region (R) and World (W) Tokens
(R:128, W:4)
98.4
98.6
96.6
95.2
(R:128, W:9)
98.4
98.8
97.0
94.8
(R:64, W:4)
98.0
99.0
97.2
94.8
(R:64, W:9)
98.6
99.4
97.4
95.6
(R:64, W:16)
98.2
98.8
96.8
95.0
TABLE VII : Ablation Study on the Region and World Tokens in PWKE. We analyze how varying the number of tokens and their attention modeling influences performance.
Method
Spatial
Object
Goal
Long
w/o noise
98.6
99.4
97.4
95.6
w/ 10%
98.4
99.4
97.2
95.4
w/ 20%
98.0
99.0
96.8
95.4
w/ 30%
98.0
98.6
96.6
95.0
w/ 50%
97.0
97.4
96.0
93.8
TABLE VIII : Robustness Evaluation with Noisy Prior Labels. We evaluate the effect of injecting different levels of noise into the prior labels. The w/o noise row provides the clean-label.
Method
Spatial
Object
Goal
Long
Representation of Future States
Full Future Modalities
95.6
96.8
95.6
92.0
Continuous Variation
96.8
97.4
96.4
93.2
Latent Variation
98.6
99.4
97.4
95.6
Shared or Separated Variation Tokens
Shared
96.8
98.6
96.0
94.4
TABLE IX : Ablation Study on the LWVQ Module. We investigate the representation of future states and the design of latent variation, comparing shared versus separated settings.
Method
Spatial
Object
Goal
Long
4
98.6
99.4
97.4
95.6
9
98.4
99.2
97.0
95.4
16
98.0
98.8
96.8
95.0
TABLE X : Ablation Study on the Number of Each Type of Variation Tokens. We select a moderate number of variation tokens to achieve better performance.
Fig. 5 : Visualization of Manipulation Region Extraction and World Variation Modeling. On the left, the manipulable regions extracted by the PWKE module are shown. On the right, the future states, reconstructed from the current state and latent variation through the LWVQ module, closely align with the real future states and accurately highlight dynamic changes, marked by red circles.
Fig. 6 : Visualization of Real-World Execution Comparison. Given the long-horizon instruction, Δ VLA successfully completes all stages of the manipulation task, while DreamVLA fails at multiple critical steps.
Fig. 7 : Visualization of Executions on Two Additional Real-World Tasks. Δ VLA successfully completes all sub-stages in each task, demonstrating reliable long-horizon planning, robust manipulation skills, and accurate reasoning.
Fig. 8 : Diagnostic Analysis of CV-Atten. During geometry-critical grasping, CV-Atten suppresses cross-modal interference by keeping attention on contact-relevant geometry. Without it, attention leaks to surrounding semantic cues, causing grasp-point drift and failure.
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics