Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation

Lingsen Zhang; Wei Li; Jie He; Henghua Zhang; Kai Mo; Shiyong Li; Yanpeng Wang; Rui Shao

cs.ROOct 19, 2026

Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation

Authors: Lingsen Zhang, Wei Li, Jie He, Henghua Zhang, Kai Mo, Shiyong Li, Yanpeng Wang, Rui Shao

Organizations: Harbin Institute of Technology, Shenzhen · Baidu, Inc. · Shenzhen Loop Area Institute

Abstract

Robotic manipulation is not merely observation-to-action generation, but the realization of task-specified state changes in the physical world. Existing vision-language-action (VLA) methods directly learn observation-to-action mappings, imposing a substantial learning burden due to the gap between semantic understanding and action execution. Although world-action models (WAMs) improve control through future prediction, modeling complete future states wastes capacity on static appearance and action-irrelevant changes. We propose Vision-Language-Delta-Action (VLDA), a paradigm that reformulates full future prediction as compact actionable transition modeling. Its core is a compact Delta variable that captures latent world-state changes, serving as an interface between vision-language understanding and action generation. VLDA learns Delta through a unified objective composed of the Local Transition Filter (LT-Filter) and Global Transition Anchor (GT-Anchor). LT-Filter captures task-relevant manipulation-centric changes while suppressing static reconstruction biases. GT-Anchor provides coarse future-level regularization to preserve the overall direction of state changes. Delta-Driven Action Refinement (DDAR) further grounds Delta in the action manifold and injects it as a structured seed to guide action denoising. We implement VLDA within LoongForge, an open-source training framework for large-scale foundation and embodied models, using its modular and distributed infrastructure for unified two-stage optimization. Experiments show that VLDA improves long-horizon consistency, robustness, and generalization, validating actionable Delta as an alternative to direct action prediction and full future-state modeling.

Figures & tables

Figure 1: Motivation and paradigm comparison of VLDA. Robotic manipulation is governed by sparse task-relevant state changes that provide essential cues for action generation, while most visual content remains static. Unlike existing asynchronous and synchronous paradigms that predict complete future states before or alongside actions, VLDA learns a compact Delta from current-to-future transitions and uses it as an actionable interface to guide action refinement, forming an understanding-transition-execution loop.

Figure 2: Overview of VLDA. Given the observation and instruction, the World Expert predicts an actionable Delta under transition supervision. In training stage I, LT-Filter and GT-Anchor learn selective and consistent transition representations. In stage II, Delta-Driven Action Refinement grounds Delta in the action manifold through Delta-to-Action Manifold Grounding and injects it into action denoising through Delta-Seeded Action Flow, enabling the Action Expert to generate action chunks.

Method	Perturbation Dimensions							Task Suites				Avg.
Method	Camera	Robot	Language	Light	Background	Noise	Layout	Spatial	Object	Goal	Long	Avg.
OpenVLA [ 26 ]	0.8	3.5	23.0	8.1	34.8	15.2	28.5	19.4	14.0	15.1	14.3	15.6
OpenVLA-OFT [ 27 ]	56.4	31.9	79.5	88.7	93.3	75.8	74.2	84.0	66.5	63.0	66.4	69.6
UniVLA [ 6 ]	1.8	46.2	69.6	69.0	81.0	21.2	31.9	55.5	36.7	40.7	39.9	43.9
WorldVLA [ 7 ]	0.1	27.9	41.6	43.7	17.1	10.9	38.0	32.5	28.6	31.8	8.2	25.0
DreamVLA [ 58 ]	65.0	40.8	63.5	85.7	82.6	84.9	74.0	79.7	79.0	61.7	59.8	69.9

Table 1: Results on LIBERO-Plus [ 13 ] . We report generalization performance across seven perturbation dimensions and four task suites. VLDA is evaluated in a zero-shot setting with 10,030 rollouts.

Figure 3: Representative executions on Ring Stacking, Conveyor Sorting, Table Sweeping, and Basket Relocation, including two OOD spatial configurations. Frames proceed from left to right, while red dashed lines and circular zoom-ins highlight the correspondence between task-relevant current-to-future transitions and actions. Complete trajectories in Appendix C.1 .

Method	Ranking RGB	Ranking Size	Grab Roller	Open Laptop	Open Microwave	Place Dual Shoes	Put Bottles Dustbin	Shake Bottle	Stacking Three	Stacking Two	Avg.
ACT [ 60 ]	0%	0%	25%	0%	0%	0%	1%	10%	0%	0%	3.6%
RDT [ 38 ]	0%	0%	43%	32%	20%	4%	4%	45%	17%	30%	19.5%
DP [ 11 ]	0%	0%	0%	0%	0%	0%	0%	8%	0%	0%	0.8%
DP3 [ 55 ]	0%	0%	2%	7%	22%	0%	21%	19%	5%	6%	8.2%
TwinVLA [ 22 ]	0%	0%	22%	17%	1%	3%	4%	58%	15%	11%	13.1%
$\boldsymbol{\pi_{0}}$ [ 4 ]	5%	1%	80%	46%	50%	0%	13%	60%	24%	41%	32.0%

Table 2: Results on selected hard tasks from RoboTwin 2.0 [ 8 ] , chosen to cover variations in layout, illumination, appearance, and other factors. VLDA is trained exclusively on the clean setting and evaluated on the randomized setting, constituting a zero-shot evaluation. VLDA achieves the best average performance, and the best result for each task is highlighted in bold . Complete results across all 50 tasks are reported in Appendix B.1 for reference.

Method	Ring Stacking		Table Sweeping				Conveyor Sorting				Basket Relocation			Avg.
Method	Put_1	+Put_2	Sweep_1	+Sweep_2	+Lift	+Empty	Pick_1	+Place_1	+Pick_2	+Place_2	Put Toys	+Grasp	+Place	Avg.
OpenVLA-OFT † [ 27 ]	64%	48%	48%	40%	24%	8%	32%	16%	8%	4%	52%	24%	4%	16.0%
$\pi_{0.5}^{\dagger}$ [ 23 ]	88%	80%	80%	76%	64%	52%	64%	52%	44%	44%	92%	72%	48%	56.0%
VLDA	96%	92%	88%	88%	80%	68%	72%	64%	60%	60%	100%	88%	72%	73.0%

Table 3: Real-world results on the GALAXEA R1 Lite platform [ 15 ] . “

\dagger

” denotes reproduced results under VLDA settings. Entries report progressive-stage success over 25 trials; Avg. averages final-checkpoint success across tasks. Checkpoint definitions and AgileX Cobot Magic [ 14 ] results are in Appendix B.1 .

Table 7

Figure 4: Visualization of Delta supervision. LT-Filter extracts sparse manipulation-centric local changes, while GT-Anchor preserves the coarse global transition trend.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Name	Number	Language Instruction	Progressive Checkpoints
1	Ring Stacking	200 (100 $\times 2$ )	Stack the yellow and purple rings on the peg.	$C_{1}$ : the yellow ring is placed around the peg; $C_{2}$ : the purple ring is placed above the yellow ring.
2	Conveyor Sorting	250 (125 $\times 2$ )	Pick the cup and bun from the conveyor and place them on the plate.	$C_{1}$ : the moving cup is successfully grasped; $C_{2}$ : the cup is placed upright perfectly on the plate; $C_{3}$ : the moving bun is successfully grasped; $C_{4}$ : the bun is placed on the plate while distractor objects remain unselected.
3	Table Sweeping	250 (125 $\times 2$ )	Sweep the paper balls into the dustpan and empty them onto the plate.	$C_{1}$ : the first paper ball is swept into the dustpan; $C_{2}$ : both paper balls are inside the dustpan; $C_{3}$ : the dustpan is lifted without losing either paper ball; $C_{4}$ : both paper balls are emptied onto the plate.
4	Basket Relocation	300 (150 $\times 2$ )	Put both toys in the basket and move the basket onto the blue mat.	$C_{1}$ : both toys are placed inside the basket; $C_{2}$ : the basket is securely grasped with both arms; $C_{3}$ : the basket is placed on the blue mat without dropping either toy.

Appendix

Table 7: Real-world Task Details. Language instructions and progressive evaluation checkpoints for the four real-world tasks. Checkpoints are cumulative: reaching checkpoint

C_{k}

requires successful completion of all preceding checkpoints.

Figure 5: GALAXEA R1 Lite (left) and AgileX Cobot Magic (right) platforms.

Task	ACT	DP	DP3	RDT-1B	$\pi_{0}$	$\pi_{0.5}$	TwinVLA	BagelVLA	HALO	VLDA
Adjust Bottle	23%	0%	3%	75%	56%	83%	35%	14%	9%	76%
Beat Block Hammer	3%	0%	8%	37%	21%	50%	10%	16%	11%	52%
Blocks Ranking RGB	0%	0%	0%	0%	5%	35%	0%	4%	7%	53%
Blocks Ranking Size	0%	0%	0%	0%	1%	14%	0%	2%	2%	16%
Click Alarmclock	4%	5%	14%	12%	11%	93%	1%	20%	14%	64%
Click Bell	3%	0%	0%	9%	3%	76%	13%	35%	10%	67%

Appendix

Table 8: Per-task success rates on the 50 RoboTwin 2.0 tasks under the Randomized setting.

Method	Ring Stacking		Table Sweeping				Conveyor Sorting				Basket Relocation			Avg.
Method	Put_1	+Put_2	Sweep_1	+Sweep_2	+Lift	+Empty	Pick_1	+Place_1	+Pick_2	+Place_2	Put Toys	+Grasp	+Place	Avg.
OpenVLA-OFT † [ 27 ]	60%	44%	48%	44%	28%	12%	32%	16%	8%	4%	56%	28%	8%	17.0%
$\pi_{0.5}^{\dagger}$ [ 23 ]	88%	80%	80%	72%	60%	48%	64%	52%	44%	44%	92%	68%	44%	54.0%
VLDA	96%	88%	92%	88%	80%	68%	76%	68%	64%	64%	100%	88%	68%	72.0%

Appendix

Table 9: Real-world task results on the ALOHA [ 14 ] . “

\dagger

” denotes results reproduced using the same settings as VLDA. Each entry reports the progressive-stage success rate over 25 trials. Avg. is the average complete-task success rate, calculated from the final checkpoint of each task.

Method	Perturbation Dimensions							Task Suites				Avg.
Method	Camera	Robot	Language	Light	Background	Noise	Layout	Spatial	Object	Goal	Long	Avg.
w/o Transition Pre-Training	66.6	71.5	80.4	87.7	89.4	84.3	81.6	83.9	87.2	74.7	72.2	79.4
w/ Full-Future Prediction	68.0	73.3	81.8	89.1	90.8	85.5	84.4	85.4	88.0	76.0	75.2	81.1
VLDA	72.4	75.3	84.0	96.0	95.5	89.4	85.7	89.6	89.8	80.1	78.3	84.5

Appendix

Table 10: Results on the LIBERO-Plus benchmark [ 13 ] . We report zero-shot generalization performance across seven perturbation dimensions and four task suites using 10,030 total rollouts. The last three rows compare VLDA with transition-paradigm and training-strategy ablations. All three variants are trained only on standard LIBERO and evaluated on LIBERO-Plus without further adaptation.

Table 14

Figure 6: Representative VLDA executions on the GALAXEA R1 Lite platform across four long-horizon tasks and two OOD configurations of Basket Relocation. Frames proceed chronologically from left to right, with colored labels indicating cumulative evaluation checkpoints.

Method	LIBERO					LIBERO-plus
Method	Spat.	Obj.	Goal	Long	Avg.	Cam.	Robot	Lang.	Light	Bkg.	Noise	Layout	Spat.	Obj.	Goal	Long	Avg.
w/o DAMG	99.2	99.4	99.4	94.6	98.2	69.5	74.1	82.5	89.7	91.8	86.9	84.6	86.7	89.1	76.9	75.5	82.0
VLDA	99.8	99.6	99.4	95.2	98.5	72.4	75.3	84.0	96.0	95.5	89.4	85.7	89.6	89.8	80.1	78.3	84.5

Appendix

Table 14: Complete ablation results of Delta-to-Action Manifold Grounding on LIBERO and LIBERO-plus.

Figure 7: Representative VLDA executions on the AgileX Cobot Magic platform across four long-horizon tasks and two OOD configurations of Basket Relocation. Frames proceed chronologically from left to right, with colored labels indicating cumulative evaluation checkpoints.

Figure 8: Visualization of Delta supervision. LT-Filter extracts sparse manipulation-centric local changes, while GT-Anchor preserves the coarse global transition trend.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,615 papers

VLA- $Δ$ : Learning Physical State Transition for Vision-Language-Action Models
Oct 15, 2026Qi Lu, Jiarun Li, Ce Zhang +5VLA Model Training Vision-Language-Action Models
Zhejiang University (zju.edu.cn) · Central South University (csu.edu.cn)
$Δ$ VLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation
Mar 9, 2026Yijie Zhu, Jie He, Rui Shao +4Vision-Language-Action Models VLMs for Robotics
Harbin Institute of Technology, Shenzhen, Shenzhen 518055, China · Great Bay University, Dongguan 523000, China · Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511400, China +1
SLIP-VLA: Single-Step Latent Imagination for Policy Learning in Vision-Language-Action Models
Sep 27, 2026Tianfu Li, Haoxuan Xu, Wenbo Chen +7World Model Learning Visuomotor Policy Learning
The Hong Kong University of Science and Technology (Guangzhou). · The Hong Kong University of Science and Technology. · Nanyang Technological University. +1

Oct 15, 2026J/K move · Enter open · S save

VLA- $Δ$ : Learning Physical State Transition for Vision-Language-Action Models

Qi Lu, Jiarun Li, Ce Zhang, Miao Pan+4

Zhejiang University (zju.edu.cn) · Central South University (csu.edu.cn)

VLA Model Training Vision-Language-Action Models VLMs for Robotics Robot Dynamics Learning

Open paper AskPDF arXiv

Vision-language-action (VLA) models transfer semantic knowledge acquired through vision-language pretraining to robot control, achieving strong performance across various benchmarks. However, most VLAs directly map observations to demonstrated actions without explicit supervision of intermediate physical state changes, making them prone to overfitting observation-action correspondences. Some methods introduce intermediate reasoning or future-state prediction in language, visual, or latent spaces to guide action generation, but do not explicitly supervise the physical state transition itself. We therefore propose VLA-

Δ

, a general framework that uses physical changes between current and future states as explicit learning targets. To support this objective, we construct a large-scale pretraining dataset and derive physical state-transition annotations covering task-relevant semantics and visual motion changes, providing more direct supervision for continuous action generation. During inference, the model generates actions directly from current observations without additional inference overhead. VLA-

Δ

achieves performance competitive with the state of the art on LIBERO and RoboTwin, and outperforms

π_{0.5}

in real-robot experiments. Our code is available at https://github.com/ZJU-OmniAI/VLA-Delta

Abstract

Robotic Manipulation

Explore similar work

VLA-ΔΔΔ: Learning Physical State Transition for Vision-Language-Action Models

ΔΔΔVLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation

SLIP-VLA: Single-Step Latent Imagination for Policy Learning in Vision-Language-Action Models

VLA- $Δ$ : Learning Physical State Transition for Vision-Language-Action Models

$Δ$ VLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation