Vision-Language Models Are Joint Denoisers in Robotic Action Generation

Xicheng Gong; Wenxuan Song; Chengze Liu; Qiwei Li; Qize Yu; Jiaolong Yang; Haoang Li; Yadong Mu

cs.ROOct 31, 2026

Vision-Language Models Are Joint Denoisers in Robotic Action Generation

Authors: Xicheng Gong, Wenxuan Song, Chengze Liu, Qiwei Li, Qize Yu, Jiaolong Yang, Haoang Li, Yadong Mu

Organizations: Peking University · The Hong Kong University of Science and Technology (Guangzhou) · Microsoft Research Asia

Abstract

Diffusion-based vision-language-action (VLA) models cascade pretrained vision-language models (VLMs) with diffusion action experts, but typically restrict the VLM to providing static conditioning. This separation limits how much pretrained perception and reasoning can guide action generation, especially in out-of-distribution settings. We introduce JADE, a Joint Action DEnoiser that enables the VLM and action expert to jointly refine actions. Our key insight is that structured action latents allow the VLM to read the current noisy action together with visual observations and language instructions. At every denoising step, the VLM contextualizes the noisy action and guides the action expert through gated cross-attention, so that its guidance evolves with the action being generated. We further supervise the VLM to recover clean action latents from noisy ones, which gives it a direct action-learning signal and couples perception with control. Additionally, joint denoising enables JADE to generate actions with fewer sampling steps, helping offset the additional cost of VLM participation. An adaptive step-size schedule further balances task performance and inference latency. On RoboDojo, JADE achieves an average success rate of 12.67%, the highest among the evaluated $π_{0.5}$ -based methods, and is competitive with recent VLA and WAM foundation models pretrained at scale. It also improves performance on RoboTwin 2.0 and raises the average success rate across four real-world bimanual tasks by 20.8 percentage points compared with $π_{0.5}$ .

Figures & tables

Figure 1: Illustration of the key difference between our proposed JADE and conventional VLA models. (a) During training, a frozen OAT encoder ( Liu et al., 2026a ) in JADE provides noisy action latents as VLM inputs and clean action latents as training targets. (b) During inference, JADE brings the VLM into the denoising loop, so its guidance to the action expert is updated with the current action at every step.

Figure 2: Architecture of JADE. A frozen OAT encoder maps the current noisy action to compact latents, which the VLM processes together with the observation prefix. The resulting register features guide each denoising block of the action expert through gated cross-attention, and during training the final features are aligned with the clean action latents.

Figure 3: Adaptive inference for joint denoising. The left panel shows uniform Euler steps, which native

\pi_{0.5}

and default JADE both use. The right panel shows the adaptive schedule, where a small

c_{i}

gives a large step and a large

c_{i}

gives a small step. Trajectories are schematic.

Method	Clean	Random	Avg.
ACT ( Zhao et al., 2023 )	29.7	1.7	15.7
RDT ( Liu et al., 2025 )	34.5	13.7	24.1
$\pi_{0}$ ( Black et al., 2024 )	46.4	16.3	31.4
GR00T-N1.7 ( Bjorck et al., 2025 )	43.6	20.7	32.2
StarVLA ( Community, 2026 )	58.1	10.6	34.4
FastWAM ( Yuan et al., 2026 )	70.2	1.2	35.7

Table 1: Results on the RoboTwin 2.0 benchmark. Success rates (%) are averaged over 50 tasks under Clean and Random settings; Avg. is their mean. JADE uses 10 denoising steps. Best values in each column are bold.

Method	Gen.	Prec.	Long.	Mem.	Open	Avg.
OpenVLA-OFT ( Kim et al., 2025 )	0.04 / 0.00	0.20 / 0.00	0.70 / 0.00	0.00 / 0.00	0.08 / 0.08	0.21 / 0.02
RDT-1B ( Liu et al., 2025 )	0.56 / 0.33	0.38 / 0.00	1.13 / 0.00	0.49 / 0.33	0.00 / 0.00	0.51 / 0.13
GR00T-N1.7 ( Bjorck et al., 2025 )	2.16 / 1.22	2.54 / 0.67	8.30 / 3.58	1.06 / 0.89	0.18 / 0.17	2.85 / 1.31
$\pi_{0}$ ( Black et al., 2024 )	3.94 / 2.56	3.56 / 0.75	6.19 / 2.00	3.47 / 2.11	0.25 / 0.25	3.48 / 1.53
Fast-WAM ( Yuan et al., 2026 )	2.34 / 1.11	1.96 / 0.00	9.14 / 5.17	3.55 / 3.44	0.42 / 0.42	3.48 / 2.03
LingBot-VLA ( Wu et al., 2026 )	6.71 / 4.28	5.33 / 1.83	10.89 / 5.25	3.82 / 2.78	0.72 / 0.67	5.50 / 2.96

Table 2: Results on the RoboDojo benchmark. Each entry reports Score / Success Rate (SR, %) . JADE uses 10 denoising steps.

		RoboTwin			RoboDojo
Conditioning	$\lambda_{\mathrm{align}}$	Clean	Random	Avg.	SR	Score
Queries	0	73.1	49.5	61.3	7.68	11.33
Queries	0.1	72.6	45.3	59.0	9.66	15.36
Action latents	0	79.1	52.9	66.0	11.15	16.59
Action latents	0.1	79.3	54.4	66.9	12.67	18.29

Table 3: Ablation studies. We examine (a) action conditioning and auxiliary alignment supervision, (b) action representations, and (c) step-wise action-conditioned semantic updates. RoboTwin reports success rates (%); RoboDojo in (a) reports SR and Score averaged over its five evaluation dimensions, while (c) reports Score and full-task success rate on RoboDojo.

Figure 4: Sampling efficiency of JADE. The left panel shows the success rates of JADE and native

\pi_{0.5}

across different numbers of function evaluations (NFE) on fixed task subsets, and purple diamonds mark JADE Balanced. The right table reports three sampling configurations on the full benchmarks, with NFE averaged per action chunk.

Figure 5: Real-robot task illustrations and success-rate comparisons. Panels (a)–(d) show the four tasks, and panel (e) compares JADE with $\pi_{0.5}$ .

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Experimental setup of the dual-arm robotic platform.

Task	Clean	Random	Task	Clean	Random
adjust bottle	100	89	place can basket	80	30
beat block hammer	90	34	place cans plasticbox	86	82
blocks ranking rgb	85	53	place container plate	95	91
blocks ranking size	43	24	place dual shoes	72	53
click alarmclock	88	76	place empty cup	93	91
click bell	95	90	place fan	73	40

Appendix

Table 4: Per-task success rates (%) of JADE on RoboTwin 2.0 with ten denoising steps. Each entry is computed over

100

trials. The last row averages all

50

tasks.

Task	Score	SR	Task	Score	SR
Generalization			Long-Horizon
stack bowls	54.60	48.00	put bottles into dustbin	96.70	94.67
push T	0.00	0.00	fill pen holder	41.90	16.67
pack objects into box	29.93	7.33	classify objects	43.63	33.33
fold clothes	59.07	54.67	play tic tac toe	15.10	2.00
hang mugs	9.40	0.67	fill egg holder	6.50	0.00

Appendix

Table 5: Per-task Score and success rate (SR, %) of JADE on RoboDojo with ten denoising steps. Each task includes

150

trials. The last row averages the five category means.

Setting	RoboTwin	RoboDojo
Training demonstrations	$2{,}500$	$3{,}500$
Optimizer updates	$20{,}000$	$60{,}000$
Global batch size	$256$	$256$
Hardware	$8\times$ A800 80GB
Precision / sharding	BF16 / FSDP
Optimizer	AdamW, $(\beta_{1},\beta_{2})=(0.9,0.95)$

Appendix

Table 6: Training configuration for JADE on the simulation benchmarks.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,202 papers

TokenPB: Enabling VLA Action Self-Verification via VLM Token Probability Bucketing
Mar 18, 2026Chen Zhao, Zhuoran Wang, Haoyang Li +6Vision-Language-Action Models VLA Model Training
School of Information, Renmin University of China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China · Beijing University of Posts and Telecommunications +1
dVLA-RL: Reinforcement Learning over Denoising Trajectories for Discrete Diffusion Vision-Language-Action Models
Jun 22, 2026Yuhao Wu, Yitian Liu, Weijie Shen +13Vision-Language-Action Models VLMs for Robotics
ScaleLab at SJTU · 1Shanghai Jiao Tong University · 3Baidu AI Cloud +3
AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Models
Mar 10, 2026Yutong Hu, Jan-Nico Zaech, Nikolay Nikolov +6Temporal Modeling for VLA Models Vision-Language-Action Models
∗INSAIT, Sofia University “St. Kliment Ohridski” · †KU Leuven, Dept. Mechanical Engineering, Research unit Robotics, Automation and Mechatronics · ‡KU Leuven, Dept. Electrical Engineering, Research unit Processing Speech and Images

Mar 18, 2026J/K move · Enter open · S save

TokenPB: Enabling VLA Action Self-Verification via VLM Token Probability Bucketing

Chen Zhao, Zhuoran Wang, Haoyang Li, Shifeng Bao+5

School of Information, Renmin University of China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China · Beijing University of Posts and Telecommunications+1

Vision-Language-Action Models VLA Model Training

Open paper AskPDF arXiv

Test-time sampling can improve Vision-Language-Action (VLA) policies, but only if the model can efficiently select a good action from multiple candidates. Prior work often relies on separately trained external verifiers, adding extra models and inference overhead. We propose Token Probability Bucketing (TokenPB), a self-verification approach that discretizes continuous action chunks into bucket tokens and co-trains a single VLA checkpoint with a joint objective: a flow-matching loss for action generation and an auxiliary bucket loss

\mathcal{L}_{\text{bucket}}

that teaches the VLM backbone to model observation-conditioned distributions over bucket-token sequences. At inference, TokenPB samples

M

candidates, scores each via teacher-forced evaluation under the learned distribution, and selects the best---enabling efficient batched scoring with observation-prefix KV-cache reuse. We provide theory connecting the training objective to a monotonic ranking property and bounds on expected selection regret that separate candidate-set effects from calibration. Across multiple backbones and benchmarks, TokenPB yields consistent success-rate improvements over single-sample inference from the same checkpoint (+2.7 in simulation, +17.1 in real-world, +16.4 under out-of-distribution shifts), and further improves over external-verifier baselines under the same sampling budget. The code is publicly available at https://github.com/RUCKBReasoning/TokenPB.

Abstract

Vision-Language-Action Models

Explore similar work

TokenPB: Enabling VLA Action Self-Verification via VLM Token Probability Bucketing

dVLA-RL: Reinforcement Learning over Denoising Trajectories for Discrete Diffusion Vision-Language-Action Models

AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Models