Concertina: Embodying Manipulation and Cognition in One Diffusion Vision-Language Model

Chengcheng Wang; Wei He; Wenhao Li; Hongguang Li; Kai Han; Jianyuan Guo; Chang Xu

cs.CVNov 5, 2026

Concertina: Embodying Manipulation and Cognition in One Diffusion Vision-Language Model

Authors: Chengcheng Wang, Wei He, Wenhao Li, Hongguang Li, Kai Han, Jianyuan Guo, Chang Xu

Organizations: University of Sydney · TokenRhythm Technologies · City University of Hong Kong

Abstract

Most vision-language-action (VLA) policies decouple manipulation from cognition by bolting an external action mechanism onto a vision-language model (VLM)--either quantizing actions into discrete tokens whose errors bear no stable correspondence to physical displacements, or delegating continuous trajectory generation to a separate action expert whose gradients compete with or are blocked from updating the VLM backbone. We argue that manipulation and cognition should instead be embodied in the same model, shifting from a decoupled VLA to an Embodied Vision-Language Model (eVLM). We introduce Concertina, an eVLM that places continuous trajectory representation, blockwise generation, and physical-coordinate supervision directly on the computation and parameter path of a shared diffusion VLM backbone. Each control timestep is encoded as one continuous latent action token in the backbone hidden space, and the shared Transformer reconstructs temporally ordered action blocks via the same corrupt-and-reconstruct operator used for multimodal pretraining. Active action projections account for only 0.21% of parameters in the 4.46B model with zero separate action-generator layers, compared with 16.5%-35.1% dedicated action-expert capacity in representative continuous VLAs. Using our proposed Action Interface Audit, we confirm that cross-time trajectory coupling emerges in the middle and late layers of the shared backbone rather than at the action interfaces. On standard benchmarks, Concertina achieves 98.15% average success on regular LIBERO, 85.9% on the 10,030-task LIBERO-Plus benchmark, and 4.142 average completed chain length on CALVIN ABC-D. Code: https://github.com/lose4578/Concertina Hugging Face: https://huggingface.co/collections/Traly/concertina

Figures & tables

	Representational Coupling		Computational & Parameter Coupling				Generation & Complexity
Paradigm / Method	Cont. action values	Coordinate supervision	Backbone generates traj.	No external action gen.	No pretrained tokenizer/codec	Single action repr. & loss	Parallel / joint generation	Causal temporal order	Iterative refinement	Sequence length $\mathcal{O}(H)$
RT-2 ( Brohan et al., 2023 ) / OpenVLA ( Kim et al., 2024 )	$\times$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\times$	$\times$
FAST / $\pi_{0}$ -FAST ( Pertsch et al., 2025 )	$\times$	$\times$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\times$		$\times$
$\pi_{0}$ ( Black et al., 2024 ) / $\pi_{0.5}$ ( Physical Intelligence et al., 2025 )	$\checkmark$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$
SmolVLA ( Shukor et al., 2025 ) / GR00T N1 ( Bjorck et al., 2025 )	$\checkmark$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$
X-VLA ( Zheng et al., 2025 )	$\checkmark$	$\checkmark$	$\times$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\checkmark$	$\checkmark$

Table 1: Architectural comparison between representative VLA paradigms and our eVLM (

\checkmark

: satisfied;

\times

: not satisfied; : partially satisfied). Columns 1–2 reflect representational coupling (whether actions enter and are supervised in physical coordinates); columns 3–4 reflect computational and parameter coupling (whether the VLM backbone directly generates trajectories without a separate action Transformer, DiT, flow expert, or post-codec generator); the remaining columns summarize training and decoding structure. Coordinate supervision marks for dual discrete-continuous objectives (HybridVLA), latent-codec policy losses (CAT), or discrete-pretrain/continuous-finetune stage switches (Dream-VLA).

Figure 1: Concertina (eVLM) reconstructs continuous action blocks inside a shared diffusion vision-language model. Multimodal inputs and continuous latent action tokens are jointly processed across all layers of the shared Transformer $F_{\theta}$ , combining position-wise continuous coordinate projections ( $E_{\phi},D_{\psi}$ ), packed blockwise teacher forcing during training, and risk-guided blockwise reconstruction at inference. Amber tiles denote complete action vectors; dashed “M” tiles denote masked vectors.

Figure 2: From decoupled VLAs to an Embodied Vision-Language Model (eVLM). Comparison of four architectural routes to robot manipulation: (A) OpenVLA quantizes coordinates into discrete vocabulary tokens (language-objective VLA); (B) $\pi_{0}$ delegates continuous trajectory generation to a separate flow-matching action expert (separate-objective VLA); (C) Dream-VLA retains a shared diffusion VLM backbone but switches to a flow-matching objective for continuous control (switched-objective VLA); (D) Concertina instantiates an eVLM that preserves both the shared diffusion VLM backbone and its native masked block reconstruction operator over continuous latent action tokens.

Discrete & Compressed-Token VLAs
	Regular LIBERO (%)					LIBERO-Plus (%)					CALVIN ABC–D
Method / checkpoint	Spatial	Object	Goal	LIBERO-10	Avg.	Spatial	Object	Goal	LIBERO-10	Overall	AvgLen	SR@1	SR@2	SR@3	SR@4	SR@5
Octo ( Octo Model Team et al., 2024 )	78.9	85.7	84.6	51.1	75.1	–	–	–	–	–	–	–	–	–	–	–
OpenVLA ( Kim et al., 2024 )	84.7	88.4	79.2	53.7	76.5	19.4	14.0	15.1	14.3	15.6	–	–	–	–	–	–
$\pi_{0}$ -Fast ( Pertsch et al., 2025 )	96.4	96.8	88.6	60.2	85.5	74.4	72.7	57.5	43.4	61.6	–	–	–	–	–	–
NORA ( Hung et al., 2025 )	–	–	–	–	–	47.6	34.4	38.8	36.3	39.0	–	–	–	–	–	–

Table 2: Main results across regular LIBERO, LIBERO-Plus, and CALVIN ABC–D, grouped by architectural paradigm. Regular-LIBERO Avg. is the arithmetic mean of its four suite scores. Bold and underline mark the best and second-best result per column. The

\pi_{0.5}

and IMLE-VLA rows follow the comparison reported by Hosseinkhani et al. ( Hosseinkhani et al., 2026 ) . EFFVLA reports suite-level LIBERO-Plus averages, while Concertina and the remaining LIBERO-Plus rows report the official 10,030-task benchmark with Long labeled LIBERO-10. CALVIN reports average completed chain length and sequence success rates (SR@

1

–

5

) on ABC

\rightarrow

Figure 3: Blockwise horizon partition, physical action coherence, and cumulative recipe progression. (a) Regular-LIBERO suite-level and overall success over 200 episodes (40 tasks

\times

5 episodes, Wilson 95% CIs) at fixed 2,400-step finetuning budgets, showing consistent gains across all four suites (

38.5\%\rightarrow 73.0\%

overall). (b) Directional cosine similarity of physical action increments across temporal lags

\Delta t=1\dots 30

over 999,830 AgiBot frames (30 Hz, 95% task-bootstrap CI); enlarging the block size from

K=5

K=10

more than doubles intra-block coherent action mass (

9.8\%\rightarrow 21.3\%

2.17\times

). (c) Official 1,000-sequence CALVIN ABC–D chain survival curves (SR@

1

–

5

) across block partitions (

K=5

vs.

K=10

) and pretraining corruption/weighting schedules (

1.333\rightarrow 1.701\rightarrow 2.087

AvgLen). (d) Cumulative CALVIN ABC–D progression from uniform coordinate weights (

0.41

) to physical weighting, state masking, 60k multi-block pretraining, aligned single-block execution (

4.135

), and two-block continued pretraining (

4.142

Pretraining configuration	Coord. weights $\omega_{d}$	State mask	Early-layer freeze	Visual RRC	AvgLen	SR@1	SR@2	SR@3	SR@4	SR@5
Full pretraining recipe	$500:10:1$	$\checkmark$	Layers 0–1	$\checkmark$	1.837	73.1	50.4	30.5	18.9	10.8
Full trainable backbone, w/o state mask	$500:10:1$	$\times$	None (all 36L)	$\checkmark$	1.589	69.9	42.7	23.9	14.4	8.0
w/o proprioceptive state mask	$500:10:1$	$\times$	Layers 0–1	$\checkmark$	1.093	61.9	30.1	11.2	4.4	1.7
w/o state mask, w/o visual RRC	$500:10:1$	$\times$	Layers 0–1	$\times$	1.027	56.4	26.7	12.3	4.9	2.4
Full trainable backbone, w/o state mask & RRC	$500:10:1$	$\times$	None (all 36L)	$\times$	0.607	40.6	15.2	3.6	1.0	0.3
Uniform coordinate weights, w/o state mask	$1:1:1$	$\times$	Layers 0–1	$\checkmark$	0.535	36.5	13.0	3.0	0.8	0.2

Table 3: Controlled pretraining ablation evaluating physical coordinate weighting (

\omega_{d}

), proprioceptive state masking, early-layer freezing, and random-resized crop (RRC). Every configuration runs 4,800 pretraining updates on AgiBot followed by 2,600 CALVIN finetuning updates (global batch 1,536) and is evaluated on all 1,000 official CALVIN ABC–D chains.

Training & execution configuration	Pretraining stage	Downstream $(H,M,K)$	Exec. $H_{\mathrm{exec}}$	AvgLen	SR@1	SR@2	SR@3	SR@4	SR@5
Multi-block finetuning ( $H=50$ )	60k ( $H=50,M=5,K=10$ )	$(50,5,10)$	$10$	3.535	92.6	82.5	70.2	59.3	48.9
Two-block finetuning ( $H=20$ ), 2-block exec.	60k ( $H=50,M=5,K=10$ )	$(20,2,10)$	$20$	3.768	95.3	85.8	74.8	64.9	56.0
Two-block finetuning ( $H=20$ ), 1-block exec.	60k ( $H=50,M=5,K=10$ )	$(20,2,10)$	$10$	3.924	96.4	89.2	79.9	69.2	57.7
Aligned 1-block finetuning ( $H=10$ )	60k ( $H=50,M=5,K=10$ )	$(10,1,10)$	$10$	4.135	98.7	93.9	84.6	72.1	64.2
Single-block +40k cont. (trainable interfaces)	60k + 40k ( $H=10,M=1,K=10$ )	$(10,1,10)$	$10$	3.786	97.6	89.0	75.5	64.3	52.2
Single-block +40k cont. (frozen interfaces)	60k + 40k ( $H=10,M=1,K=10$ )	$(10,1,10)$	$10$	3.934	97.9	90.3	79.7	67.2	58.3

Table 4: Synergy between multi-block pretraining and receding-horizon closed-loop execution on CALVIN ABC–D (1,000 chains per row, 5,000 finetuning updates). Aligning the downstream finetuning horizon to the

K=10

receding-horizon execution block raises AvgLen from

3.535

4.135

, while retaining multi-block causal-bidirectional context during continued pretraining (

H=20,M=2,K=10

) outperforms single-block continued pretraining (

H=10,M=1,K=10

) by

+0.208

AvgLen (95% paired bootstrap CI

[+0.119,+0.298]

Figure 4: Co-design of action representation and supervision objective in the shared hidden space. (a) Shared-hidden gradient cosine between the main relative-action objective and auxiliary absolute-action objective across eight fixed windows is

-0.891

(

99\%

of action slots

<0

), reducing combined-vs-main alignment to

+0.474

(auxiliary/main norm ratio

0.88\times

). (b) Layerwise parameter update divergence across all 26 active Transformer layers (L2–L27): while relative update norm

\Delta W/W

rises from

4.6\%

9.3\%

with identical overall magnitude (

1.003\times

), the parameter update direction cosine plunges from

0.604

at Layer 2 to

0.138

at Layer 22 (pooled cosine

0.225

). (c) Resulting readout degradation (

5.12\times

teacher action loss,

2.47\times

position error,

1.60\times

normalized MAE) and

92.0\%

closed-loop CALVIN collapse (

2.970\rightarrow 0.237

). (d) Complete

2\times 2\times 2

factorial co-design across 8,000 CALVIN chains on 4.46B BARD-VL-B4, showing consistent gains from probability-valued gripper BCE (

+0.427

AvgLen) and trainable action interfaces (

+0.311

Gripper coordinate objective	Backbone training	Action interfaces ( $E_{\phi},D_{\psi}$ )	AvgLen	SR@1	SR@2	SR@3	SR@4	SR@5
Probability-valued Gripper BCE	Freeze layers 0–1 (34L trainable)	Trainable	3.535	92.6	82.5	70.2	59.3	48.9
Probability-valued Gripper BCE	Full trainable backbone (all 36L)	Trainable	3.525	92.7	82.9	71.1	58.3	47.5
Probability-valued Gripper BCE	Freeze layers 0–1 (34L trainable)	Frozen	3.441	91.1	80.8	68.7	57.0	46.5
Smooth- $L_{1}$ on all coordinates	Freeze layers 0–1 (34L trainable)	Trainable	3.394	91.2	81.6	67.4	55.1	44.1
Probability-valued Gripper BCE	Full trainable backbone (all 36L)	Frozen	3.129	87.0	74.5	60.5	50.0	40.9
Smooth- $L_{1}$ on all coordinates	Full trainable backbone (all 36L)	Trainable	2.944	87.7	71.6	57.3	43.8	34.0

Table 5: Full

2\times 2\times 2

factorial ablation on 4.46B BARD-VL-B4 (initialized from the 60k pretraining checkpoint at

H=50,M=5,K=10

; 5,000 CALVIN updates at batch size 1,536; evaluated on all 1,000 official CALVIN ABC–D chains per configuration, 8,000 chains total).

Checkpoint	Action representation	Trajectory generator	$H$	Valid $D$	Interface NLR	Full NLR	ASCR	Trajectory coupling locus
$\pi_{0}$ base ( Black et al., 2024 )	32D continuous	External flow expert	50	32	0	8.141%	16.51%	External action expert
$\pi_{0.5}$ base ( Physical Intelligence et al., 2025 )	32D continuous	External flow expert	50	32	0	7.141%	19.17%	External action expert
SmolVLA base ( Shukor et al., 2025 )	32D continuous	External flow expert	50	32	–	–	22.19%	External action expert
X-VLA base ( Zheng et al., 2025 )	20D continuous	External 24L Transformer	50	20	–	–	35.10%	External 24L Transformer
Concertina (eVLM) pretraining	20D canonical latent	Shared VLM backbone (36L)	50	20	0	4.058%	0.21%	Shared VLM backbone
Concertina (eVLM) CALVIN	10D absolute latent	Shared VLM backbone (36L)	50	10	0	4.103%	0.21%	Shared VLM backbone

Table 6: Action Interface Audit: Trajectory Coupling Attribution (TCA) and Action-Specific Capacity Ratio (ASCR) across external-expert VLAs and our eVLM (Concertina). All three Concertina rows evaluate 4.46B BARD-VL-B4 checkpoints at

H=50

under deterministic float32 math-only SDPA (

\epsilon=0.02

). Under packed blockwise BD3 attention (

M=5,K=10

), the three Concertina checkpoints likewise yield zero interface NLR and

2.595\%

3.163\%

, and

1.078\%

full-path NLR, respectively.

Figure 5: Action Interface Audit across representational, computational, and parameter coupling. (a) Physical action error span (

p_{05}

–

p_{50}

–

p_{95}

) and structural decoding fragility under single-token substitutions: continuous coordinates (Concertina) preserve unit error geometry (

\mathrm{EGC}=1.0\times

0\%

invalid), whereas 256-bin quantization induces a

25.1\times

error span and compressed DCT+BPE tokenizers (FAST+, MolmoAct2-FAST) suffer

82.2\%

and

77.9\%

invalid trajectory shapes while perturbing

30/30

steps on valid decodes. (b) Normalized Jacobian singular value spectrum (

\sigma_{i}/\sigma_{1}

) of Concertina’s continuous action encoder

\partial E_{\phi}(a)/\partial a\in\mathbb{R}^{2560\times 20}

across pretraining updates, maintaining a full-rank 20D manifold (effective rank

18.63/20

, Spearman

\rho=0.86

–

0.93

). (c) Layerwise emergence of cross-time trajectory coupling (

\mathrm{NLR}

|t-t_{0}|\geq 6

) across the 36-layer BARD-VL-B4 backbone versus

0.0\%

for the position-wise interface (

D_{\psi}\circ E_{\phi}

) alone. (d) Action-Specific Capacity Ratio (

\mathrm{ASCR}

) across VLA families, showing that Concertina uses

78\times

–

167\times

fewer action-specific parameters (

0.21\%

) than external-expert architectures (

16.5\%

–

35.1\%

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Validity masks separate embodiment coordinates, reconstruction selection, and time padding. The canonical 20D layout provides fixed slots for single-arm and bimanual embodiments; coordinate validity

q

, masking indicator

m

, and timestep validity

v

play orthogonal roles in the supervised objective.

Figure 7: Risk-guided decoding scores candidate action tokens and supports selective iterative refinement. The token-risk head predicts log reconstruction error from detached hidden states; candidate slots are committed in order of confidence, and optional refinement rounds remask and update low-scoring positions when the predicted score improves.

Dataset source	Subsets / views	Episodes	Frames	Sampling weight
AgiBotWorld–Alpha	34 / 34	32,735	46,156,708	$44.44\%$
DROID (30 Hz aligned)	1 / 2	97,685	27,348,488	$33.33\%$
RoboMIND Franka	1 / 1	5,333	943,842	$11.11\%$
RoboMIND AgileX	1 / 1	8,906	5,387,380	$7.78\%$
RoboMIND FR3	1 / 1	1,774	375,807	$3.33\%$
Total pretraining mixture	38 / 39	146,433	80,212,225	100.00%

Appendix

Table 7: Multi-embodiment pretraining dataset mixture totaling 146,433 episodes and 80.2M frames across five robot manipulation sources.

Stage	Dataset	Updates	Configuration
Stage-1 Pretraining	Multi-embodiment mixture (Table 7 )	60,000	Canonical 20D actions ( $H=50,M=5,K=10$ ); active-dimension validity masks; 30 Hz temporal alignment
Stage-2 Continued Pretraining	Multi-embodiment mixture (Table 7 )	30,000	Two-block continuation ( $H=20,M=2,K=10$ ); mask-only inverse- $t$ objective; probability-valued gripper BCE
Downstream Finetuning	CALVIN ABC–D	5,000	Absolute end-effector actions; single-block horizon ( $H=10,M=1,K=10$ )
Downstream Finetuning	Regular LIBERO / LIBERO-Plus	5,000	Relative end-effector actions; single-block horizon ( $H=10,M=1,K=10$ )

Appendix

Table 8: Overview of pretraining and downstream finetuning stages.

Hyperparameter	Value
Compute & batch size	8 GPUs; global batch size 1,536 (192/GPU) for pretraining and CALVIN finetuning; global batch size 448 (56/GPU) for LIBERO finetuning
Closed-loop execution	$H=10,K=10,M=1$ ; proposal budget $D=4$ ; refinement steps $R=5$ ; executed horizon $H_{\mathrm{exec}}=10$
Warmup & checkpointing	80-step linear warmup; checkpoints saved every 1,250 updates; seed 42

Appendix

Table 9: Shared optimization and closed-loop inference configuration.

			Offline 32-Window MAE		CALVIN Closed-Loop (100 chains)
Decoding strategy	Mean BB passes	Latency $p_{50}$ (ms)	CALVIN MAE	LIBERO MAE	AvgLen	SR@1 (%)	SR@3 (%)	SR@5 (%)
$D=1,R=0$ (single-pass parallel)	$1.00$	$65.8$	$0.06282$	$0.27692$	$3.92$	$95.0$	$80.0$	59.0
$D\leq 4,R=0$ (proposals only)	$2.00$	$91.7$	$0.06279$	$0.27326$	$3.90$	$93.0$	$79.0$	$56.0$
$D\leq 4,R=5$ (with risk refinement)	$7.00$	$223.0$	0.06196	0.26852	4.04	97.0	82.0	59.0

Appendix

Table 10: Offline reconstruction accuracy (32 windows,

2{,}880

non-gripper coordinates per arm) and 100-sequence closed-loop CALVIN ABC–D performance across decoding budgets at fixed checkpoint identity (

H=10,K=10,M=1

). Risk-guided refinement (

D\leq 4,R=5

) reduces offline MAE on 25/32 CALVIN windows (

\Delta=-0.00084

, 95% paired bootstrap CI

[-0.00125,-0.00043]

) and 25/32 LIBERO windows (

\Delta=-0.00840

vs.

D=1,R=0

, 95% CI

[-0.01316,-0.00400]

), while single-pass parallel decoding (

D=1,R=0

) already achieves

3.92

AvgLen at

65.8\text{\,ms}

median latency (

6.58\text{\,ms}

per executed control step).

Method	Decoding budget	Chunk $H$	Exec. $E$	Success (50 eps)	Mean capped steps
Concertina	$D=1,R=0$ (1 pass)	$10$	$10$	50/50 (100.0%)	102.40
Concertina	$D\leq 4,R=5$ (7 passes)	$10$	$10$	50/50 (100.0%)	$102.84$
Concertina	$D\leq 4,R=5$ (replan every step)	$10$	$1$	46/50 (92.0%)	$118.82$
$\pi_{0.5}$ ( Physical Intelligence et al., 2025 )	Flow 2 steps	$50$	$10$	49/50 (98.0%)	$105.92$
$\pi_{0.5}$ ( Physical Intelligence et al., 2025 )	Flow 5 steps	$50$	$10$	49/50 (98.0%)	$106.08$
$\pi_{0.5}$ ( Physical Intelligence et al., 2025 )	Flow 10 steps (default)	$50$	$10$	48/50 (96.0%)	$115.26$

Appendix

Table 11: Closed-loop decoding budget and replanning frequency on LIBERO-Spatial (

10\text{ tasks}\times 5\text{ initial states}=50\text{ episodes}

per arm; failed episodes capped at 280 steps). Executing a full

E=10

block before replanning outperforms step-by-step replanning (

E=1

+8.0\text{ pp}

) by preserving intra-block motion coherence, whereas open-loop 50-step execution in

\pi_{0.5}

(

E=50

) degrades success by

-12.0\text{ pp}

(95% paired bootstrap CI

[-24.0,-2.0]

Representation	Unperturbed MAE	Mean tokens	EGC ( $p_{95}/p_{05}$ )	Invalid shape (%)	Valid RMSE median
Continuous coordinates (Concertina)	$0.0000$	$30.0$	$1.00$	$0.00$	n/a
Uniform quantizer (256 bins)	$0.0023$	$210.0$	$25.14$	$0.00$	n/a
FAST+ ( Pertsch et al., 2025 )	$0.0200$	$57.7$	–	$82.23$	$0.0526$
MolmoAct2-FAST (repaired)	$0.0200$	$90.2$	–	$77.93$	$0.0690$

Appendix

Table 12: Error-Geometry Consistency (EGC) and representation diagnostics on 512 CALVIN windows (

H=30,D=7

). Unperturbed MAE measures round-trip reconstruction error; EGC, invalid shape, and valid RMSE refer to single-token substitution.

Run	Windows	FAST+ invalid (%)	FAST+ RMSE median	Molmo invalid (%)	Molmo RMSE median	Time support
Canonical	512	82.23	0.0526	77.93	0.0690	30/30
Expanded	8,192	84.52	0.0590	79.24	0.1173	30/30
Four-seed range	$4\times 2{,}048$	81.30–85.11	0.0552–0.0631	76.22–79.00	0.0966–0.1171	30/30

Appendix

Table 13: Expanded representation stress test across 512,

4\times 2{,}048

, and 8,192 windows. Invalid shape is the fraction of single-token edits that fail to decode into

30\times 7

coefficients. Time support counts affected steps with a coordinate change exceeding 0.01 normalized units.

Checkpoint (4.46B BARD-VL-B4)	$\epsilon=0.01$	$\epsilon=0.02$	$\epsilon=0.04$	Interface NLR
Concertina pretraining ( $H=50,D=20$ )	$4.058\%$	$4.058\%$	$4.056\%$	$0$
Concertina downstream (CALVIN, $D=10$ )	$4.103\%$	$4.103\%$	$4.102\%$	$0$
Concertina downstream (LIBERO, $D=10$ )	$1.215\%$	$1.217\%$	$1.216\%$	$0$

Appendix

Table 14: Full-path NLR stability across finite-difference step sizes

\epsilon\in\{0.01,0.02,0.04\}

on the 4.46B BARD-VL-B4 checkpoints. Each cell averages 11 coordinate probes with zero repeated-forward jitter; interface-only NLR is identically zero.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,216 papers

Jun 29, 2026

Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervision

Cross-embodiment transfer in vision-language-action (VLA) models remains challenging because low-level state and action spaces differ fundamentally across robot platforms. We observe that the high-level cognitive process underlying manipulation, including scene perception, object identification, task planning, and sub-task decomposition, is largely shared across embodiments. Based on this observation, we present ZR-0, a 2.6 billion parameter end-to-end VLA model that uses dense Embodied Chain-of-Thought (ECoT) supervision to align cross-embodiment representations within the vision-language model (VLM). ZR-0 adopts a dual-stream architecture: a pre-trained VLM (System 2) generates structured ECoT reasoning during training, while a Diffusion Transformer-based action expert (System 1) produces continuous action chunks via flow matching. The two components are coupled through cross-attention, with an attention mask that restricts the action expert to input prompt features only, enabling ECoT generation to be entirely skipped at inference without any performance loss. ZR-0 is pre-trained on ProcCorpus-60M, a large-scale dataset comprising approximately 60 million frames (approximately 1,000 hours) from over 400K trajectories, with dense ECoT annotations covering 96.8% of all frames. We evaluate ZR-0 on three simulation benchmarks spanning single-arm (LIBERO), bimanual (RoboTwin 2.0), and humanoid (RoboCasa GR-1 Tabletop) embodiments, as well as real-world experiments on the xArm platform, demonstrating strong performance across all settings. Code and model checkpoints are available at https://github.com/RUCKBReasoning/ZR-0.

Haoyang Li, Guanlin Li, Youhe Feng +9

1Renmin University of China · 2Zhipu AI

Cross-Embodiment Robot Learning Robotic Manipulation Vision-Language-Action Models Reasoning for VLA Models

Ask PDF

Mar 19, 2026

Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models

A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning. However, if the basket is shifted five centimeters to the left, it drops the soup off the table. We developed an interpretability pipeline to investigate what such a policy is computing. Injecting task A's activations into the action expert while the policy looks at task B's scene makes

π_{0.5}

execute task A's motor trajectory in 99.6% of episodes (

n{=}1{,}968

). The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report. The binding holds on a physical UR10e: directed replay redirects the commanded task in 26/26 rollouts in one direction and 18/26 in the other, with no-injection and random-direction controls at the floor, and once the source object is moved 10-15 cm the arm grasps and releases at the recorded coordinates in 50/50 trials, picking up the wrong block when one sits there and closing on empty air when none does. Applying the same interventions to five VLAs on LIBERO, MetaWorld, SimplerEnv, and ALOHA over

420{,}000+

rollouts yields three further findings: every architecture encodes the instruction, and four of the five weight it below vision whenever the scene alone identifies the goal; under the standard replacement write-back the SAE pooling that preserves behavior is architecture-specific (

π_{0.5}

per-token, SmolVLA indifferent), and goal selection and motor generation separate into VLM and expert pathways wherever the two are distinguishable. We release Action Atlas (https://action-atlas.com), an interactive platform covering all 520+ trained SAEs, 79 identified concepts, and the scored UR10e rollouts. Code: https://cwru-aism.github.io/vla-interp-page/

Bryce Grant, Xijia Zhao, Peng Wang

Case Western Reserve University

Vision-Language-Action Models Robotics

Ask PDF

Jul 9, 2026

CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding

Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult to isolate what the VLM contributes to control. Directly converting pretrained VLMs into VLAs with minimal architectural change offers a more transparent path to understanding how VLM capabilities transfer across model scales. The core obstacle is output-distribution mismatch: predicting actions as bare numeric token sequences moves generation away from the VLM's pretrained language distribution, degrading the capabilities we seek to preserve. To address this, we propose CLAP (Causal Language-Action Prediction), which prepends each numeric action sequence with a natural-language action description, causally conditioning precise action-token prediction on a language-action plan without modifying the backbone architecture. With only 4-5 hours of training, 2B CLAP achieves 90.8% on LIBERO (+14.9 pt over VLA-0) and improves robustness on LIBERO-PRO under language, object, and spatial perturbations. Ablations attribute the gains to action-grounded language semantics, and on a real UR5e robot 2B CLAP reaches 60% success against 15% for matched VLA-0. We release CLAP at 0.8B, 2B, and 4B as an open-weight, multi-scale compact VLA family from a single VLM lineage, enabling controlled analysis of VLM-to-VLA capability transfer.

Yuri Ishitoya, Jeremy Siburian, Masashi Hamaya +5

Ochanomizu University · University of Tokyo · OMRON SINIC X Corp.

Robotic Manipulation VLM Adaptation Vision-Language-Action Models Robustness of VLA Models VLA Model Training VLA Model Adaptation+1

Ask PDF

Abstract

Vision-Language-Action Models

Explore similar work

Training Vision-Language-Action Models with Dense Embodied Chain-of-Thought Supervision

Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models

CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding