The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface

Yuxiang Liu; Lizhi Yang; Fengze Xie; Aaron Ames; Yisong Yue

cs.AISep 28, 2026

The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface

Authors: Yuxiang Liu, Lizhi Yang, Fengze Xie, Aaron Ames, Yisong Yue

Organizations: University of California, Berkeley · California Institute of Technology

Abstract

Vision-language-action (VLA) policies connect a pretrained vision-language backbone to an action head through a latent interface, but which backbone layers this interface should expose remains unclear. We study single-layer selection and multi-layer fusion for frozen backbones across three pretrained models and two manipulation benchmarks, LIBERO and CALVIN, with three policy-training seeds per configuration. Across three fusion mechanisms and three layer-subset strategies, 47 of 54 configurations underperform the best observed single-layer policy. Our stastical analysis further confirms that fusion's advantage is very limited. However, the best layer varies substantially across backbones and benchmarks, making layer selection consequential and exhaustive policy sweeps expensive. We further derive a reweighting equivalence between the proposed information-bottleneck objectives for action-conditioned InfoNCE and action prediction, motivating InfoNCE as a proxy for layer quality. Empirically, InfoNCE provides the most consistent positive association with policy success among four evaluated proxies. Selecting the layer with the highest InfoNCE score requires 9-33 times less GPU compute than exhaustive policy sweeps and reduces mean selection regret from 17.89 percentage points for deepest-layer selection to 3.71 points across six settings. Its mean regret is close to the 3.28-3.50 points achieved by fixed-layer heuristics optimized retrospectively using all six oracle sweeps, without requiring closed-loop evaluations during selection.

Figures & tables

LIBERO
	Cosmos-Reason2-2B			Qwen3-VL-2B			ABot-Pretrain-4B
Layer config	APT	VLA-Ad.	BEHAV.	APT	VLA-Ad.	BEHAV.	APT	VLA-Ad.	BEHAV.
best	$-1.8$	$-3.4$	$-4.5$	$-4.1$	$-4.1$	$-5.4$	$-2.1$	$-1.1$	$-0.1$
even	$-3.2$	$-4.7$	$-9.5$	$-7.4$	$-13.4$	$-4.5$	$-3.9$	$-53.0$	$-32.9$
last	$-3.6$	$-14.9$	$-20.2$	$-8.0$	$-21.1$	$-26.7$	$-1.4$	$\mathbf{+0.2}$	$\mathbf{+0.0}$
CALVIN

TABLE I: Layer fusion minus the single best layer, across benchmarks and base models

	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
LIBERO
Oracle	129.1	127.4	152.5
InfoNCE	10.6	14.1	4.6
CALVIN
Oracle	119.1	94.0	133.7
InfoNCE	7.4	7.5	13.4

TABLE II: GPU-Hours

	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
LIBERO
Oracle	129.1	127.4	152.5
InfoNCE	10.6	14.1	4.6
CALVIN
Oracle	119.1	94.0	133.7
InfoNCE	7.4	7.5	13.4

TABLE II: GPU-Hours

LIBERO
	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
Optimal layer $l^{\star}$	15	3	31
Succ. at $l^{\star}$ (%)	76.3	76.8	98.4
Succ. at deepest (%)	62.9	55.6	96.6
CALVIN
Optimal layer $l^{\star}$	11	16	12

TABLE III: Optimal layer per benchmark and base model

Fig. 1: The $\log N-\mathcal{L}_{\text{infoNCE}}$ -versus-layer curve closely tracks the non-monotone LIBERO-success-versus-layer curve. The left column shows LIBERO success and the right column shows InfoNCE for Cosmos-2B (top) and Abot-Pretrain (bottom).

LIBERO
	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
InfoNCE	$\mathbf{+0.82}$	$\mathbf{+0.85}$	$\mathbf{+0.94}$
DiME	$+0.72$	$+0.62$	$+0.85$
Prompt-level entropy	$-0.82$	$-0.79$	$-0.03$
Dataset-level entropy	$-0.69$	$-0.53$	$+0.77$
CALVIN

TABLE IV: Spearman correlation of proxy metrics with oracle success across layers

Regret (success %)	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
LIBERO
InfoNCE-argmax	2.1 (L10)	7.3 (L10)	1.8 (L36)
Deepest layer	13.4 (L28)	21.3 (L28)	1.8 (L36)
Oracle constant, min-mean (L16)	4.0	8.3	3.2
Oracle constant, min-max (L15)	0.0	6.4	5.8
CALVIN

TABLE V: Regret of layer-selection schemes

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

LIBERO
	Cosmos-Reason2-2B			Qwen3-VL-2B			ABot-Pretrain-4B
Layer config	APT	VLA-Ad.	BEHAV.	APT	VLA-Ad.	BEHAV.	APT	VLA-Ad.	BEHAV.
best	$+7.2$	$-2.0$	$-3.1$	$+2.2$	$-2.2$	$-3.6$	$-0.6$	$-0.7$	$+0.3$
even	$+5.8$	$+4.3$	$-0.6$	$-1.1$	$-7.1$	$+1.8$	$+60.0$	$+10.9$	$+31.0$
last	$-1.7$	$-4.8$	$-10.2$	$-3.2$	$-2.5$	$-8.1$	$+0.5$	$+1.0$	$+0.8$
CALVIN

Appendix

TABLE VI: Layer fusion minus its shallowest constituent layer, across benchmarks and base models

Fig. 2: Layer quality depends non-monotonically on depth and varies across training seeds. Overall success rate versus VLM layer for three backbones (rows) on LIBERO (left) and CALVIN (right). Bold curves show the mean across three training seeds, shaded regions show one standard deviation, and faint curves show individual seeds. See Sec. IV-A for experiment details.

Fig. 3: APT-Repeat[ $i$ ] versus the single-layer GR00T policy across all 36 layers of Abot-Pretrain on CALVIN. Curves show the mean over the three policy seeds and shaded bands show one standard deviation. Success is reported in percent. Dotted lines mark the three APT fusion configurations (APT-Last, APT-best, APT-even) under the same backbone and benchmark. GR00T values are taken from the single-layer oracle sweep of Sec. IV-A ; APT-Repeat values are measured with the protocol os Appendix A-F .

Fig. 4: Finite- $N$ slack of the InfoNCE score for Abot-Pretrain on LIBERO. The line shows $\log N-\mathcal{L}_{\text{infoNCE}}$ averaged over three proxy seeds, and the shaded band extends it upward by the direct slack estimate $\widehat{\Delta}_{N}$ of Eq. equation 23 , also averaged over the seeds. Smaller $\Delta_{N}$ means empirically tighter bound in Eq. 6 . Each critic uses $N=512$ and is scored on $n=16{,}384$ held-out pairs, with a normalization set of $m_{1}=15{,}383$ of the other $16{,}383$ held-out latents per query.

Benchmark	Backbone	Method	Best	Even	Last
LIBERO	Cosmos	APT	.666	.969	.790
		VLA-Adapter	.877	.985	1.000
		BEHAVIOR	.983	.923	1.000
LIBERO	Qwen	APT	.900	.990	.870
		VLA-Adapter	.942	.998	.999
		BEHAVIOR	.902	.898	.999

Appendix

TABLE VII: All fusion methods other than APT do not demonstrate statistically significant advantage over the single optimal layer. Each cell reports the p-value for a fusion configuration compared with the optimal single layer. Lower values indicate stronger statistical evidence that fusion outperforms the optimal single layer;

p<0.05

indicates a statistically nominal advantage. See Appendix A-H for calculation of p-value.

Dataset	Demos	Frames	Samples	Train demos	Val. demos	Train/val. samples
LIBERO-Spatial	432	52,970	46,490	410	22	44,116 / 2,374
LIBERO-Object	454	66,984	60,174	431	23	57,192 / 2,982
LIBERO-Goal	428	52,042	45,622	407	21	43,497 / 2,125
LIBERO-10	379	101,469	95,784	360	19	90,984 / 4,800
LIBERO total	1,693	273,465	248,070	1,608	85	235,789 / 12,281
CALVIN ABC subset	4,608	274,466	205,346	4,378	230	195,438 / 9,908

Appendix

TABLE VIII: Training data and fixed trajectory-level validation splits. “Samples” are valid one-observation, 16-action windows.

Benchmark	Backbone checkpoint	Layers	Width	Oracle seeds	Episodes/task
LIBERO	nvidia/Cosmos-Reason2-2B	28	2048	42, 1234, 2718	10
LIBERO	Qwen/Qwen3-VL-2B-Instruct	28	2048	42, 1234, 2718	10
LIBERO	local ABot-Pretrain VLM	36	2560	42, 1234, 4321	10
CALVIN	nvidia/Cosmos-Reason2-2B	28	2048	42, 1234, 2718	15
CALVIN	Qwen/Qwen3-VL-2B-Instruct	28	2048	42, 1234, 2718	15
CALVIN	local ABot-Pretrain VLM	36	2560	42, 1234, 2718	15

Appendix

TABLE IX: Actual backbone–benchmark settings. Width is the language-tower hidden size. The seed order is the order used in per-seed tables below.

Pair	Best-16	Best-8
LIBERO/Cosmos	1,2,4,5,6,7,8,9,10,11,13,14,15,16,17,19	5,6,8,11,12,14,15,17
LIBERO/Qwen	1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17	2,3,4,5,8,9,11,13
LIBERO/ABot	18,21,22,24,25,26,27,28,29,30,31,32,33,34,35,36	26,27,28,29,31,32,33,35
CALVIN/Cosmos	3,5,8,9,10,11,12,13,14,15,16,17,18,19,20,22	10,11,12,13,14,15,16,17
CALVIN/Qwen	2,3,9,10,11,12,13,14,15,16,17,18,19,20,21,22	12,13,14,15,16,17,18,19
CALVIN/ABot	5,10,11,12,13,14,15,16,17,18,19,20,21,22,23,25	11,12,13,15,16,18,21,22

Appendix

TABLE X: Exact Best layer sets. APT uses Best-16; VLA-Adapter and BEHAVIOR use Best-8.

Benchmark	Backbone	$l^{*}$	Per-seed success	Mean	Sample SD
LIBERO	Cosmos	15	76.8 / 74.5 / 77.6	76.3	1.6
LIBERO	Qwen	3	75.6 / 79.8 / 75.1	76.8	2.6
LIBERO	ABot	31	98.2 / 98.3 / 98.8	98.4	0.3
CALVIN	Cosmos	11	58.3 / 67.7 / 67.3	64.4	5.3
CALVIN	Qwen	16	63.7 / 68.7 / 61.7	64.7	3.6
CALVIN	ABot	12	68.1 / 65.1 / 65.1	66.1	1.7

Appendix

TABLE XI: Per-seed success at the layer selected by the three-seed oracle curve. Values are percent. The triple follows the seed order in Table IX .

Explore similar work

Jun 12, 2026cs.CV

Encoder Winners Do Not Reliably Transfer Across VLA Backbone Scale: A Frozen-Backbone Grafting Diagnostic

Vision-language-action (VLA) policies typically inherit their vision encoder from upstream VLM releases, but it is unclear whether an encoder choice validated on a small VLA transfers to a larger backbone. We introduce a frozen-backbone grafting diagnostic: the vision tower of a released VLA is replaced by a candidate encoder under a fixed protocol (adaptive average pooling, LayerNorm, and a single trainable linear projector), with the language model and action expert frozen. Across four encoders, two LIBERO suites, two backbones (SmolVLA-450M and

π_{0.5}

-3.3B), and two-to-three seeds per cell (40 main grafting runs plus native, LoRA, pooling, and zero-/shuffled-image controls, all scored by offline action MSE), the small-backbone winner does not reliably select the large-backbone top tier: SigLIP is best on SmolVLA across both suites, while on

π_{0.5}

DINOv2-small leads the spatial suite and the object suite is a seed-sensitive near-tie band; three of the four backbone-suite comparisons (and 11 of 12 seed-level cells) support backbone-dependent rankings. The grafting wrapper is itself non-neutral with opposite sign across backbones (+45-56% MSE on the SmolVLA native tower, -50-52% on

π_{0.5}

), so all conclusions are conditional on the fixed grafting protocol. We position frozen grafting as a cheap target-backbone diagnostic to run before committing to an encoder at scale, not as a closed-loop deployment claim.

Qingping Zeng, Fei She

Tsinghua University

Sentence Encoders Encoders Diagnosis

Ask PDF

Sep 5, 2026cs.AI

LayerRoute: Action-Conditioned Mixture-of-Layers Routing for Vision-Language-Action Policies

Vision-Language-Action (VLA) policies leverage pretrained vision-language models (VLMs) to guide action generation for robot control. VLMs provide hierarchical visual-semantic representations that evolve across layers, from local visual geometry to abstract, language-aligned semantics; different manipulation tasks may therefore require different mixtures of layer representations. Meanwhile, the action module maintains intermediate representations that evolve throughout action computation and may provide useful information for subsequent decisions. However, existing VLA interfaces offer limited flexibility in representation access: VLM information is exposed through fixed layer assignments for each action layer, while intermediate action states are only propagated implicitly through residual streams without explicit reuse. We introduce LayerRoute, an action-conditioned representation routing interface that enables adaptive access to VLM layers and action representations. The Layer Mixture Router dynamically forms mixtures of cached VLM representations, while Action-State Reread reuses earlier action representations. Across diverse simulation and real-world benchmarks, LayerRoute consistently improves StarVLA-

π

and

π_{0.5}

, achieving up to 7.2 gains on LIBERO Long with only 0.31% / 3.87% additional parameters. Ablation studies validate the benefit of action-conditioned layer routing, while routing analyses reveal structured allocation patterns across action layers and task settings.

Zheng Lu, Haoran Liao, Wanqi Zhong +9

Tsinghua University · Zhejiang University · Nanyang Technological University +2

Topic pending

Ask PDF

Aug 6, 2026cs.RO

LIRA: Local Cross-Layer Information Routing for Vision-Language-Action Decoding

Vision-Language-Action (VLA) models transform representations from pretrained vision-language models (VLMs) into robot actions, yet the interface that routes intermediate VLM features into action decoders remains underexplored. Existing designs either expose only a narrow part of the representation hierarchy or rigidly match each decoder block to one VLM layer, restricting access to complementary task evidence across depths. We introduce LIRA, a local cross-layer action-conditioning mechanism that formulates VLM-to-action conditioning as depth-aware information routing. LIRA operates on task-token features and LIRA Query features derived from intermediate VLM states, then assigns each Parallel Fusion Block a depth-aligned local window centered on its corresponding VLM layer. Parallel Fusion Blocks aggregate neighboring LIRA Query features and integrate them with task-token features and proprioceptive inputs before action prediction. This routing interface leaves the backbone architecture, action decoder, and supervised training recipe unchanged. Across LIBERO, LIBERO-Plus, CALVIN ABC

\rightarrow

D, and real-world manipulation, LIRA improves the principal aggregate metrics over the VLA-Adapter baseline under the same 0.5B-parameter configuration. In zero-shot transfer to LIBERO-Plus, LIRA increases average success from 59.1% to 78.0%, an 18.9-point gain indicating improved robustness under controlled distribution shifts.

Zhewei Zhang, Puyue Wang, Guanren Qiao +10

Cross-Layer Interactions

Ask PDF