The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface

Yuxiang Liu; Lizhi Yang; Fengze Xie; Aaron Ames; Yisong Yue

cs.AISep 28, 2026

The Layer Mystery of VLA: An Information-Theoretical Analysis of VLA Latent Interface

Authors: Yuxiang Liu, Lizhi Yang, Fengze Xie, Aaron Ames, Yisong Yue

Organizations: University of California, Berkeley · California Institute of Technology

Abstract

Vision-language-action (VLA) policies connect a pretrained vision-language backbone to an action head through a latent interface, but which backbone layers this interface should expose remains unclear. We study single-layer selection and multi-layer fusion for frozen backbones across three pretrained models and two manipulation benchmarks, LIBERO and CALVIN, with three policy-training seeds per configuration. Across three fusion mechanisms and three layer-subset strategies, 47 of 54 configurations underperform the best observed single-layer policy. Our stastical analysis further confirms that fusion's advantage is very limited. However, the best layer varies substantially across backbones and benchmarks, making layer selection consequential and exhaustive policy sweeps expensive. We further derive a reweighting equivalence between the proposed information-bottleneck objectives for action-conditioned InfoNCE and action prediction, motivating InfoNCE as a proxy for layer quality. Empirically, InfoNCE provides the most consistent positive association with policy success among four evaluated proxies. Selecting the layer with the highest InfoNCE score requires 9-33 times less GPU compute than exhaustive policy sweeps and reduces mean selection regret from 17.89 percentage points for deepest-layer selection to 3.71 points across six settings. Its mean regret is close to the 3.28-3.50 points achieved by fixed-layer heuristics optimized retrospectively using all six oracle sweeps, without requiring closed-loop evaluations during selection.

Figures & tables

LIBERO
	Cosmos-Reason2-2B			Qwen3-VL-2B			ABot-Pretrain-4B
Layer config	APT	VLA-Ad.	BEHAV.	APT	VLA-Ad.	BEHAV.	APT	VLA-Ad.	BEHAV.
best	$-1.8$	$-3.4$	$-4.5$	$-4.1$	$-4.1$	$-5.4$	$-2.1$	$-1.1$	$-0.1$
even	$-3.2$	$-4.7$	$-9.5$	$-7.4$	$-13.4$	$-4.5$	$-3.9$	$-53.0$	$-32.9$
last	$-3.6$	$-14.9$	$-20.2$	$-8.0$	$-21.1$	$-26.7$	$-1.4$	$\mathbf{+0.2}$	$\mathbf{+0.0}$
CALVIN

TABLE I: Layer fusion minus the single best layer, across benchmarks and base models

	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
LIBERO
Oracle	129.1	127.4	152.5
InfoNCE	10.6	14.1	4.6
CALVIN
Oracle	119.1	94.0	133.7
InfoNCE	7.4	7.5	13.4

TABLE II: GPU-Hours

	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
LIBERO
Oracle	129.1	127.4	152.5
InfoNCE	10.6	14.1	4.6
CALVIN
Oracle	119.1	94.0	133.7
InfoNCE	7.4	7.5	13.4

TABLE II: GPU-Hours

LIBERO
	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
Optimal layer $l^{\star}$	15	3	31
Succ. at $l^{\star}$ (%)	76.3	76.8	98.4
Succ. at deepest (%)	62.9	55.6	96.6
CALVIN
Optimal layer $l^{\star}$	11	16	12

TABLE III: Optimal layer per benchmark and base model

Fig. 1: The $\log N-\mathcal{L}_{\text{infoNCE}}$ -versus-layer curve closely tracks the non-monotone LIBERO-success-versus-layer curve. The left column shows LIBERO success and the right column shows InfoNCE for Cosmos-2B (top) and Abot-Pretrain (bottom).

LIBERO
	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
InfoNCE	$\mathbf{+0.82}$	$\mathbf{+0.85}$	$\mathbf{+0.94}$
DiME	$+0.72$	$+0.62$	$+0.85$
Prompt-level entropy	$-0.82$	$-0.79$	$-0.03$
Dataset-level entropy	$-0.69$	$-0.53$	$+0.77$
CALVIN

TABLE IV: Spearman correlation of proxy metrics with oracle success across layers

Regret (success %)	Cosmos-2B (28L)	Qwen-2B (28L)	Abot-4B (36L)
LIBERO
InfoNCE-argmax	2.1 (L10)	7.3 (L10)	1.8 (L36)
Deepest layer	13.4 (L28)	21.3 (L28)	1.8 (L36)
Oracle constant, min-mean (L16)	4.0	8.3	3.2
Oracle constant, min-max (L15)	0.0	6.4	5.8
CALVIN

TABLE V: Regret of layer-selection schemes

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

LIBERO
	Cosmos-Reason2-2B			Qwen3-VL-2B			ABot-Pretrain-4B
Layer config	APT	VLA-Ad.	BEHAV.	APT	VLA-Ad.	BEHAV.	APT	VLA-Ad.	BEHAV.
best	$+7.2$	$-2.0$	$-3.1$	$+2.2$	$-2.2$	$-3.6$	$-0.6$	$-0.7$	$+0.3$
even	$+5.8$	$+4.3$	$-0.6$	$-1.1$	$-7.1$	$+1.8$	$+60.0$	$+10.9$	$+31.0$
last	$-1.7$	$-4.8$	$-10.2$	$-3.2$	$-2.5$	$-8.1$	$+0.5$	$+1.0$	$+0.8$
CALVIN

Appendix

TABLE VI: Layer fusion minus its shallowest constituent layer, across benchmarks and base models

Fig. 2: Layer quality depends non-monotonically on depth and varies across training seeds. Overall success rate versus VLM layer for three backbones (rows) on LIBERO (left) and CALVIN (right). Bold curves show the mean across three training seeds, shaded regions show one standard deviation, and faint curves show individual seeds. See Sec. IV-A for experiment details.

Fig. 3: APT-Repeat[ $i$ ] versus the single-layer GR00T policy across all 36 layers of Abot-Pretrain on CALVIN. Curves show the mean over the three policy seeds and shaded bands show one standard deviation. Success is reported in percent. Dotted lines mark the three APT fusion configurations (APT-Last, APT-best, APT-even) under the same backbone and benchmark. GR00T values are taken from the single-layer oracle sweep of Sec. IV-A ; APT-Repeat values are measured with the protocol os Appendix A-F .

Fig. 4: Finite- $N$ slack of the InfoNCE score for Abot-Pretrain on LIBERO. The line shows $\log N-\mathcal{L}_{\text{infoNCE}}$ averaged over three proxy seeds, and the shaded band extends it upward by the direct slack estimate $\widehat{\Delta}_{N}$ of Eq. equation 23 , also averaged over the seeds. Smaller $\Delta_{N}$ means empirically tighter bound in Eq. 6 . Each critic uses $N=512$ and is scored on $n=16{,}384$ held-out pairs, with a normalization set of $m_{1}=15{,}383$ of the other $16{,}383$ held-out latents per query.

Benchmark	Backbone	Method	Best	Even	Last
LIBERO	Cosmos	APT	.666	.969	.790
		VLA-Adapter	.877	.985	1.000
		BEHAVIOR	.983	.923	1.000
LIBERO	Qwen	APT	.900	.990	.870
		VLA-Adapter	.942	.998	.999
		BEHAVIOR	.902	.898	.999

Appendix

TABLE VII: All fusion methods other than APT do not demonstrate statistically significant advantage over the single optimal layer. Each cell reports the p-value for a fusion configuration compared with the optimal single layer. Lower values indicate stronger statistical evidence that fusion outperforms the optimal single layer;

p<0.05

indicates a statistically nominal advantage. See Appendix A-H for calculation of p-value.

Dataset	Demos	Frames	Samples	Train demos	Val. demos	Train/val. samples
LIBERO-Spatial	432	52,970	46,490	410	22	44,116 / 2,374
LIBERO-Object	454	66,984	60,174	431	23	57,192 / 2,982
LIBERO-Goal	428	52,042	45,622	407	21	43,497 / 2,125
LIBERO-10	379	101,469	95,784	360	19	90,984 / 4,800
LIBERO total	1,693	273,465	248,070	1,608	85	235,789 / 12,281
CALVIN ABC subset	4,608	274,466	205,346	4,378	230	195,438 / 9,908

Appendix

TABLE VIII: Training data and fixed trajectory-level validation splits. “Samples” are valid one-observation, 16-action windows.

Benchmark	Backbone checkpoint	Layers	Width	Oracle seeds	Episodes/task
LIBERO	nvidia/Cosmos-Reason2-2B	28	2048	42, 1234, 2718	10
LIBERO	Qwen/Qwen3-VL-2B-Instruct	28	2048	42, 1234, 2718	10
LIBERO	local ABot-Pretrain VLM	36	2560	42, 1234, 4321	10
CALVIN	nvidia/Cosmos-Reason2-2B	28	2048	42, 1234, 2718	15
CALVIN	Qwen/Qwen3-VL-2B-Instruct	28	2048	42, 1234, 2718	15
CALVIN	local ABot-Pretrain VLM	36	2560	42, 1234, 2718	15

Appendix

TABLE IX: Actual backbone–benchmark settings. Width is the language-tower hidden size. The seed order is the order used in per-seed tables below.

Pair	Best-16	Best-8
LIBERO/Cosmos	1,2,4,5,6,7,8,9,10,11,13,14,15,16,17,19	5,6,8,11,12,14,15,17
LIBERO/Qwen	1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,17	2,3,4,5,8,9,11,13
LIBERO/ABot	18,21,22,24,25,26,27,28,29,30,31,32,33,34,35,36	26,27,28,29,31,32,33,35
CALVIN/Cosmos	3,5,8,9,10,11,12,13,14,15,16,17,18,19,20,22	10,11,12,13,14,15,16,17
CALVIN/Qwen	2,3,9,10,11,12,13,14,15,16,17,18,19,20,21,22	12,13,14,15,16,17,18,19
CALVIN/ABot	5,10,11,12,13,14,15,16,17,18,19,20,21,22,23,25	11,12,13,15,16,18,21,22

Appendix

TABLE X: Exact Best layer sets. APT uses Best-16; VLA-Adapter and BEHAVIOR use Best-8.

Benchmark	Backbone	$l^{*}$	Per-seed success	Mean	Sample SD
LIBERO	Cosmos	15	76.8 / 74.5 / 77.6	76.3	1.6
LIBERO	Qwen	3	75.6 / 79.8 / 75.1	76.8	2.6
LIBERO	ABot	31	98.2 / 98.3 / 98.8	98.4	0.3
CALVIN	Cosmos	11	58.3 / 67.7 / 67.3	64.4	5.3
CALVIN	Qwen	16	63.7 / 68.7 / 61.7	64.7	3.6
CALVIN	ABot	12	68.1 / 65.1 / 65.1	66.1	1.7

Appendix

TABLE XI: Per-seed success at the layer selected by the three-seed oracle curve. Values are percent. The triple follows the seed order in Table IX .

Explore similar work

Encoder Winners Do Not Reliably Transfer Across VLA Backbone Scale: A Frozen-Backbone Grafting Diagnostic
Jun 12, 2026Qingping Zeng, Fei SheSentence Encoders Encoders
Tsinghua University
LayerRoute: Action-Conditioned Mixture-of-Layers Routing for Vision-Language-Action Policies
Sep 5, 2026Zheng Lu, Haoran Liao, Wanqi Zhong +9
Tsinghua University · Zhejiang University · Nanyang Technological University +2
LIRA: Local Cross-Layer Information Routing for Vision-Language-Action Decoding
Aug 6, 2026Zhewei Zhang, Puyue Wang, Guanren Qiao +10Cross-Layer Interactions

Jun 12, 2026, cs.CVJ/K move · Enter open · S save

Encoder Winners Do Not Reliably Transfer Across VLA Backbone Scale: A Frozen-Backbone Grafting Diagnostic

Qingping Zeng, Fei She

Tsinghua University

Sentence Encoders Encoders Diagnosis

Open paper AskPDF arXiv

Vision-language-action (VLA) policies typically inherit their vision encoder from upstream VLM releases, but it is unclear whether an encoder choice validated on a small VLA transfers to a larger backbone. We introduce a frozen-backbone grafting diagnostic: the vision tower of a released VLA is replaced by a candidate encoder under a fixed protocol (adaptive average pooling, LayerNorm, and a single trainable linear projector), with the language model and action expert frozen. Across four encoders, two LIBERO suites, two backbones (SmolVLA-450M and

π_{0.5}

-3.3B), and two-to-three seeds per cell (40 main grafting runs plus native, LoRA, pooling, and zero-/shuffled-image controls, all scored by offline action MSE), the small-backbone winner does not reliably select the large-backbone top tier: SigLIP is best on SmolVLA across both suites, while on

π_{0.5}

DINOv2-small leads the spatial suite and the object suite is a seed-sensitive near-tie band; three of the four backbone-suite comparisons (and 11 of 12 seed-level cells) support backbone-dependent rankings. The grafting wrapper is itself non-neutral with opposite sign across backbones (+45-56% MSE on the SmolVLA native tower, -50-52% on

π_{0.5}

), so all conclusions are conditional on the fixed grafting protocol. We position frozen grafting as a cheap target-backbone diagnostic to run before committing to an encoder at scale, not as a closed-loop deployment claim.

Abstract

Explore similar work

Encoder Winners Do Not Reliably Transfer Across VLA Backbone Scale: A Frozen-Backbone Grafting Diagnostic

LayerRoute: Action-Conditioned Mixture-of-Layers Routing for Vision-Language-Action Policies

LIRA: Local Cross-Layer Information Routing for Vision-Language-Action Decoding