Causal Analysis of Robot Foundation Models Enables Efficiency Gains

Aiden Swann; Mathilde Papillon; Mozes Jacobs; Siddharth Boppana; Tal Haklay; Eric Bigelow; Andrew Lee; Conor Watts

cs.ROOct 14, 2026

Causal Analysis of Robot Foundation Models Enables Efficiency Gains

Authors: Aiden Swann, Mathilde Papillon, Mozes Jacobs, Siddharth Boppana, Tal Haklay, Eric Bigelow, Andrew Lee, Conor Watts,

Organizations: •Goodfire · aStanford University · bUniversity of California, Santa Barbara · cHarvard University

Abstract

Vision-language-action (VLA) models are a promising approach to general purpose robot manipulation. While the internal computation of vision-language models, which compose the backbone of VLAs, has been studied extensively, less is known about how their perceptual and semantic representations are transformed into action in VLA models. We study this question mechanistically in three frontier VLA models ( $π_{0.5}$ , MolmoAct2, and MolmoBot) by analyzing the causal pathways through which the input image and the instruction shape actions. Across the three models, activation patching shows that causally relevant information influences the action expert (AE) through a narrow band of mid-to-late cross-attention layers, which we term the action-conditioning interface. The content of this information changes with token and layer within the model. Early-to-mid layer representations of the target noun token determine which object is selected, whereas mid-to-late layers carry the spatial information in text tokens that redirects the movement when patched. Where spatial information is stored in prompt tokens various based on the architecture. In MolmoAct2 the model forms task-conditioned representations at the text positions that follow the instruction, whereas $π_{0.5}$ additionally stores this information at image token positions. We show that these findings can be translated directly into improving model efficiency. We find that restricting MolmoBot's action expert conditioning to only 3 out of 36 layers reduces inference FLOPs by up to $33\%$ while maintaining comparable task success with the default model across 3,000 evaluation episodes ( $46.7\%$ vs. $46.9\%$ ), and the same principle carries over to real-robot deployment of $π_{0.5}$ .

Figures & tables

Figure 1: Activation patching reveals how a vision-language-action (VLA) model turns perception into action. ( Left ) We copy activations from a donor episode (“Pick up the banana”) into a target episode (“Pick up the apple”) with the position of the objects swapped. Visual information (purple) sits in early image tokens, semantic information (blue) in the early noun token, and geometric information (orange) in later text tokens. The action expert (AE) reads from a narrow band of later layer. ( Top right ) Schematic of these effects by layer. The action expert reads from a narrow band of layers.( Bottom right ) Keeping only this interface in MolmoBot, 3 of 36 connections , cuts compute by 33% with comparable average success (46.7% vs. 46.9%) over 3000 trials.

Figure 2: Action conditioning concentrates in model-specific layer bands. Curves show normalized trajectory shift $E$ toward the reference trajectory (same scene other object requested) after patching one action-expert connection (left) or three consecutive connections (right, indexed by the center layer). The results shown are computed across 144,768 patches over 26 scenes via averaging comparisons B and C ( Section 4.1 ).

	Pick	Pick & Place		PnP Next-To		Avg
Cross-attention layers	Success	Oracle	Final	Oracle	Final	(oracle)
All 36 (released MolmoBot-DROID)	62.0	57.4	49.6	21.2	15.1	46.9 $\pm$ 1.7
Layers 23–25, pretrained	57.7	59.2	50.7	23.3	18.0	46.7 $\pm$ 1.7
Layer 24, pretrained	63.0	53.0	45.7	20.8	15.9	45.6 $\pm$ 1.7
Layers 23–25, adapt (50K FT)	56.8	55.4	45.9	23.1	17.8	45.1 $\pm$ 1.7
Layer 24, adapt (50K FT)	54.7	54.0	44.9	24.9	19.4	44.5 $\pm$ 1.7

Table 1: Keeping only the action-conditioning interface preserves performance Success rate (%) on the MolmoBot paper’s simulation benchmarks (1,000 identical episodes per benchmark). Reading only layer 24 (1 of 36 connections) comes within 1.3 points of the released all-layer model when trained from scratch. Reading only L23–25 comes within 0.2 points and gives the best Pick & Place success of any model.

\pm

is a 95% binomial confidence interval. Final measures success state at end of episode. Oracle terminates at the first success state. Shading ranks each column (darkest = best).

$\pi_{0.5}$ policy	Success	Task time (s)
Original (all 18 layers)	64/100 (64%)	34.9
Restricted (layers 14–16)	67/100 (67%)	26.5

Table 2: Real-robot evaluation of

\pi_{0.5}

. Over 100 trials per policy on seven tabletop tasks ( Figure 9 ; per-task results in Table 6 ), restricting action-expert attention to layers 14–16 preserves task success. Task time is the median duration of successful trials for each task, averaged over the seven tasks. It measures full task execution, not inference latency.

Panel of Figure 3 — Figure 3: Three counterfactual comparisons test hypothesis about target selection We patch activations from a donor run into a recipient run. (a) The input changes in A , B and C produce distinct causal signatures for each interchange intervention type. (b) Green arrows show the unpatched behavior; red arrows show predicted recipient behavior if the patch transfers the labeled information. These labels denote transfer hypotheses , not information uniquely isolated by single comparisons. For example, in A , patching scene-layout information through an image patch can produce the same behavioral shift as patching target-object identity through a noun patch. We therefore interpret effects jointly across the comparisons and patched token positions.

Figure 4: Backbone interventions trace the construction of the geometric variable in MolmoAct2 and

\pi_{0.5}

. Colors show normalized trajectory shift

E

toward the other-object trajectory in the original scene. Each cell averages 416 interventions: 26 original–counterfactual pairs, each tested in both directions under eight noise seeds. The visual, semantic, and geometric hypotheses predict the signatures in Figure 3(a) . We omit MolmoAct2 layers 27–35 as they have

|E|<0.001

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

	A	B	C
Counterfactual layout	$V_{1}$	$V_{0}$	$V_{1}$
Counterfactual request	$b$	$b$	$a$
Clean original output	$p_{\mathrm{L}}$	$p_{\mathrm{L}}$	$p_{\mathrm{L}}$
Clean counterfactual output	$p_{\mathrm{L}}$	$p_{\mathrm{R}}$	$p_{\mathrm{R}}$
Visual intervention $Y^{V}$	$p_{\mathrm{R}}$	$p_{\mathrm{L}}$	$p_{\mathrm{R}}$
Semantic intervention $Y^{S}$	$p_{\mathrm{R}}$	$p_{\mathrm{R}}$	$p_{\mathrm{L}}$

Appendix

Table 3: High-level predictions for the original input

(V_{0},a)

. Each entry is the location the actions target.

p_{\mathrm{L}}

preserves the original target (

E=0

), and

p_{\mathrm{R}}

redirects the reach to the other object’s location in the original scene (

E=1

). The intervention rows give the signatures in Figure 3(a) .

Panel of Figure 5 — Figure 5: Complementary and full-trajectory interface patching. (a) Normalized trajectory shift $E$ when patching all connections except one (left) or except three consecutive connections (right). Horizontal axes index the unchanged connection or window center. (b) RMS displacement in millimeters after patching only a single connection (left) or three consecutive connections (right, indexed by their center). Trajectories are reconstructed from predicted actions using forward kinematics. Both panels average comparisons B and C with equal weighting across object-pair families; shading denotes 95% family-bootstrap confidence intervals.

Figure 6: MolmoBot backbone patching across 13 object pairs. Same comparisons, token groups, and metric as Figure 4 . Each cell averages 416 patches: 26 donor–recipient pairs, each patched in both directions under eight noise seeds. Image patches in B are exactly zero because MolmoBot’s image positions cannot attend to the instruction. Omitted layers 29–35 have

|E|<0.005

Figure 7: Small token sets that carry location in MolmoAct2, across all 13 object pairs. For each layer, filled cells mark the smallest set of text tokens, found by greedy forward selection, whose joint patch keeps at least

75\%

of the effect of patching all 79 non-state text tokens in comparison C; the right column gives the number of tokens. Effects are averaged over all 52 comparison-C donor–recipient pairs with equal weight per object pair. The sets keep

76

–

89\%

of the effect on the noise seed used for selection and

72

–

90\%

on three held-out seeds. The target noun, which differs across object pairs, is labeled <object> ; state tokens were not searched.

	Vision	VLM	Action expert	Total	Change
MolmoBot, all 36 layers	2.46	7.25	2.22	11.93	–
MolmoBot, layer 24	2.46	5.04	0.14	7.64	$-36\%$
MolmoBot, layers 23–25	2.46	5.24	0.26	7.96	$-33\%$
$\pi_{0.5}$ , all 18 layers	0.66	3.98	0.13	4.76	–
$\pi_{0.5}$ , layers 14–16	0.66	3.75	0.11	4.52	$-5\%$

Appendix

Table 4: Estimated TFLOPs per action chunk. Restricted policies stop the VLM after the deepest layer the action expert reads.

MolmoBot	Latency (ms)	Throughput (chunks/s)	Memory (GB)
All 36 layers	$311\pm 3$	21.5	42.7
Layer 24	$204\pm 16$ ( $-35\%$ )	31.8 ( $+48\%$ )	31.7
Layers 23–25	$209\pm 2$ ( $-33\%$ )	30.9 ( $+44\%$ )	32.2

Appendix

Table 5: Measured inference cost on one H100 (bfloat16, ten flow-matching steps). Latency is at batch size 1; throughput and peak memory are at batch size 64. Restricted policies stop the VLM after the deepest layer the action expert reads.

Figure 8: Counterfactual scenes. Object pairs used each shown in both layouts with the objects’ left–right positions exchanged. Images show the exterior DROID camera, cropped to the table (during experiments the full frame is used).

Task	Trials	Success	Time (s)	Success	Time (s)
		Original (all 18 layers)		Restricted (layers 14–16)
Fruit in basket	25	22	53.6	22	45.4
Marker in mug	20	15	16.1	10	16.4
Stack the bowls	15	8	71.8	11	50.0
Screwdriver in bin	10	5	37.8	7	20.0
Mug in bin	10	3	18.9	8	21.4

Appendix

Table 6: Real-robot results for

\pi_{0.5}

by task. Success counts and median duration of successful trials. The last row gives total successes and the median task time averaged over the seven tasks.

Figure 9: Real-robot tasks for $\pi_{0.5}$ . The first exterior-camera frame of each task, with the instruction given to both policies. For the [noun] in bin task,[noun] is replaced with each of the target objects for 10 trials each.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,194 papers

Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models
Mar 19, 2026Bryce Grant, Xijia Zhao, Peng WangVision-Language-Action Models Robotics
Case Western Reserve University
Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models
Mar 16, 2026Yulin Luo, Hao Chen, Zhuangzhe Wu +10Spatial Grounding for VLA Models Vision-Language-Action Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · The Chinese University of Hong Kong · Simplexity Robotics
A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model
Apr 7, 2026Kaidong Zhang, Jian Zhang, Rongtao Xu +20Flow-Based VLA Models Vision-Language-Action Models
SYSU · Spatialtemporal AI · MBZUAI

Mar 19, 2026J/K move · Enter open · S save

Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models

Bryce Grant, Xijia Zhao, Peng Wang

Case Western Reserve University

Vision-Language-Action Models Robotics

Open paper AskPDF arXiv

A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning. However, if the basket is shifted five centimeters to the left, it drops the soup off the table. We developed an interpretability pipeline to investigate what such a policy is computing. Injecting task A's activations into the action expert while the policy looks at task B's scene makes pi0.5 execute task A's motor trajectory in 99.6% of episodes (

n{=}1{,}968

). The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report. The binding holds on a physical UR10e: directed replay redirects the commanded task in 26/26 rollouts in one direction and 18/26 in the other, with no-injection and random-direction controls at the floor, and once the source object is moved 10--15,cm the arm grasps and releases at the recorded coordinates in 50/50 trials, picking up the wrong block when one sits there and closing on empty air when none does. Applying the same interventions to six VLAs on LIBERO, MetaWorld, SimplerEnv, and ALOHA over 420{,}000+ rollouts yields three further findings: every architecture encodes the instruction, and four of the five weight it below vision whenever the scene alone identifies the goal; under the standard replacement write-back the SAE pooling that preserves behavior is architecture-specific (pi0.5 per-token, SmolVLA indifferent), and goal selection and motor generation separate into VLM and expert pathways wherever the two are distinguishable. We release Action Atlas (https://action-atlas.com), an interactive platform covering all 520+ trained SAEs, 79 identified concepts, and the scored UR10e rollouts. Code: https://github.com/CWRU-AISM/action-atlas

Abstract

Vision-Language-Action Models

Explore similar work

Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models

Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models

A1: A Fully Transparent Open-Source, Adaptive and Efficient Truncated Vision-Language-Action Model