Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success

Yunfei Xie; Howard H. Qian; Shiyi Lan; Kaiyu Hang; Chen Wei

cs.CVOct 22, 2026

Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success

Authors: Yunfei Xie, Howard H. Qian, Shiyi Lan, Kaiyu Hang, Chen Wei

Organizations: Rice University · NVIDIA

Abstract

Robotic foundation models are typically evaluated through costly closed-loop rollouts, which motivates cheap offline proxies for model capability. The Platonic Representation Hypothesis (PRH) suggests that as representations improve, different modalities such as vision and language converge toward a shared statistical model of reality. We ask whether motion trajectories form another such modality: as embodied models become more capable, should their sensory representations converge toward the structure of physical motion? To examine this, we propose Sensorimotor Alignment ( $S^2$ ), a training-free offline metric that measures how well a model's sensory representation aligns with the geometry of physical motion. We find that $S^2$ correlates positively with downstream robot task success across embodied model families, making it a low-cost offline proxy for prioritizing models before closed-loop evaluation. $S^2$ can also identify promising vision-language model (VLM) backbones even before costly fine-tuning into vision-language-action models (VLAs). Beyond this correlation, $S^2$ offers representation-level explanations for phenomena observed in VLAs and world action models (WAMs). With $S^2$ , we extend PRH to embodied models and make sensorimotor alignment empirically measurable, offering insight into why an embodied model succeeds beyond its closed-loop success rate.

Figures & tables

Figure 1: Left: We test the Platonic Representation Hypothesis (PRH) in robotics, using image–text features as the sensory representation and end-effector trajectories as the motor representation: a more capable embodied model should have sensory representations that align more closely with this motor representation. To validate this, we define Sensorimotor Alignment ( $\mathrm{S}^{2}$ ). Right: We measure $\mathrm{S}^{2}$ across 12 vision-language-action models (VLAs) and find that $\mathrm{S}^{2}$ correlates positively with SimplerEnv task success rate (Pearson $r=+0.717$ ), providing empirical support for this PRH-inspired view of embodiment.

Figure 2: Measuring Sensorimotor Alignment ( $\mathrm{S}^{2}$ ) by nearest-neighbor overlap. For the query episode (left), sensory neighbors (bottom right) have similar vision-text features, and motor neighbors (top right) have similar trajectories. Green checks mark shared neighbors; red crosses mark neighbors found only in the sensory set. A larger overlap means that episodes with similar sensory representations also follow similar motor trajectories.

Figure 3: Sensorimotor Alignment (

\mathrm{S}^{2}

) correlates with task success across model families and in simulated and real-world evaluations. Each panel plots offline

\mathrm{S}^{2}

(

x

-axis) against downstream task success rate (

y

-axis) for (a) fine-tuned vision-language-action models (VLAs), (b) raw vision-language models (VLMs), (c) world action models (WAMs), and (d) real-robot policies. For raw VLMs, task success is measured after VLA fine-tuning. These correlations (Pearson’s

r

; unadjusted one-tailed

p

-values) support

\mathrm{S}^{2}

as an offline proxy across model families.

Figure 4: Sensorimotor Alignment (

\mathrm{S}^{2}

) correlates with VLA success rate where VQA scores do not. Across 8 raw vision-language models (VLMs) [ 57 ] ,

\mathrm{S}^{2}

(left) correlates strongly with SimplerEnv task success after fine-tuning into vision-language-action models (VLAs), whereas visual question answering (VQA) scores (right) show virtually no correlation.

Figure 5: Sensorimotor Alignment (

\mathrm{S}^{2}

) correlates with WAM success rate more closely than video-quality metrics. For 3 world action models (WAMs) evaluated on RoboTwin, video-quality metrics (SSIM, PSNR, FID) show no significant correlations with task success. In contrast, only

\mathrm{S}^{2}

correctly recovers the success-rate ordering. Only 3 of the 6 WAMs are compared because the others do not generate future video during closed-loop task evaluation.

Figure 8: Fine-tuned VLAs rely mainly on visual rather than language representations. We correlate Sensorimotor Alignment (

\mathrm{S}^{2}

) from vision, text, and joint vision-text features (left to right) with SimplerEnv task success rate. Top: For fine-tuned vision-language-action models (VLAs), vision and joint features correlate similarly strongly, while text features correlate weakly. Bottom: For raw vision-language models (VLMs), joint-feature

\mathrm{S}^{2}

correlates with task success after VLA fine-tuning, while neither modality alone reaches significance.

Figure 9: Freezing the vision encoder reduces both Sensorimotor Alignment (

\mathrm{S}^{2}

) and task success rate. For Qwen2.5-VL-3B vision-language-action models fine-tuned on BridgeDataV2, we compare end-to-end training with a frozen vision encoder. Freezing lowers

\mathrm{S}^{2}

from joint vision-text features (top) and SimplerEnv success rate (bottom), linking lower success to weaker alignment. Trajectory horizon

h=7

Panel of Figure 10 — Figure 10: Sensorimotor Alignment ( $\mathrm{S}^{2}$ ) is already high at early denoising steps and peaks at mid-depth blocks. For 3 world action models (WAMs), (a) $\mathrm{S}^{2}$ remains nearly flat across denoising, suggesting early-step features already align with robot trajectories; (b) schedule-averaged $\mathrm{S}^{2}$ peaks at Block 14, showing peak alignment at mid-depth.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Removed group	Pearson’s $r$	One-tailed $p$
StarVLA	$+0.526$	$0.113$
CogACT	$+0.785$	$0.006$
GR00T	$+0.745$	$0.007$
Pi0	$+0.783$	$0.002$
SpatialVLA	$+0.687$	$0.010$

Appendix

Table 1: Leave-one-out over architecture groups for 12 fine-tuned vision-language-action models. Each row removes one group and recomputes the correlation between Sensorimotor Alignment (

\mathrm{S}^{2}

) and SimplerEnv-WidowX task success rate. The remaining model counts

n

are 7, 9, 10, 11, and 11 in row order.

Figure 11: Sensorimotor Alignment (

\mathrm{S}^{2}

) of frozen vision encoders correlates with downstream policy success rate. Points represent the 14 frozen backbones of Hu et al. [19] , scored by

\mathrm{S}^{2}

on DROID without policy training. The vertical axis shows the interquartile mean (IQM) of task success rates over 21 tasks, using low-data behavior cloning (left) or reinforcement learning with visual reward functions (right).

Model	SR (%)	$h\!=\!1$	$h\!=\!3$	$h\!=\!7$	$h\!=\!15$	$h\!=\!25$	$h\!=\!40$	$h\!=\!75$	$h\!=\!110$	$h\!=\!150$	$h\!=\!220$
Qwen2.5-GR00T-Bridge	71.4	0.034	0.034	0.036	0.040	0.043	0.051	0.065	0.076	0.086	0.097
Qwen3-GR00T-RT-1-Bridge	65.3	0.028	0.027	0.029	0.032	0.036	0.044	0.057	0.071	0.083	0.096
Qwen2.5-GR00T-RT-1-Bridge	63.6	0.034	0.034	0.035	0.040	0.042	0.048	0.062	0.071	0.080	0.090
CogACT-Large	58.3	0.031	0.031	0.033	0.037	0.040	0.048	0.062	0.075	0.086	0.097
GR00T-N1.6-Bridge	57.1	0.023	0.024	0.026	0.029	0.031	0.038	0.051	0.064	0.076	0.091
CogACT-Base	51.3	0.031	0.031	0.032	0.037	0.040	0.049	0.062	0.074	0.086	0.098

Appendix

Table 2: Sensorimotor Alignment (

\mathrm{S}^{2}

) from joint vision-text features at the language-model output versus SimplerEnv-WidowX task success rate (SR) across trajectory horizons

h

. We score 12 fine-tuned vision-language-action models on DROID (

N=6{,}000

episodes,

k=10

neighbors). Bottom rows report Spearman’s

\rho

and Pearson’s

r

with their one-tailed

p

-values. Bold marks the horizon of peak Pearson

r

(

h=7

Setting	$r$	$p$
Number of neighbors $k$
$k\!=\!2$	$+0.679$	$0.0076$
$k\!=\!5$	$+0.667$	$0.0089$
$k\!=\!10^{\ast}$	$+0.717$	$0.0043$
$k\!=\!15$	$+0.665$	$0.0091$
$k\!=\!20$	$+0.644$	$0.0119$

Appendix

Table 5: Sensitivity of the Sensorimotor Alignment (

\mathrm{S}^{2}

)–success-rate correlation to implementation choices. Pearson’s

r

and one-tailed

p

compare

\mathrm{S}^{2}

with SimplerEnv-WidowX task success rate across 12 fine-tuned vision-language-action models. Each block varies one choice; ∗ marks the default.

	$\mathrm{S}^{2}$ (mean over 12 VLAs)			$\mathrm{S}^{2}$ –SR Pearson $r$
Horizon $h$	w/ DTW	w/o DTW	rel. change	w/ DTW	w/o DTW	rel. change
$1$	$0.0268$	$0.0268$	$+0.0\%$	$+0.692$	$+0.692$	$+0.0\%$
$3$	$0.0268$	$0.0269$	$-0.6\%$	$+0.699$	$+0.700$	$-0.1\%$
$7$	$0.0280$	$0.0277$	$+1.2\%$	$+0.717$	$+0.713$	$+0.6\%$
$15$	$0.0314$	$0.0302$	$+4.1\%$	$+0.696$	$+0.710$	$-2.0\%$
$25$	$0.0342$	$0.0317$	$+7.7\%$	$+0.689$	$+0.711$	$-3.2\%$

Appendix

Table 6: Comparing Sensorimotor Alignment (

\mathrm{S}^{2}

) with and without Dynamic Time Warping (DTW). DTW raises absolute

\mathrm{S}^{2}

more at longer trajectory horizons

h

, while its Pearson correlation

r

with SimplerEnv-WidowX task success rate (SR) stays stable. Each relative change is

100(v_{\mathrm{DTW}}/v_{\mathrm{no\,DTW}}-1)\%

for the quantity

v

in that block.

Figure 12: Sensorimotor Alignment (

\mathrm{S}^{2}

) is sensitive to a mismatch in arm configuration. For 3 world action models (WAMs) evaluated on bimanual RoboTwin, the matched bimanual ALOHA reference (left) recovers the success-rate ordering, while the single-arm DROID reference (right) weakens the correlation.

\mathrm{S}^{2}

thus reflects whether reference data matches the target arm configuration.

Feature position	Pearson’s $r$	One-tailed $p$
ViT-out	$+0.567$	$0.0274$
LLM-in vision-text	$+0.580$	$0.0239$
LLM-out vision-only	$+0.701$	$0.0056$
LLM-out vision-text ∗	$+0.717$	$0.0043$
LLM-in text-only	$-0.325$	$0.8486$
LLM-out text-only	$+0.245$	$0.2215$

Appendix

Table 7: Correlation between Sensorimotor Alignment (

\mathrm{S}^{2}

) and task success by feature position. Pearson’s

r

and one-tailed

p

compare

\mathrm{S}^{2}

with SimplerEnv-WidowX task success rate across 12 fine-tuned vision-language-action models on DROID (

N=6{,}000

episodes, trajectory horizon

h=7

). ViT-out, LLM-in, and LLM-out denote the vision-encoder output, language-model input, and language-model output; ∗ marks the default feature.

Feature	Vision encoder	Raw VLM	Finetuned VLA	Relative change
	(mean, $n\!=\!3$ )	(mean, $n\!=\!8$ )	(mean, $n\!=\!12$ )
ViT-out	$0.0246$	$0.0247$	$0.0278$	$+12.7\%$
LLM-in vision-only	–	$0.0244$	$0.0275$	$+12.7\%$
LLM-in vision-text	–	$0.0233$	$0.0275$	$+18.1\%$
LLM-out vision-only	–	$0.0212$	$0.0275$	$+29.9\%$
LLM-out text-only	–	$0.0140$	$0.0202$	$+45.0\%$

Appendix

Table 8: Mean Sensorimotor Alignment (

\mathrm{S}^{2}

) by feature position on DROID (

N=6{,}000

episodes, trajectory horizon

h=7

). ViT-out, LLM-in, and LLM-out denote the vision-encoder output, language-model input, and language-model output; standalone vision encoders have only ViT-out features. The last column is

100(\overline{\mathrm{S}^{2}}_{\mathrm{VLA}}/\overline{\mathrm{S}^{2}}_{\mathrm{VLM}}-1)\%

, where each bar denotes the model-group mean.

Model	ViT-out	LLM-in vision-only	LLM-in text-only	LLM-in vision-text	LLM-out vision-only	LLM-out text-only	LLM-out vision-text
Pretrained vision encoders, no LLM attached ( $n\!=\!3$ )
CLIP-ViT-L/14 [ 37 ]	$0.0240$	–	–	–	–	–	–
SigLIP-SO400M [ 56 ]	$0.0247$	–	–	–	–	–	–
SigLIP-2 Large [ 46 ]	$0.0250$	–	–	–	–	–	–
Raw VLMs from VLM4VLA ( $n\!=\!8$ )
KosMos-2	$0.0257$	$0.0238$	$0.0093$	$0.0239$	$0.0214$	$0.0151$	$0.0216$

Appendix

Table 9: Per-model Sensorimotor Alignment (

\mathrm{S}^{2}

) by feature position on DROID (

N=6{,}000

episodes, trajectory horizon

h=7

). ViT-out, LLM-in, and LLM-out denote the vision-encoder output, language-model input, and language-model output. Vision-only, text-only, and vision-text select image tokens, instruction tokens, or both from the same model input. Group means are in Table 8 .

Horizon $h$	Unfrozen $\mathrm{S}^{2}$	Frozen $\mathrm{S}^{2}$	Relative change
$1$	$0.0231$	$0.0143$	$-38.1\%$
$3$	$0.0232$	$0.0141$	$-39.0\%$
$7$	$0.0245$	$0.0144$	$-41.1\%$
$15$	$0.0268$	$0.0158$	$-41.2\%$
$25$	$0.0295$	$0.0166$	$-43.8\%$
$40$	$0.0352$	$0.0191$	$-45.7\%$

Appendix

Table 10: Sensorimotor Alignment (

\mathrm{S}^{2}

) with a frozen and an unfrozen vision encoder across trajectory horizons

h

. Qwen2.5-VL-3B vision-language-action models trained with the same recipe, evaluated using joint vision-text features at the language-model output on DROID (

N=6{,}000

episodes). Relative change is

100(\mathrm{S}^{2}_{\mathrm{frozen}}/\mathrm{S}^{2}_{\mathrm{unfrozen}}-1)\%

Feature	0 %	25 %	50 %	75 %	100 %
	earliest				latest
Motus random baseline $=0.012\pm 0.002$
Block 0	0.044	0.203	0.307	0.321	0.324
Block 7	0.342	0.388	0.399	0.390	0.378
Block 14	0.479	0.492	0.482	0.479	0.471
Block 21	0.454	0.448	0.447	0.436	0.424

Appendix

Table 11: Layer-wise Sensorimotor Alignment (

\mathrm{S}^{2}

) along the denoising schedule for 3 world action models (WAMs). Columns label 5 selected positions in each model’s native schedule, not matched noise levels. Motus and LingBot-VA use 500 episodes over 50 RoboTwin tasks; Vidar uses 480 episodes over 48 tasks. Rows give 6 video-backbone positions and the text embedding. Block 14 (bold) has the highest

\mathrm{S}^{2}

at 13 of the 15 model and schedule positions. The random baseline is the mean

\pm

standard deviation (divisor 10) over 10 draws of independent standard-normal features with the same shape.

Robot Foundation Models

Robot Foundation Models are pretrained systems that combine perception, reasoning, and action to support diverse tasks and robot embodiments.

See all 130 papers

Jul 20, 2026

RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model

We present RynnBrain 1.1, a family of embodied foundation models spanning 2B, 9B, and 122B-A10B scales. Trained with a unified spatio-temporal and physically grounded framework, RynnBrain 1.1 supports embodied perception, spatial reasoning, localization, and planning. Compared with RynnBrain 1.0, it further introduces contact-point prediction across the model family and native 3D grounding for the 2B and 9B models, yielding representations and outputs that are more directly aligned with robot manipulation. We also develop RynnBrain-VLA with a unified cross-embodiment action space and embodiment-specific masking, and deploy it on Unitree G1, Astribot-S1, and Tianji-Wuji. RynnBrain 1.1 achieves strong results on embodied cognition, localization, and 3D grounding, with the 122B-A10B model outperforming all evaluated proprietary and open-source models on VSI-Bench, MMSI, and RefSpatial-Bench. Real-robot experiments show that RynnBrain-initialized policies outperform Qwen-based and representative generalist VLAs, while joint multi-task and multi-embodiment training improves process scores and success rates over per-task training.

Kehan Li, Bohan Hou, Minghao Zhu +28

1DAMO Academy, Alibaba Group

3D Visual Grounding Cross-Embodiment Robot Learning Robot Foundation Models Robotic Manipulation Vision-Language-Action Models Multimodal Foundation Models+2

Ask PDF

Mar 22, 2026

RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models

Improving embodied reasoning in multimodal-large-language models (MLLMs) is essential for building vision-language-action models (VLAs) on top of them to readily translate multimodal understanding into low-level actions. Accordingly, recent work has explored enhancing embodied reasoning in MLLMs through supervision of vision-question-answering type. However, these approaches have been reported to result in unstable VLA performance, often yielding only marginal or even negative gains. In this paper, we propose a more systematic MLLM training framework RoboAlign that reliably improves VLA performance. Our key idea is to sample action tokens via zero-shot natural language reasoning and refines this reasoning using reinforcement learning (RL) to improve action accuracy. As a result, RoboAlign bridges the modality gap between language and low-level actions in MLLMs, and facilitate knowledge transfer from MLLM to VLA. To validate the effectiveness of RoboAlign, we train VLAs by adding a diffusion-based action head on top of an MLLM backbone and evaluate them on major robotics benchmarks. Remarkably, by performing RL-based alignment after SFT using less than 1% of the data, RoboAlign achieves performance improvements of 17.5%, 18.9%, and 106.6% over SFT baselines on LIBERO, CALVIN, and real-world environments, respectively.

Dongyoung Kim, Sumin Park, Woomin Song +6

KAIST · RLWRLD · Yonsei University +1

Vision-Language-Action Models Reasoning for VLA Models Modality Gap in VLMs RL for VLMs

Ask PDF

Jul 6, 2026

Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning

Embodied Chain-of-Thought has emerged as a promising mechanism to enhance robot decision-making and interpretability in black-box Vision-Language Action (VLA) models. However, whether this verbalized Chain-of-Thought truthfully reflects the policy's underlying decision process remains poorly understood. We distinguish between functional reasoning, in which reasoning improves task performance, and faithful reasoning, in which reasoning truly reflects the policy's internal decision process. We argue that SoTA alignment strategies offer a necessary but insufficient notion of faithfulness, admitting reasoning whose intermediate steps can mask the causal links in action prediction through confounding factors (e.g., reasoning that is ungrounded in the environment and internally disconnected or inconsistent), restricting policy generalization. We study this gap through a human evaluation of a SoTA reasoning model for autonomous driving, revealing an inconsistent coupling between reasoning quality and downstream trajectory improvement. We then operationalize a behavioral surrogate for embodied faithfulness through a learned critic, Pinocchio, scoring observation grounding and stepwise coherence, and use this critic as a dense reward signal in post-training an embodied policy with reinforcement learning. Across withheld driving benchmarks, our post-trained planner improves faithfulness by 4% and 18% over SoTA alignment and trajectory error post-training baselines, respectively, while maintaining competitive downstream task performance. Finally, on a synthetic out-of-distribution test set, post-training for faithfulness improves policy responsiveness to rare counterfactual scenarios by 1.6x that of a SoTA policy, suggesting that faithful reasoning traces contribute to more robust, generalizable, and interpretable embodied intelligence. Project page: https://mjf-su.github.io/pinocchio/

Matthew Foutter, Matteo Cercola, Lena Wild +4

Stanford University · Politecnico di Milano · KTH Royal Institute of Technology +2

VLMs for Autonomous Driving Vision-Language-Action Models Reasoning for VLA Models Embodied Reasoning

Ask PDF

Abstract

Robot Foundation Models

Explore similar work

RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model

RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models

Do Vision-Language-Action Models Mean What They Say? On the Role of Faithfulness in Embodied Reasoning