PredVLA: Predictive Sensorimotor Modeling for Sub-Million-Parameter Robot Manipulation

Hiroki Sawada; Shunichi Kasahara

cs.ROAug 27, 2026

PredVLA: Predictive Sensorimotor Modeling for Sub-Million-Parameter Robot Manipulation

Authors: Hiroki Sawada, Shunichi Kasahara

Organizations: Sony Computer Science Laboratory Tokyo, Japan

Abstract

Large pretrained vision-language-action models achieve strong robot-manipulation performance, while compact alternatives have largely pursued efficiency by compressing the prevailing observation-to-action paradigm. We investigate whether predictive sensorimotor modeling can make more effective use of a limited parameter budget than direct observation-to-action mapping. We present PredVLA, a language-conditioned predictive-coding policy with only 0.68 million trainable network parameters and no robot-data pretraining. Its hierarchical recurrent dynamics predict visual features and proprioception, while observations influence latent state only through prediction-error-driven online inference. On LIBERO, PredVLA achieves an 86.9% mean success rate across the three short-horizon suites and 75.4% across all four suites. Under a controlled comparison using the same frozen front end, demonstrations, action decoder, and evaluation protocol, PredVLA achieves 3.7x and 7.4x the three-suite mean success rates of parameter-matched Transformer and LSTM behavior-cloning policies, respectively. A mechanism-by-mechanism transition to the recurrent behavior-cloning baseline shows that replacing the predictive pathway with direct observation input produces the largest single performance drop, accounting for approximately $70\%$ of the endpoint gap. Further ablations identify distinct contributions from training-time latent inference, test-time error regression, hierarchical timescales, and sensory prediction-error channels. Together, these results support predictive sensorimotor modeling as a strong inductive bias for compact language-conditioned robot control.

Figures & tables

Figure 1: Structure of PredVLA. Blue: forward propagation. Red: observations. Dashed black: error propagation. The shaded region marks the trainable recurrent controller, while the multimodal front end is frozen. A shared top module T receives the language representation and feeds both the visual branch V and the action branch composed of A top and A bottom . The two branches are coupled by a visual-to-action bottleneck $V\!\to\!A$ and an efference-copy pathway $A\!\to\!V$ that carries the model’s own previous action and proprioceptive predictions rather than observations. Right: the per-step relation between the prior $c_{p}$ , posterior $c_{q}$ , and deterministic state $h$ , shown in the conventional PV-RNN form; the prior and posterior streams are displayed side by side for contrast rather than as two concurrent recurrences.

Setting	Value
Language encoder output / PCA dimension	$384\rightarrow 64$
Visual encoder output / PCA dimension per camera	$512\rightarrow 96$
Number of camera views	$2$
Concatenated visual dimension	$192$
Visual refresh interval	$4$ control steps
Proprioceptive dimension	$9$

Table 1: PredVLA model configuration used.

Setting	Value
Training
Training steps	$30{,}000$
Batch size	$32$
Optimizer	Adam
Network learning rate	$6\times 10^{-5}$
Posterior-variable learning rate	$10^{-2}$

Table 2: PredVLA training and online-inference configuration.

Model	spatial	goal	object	3-suite mean	long	4-suite mean
PredVLA	$83.19\pm 6.10$	$88.40\pm 3.83$	$89.24\pm 6.13$	$\mathbf{86.94}$	$40.57\pm 6.44$	$\mathbf{75.35}$

Table 3: PredVLA performance on LIBERO. Results are mean success rates (%)

\pm

standard deviation across

14

independently trained seeds.

	Model	Params	vs. ours	spatial	goal	object	long	Mean
B1	BC-Transformer	$646{,}628$	$0.957\times$	$24.89\pm 4.19$	$30.83\pm 5.98$	$15.49\pm 12.15$	$7.71\pm 2.43$	$19.73$
B2	BC-RNN (LSTM)	$674{,}276$	$0.998\times$	$15.89\pm 4.25$	$16.94\pm 3.42$	$2.20\pm 2.27$	$6.00\pm 6.00$	$10.26$
B3	BC-TF, chunk $8$	$772{,}558$	$1.143\times$	$21.11\pm 3.78$	$32.57\pm 2.98$	$12.83\pm 13.64$	$0.29\pm 0.76$	$16.70$
B4	BC-TF, chunk $16$	$916{,}478$	$1.356\times$	$3.77\pm 1.33$	$9.40\pm 2.67$	$0.31\pm 0.66$	$0.29\pm 0.76$	$3.44$
–	PredVLA (ours)	$\mathbf{675{,}732}$	$1.000\times$	$\mathbf{83.19}$	$\mathbf{88.40}$	$\mathbf{89.24}$	$\mathbf{40.57}$	$\mathbf{75.35}$

Table 4: Controlled baseline comparison.

Figure 2: Trainable policy parameters versus LIBERO benchmark mean success rate. Shaded region indicates the parameter range of the controlled baselines in Table 4 .

	Modification	Params	spatial	goal	object	Mean	$\Delta$
L0	— (intact PredVLA)	675,732	$82.71\pm 6.50$	$88.34\pm 4.00$	$87.23\pm 7.10$	86.09	—
L1	$-$ Online error regression	675,732	$72.36\pm 6.00$	$81.50\pm 3.67$	$77.93\pm 4.94$	77.26	$-8.83$
L2	$-$ Predictive Coding Inference	675,732	$58.14\pm 5.89$	$66.71\pm 6.42$	$66.91\pm 5.42$	63.92	$-13.34$
L3	$-$ Prediction Pathway	674,560	$10.94\pm 3.96$	$23.46\pm 6.23$	$\phantom{0}0.29\pm 0.60$	11.56	$\mathbf{-52.36}$
L4	$-$ Temporal hierarchy ( $\tau{=}1$ )	674,560	$\phantom{0}6.83\pm 3.88$	$24.06\pm 7.17$	$\phantom{0}0.11\pm 0.16$	10.33	$-1.23$
L5	PV-RNN cell $\to$ LSTM	676,928	$16.97\pm 3.96$	$25.94\pm 5.05$	$\phantom{0}0.66\pm 0.91$	14.52	$+4.19$

Table 5: The ladder from PredVLA to BC-RNN. Success rates are mean (%)

\pm

standard deviation across seven seeds.

\Delta

is the change in the three-suite mean relative to the preceding rung. L2b is a side branch from L2 and is not part of the main chain.

Figure 3: Performance across the mechanism-by-mechanism ladder from PredVLA (L0) to BC-RNN (L6). Thin lines show success rates on spatial , goal , and object , while the thick black line shows their three-suite mean. Each successive rung removes or replaces one component of the preceding model. The dominant performance decrease occurs when sensory prediction is removed (L2

\rightarrow

L3), after which flattening the temporal hierarchy and replacing the recurrent cell produce comparatively small changes. L2b is a side branch from L2 in which visual features are additionally supplied directly to the action pathway; dashed segments distinguish this branch from the main ladder.

	Change from PredVLA	spatial	goal	object	long
A1	$-$ Temporal hierarchy ( $\tau$ set to its mean)	$61.57\pm 8.73$	$71.07\pm 12.09$	$61.29\pm 12.99$	—
	$\Delta$	$-18.66$	$-16.99$	$-29.03$	—
A2	$-$ Visual-to-action bottleneck	$56.00\pm 15.64$	$72.00\pm 2.75$	$90.36\pm 3.36$	—
	$\Delta$	$-24.23$	$-16.06$	$+0.04$	—
A3	$-$ Efference copy A $\to$ V	$73.71\pm 8.87$	$76.21\pm 6.18$	$87.00\pm 6.58$	—
	$\Delta$	$-6.51$	$-11.84$	$-3.31$	—

Table 6: Ablation results. A1–A3 require retraining, whereas A4–A6 are inference-time changes applied to trained checkpoints.

\Delta

denotes the mean paired difference from the intact model evaluated using the same seven seeds.

Figure 4: Task-level effects of removing the visual prediction-error term ( $\lambda_{v}=0$ ) or the proprioceptive prediction-error term ( $\lambda_{q}=0$ ) during online inference. Within each suite, tasks are sorted by the success-rate change induced by the ablation. Gray bars indicate the intact model, while colored bars indicate the change after removing the corresponding error term. Although the suite-level averages in Table 6 are similar for some suites, the task-level patterns differ substantially, especially on object , where performance is much more sensitive to the proprioceptive prediction-error term.

Figure 5: Effect of corrupting the visual or proprioceptive observations used for error regression. $\alpha$ denotes the fraction of observations replaced by those from an unrelated episode. Bands indicate $\pm 1$ seed standard deviation, and the dashed line indicates the corresponding open-loop score.

Explore similar work

Aug 10, 2026

SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation

Vision-language-action policies rely on large multimodal backbones to jointly perform perception, language conditioning, and action generation at every control step. Much of this capacity supports open-domain semantics, whereas continuous robot manipulation primarily requires compact representations of observations, actions, and the transitions induced by actions. Pixel-level world models provide another route, but predicting visual details irrelevant to control can be unnecessarily expensive. We propose SLIM (Self-supervised Latent Interaction Model), a compact 0.5B-parameter latent interaction policy. SLIM learns action-grounded predictive latents that capture both action-conditioned future transitions and the actions that explain observed changes. SLIM learns these representations through self-supervised masked trajectory prediction, combining action reconstruction with future-latent prediction. A compact Mixture-of-Transformers (MoT) backbone models interactions between observation latents and action tokens. The resulting policy is trained with flow matching for language-conditioned action generation. Across simulation benchmarks and real-world evaluation, SLIM matches or exceeds representative large-scale VLA and world-action-model baselines with fewer parameters, no additional embodied pretraining, lower inference latency, and substantially lower GPU memory usage.

Jingkai Wang, Zihan Tang, Gu Zhang +7

1Fudan University · 2Beijing Academy of Artificial Intelligence · 3Tsinghua University +1

Latent Dynamics Modeling Language-Conditioned Robot Manipulation Vision-Language-Action Models Latent World Models

Ask PDF

Oct 6, 2026

Compact Robot Policies Need Fine-Grained Visual Representations

Multi-task manipulation policies differ in architecture, scale, and pretrained priors all at once, so published comparisons cannot attribute performance to any single component. We argue that most of it comes from the visual representation, and that parameter scale and generative priors are largely incidental. To test this, we build CoRP (Compressed Representation Policy), a deliberately compact policy (48.9M parameters, no vision-language model and no video-generative prior) that factorizes into a representation extractor and a flow-matching action generator. It reaches 97.0% on LIBERO and 75.78%/73.36% on RoboTwin 2.0 Clean/Randomized, matching systems 40.9-163.6x larger. Holding the action generator fixed, we then vary one extractor property at a time. Pretrained initialization is decisive: a random ViT-S/14 drops to 78.1% and an ImageNet ResNet-34 to 74.5% on LIBERO. Pretraining alone is not enough, as freezing the encoder costs 19.8 points. Compression matters as much: resampling each view to 48 tokens beats passing all patch tokens (97.0% vs 83.2%), and a variational information bottleneck over those tokens is worse than a hard token budget, cutting LIBERO-Goal from 95.8% to 33.0% by suppressing the instruction-dependent token selection the policy relies on. Language conditioning contributes only where the observation leaves the goal ambiguous (LIBERO-Goal: 9.2% to 95.8%), while on RoboTwin 2.0, where observations are unambiguous, removing it slightly improves success. Therefore, we argue that a compact policy works when its representation is pretrained, task-adapted, and compressed. Project page: https://corp-policy.github.io/

Nanhe Chen, Runqiu Yang, Jiawei Tang +2

X Square Robot · KTH

Robot Policy Learning Visual Representation Learning Multi-Task Robotic Manipulation

Ask PDF

Sep 3, 2026

MINERVA: How Small Can a Manipulation Policy Be and Still Solve LIBERO?

Vision-language-action (VLA) models with billions of parameters now dominate the LIBERO manipulation benchmark, but the model capacity actually required by the benchmark remains unclear. We introduce MINERVA (MINimal Efficient Robotic Vision-Action policy), a family of deliberately compact visuomotor policies designed to measure this task-specific capacity floor. A 0.54M-parameter policy achieves 95.1% average success over 2,000 rollouts on the four standard LIBERO suites, only 2.4 points below the reported LeRobot

π_{0.5}

result despite using 7,700

\times

fewer parameters. Performance saturates near 1M parameters and collapses below 0.25M. Across broad architectural, training, and inference sweeps, only action-chunk length and vision capacity consistently exceed a

\pm

1-point training-seed band. Flow matching provides no detectable advantage over direct L1 regression across three seeds, while regression is up to 3.8

\times

faster on GPU. A task-ID permutation probe shows that standard LIBERO instruction conditioning primarily selects among memorized tasks: changing only the task-ID mapping reduces success to near chance. The same recipe achieves 94.6% success across 89 LIBERO-90 tasks, while LIBERO-Plus perturbations reduce performance to 46--56%, with near-zero robustness to photometric shifts. The 0.54M policy replans every control step in 5--9 ms per chunk on a laptop CPU, 113

\times

faster than SmolVLA and 1,400

\times

faster than

π_{0.5}

, without a GPU. These results establish a first empirical estimate of LIBERO's task-specific capacity floor and motivate capacity-aware design and distillation for deployment-efficient robot policies.

Kohei Sendai, Tatsuya Matsushima, Yusuke Iwasawa

Matsuo-Iwasawa Lab, Graduate School of Engineering, The University of Tokyo, Japan

Robot Skill Learning Visuomotor Policy Learning Robot Policy Generalization

Ask PDF