PredVLA: Predictive Sensorimotor Modeling for Sub-Million-Parameter Robot Manipulation

Hiroki Sawada; Shunichi Kasahara

cs.ROAug 27, 2026

PredVLA: Predictive Sensorimotor Modeling for Sub-Million-Parameter Robot Manipulation

Authors: Hiroki Sawada, Shunichi Kasahara

Organizations: Sony Computer Science Laboratory Tokyo, Japan

Abstract

Large pretrained vision-language-action models achieve strong robot-manipulation performance, while compact alternatives have largely pursued efficiency by compressing the prevailing observation-to-action paradigm. We investigate whether predictive sensorimotor modeling can make more effective use of a limited parameter budget than direct observation-to-action mapping. We present PredVLA, a language-conditioned predictive-coding policy with only 0.68 million trainable network parameters and no robot-data pretraining. Its hierarchical recurrent dynamics predict visual features and proprioception, while observations influence latent state only through prediction-error-driven online inference. On LIBERO, PredVLA achieves an 86.9% mean success rate across the three short-horizon suites and 75.4% across all four suites. Under a controlled comparison using the same frozen front end, demonstrations, action decoder, and evaluation protocol, PredVLA achieves 3.7x and 7.4x the three-suite mean success rates of parameter-matched Transformer and LSTM behavior-cloning policies, respectively. A mechanism-by-mechanism transition to the recurrent behavior-cloning baseline shows that replacing the predictive pathway with direct observation input produces the largest single performance drop, accounting for approximately $70\%$ of the endpoint gap. Further ablations identify distinct contributions from training-time latent inference, test-time error regression, hierarchical timescales, and sensory prediction-error channels. Together, these results support predictive sensorimotor modeling as a strong inductive bias for compact language-conditioned robot control.

Figures & tables

Figure 1: Structure of PredVLA. Blue: forward propagation. Red: observations. Dashed black: error propagation. The shaded region marks the trainable recurrent controller, while the multimodal front end is frozen. A shared top module T receives the language representation and feeds both the visual branch V and the action branch composed of A top and A bottom . The two branches are coupled by a visual-to-action bottleneck $V\!\to\!A$ and an efference-copy pathway $A\!\to\!V$ that carries the model’s own previous action and proprioceptive predictions rather than observations. Right: the per-step relation between the prior $c_{p}$ , posterior $c_{q}$ , and deterministic state $h$ , shown in the conventional PV-RNN form; the prior and posterior streams are displayed side by side for contrast rather than as two concurrent recurrences.

Setting	Value
Language encoder output / PCA dimension	$384\rightarrow 64$
Visual encoder output / PCA dimension per camera	$512\rightarrow 96$
Number of camera views	$2$
Concatenated visual dimension	$192$
Visual refresh interval	$4$ control steps
Proprioceptive dimension	$9$

Table 1: PredVLA model configuration used.

Setting	Value
Training
Training steps	$30{,}000$
Batch size	$32$
Optimizer	Adam
Network learning rate	$6\times 10^{-5}$
Posterior-variable learning rate	$10^{-2}$

Table 2: PredVLA training and online-inference configuration.

Model	spatial	goal	object	3-suite mean	long	4-suite mean
PredVLA	$83.19\pm 6.10$	$88.40\pm 3.83$	$89.24\pm 6.13$	$\mathbf{86.94}$	$40.57\pm 6.44$	$\mathbf{75.35}$

Table 3: PredVLA performance on LIBERO. Results are mean success rates (%)

\pm

standard deviation across

14

independently trained seeds.

	Model	Params	vs. ours	spatial	goal	object	long	Mean
B1	BC-Transformer	$646{,}628$	$0.957\times$	$24.89\pm 4.19$	$30.83\pm 5.98$	$15.49\pm 12.15$	$7.71\pm 2.43$	$19.73$
B2	BC-RNN (LSTM)	$674{,}276$	$0.998\times$	$15.89\pm 4.25$	$16.94\pm 3.42$	$2.20\pm 2.27$	$6.00\pm 6.00$	$10.26$
B3	BC-TF, chunk $8$	$772{,}558$	$1.143\times$	$21.11\pm 3.78$	$32.57\pm 2.98$	$12.83\pm 13.64$	$0.29\pm 0.76$	$16.70$
B4	BC-TF, chunk $16$	$916{,}478$	$1.356\times$	$3.77\pm 1.33$	$9.40\pm 2.67$	$0.31\pm 0.66$	$0.29\pm 0.76$	$3.44$
–	PredVLA (ours)	$\mathbf{675{,}732}$	$1.000\times$	$\mathbf{83.19}$	$\mathbf{88.40}$	$\mathbf{89.24}$	$\mathbf{40.57}$	$\mathbf{75.35}$

Table 4: Controlled baseline comparison.

Figure 2: Trainable policy parameters versus LIBERO benchmark mean success rate. Shaded region indicates the parameter range of the controlled baselines in Table 4 .

	Modification	Params	spatial	goal	object	Mean	$\Delta$
L0	— (intact PredVLA)	675,732	$82.71\pm 6.50$	$88.34\pm 4.00$	$87.23\pm 7.10$	86.09	—
L1	$-$ Online error regression	675,732	$72.36\pm 6.00$	$81.50\pm 3.67$	$77.93\pm 4.94$	77.26	$-8.83$
L2	$-$ Predictive Coding Inference	675,732	$58.14\pm 5.89$	$66.71\pm 6.42$	$66.91\pm 5.42$	63.92	$-13.34$
L3	$-$ Prediction Pathway	674,560	$10.94\pm 3.96$	$23.46\pm 6.23$	$\phantom{0}0.29\pm 0.60$	11.56	$\mathbf{-52.36}$
L4	$-$ Temporal hierarchy ( $\tau{=}1$ )	674,560	$\phantom{0}6.83\pm 3.88$	$24.06\pm 7.17$	$\phantom{0}0.11\pm 0.16$	10.33	$-1.23$
L5	PV-RNN cell $\to$ LSTM	676,928	$16.97\pm 3.96$	$25.94\pm 5.05$	$\phantom{0}0.66\pm 0.91$	14.52	$+4.19$

Table 5: The ladder from PredVLA to BC-RNN. Success rates are mean (%)

\pm

standard deviation across seven seeds.

\Delta

is the change in the three-suite mean relative to the preceding rung. L2b is a side branch from L2 and is not part of the main chain.

Figure 3: Performance across the mechanism-by-mechanism ladder from PredVLA (L0) to BC-RNN (L6). Thin lines show success rates on spatial , goal , and object , while the thick black line shows their three-suite mean. Each successive rung removes or replaces one component of the preceding model. The dominant performance decrease occurs when sensory prediction is removed (L2

\rightarrow

L3), after which flattening the temporal hierarchy and replacing the recurrent cell produce comparatively small changes. L2b is a side branch from L2 in which visual features are additionally supplied directly to the action pathway; dashed segments distinguish this branch from the main ladder.

	Change from PredVLA	spatial	goal	object	long
A1	$-$ Temporal hierarchy ( $\tau$ set to its mean)	$61.57\pm 8.73$	$71.07\pm 12.09$	$61.29\pm 12.99$	—
	$\Delta$	$-18.66$	$-16.99$	$-29.03$	—
A2	$-$ Visual-to-action bottleneck	$56.00\pm 15.64$	$72.00\pm 2.75$	$90.36\pm 3.36$	—
	$\Delta$	$-24.23$	$-16.06$	$+0.04$	—
A3	$-$ Efference copy A $\to$ V	$73.71\pm 8.87$	$76.21\pm 6.18$	$87.00\pm 6.58$	—
	$\Delta$	$-6.51$	$-11.84$	$-3.31$	—

Table 6: Ablation results. A1–A3 require retraining, whereas A4–A6 are inference-time changes applied to trained checkpoints.

\Delta

denotes the mean paired difference from the intact model evaluated using the same seven seeds.

Figure 4: Task-level effects of removing the visual prediction-error term ( $\lambda_{v}=0$ ) or the proprioceptive prediction-error term ( $\lambda_{q}=0$ ) during online inference. Within each suite, tasks are sorted by the success-rate change induced by the ablation. Gray bars indicate the intact model, while colored bars indicate the change after removing the corresponding error term. Although the suite-level averages in Table 6 are similar for some suites, the task-level patterns differ substantially, especially on object , where performance is much more sensitive to the proprioceptive prediction-error term.

Figure 5: Effect of corrupting the visual or proprioceptive observations used for error regression. $\alpha$ denotes the fraction of observations replaced by those from an unrelated episode. Bands indicate $\pm 1$ seed standard deviation, and the dashed line indicates the corresponding open-loop score.

Explore similar work

SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation
Aug 10, 2026Jingkai Wang, Zihan Tang, Gu Zhang +7Latent Dynamics Modeling Language-Conditioned Robot Manipulation
1Fudan University · 2Beijing Academy of Artificial Intelligence · 3Tsinghua University +1
Compact Robot Policies Need Fine-Grained Visual Representations
Oct 6, 2026Nanhe Chen, Runqiu Yang, Jiawei Tang +2Robot Policy Learning Visual Representation Learning
X Square Robot · KTH
MINERVA: How Small Can a Manipulation Policy Be and Still Solve LIBERO?
Sep 3, 2026Kohei Sendai, Tatsuya Matsushima, Yusuke IwasawaRobot Skill Learning Visuomotor Policy Learning
Matsuo-Iwasawa Lab, Graduate School of Engineering, The University of Tokyo, Japan

Aug 10, 2026J/K move · Enter open · S save

SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation

Jingkai Wang, Zihan Tang, Gu Zhang, Mingyu Cao+6

1Fudan University · 2Beijing Academy of Artificial Intelligence · 3Tsinghua University+1

Latent Dynamics Modeling Language-Conditioned Robot Manipulation Vision-Language-Action Models Latent World Models

Open paper AskPDF arXiv

Vision-language-action policies rely on large multimodal backbones to jointly perform perception, language conditioning, and action generation at every control step. Much of this capacity supports open-domain semantics, whereas continuous robot manipulation primarily requires compact representations of observations, actions, and the transitions induced by actions. Pixel-level world models provide another route, but predicting visual details irrelevant to control can be unnecessarily expensive. We propose SLIM (Self-supervised Latent Interaction Model), a compact 0.5B-parameter latent interaction policy. SLIM learns action-grounded predictive latents that capture both action-conditioned future transitions and the actions that explain observed changes. SLIM learns these representations through self-supervised masked trajectory prediction, combining action reconstruction with future-latent prediction. A compact Mixture-of-Transformers (MoT) backbone models interactions between observation latents and action tokens. The resulting policy is trained with flow matching for language-conditioned action generation. Across simulation benchmarks and real-world evaluation, SLIM matches or exceeds representative large-scale VLA and world-action-model baselines with fewer parameters, no additional embodied pretraining, lower inference latency, and substantially lower GPU memory usage.

Abstract

Explore similar work

SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation

Compact Robot Policies Need Fine-Grained Visual Representations

MINERVA: How Small Can a Manipulation Policy Be and Still Solve LIBERO?