Behavioral Steering of Vision-Language-Action Models in the Low-Parameter Regime

James Tcheng; Feiyu Gavin Zhu

cs.ROOct 31, 2026

Behavioral Steering of Vision-Language-Action Models in the Low-Parameter Regime

Authors: James Tcheng, Feiyu Gavin Zhu

Organizations: Carnegie Mellon University

Abstract

Reinforcement learning (RL) is increasingly used to adapt pretrained vision-language-action models (VLAs). We investigate the problem of behavioral steering: adapting a VLA's execution style to preferences such as carrying objects higher. To investigate how little trainable capacity such steering requires and how much pretrained capability can be retained, we freeze an OpenVLA-OFT 7B policy and optimize TinyLoRA adapters with 8 to 384 trainable parameters using either policy-gradient RL (GRPO) or evolutionary search (CMA-ES). With just 256 trainable parameters, we raise carry height by 120 mm on trained tasks at 92% success and by 74 mm on a held-out task at 80% success. By rescaling the learned update, we can tune the trade-off between style gains and capability retention without retraining. Increasing adapter capacity generally improves this trade-off, retaining more pretrained capability at comparable style gains. The choice of optimizer also depends on capacity, with CMA-ES being competitive at low parameter counts, while GRPO offers better trade-offs as capacity grows. These findings show that extremely low-dimensional updates can steer pretrained VLA behavior, with adapter capacity, optimizer choice, and update scaling offering practical control over adaptation and retention.

Figures & tables

Figure 1: Style transfer to a held-out task. Our $64$ -parameter update was trained on two other LIBERO-Goal tasks, never on “put the bowl on the stove” (task 1). Top: the unsteered, frozen 7B OpenVLA-OFT policy. Bottom: the same policy with the update learned with evolutionary strategies, which raises its peak carry height by $+188$ mm.

	as trained					h.o. S.R. $=0.90$		h.o. S.R. $=0.75$
	trained tasks		task 1 (h.o.)		h.o.	trained	task 1	trained	task 1
	style	S.R.	style	S.R.	S.R.	style	style	style	style
Unsteered	$0.00$	$0.99$	$0.00$	$1.00$	$0.96$	—	—	—	—
Prompt “high”	$+0.09$	$1.00$	$+0.04$	$0.98$	$0.96$	—	—	—	—
Activation, up, strength $10$	$-0.31$	$0.88$	$+0.12$	$0.98$	$0.63$	—	—	—	—
Activation, up, strength $20$	—	$0.00$	$-0.11$	$0.16$	$0.02$	—	—	—	—

Table 1: Carry-height steering on LIBERO-Goal. Style is the carry height style score ( 3 ) in units of each task’s own spread

\sigma_{k}

(

\sigma_{4}{=}22.1

\sigma_{8}{=}22.0

\sigma_{1}{=}27.7

mm). This is averaged over the two trained tasks, task 4 and task 8. S.R. is the task completion success rate. h.o. is the mean over the eight held-out tasks. Left: the update as trained. Right: the same update scaled by

\alpha

until held-out S.R. drops to

0.90

and

0.75

, read from its

\alpha

sweep.

Panel of Figure 3 — Figure 3: At a fixed weight change, style grows sub-linearly with parameter count for both optimizers, while GRPO’s style grows faster with parameter count than ES’s. Carry-height style score on the trained tasks at a fixed weight change, against parameter count $N_{\mathrm{train}}$ . Style grows sub-linearly in both, at $N^{0.52\pm 0.02}$ ( $R^{2}=0.99$ ) for GRPO and $\propto N^{0.23\pm 0.03}$ ( $R^{2}=0.90$ ) for ES. Faint dashed lines (left panel, right axis) show success on the eight held-out tasks at the same weight change. The 8-parameter cells are absent because their trained updates stop short of $85$ units.

$N_{\mathrm{train}}$	32	64	96	128	256	384
GRPO	0.36	0.41	0.37	0.49	0.46	0.38
ES	0.23	0.20	0.21	0.17	0.11	0.10

Table 2: GRPO’s updates stay aligned across parameter counts while ES’s become less aligned as the count grows. Each entry reports the mean cosine similarity between a given optimizer’s weight update

\Delta W

at parameter count

N_{\mathrm{train}}

and that same optimizer’s updates across the remaining five parameter counts.

1

means the same direction,

0

unrelated.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: Learned updates raise carry height on task 4, “put the bowl on top of the cabinet”. One episode per policy from the same initial state (state $6$ ). Top: unsteered policy. Middle: ES with $N_{\mathrm{train}}{=}256$ , carrying the bowl $108.3$ mm higher at peak. Bottom: GRPO with $N_{\mathrm{train}}{=}256$ , $95.9$ mm higher at peak. Both updates are shown as trained ( $\alpha{=}1$ ).

VLA Model Adaptation

See all 48 papers

Adaptive Capacity Allocation for Vision Language Action Fine-tuning
Mar 8, 2026Donghoon Kim, Minji Bae, Unghui Nam +4VLA Model Training Nonlinear LoRA
Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, Republic of Korea · Department of Computer Science and Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
Not All Layers Need Tuning: Diagnosing and Directing Adaptation in Vision-Language-Action Models
Sep 16, 2026Shahram Najam Syed, Arthur Jakobsson, Prayuj Sachdev +1VLA Model Adaptation Low-Rank Adaptation
Robotics Institute, Carnegie Mellon University, Pittsburgh, USA · Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, USA
Rethinking VLM Representation for VLA Initialization
May 25, 2026Weifeng Lin, Siyuan Huang, Hao Li +5VLM Adaptation Low-Rank Adaptation
1CUHK · 4ACE Robotics · 3Peking University +1

Mar 8, 2026J/K move · Enter open · S save

Adaptive Capacity Allocation for Vision Language Action Fine-tuning

Donghoon Kim, Minji Bae, Unghui Nam, Gyeonghun Kim+3

Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, Republic of Korea · Department of Computer Science and Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea

VLA Model Training Nonlinear LoRA Parameter-Efficient Fine-Tuning VLA Model Adaptation Low-Rank Adaptation Vision-Language-Action Models+1

Open paper AskPDF arXiv

Vision language action models (VLAs) are increasingly used for Physical AI, but deploying a pre-trained VLA model to unseen environments, embodiments, or tasks still requires adaptation. Parameter-efficient fine-tuning (PEFT), especially LoRA, is common for VLA policies, yet the exposed capacity knob, the rank, does not transfer uniformly: robotics transfer exhibits a higher and task-varying intrinsic rank than language fine-tuning. Small ranks suffice for LLMs (e.g.,

r \in \{4, 8\}

), while spectral analyses indicate VLAs may require much larger ranks (e.g.,

r \approx 128

) or near-full rank, a mismatch that worsens in multi-task settings. We present LoRA-SP (Select-Prune), a rank-adaptive fine-tuning method that replaces fixed-rank updates with input- and layer-wise capacity. LoRA-SP uses an SVD-style parameterization with a small router whose nonnegative scores act as singular values over a shared vector bank. The active set is chosen by an energy target on the cumulative squared scores

E(k) \ge η

, providing a direct link to approximation error via our spectral analysis. During training,

η

concentrates energy on a few directions and teaches the router to rely on fewer vectors while preserving accuracy. This yields compact adapters that reduce cross-task interference and improve generalization. On four real-robot manipulation tasks collected on an unseen AgileX PiPER arm, across two VLA backbones (

π_0

and SmolVLA), LoRA-SP matches or exceeds full fine-tuning with far fewer trainable parameters, and improves multi-task success by up to 31.6% over standard LoRA while remaining robust to rank choice.

Abstract

VLA Model Adaptation

Explore similar work

Adaptive Capacity Allocation for Vision Language Action Fine-tuning

Not All Layers Need Tuning: Diagnosing and Directing Adaptation in Vision-Language-Action Models

Rethinking VLM Representation for VLA Initialization