VLA-Stem: A Lightweight Safety Mechanism for Vision-Language-Action Models via Reinforcement Learning

Bojun Zou; Wenhao Yu; Jiaohao Li; Jiaxuan Lin; Guanting Ye; Haohan Geng; Weihao Yu; Yanyong Zhang

cs.ROOct 27, 2026

VLA-Stem: A Lightweight Safety Mechanism for Vision-Language-Action Models via Reinforcement Learning

Authors: Bojun Zou, Wenhao Yu, Jiaohao Li, Jiaxuan Lin, Guanting Ye, Haohan Geng, Weihao Yu, Yanyong Zhang,

Organizations: University of Science and Technology of China (USTC) · Peking University (PKU) · University of Macau (UM)

Abstract

Vision-Language-Action (VLA) models integrate high-level reasoning with low-level action execution for end-to-end robotic control. However, most VLA models target static manipulation in unconstrained environments, overlooking spatial constraints and dynamic obstacles. Fine-tuning VLAs on obstacle-avoidance data is a straightforward solution, but collecting such data is costly, especially for dynamic scenarios, and may degrade their original task capabilities. To address these limitations, we propose VLA-Stem, a lightweight, reinforcement-learning-based safety mechanism for VLA systems. Without fine-tuning the underlying VLA policy, VLA-Stem converts its action chunks into motion references and uses trajectory-level guidance to preserve their spatial intent while allowing flexible motion and timing adjustments. Conditioned on these references, robot proprioception, and local obstacle information, the Stem policy adaptively modifies execution at inference time, preserving the original task intent while enabling responsive adaptation to spatial constraints and dynamic obstacles. For robust and sample-efficient training, we combine teacher-student distillation with curriculum learning. We evaluate VLA-Stem on reference following, static spatial constraints, and dynamic obstacle avoidance. Across six constrained simulation settings, VLA-Stem achieves 52.8% average collision-free task success versus 23.7% for the strongest safety baseline, while preserving base-policy performance in obstacle-free settings; real-world experiments further show consistent collision reductions under static and dynamic obstacles. These results demonstrate that VLA-Stem couples high-level VLA task intent with rapid, reflex-like responses to immediate physical risks, providing a brainstem-inspired mechanism for robust robot execution.

Figures & tables

Fig. 1: This figure illustrates the biological motivation behind our method. The left side depicts the interaction between higher-level motor control and lower-level sensorimotor regulation: task-directed motor commands are continuously shaped by rapid responses to immediate physical conditions. In this functional analogy, higher-level task control and motor coordination correspond to the base VLA, while the brainstem-inspired lower-level regulation corresponds to our proposed VLA-Stem . The right side shows how VLA-Stem adapts the VLA-directed motion to bypass an obstacle while continuing toward the target object.

Fig. 2: VLA-Stem pipeline. The teacher is trained using curriculum learning, progressing from reference trajectory following to static obstacle avoidance and then to dynamic obstacle avoidance, with gradually increasing difficulty. The teacher policy takes as input the trajectory sequence produced by the base VLA policy, robot proprioception, and obstacle information, and additionally leverages privileged task information about the target object to accelerate training. Finally, the teacher policy is used to distill a student policy, removing the dependence on task-specific privileged information at deployment.

Method	Metric	PickCube			Object Stowing			StackCube
		Normal	Static	Dynamic	Normal	Static	Dynamic	Normal	Static	Dynamic
$\pi_{0}$	SUCC $\uparrow$	0.6884	0.0106	0.0044	0.4322	0.0006	0.0559	0.6506	0.1316	0.1453
	COLL $\downarrow$	–	0.9422	0.9703	–	0.5363	0.6809	–	0.6478	0.5484
$\pi_{0.5}$	SUCC $\uparrow$	0.7166	0.0325	0.0131	0.6138	0.0000	0.0256	0.7450	0.1725	0.1247
	COLL $\downarrow$	–	0.8781	0.9328	–	0.5897	0.6156	–	0.7319	0.7012
VLA-finetune	SUCC $\uparrow$	0.7166	0.2847	0.1869	0.6138	0.0072	0.4784	0.7450	0.0062	0.0641

TABLE I: Experimental results on simulation tasks. Higher SUCC is better and lower COLL is better.

Fig. 3: Dynamic obstacle avoidance in simulation. Compared with $\pi_{0.5}$ , VLA-Stem waits for the moving obstacle to pass before resuming task execution.

Fig. 4: Reference-conditioned execution on StackCube: trajectory visualization and quantitative results over 30 paired trials per direction.

Method	Normal		Static		Dynamic
	SUCC $\uparrow$	COLL $\downarrow$	SUCC $\uparrow$	COLL $\downarrow$	SUCC $\uparrow$	COLL $\downarrow$
VLA-Stem w/o reference	0.0000	–	0.0000	0.1300	0.0000	0.0156
VLA-Stem	0.7325	–	0.5219	0.2316	0.4153	0.3122

TABLE II: VLA reference ablation on PickCube. Results are reported for student policies.

Panel of Figure 7 — Fig. 5: Real-world evaluation of VLA-Stem. (a) Real-world experimental examples. (b–c) Quantitative results on Object Stowing and Stacking, respectively.

Method	Normal		Static		Dynamic
	SUCC $\uparrow$	COLL $\downarrow$	SUCC $\uparrow$	COLL $\downarrow$	SUCC $\uparrow$	COLL $\downarrow$
VLA-Stem w/ Step-wise	0.2888	–	0.4778	0.0366	0.1453	0.2194
VLA-Stem	0.7061	–	0.5784	0.1756	0.5950	0.1203

TABLE III: Ablation results of Trajectory-level guidance on Object Stowing.

Method	Normal		Static		Dynamic
	SUCC $\uparrow$	COLL $\downarrow$	SUCC $\uparrow$	COLL $\downarrow$	SUCC $\uparrow$	COLL $\downarrow$
VLA-Stem w/o distillation	0.2903	–	0.0525	0.3066	0.0900	0.4419
VLA-Stem	0.7061	–	0.5784	0.1756	0.5950	0.1203

TABLE IV: Ablation results of Teacher–student distillation on Object Stowing.

Stage	$w_{\mathrm{task}}$	$w_{\mathrm{coll}}$	$w_{\mathrm{follow}}$	$w_{\mathrm{step}}$
Following	0.6	0.0	0.4	0.2
Static	1.2	1.0	0.4	0.1
Dynamic	1.2	1.0	0.6	0.0

TABLE V: Reward weights used in the three curriculum stages.

Parameter	Following	Static	Dynamic
Parallel environments	256	192	192
Learning rate	$2\!\times\!10^{-4}$	$1.5\!\times\!10^{-4}$	$1.5\!\times\!10^{-4}$
PPO update epochs	8	12	12
Minibatch size	3,200	19,200	19,200
Rollout horizon	100	100	100
Discount factor $\gamma$	0.8	0.8	0.8

TABLE VI: Stage-wise PPO training configurations and simulation budgets. Transition counts are approximate and include both teacher and student training.

Vision-Language-Action Models

Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.

See all 1,190 papers

ShieldVLA: Feasibility-Aware Safety Alignment for Vision-Language-Action Models
Sep 2, 2026Manan Tayal, Akshay NambiVision-Language-Action Models Safe RL
Indian Institute of Science (IISc) · Microsoft Research
VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer
Dec 9, 2025Songqiao Hu, Zeyi Liu, Shuang Liu +5Robot Safety Vision-Language-Action Models
Department of Automation, Tsinghua University, Beijing 100084, China. · Institute for Embodied Intelligence and Robotics, Tsinghua University, Beijing 100084, China. · 3TetraBOT. +1
Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering
Jun 10, 2026Hyun Joe Jeong, Gokul Swamy, Andrea BajcsyLanguage Model-Based Control Vision-Language-Action Models
Robotics Institute Carnegie Mellon University

Sep 2, 2026J/K move · Enter open · S save

ShieldVLA: Feasibility-Aware Safety Alignment for Vision-Language-Action Models

Manan Tayal, Akshay Nambi

Indian Institute of Science (IISc) · Microsoft Research

Vision-Language-Action Models Safe RL VLMs for Robotics Reachability Analysis

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models demonstrate strong generalization in robotic manipulation and navigation, but existing fine-tuning methods provide limited safety guarantees. Current approaches primarily rely on Lagrangian optimization that enforces safety through soft penalties on expected cumulative cost, often resulting in residual constraint violations or overly conservative behavior. Moreover, learning safety in visual domains is challenging due to the absence of dense per-step safety annotations. We propose ShieldVLA, a safety-aligned fine-tuning framework for VLA models based on Hamilton-Jacobi (HJ) reachability. ShieldVLA learns a model-free approximation of the HJ reachability value function directly from visual observations to estimate the safe operating region. The learned safety critic gates policy optimization by separating reward maximization within feasible regions from recovery near unsafe states, avoiding persistent reward-cost trade-offs. To enable scalable supervision in visual environments, we introduce rubric-based VLM safety scores that convert semantic safety feedback into structured critic targets without requiring manual cost labels. Across five navigation and manipulation benchmarks spanning multiple VLA backbones, ShieldVLA reduces cumulative safety cost by 57% on average and improves task success rate by +0.13 over SafeVLA.

Abstract

Vision-Language-Action Models

Explore similar work

ShieldVLA: Feasibility-Aware Safety Alignment for Vision-Language-Action Models

VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer

Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering