ShieldVLA: Feasibility-Aware Safety Alignment for Vision-Language-Action Models
Organizations: Indian Institute of Science (IISc) · Microsoft Research
Abstract
Vision-Language-Action (VLA) models demonstrate strong generalization in robotic manipulation and navigation, but existing fine-tuning methods provide limited safety guarantees. Current approaches primarily rely on Lagrangian optimization that enforces safety through soft penalties on expected cumulative cost, often resulting in residual constraint violations or overly conservative behavior. Moreover, learning safety in visual domains is challenging due to the absence of dense per-step safety annotations. We propose ShieldVLA, a safety-aligned fine-tuning framework for VLA models based on Hamilton-Jacobi (HJ) reachability. ShieldVLA learns a model-free approximation of the HJ reachability value function directly from visual observations to estimate the safe operating region. The learned safety critic gates policy optimization by separating reward maximization within feasible regions from recovery near unsafe states, avoiding persistent reward-cost trade-offs. To enable scalable supervision in visual environments, we introduce rubric-based VLM safety scores that convert semantic safety feedback into structured critic targets without requiring manual cost labels. Across five navigation and manipulation benchmarks spanning multiple VLA backbones, ShieldVLA reduces cumulative safety cost by 57% on average and improves task success rate by +0.13 over SafeVLA.
Figures & tables
| Rubric | What it asks the VLM | |
|---|---|---|
| Proximity | How close is the nearest obstacle? | 5 |
| Heading | Is the robot moving toward an obstacle? | 4 |
| Contact | Is the robot in contact with an obstacle now? | 5 |
| Occlusion | How occluded is the relevant obstacle? | 2 |
| Clearance | How much room is there to manoeuvre? | 3 |
| Dubins-VL | TurtleBot-Nav | S-CHORES Nav | S-CHORES Fetch | Franka-Reach | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Method | SR | CSC | SR | CSC | SR | CSC | SR | CSC | SR | CSC |
| Unconstrained VLA | 0.08 | 45.7 | 0.30 | 22.2 | 0.45 | 13.2 | 0.15 | 14.1 | 0.10 | 45.0 |
| SafeVLA ( Zhang et al., 2025a ) | 0.22 | 2.66 | 0.34 | 14.6 | 0.59 | 1.82 | 0.34 | 8.98 | 0.34 | 13.4 |
| VisionCBF ( Tayal et al., 2025a ) | 0.18 | 1.31 | 0.38 | 12.1 | 0.24 | 1.22 | 0.18 | 6.34 | 0.29 | 6.8 |
| HJ (GT cost) | 0.38 | 1.13 | – | – | – | – | – | – | – | – |
| ShieldVLA (Ours) | 0.32 | 1.3 | 0.54 | 6.90 | 0.68 | 1.44 | 0.49 | 5.50 | 0.45 | 3.50 |
| Metric | Dubins-VL | TurtleBot-Nav | S-CHORES Nav | S-CHORES Fetch | Franka-Reach |
|---|---|---|---|---|---|
| SR | |||||
| CSC |
| TurtleBot-Nav | Safety-CHORES Nav | Safety-CHORES Fetch | Franka-Reach | |||||
|---|---|---|---|---|---|---|---|---|
| Cost signal | SR | CSC | SR | CSC | SR | CSC | SR | CSC |
| Binary | 0.31 | 2.80 | 0.65 | 1.60 | 0.60 | 7.14 | 0.25 | 1.20 |
| VLM Rubric | 0.54 | 6.90 | 0.68 | 1.44 | 0.49 | 5.50 | 0.45 | 3.50 |
| Critic use | SR | CSC |
|---|---|---|
| Lagrangian penalty | 0.33 | 7.80 |
| ShieldVLA gating | 0.54 | 6.90 |
| TurtleBot-Nav | Safety-CHORES Nav | Safety-CHORES Fetch | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SafeVLA | Ours | SafeVLA | Ours | SafeVLA | Ours | |||||||
| Condition | SR | CSC | SR | CSC | SR | CSC | SR | CSC | SR | CSC | SR | CSC |
| In-dist | 0.34 | 14.6 | 0.54 | 6.9 | 0.59 | 1.82 | 0.68 | 1.44 | 0.34 | 8.98 | 0.49 | 5.50 |
| +Color | 0.29 | 22.6 | 0.51 | 7.8 | 0.54 | 3.42 | 0.70 | 1.17 | 0.31 | 15.1 | 0.45 | 6.20 |
| +Light | 0.30 | 22.7 | 0.59 | 5.7 | 0.59 | 2.33 | 0.64 | 0.91 | 0.32 | 8.33 | 0.45 | 6.33 |
| +All | 0.30 | 22.7 | 0.51 | 10.7 | 0.56 | 3.27 | 0.70 | 0.77 | 0.30 | 12.66 | 0.45 | 7.18 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| SR | 0.42 | 0.46 | 0.43 | 0.41 | 0.37 |
| CSC | 4.24 | 0.82 | 0.57 | 0.08 | 0.0 |
| 5-rubric (base) | 7-rubric (post-RTD) | |
|---|---|---|
| Qwen3-VL-2B | 0.500 (mode collapse) | 0.550 |
| Qwen3-VL-8B | 0.512 | 0.608 |
| Scorer | CSC | SR |
|---|---|---|
| Unshielded BC (no critic) | – | 0.30 |
| ShieldVLA, Qwen3-VL-2B | 7.59 | 0.56 |
| ShieldVLA, Qwen3-VL-8B | 6.90 | 0.54 |
| Qwen3-VL-2B | Qwen3-VL-8B | |||
|---|---|---|---|---|
| Condition | CSC | SR | CSC | SR |
| In-Dist | 7.59 | 0.56 | 6.90 | 0.54 |
| +Color | 8.52 | 0.47 | 7.80 | 0.51 |
| +Light | 7.60 | 0.51 | 5.70 | 0.59 |
| +All | 11.90 | 0.48 | 10.70 | 0.54 |
| Scorer | Per-frame AUROC | Spearman ( , log 1+ cost) |
|---|---|---|
| Qwen3-VL-2B | 0.539 | +0.094 |
| Qwen3-VL-8B | 0.548 | +0.272 |
| Votes | Weight | Criterion | Definition |
|---|---|---|---|
| 23 | 4 | Obstruction | Path forward obstructed by furniture / clutter / walls |
| 15 | 4 | Obstacle_Proximity | Distance to nearest obstacle in immediate vicinity |
| Environment | Training Steps | Wall-Clock Time | GPU-Hours |
|---|---|---|---|
| Dubins-VL | 2M | 0.5h | 4 |
| TurtleBot-Nav (HJ critic) | 12M | 8h | 64 |
| Safety-CHORES Nav | 15M | 50h | 400 |
| Safety-CHORES Fetch | 25M | 85h | 700 |
| Franka-Reach | 15M | 24h | 192 |
| VLM labelling (8B, per env) | – | 26 min | 8 |
| Hyperparameter | Dubins-VL | TurtleBot-Nav | S-CHORES Nav | S-CHORES Fetch | Franka-Reach |
|---|---|---|---|---|---|
| Safety critic | |||||
| Discount factor | 0.99 | 0.95 | 0.99 | 0.99 | 0.99 |
| Safety critic LR | 3e-4 | 3e-4 | 1e-4 | 1e-4 | 1e-4 |
| Target update rate | 0.005 | 0.001 | 0.005 | 0.005 | 0.005 |
| Replay buffer size | 200K | 500K | 500K | 500K | 500K |
| Critic hidden dims | [256, 256] | [512, 512] | [512, 512] | [512, 256] | [512, 256] |
Vision-Language-Action Models
Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.
See all 1,154 papers