VLPSA: Vision-Language-Poisson-Safe Actions for Full-Body Safety of Learned Policies

Meg Wilkinson; Emily Fourney; Joel W. Burdick; Aaron D. Ames

cs.ROSep 18, 2026

VLPSA: Vision-Language-Poisson-Safe Actions for Full-Body Safety of Learned Policies

Authors: Meg Wilkinson, Emily Fourney, Joel W. Burdick, Aaron D. Ames

Organizations: Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, CA 91125, USA

Abstract

Vision-language-action (VLA) models enable increasingly general-purpose robotic manipulation, but such learned policies do not provide safety guarantees for collision avoidance---especially in environments outside of training distributions. This work presents Vision-Language-Poisson-Safe Actions (VLPSA), a safety filtering framework that provides full-body safety for VLA policies in cluttered and dynamic environments without retraining. VLPSA synthesizes Poisson Safety Functions (PSF) online from perception data, yielding a Control Barrier Function (CBF) that is enforced through a CBF-QP safety filter over the full body and any grasped object, treated as an extension of the final robot link. To enable real-time deployment while maintaining fine spatial resolution in critical task regions, VLPSA combines dual resolutions of this PSF using Boolean CBF compositions. We evaluate VLPSA on SafeLIBERO against safety-filtering baselines, where it achieves the highest collision avoidance rate among the evaluated methods, increasing collision avoidance from 23.1% for the base $π_{0.5}$ policy to 91.2% while surpassing its task success rate. We further deploy VLPSA on a Franka FR3 in cluttered scenes with dynamic obstacles and human interference, demonstrating real-time full-body safety during manipulation tasks.

Figures & tables

Fig. 1 : Pipeline for VLPSA: Sensory and semantic input and robot state are sent to the VLA for a nominal trajectory and gripper state. Depth sensors are fused with robot state information to create an occupancy map for the synthesis of PSFs at varying resolutions for safety filtering. When grasping, live samples are generated and updated to protect the held object, incorporated into the same safety filter.

Fig. 2 : Multigrid Usage. (Left) Fine-grid usage (% of sample points) for the grasped object during a trial in static clutter and with dynamic obstacles. The grasped object shows consistent moderate usage under clutter but sparser, higher peaks under moving obstacles. (Right) Zero-level set of the fine PSF $h_{f}$ in a cluttered environment showing which grid wins at each point.

Single uniform
Grid ( $\varepsilon$ , $N^{3}$ )	Poisson [ms]	QP [ms]	Total [ms]
$3.0\text{\,}\mathrm{cm}$ , $80^{3}$	$3.44\pm 0.23$	$0.72\pm 0.02$	$6.5\pm 1.9$
$2.0\text{\,}\mathrm{cm}$ , $120^{3}$	$9.12\pm 0.44$	$0.76\pm 0.04$	$23.1\pm 6.6$
$1.5\text{\,}\mathrm{cm}$ , $160^{3}$	$20.20\pm 0.46$	$0.78\pm 0.12$	$38.5\pm 5.7$
$1.0\text{\,}\mathrm{cm}$ , $240^{3}$	$91.80\pm 0.95$	$0.61\pm 0.07$	$261.6\pm 9.0$
$0.5\text{\,}\mathrm{cm}$ , $480^{3}$	$724.19\pm 3.71$	$0.59\pm 0.03$	$2059\pm 33$

TABLE I : Solve cost for a fixed workspace extent (

2.4\text{\,}\mathrm{m}

per side) and a fixed

100

-iteration budget. Mean

\pm

std over

27

–

1708

solves per configuration on an RTX 4090, with

|\mathcal{Y}_{\mathrm{aug}}|=679

. The total time includes all numerical processing. The multigrid scheme resolves

1.0\text{\,}\mathrm{cm}

over the region of interest at coarse-grid cost; it doubles the number of QP constraints, but latency is dominated by the Poisson solve.

Spatial
Method	SR $\uparrow$	CAR $\uparrow$	SSR $\uparrow$	Steps $\downarrow$
Nominal	57.8 $\pm$ 4.8	20.8 $\pm$ 4.0	19.2 $\pm$ 3.9	119.5 $\pm$ 6.3
VLSA ∗	73.8 $\pm$ 4.3	70.5 $\pm$ 4.5	59.8 $\pm$ 4.8	133.4 $\pm$ 4.2
VLPSA-NI	71.5 $\pm$ 4.4	75.8 $\pm$ 4.2	54.5 $\pm$ 4.9	123.0 $\pm$ 5.9
VLPSA	72.2 $\pm$ 4.4	89.8 $\pm$ 3.0	66.5 $\pm$ 4.6	136.9 $\pm$ 5.9
Goal

TABLE II : VLPSA vs. VLSA ∗ [ 11 ] (full action space) and nominal

\pi_{0.5}

on SafeLIBERO (Levels I+II). VLPSA-NI ablates injection of held-object samples into the safe set. Success (SR), Avoidance (CAR,

100-

collision rate), and Safe Success (SSR) rates in %; Steps is the mean number of steps to reach success, over successful episodes.

\pm

denotes a 95% confidence interval (normal approximation;

n{=}400

episodes per suite). Bold marks the best method where its interval does not overlap that of the runner-up.

Fig. 3 : (A) Hardware validation of the VLA policy with and without the proposed safety layer in a cluttered tabletop environment for the task “Pick up the banana.” (Top) The cluttered environment is out-of-distribution for the nominal VLA policy, which completes the task but collides with multiple objects in the scene. (Middle) With the proposed full-body safety architecture, the VLA policy completes the same task while avoiding all collisions. (Bottom Left) The minimum active Poisson Safety Function value per link remains positive through-out the trial. (Bottom Right) The nominal trajectory versus the safety filtered trajectory throughout the trial. (B) The minimum value of the PSF per-link throughout a long horizon trial with the same text prompt as above. This dictates two successful executions of the task in clutter with dynamical obstacles.

Explore similar work

Your Model Already Knows: Attention-Guided Safety Filter for Vision-Language-Action Models
Jun 8, 2026Seongbin Park, Fan Zhang, Baharan Mirzasoleiman +2Robot Safety Safety Filtering
University of California Los Angeles United States
VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer
Dec 9, 2025Songqiao Hu, Zeyi Liu, Shuang Liu +5Robot Safety Vision-Language-Action Models
Department of Automation, Tsinghua University, Beijing 100084, China. · Institute for Embodied Intelligence and Robotics, Tsinghua University, Beijing 100084, China. · 3TetraBOT. +1
SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models
May 30, 2026Jialiang Fan, Weizhe Xu, Zijun Wang +3Robot Safety Robotic Manipulation
University of Notre Dame · University of Pennsylvania

Jun 8, 2026J/K move · Enter open · S save

Your Model Already Knows: Attention-Guided Safety Filter for Vision-Language-Action Models

Seongbin Park, Fan Zhang, Baharan Mirzasoleiman, Shahriar Talebi+1

University of California Los Angeles United States

Robot Safety Safety Filtering Vision-Language-Action Models Dynamic Obstacle Avoidance Robot Collision Avoidance

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models have demonstrated impressive end-to-end performance across a variety of robotic manipulation tasks. However, these policies offer no guarantees against collisions with task-irrelevant objects in the scene. Existing safety filters sidestep this problem by querying a vision-language model (VLM) to identify obstacles and their locations. This, however, is too slow to run in the control loop and can only be invoked at episode initialization, leaving the filter unable to track moving obstacles. We discover that a small number of attention heads within a VLA model reliably localize the object the policy intends to approach. These heads can be exploited within a training-free safety framework that obtains the active target from the attention heads at every step, treats the remainder of the scene as obstacles, and feeds these into a Control Barrier Function (CBF) filter. Together with a lightweight real-time object tracker, this allows for collision avoidance for non-static obstacles. We evaluate our framework on SafeLIBERO, which we extend with moving obstacles. On the original static benchmark, our method performs comparably to an oracle that uses privileged simulator state to identify the target, emulating a VLM-based identification step run once at episode initialization. On the dynamic variant, where the oracle's init-time target assignment becomes stale, our method substantially outperforms it by 43%, on average. Our findings suggest that the perceptual signals needed for real-time safety filtering are already present within VLA policies and can be exploited without additional training or heavy auxiliary models.

Abstract

Explore similar work

Your Model Already Knows: Attention-Guided Safety Filter for Vision-Language-Action Models

VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer

SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models