A Safe Action Is Not Enough: Feasible-Future Decoding for Vision-Language-Action Policies

Tu Nguyen; Matthieu Zimmer; Vu Anh Vu; Ziyi Wang; Jannik Hammel Nielsen; Xuebing Zhou; Haitham Bou Ammar

cs.AIOct 4, 2026

A Safe Action Is Not Enough: Feasible-Future Decoding for Vision-Language-Action Policies

Authors: Tu Nguyen, Matthieu Zimmer, Vu Anh Vu, Ziyi Wang, Jannik Hammel Nielsen, Xuebing Zhou, Haitham Bou Ammar

Organizations: Huawei Heisenberg Research Center · Huawei Noah’s Ark Lab · TU Berlin · UCL Center for AI

Abstract

A safe action is not necessarily a viable one. A frozen vision-language-action (VLA) policy can favor a locally admissible move that leaves no policy-supported route to safe task completion. We call this the feasibility-likelihood gap: likelihood ranks the next move, while feasibility depends on the futures it leaves open. To bring those futures into the decision, we derive the exact next-block marginal of the history-conditioned policy-environment trajectory law restricted to safe task completion. The derivation reveals a candidate-dependent feasible-future mass: its support records whether safe completion remains possible under the frozen continuation process, while its magnitude measures how much weighted safe-completion mass remains. Since exact evaluation is impractical online, we develop a selective finite-candidate approximation and establish conditions for recovering the best retained viable candidate. Our alarm-triggered, training-free reranker VICS-G lowers mean cumulative safety cost by 1.9%-57.5% across six Safety-CHORES settings while remaining within 2.5 percentage points of policy sampling in success and 0.82 steps in mean episode length. Our approach offers a promising and practical path toward safer task completion, grounded in an exact policy-relative target yet requiring neither policy retraining nor online rollouts.

Figures & tables

Figure 1: When progress stalls, find a way forward. In a selected ObjectNav episode ( navigate to an apple ), policy sampling fails at cost $23$ , while the local-robustness decoder RCD ends early at cost $3$ . VICS-G breaks the cycle of failed retries and reaches the goal at cost $11$ . Circles mark the region of failed forward moves. All three incur safety cost; the full trajectories and intervention trace appear in Appendix A .

Figure 2: Local safety is not enough. Local admissibility removes the unsafe shortcut

a_{U}

, but only future support removes the safe dead end

a_{D}

; among viable actions, feasible-future magnitude then favors

a_{H}

over

a_{L}

. The sequence

a_{U}\!\to a_{D}\!\to a_{L}\!\to a_{H}

shows successive ideal scoring factors, not physical time.

Setting	Decoder	Success $\uparrow$	$\Delta$ S (pp)	Cost $\downarrow$	Cost red. $\uparrow$	Viol. $\downarrow$	Steps
PickUp / base	Policy sample	.894 (143/160)	–	1.656	–	1.656	41.78
	RCD	.775 (124/160)	$-11.9$	.931	43.8%	.906	33.66
	VICS-G	.894 (143/160)	$0.0$	1.625	1.9%	1.625	41.83
	VICS-S	.900 (144/160)	$+0.6$	1.175	29.1%	1.175	40.09
PickUp / safe	Policy sample	.925 (148/160)	–	.400	–	.400	45.19
	RCD	.850 (136/160)	$-7.5$	.256	36.0%	.244	40.76

Table 1: VICS-G consistently lowers safety cost while retaining near-policy success and exposure on Safety-CHORES. Success is rate (count/

N

);

\Delta

S and cost reduction are relative to policy sampling. Steps report exposure. VICS-S denotes task-specific diagnostic profiles (Appendix F ). Values are descriptive aggregates.

Decoder	Success (%) $\uparrow$	$\Delta$ S (pp)	Safe success (%) $\uparrow$	Cost $\downarrow$	Cost red. $\uparrow$	Viol. $\downarrow$	Steps
Policy sample	55.09	–	28.63	4.33	–	3.42	116.9
RCD †	37.79	$-17.30$	23.11	1.98	54.3%	1.53	104.7
Switch after failure †	53.92	$-1.16$	27.76	4.27	1.3%	3.39	118.5
VICS-G (feedback masked)	55.09	$0.00$	28.49	4.03	6.8%	3.18	117.0
VICS-G + execution feedback	55.67	$+0.58$	28.92	3.64	15.9%	2.85	116.9

Table 2: Execution feedback preserves near-policy task success while reducing observed safety cost and violations under perturbed execution. Safety-aligned Fetch,

\rho=0.1

; Reranking uses

K=12

. IID and episode-persistent perturbations have equal weight (

688

executions per method,

172

paired tasks). Means are descriptive; safe success requires zero violations.

\Delta

S (pp) and cost reduction are relative to policy sampling and use unrounded values. Bold: best; underline: second-lowest cost and violations, including ties; steps are unranked.

\dagger

marks post-outcome supplementary baselines. Full conditions and paired inference appear in Appendix I .

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: VICS-G escapes the failed-action loop and completes the task. In ObjectNav (“navigate to an apple”), policy sampling accumulates cost through failed retries and ends unsuccessfully at cost $23$ , while RCD stops early at cost $3$ . VICS-G replaces the marked actions, leaves the failure region, and completes the task at cost $11$ .

Figure 4: VICS-G preserves task completion while the alternatives fail differently. From the same initial Fetch view, VICS-G finishes holding the target plant at cost $12$ ; policy sampling ends with the wrong object (a book) at cost $19$ , while RCD ends empty-handed at cost $3$ . Later frames are aligned by interaction phase rather than elapsed time.

Support rule	Magnitude	Selected	Safe success	Regret
No rejection	Flat	$b_{D}$	$0$	$+\infty$
No rejection	Exact	$b_{D}$	$0$	$+\infty$
Exact support	Flat	$b_{L}$	$1$	$1$
Exact support	Exact	$b_{H}$	$1$	$0$

Appendix

Table 3: Effect of support filtering and continuation weighting in the three-action example. Both viable actions succeed with probability one, but their suffix rewards differ. Regret is measured in base-two score units.

Action group	Motions or events	$h$	$q$	$\alpha$	$\gamma$
Forward	Move forward	$.20$	$3$	$.125$	$.075$
Turn	Rotate left or right	$.35$	$2$	$.25$	$.175$
Arm translation	Raise, lower, extend, or retract	$.05$	$1$	$.75$	$.625$
Backward	Move backward	$-.10$	$1$	$.625$	$.50$
Other	Grasp, release, wrist rotation, or termination	$0$	$1$	$.50$	$.50$

Appendix

Table 4: Shared action features for PickUp, ObjectNav, and Fetch with both checkpoints.

h

defines the continuation preference;

(q,\alpha,\gamma)

define the repetition term

P_{t}(a)

in Equation 47 . Small-motion variants share their group’s values.

Setting	$\Delta$ Success (pp)	$\Delta$ Cost	$\Delta$ Violations	$\Delta$ Steps
PickUp / base	$0.00$ $[0.00,0.00]$	$-0.03$ $[-0.09,0.00]$	$-0.03$ $[-0.09,0.00]$	$0.00$ $[-0.02,0.02]$
PickUp / safe	$0.00$ $[0.00,0.00]$	$-0.01$ $[-0.04,0.02]$	$-0.01$ $[-0.04,0.02]$	$0.01$ $[-0.02,0.03]$
ObjectNav / base	$-3.00$ $[-6.00,-0.50]$	$0.91$ $[-1.97,3.57]$	$1.09$ $[-0.67,2.96]$	$3.25$ $[-0.22,7.31]$
ObjectNav / safe	$-1.50$ $[-3.50,0.00]$	$-0.04$ $[-0.23,0.14]$	$-0.07$ $[-0.23,0.07]$	$0.64$ $[-1.22,2.77]$
Fetch / base	$1.16$ $[-1.74,4.07]$	$-0.99$ $[-2.76,0.10]$	$-0.54$ $[-1.57,0.20]$	$0.35$ $[-1.34,2.28]$
Fetch / safe	$0.00$ $[0.00,0.00]$	$0.01$ $[-0.15,0.13]$	$-0.04$ $[-0.17,0.06]$	$0.51$ $[-0.13,1.40]$

Appendix

Table 5: Incremental effect of the

z_{G}

correction:

\eta_{v}=0.10

minus zero. The gate and history-dependent local score remain active in both arms. Estimates have

95\%

pointwise bootstrap intervals; success uses percentage points and other outcomes use mean episode differences. Positive success and negative cost/violations favor

0.10

. Fetch uses all

20

actions.

Figure 5: The continuation correction reduces observed violations in five of six sweep settings at the nominal weight. Curves show changes from

\eta_{v}=0

: success in percentage points and other outcomes in episode-mean units. The vertical line marks

0.10

; markers are evaluated weights and lines guide the eye. These descriptive trends are interpreted with the paired intervals in Table 5 . Fetch uses all

20

actions. Safe denotes a safety-aligned checkpoint.

Method	Task success (%) $\uparrow$	Safe success (%) $\uparrow$	Cost $\downarrow$	Steps
Policy sample	56.98	28.49	4.052	118.89
RCD	40.12	28.49	1.436	106.11
VICS-G	58.72	27.91	3.233	117.65
VICS-R	58.72	30.23	3.116	118.34

Appendix

Table 6: Rollout-augmented VICS-G on safety-aligned Fetch. All methods use the same

172

tasks and modified simulator. VICS-R adds one early rollout comparison to VICS-G , using four continuations per candidate. Safe success requires completion with zero cumulative cost. Cost and steps are episode means. Bold marks the highest success rates and lowest cost, including ties.

Clean † ( $172$ executions per method)
Decoder	Success (%) $\uparrow$	Safe success (%) $\uparrow$	Safety cost $\downarrow$	Violations $\downarrow$	Steps
Policy sample	58.14	30.81	3.628	2.773	116.63
RCD	40.70	25.58	1.797	1.407	104.34
Switch after failure	58.72	30.81	3.058	2.541	116.50
VICS-G (feedback masked)	59.88	31.98	3.151	2.535	115.45
VICS-G + execution feedback	59.30	31.40	3.529	2.797	115.65

Appendix

Table 7: Under both shifts, the feedback-aware decoder lowers observed cost at near-policy completion. All five decoders use the same

172

tasks. Values are descriptive means; safe success requires zero violations. Within each condition, bold marks the best values and underlining marks second-lowest cost and violations, including ties. Steps report exposure and are unranked.

Outcome	Clean †	IID shift	Persistent shift
Success (pp)	$-0.58$ $[-2.91,\,1.16]$	$+1.16$ $[-1.16,\,3.20]$	$0.00$ $[-2.33,\,2.33]$
Safe success (pp)	$-0.58$ $[-1.74,\,0.00]$	$+0.58$ $[-1.45,\,2.91]$	$+0.29$ $[-1.45,\,2.03]$
Safety cost	$+0.378$ $[-0.006,\,0.878]$	$-0.424$ $[-1.259,\,0.363]$	$-0.358$ $[-1.794,\,0.683]$
Violations	$+0.262$ $[-0.006,\,0.576]$	$-0.320$ $[-0.866,\,0.198]$	$-0.334$ $[-1.375,\,0.462]$
Steps	$+0.20$ $[-0.41,\,0.87]$	$-0.57$ $[-3.49,\,2.44]$	$+0.38$ $[-2.10,\,2.96]$

Appendix

Table 8: Paired effects of execution feedback. Each cell gives the feedback-minus-masked difference and its two-sided

95\%

paired task-bootstrap interval. Positive differences favor feedback for success outcomes; negative differences favor feedback for cost and violations. Success differences use percentage points (pp).

Step / outcome	Means: feedback / ref.	Difference [ $95\%$ CI]	Required bound	Decision
A. Primary tests: reference is the same decoder without feedback
1 / Success	$55.67\%\,/\,55.09\%$	$+0.58\;[-1.16,\,2.33]$ pp	Lower $>-5$ pp	Pass
2 / Safety cost	$3.642\,/\,4.033$	$-0.391\;[-1.211,\,0.278]$	Upper $<0$	Not established; stop
B. Descriptive comparisons: reference is policy sampling
3 / Success	$55.67\%\,/\,55.09\%$	$+0.58\;[-1.16,\,2.33]$ pp	Lower $>-5$ pp	Not tested
4 / Safety cost	$3.642\,/\,4.330$	$-0.688\;[-1.329,\,-0.077]$	Upper $<0$	Not tested

Appendix

Table 9: Prespecified sequential tests of execution feedback. Original

2{,}064

-execution study, with IID and persistent shifts weighted equally. Differences are feedback-aware minus the reference named in each panel; brackets give two-sided

95\%

paired task-bootstrap intervals. The success tolerance is

5

percentage points.

Explore similar work

May 30, 2026cs.RO

SafeVLA-Bench: A Benchmark for the Success-Safety Gap in Vision-Language-Action Models

Vision-language-action (VLA) benchmarks measure whether a policy completes a requested manipulation task, but binary success can hide safety violations along the trajectory: a policy may reach the goal while applying excessive contact, disturbing bystander objects, destabilizing a held object, or entering robot self-contact. We present SafeVLA-Bench, a post-hoc safety-evaluation framework for existing simulator-based VLA benchmarks that reveals violations missed by success-only evaluation. It encodes task-aware safety requirements as Signal Temporal Logic (STL) invariants with quantitative robustness semantics. Alongside native success, it reports the safety rate and the success-but-unsafe rate (SBU) used in prior safety evaluations, and introduces the Violation Severity Index (VSI), a bounded worst-violation depth score. We instantiate SafeVLA-Bench on LIBERO and RoboCasa-365, evaluating twenty-seven policy-benchmark entries across tabletop and kitchen manipulation tasks. High task success does not imply safe execution: the fifteen tabletop policies above 90% mean success still have 18-28% unsafe-episode rates, and 38-56% of successful RoboCasa-365 rollouts violate at least one active safety clause. A post-training case study further shows that SafeVLA-Bench can be used to improve policy safety. Project page: https://safevla.org

Jialiang Fan, Weizhe Xu, Zijun Wang +3

University of Notre Dame · University of Pennsylvania

Topic pending

Ask PDF

Sep 28, 2026cs.LG

Vision--Language Signals in Constrained RL: Safety Gains Without Anticipation

Safe reinforcement learning seeks policies that maximise task performance while satisfying safety constraints. In driving benchmarks, however, collision costs typically appear only at the time of collision, providing no advance warning of an approaching hazard. Frozen vision--language models can provide dense semantic feedback, yet it remains unclear whether their scores anticipate collisions and which component drives an observed safety improvement. Episodic cost can also favour policies that make little task progress. To address these gaps, we propose VLM-Safe-RL, a framework that integrates frozen CLIP signals into PPO-Lagrangian through reward shaping and an augmented multiplier update. On MetaDrive Hard, which combines the densest traffic with the largest map, the catastrophe rate falls from 31.6% to 19.4%. FormulaOne-L2 analysis finds no evidence that the CLIP signals anticipate collisions and shows that the VLM term has a negligible effect on the Lagrange multiplier. These findings show a conditional reduction in observed catastrophe rate without evidence of collision anticipation.

Samuel Tetteh, Cody Fleming

Iowa State University Ames, Iowa, USA

Frozen Vision-Language Models Safety Alignment Trajectory-Level Credit Collision Prediction

Ask PDF

Jun 25, 2026cs.RO

ForesightSafety-VLA: A Unified Diagnostic Safety Benchmark for Vision-Language-Action Models

In embodied intelligence, safety is a prerequisite for reliable robot deployment in the physical world. Current vision-language-action (VLA) models continue to advance toward general-purpose task capability, yet their embodied safety limits remain poorly understood. To address this gap, we introduce ForesightSafety-VLA, a diagnostic benchmark that makes safety the primary evaluation target for VLA systems. We define a 13-category safety taxonomy covering physical interaction safety (Safe-Core), instruction-side safety (Safe-Lang), and perception-side safety (Safe-Vis), and evaluate policies under three controlled dimensions of variation -- scene structure, language command, and visual observation -- so that failure sources can be diagnosed rather than hidden in a single aggregate score. Beyond binary task success, ForesightSafety-VLA measures process-level risk through cumulative safety cost (CC) and risk exposure time (RET), together with a four-quadrant decomposition of safe/unsafe success and failure. We instantiate 66 safety-augmented base scenarios in RoboTwin across 5 embodiments and report results on representative VLA baselines. Across the evaluated baselines, even the strongest policy incurs non-trivial safety cost and unsafe nominal success, while structure and visual variation induce substantially stronger safety degradation than ordinary language variation. These results suggest that embodied safety is tightly coupled to perception, grounding, and control competence rather than being reducible to post-hoc safety filtering alone.

Mingyang Lyu, Yinqian Sun, Yiyang Jia +5

Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China. · Beijing Key Laboratory of Safe AI and Superalignment, China. · Beijing Institute of AI Safety and Governance, China. +2

Foresight

Ask PDF