Vision-and-language navigation (VLN) agents are typically evaluated by whether they reach a target and how efficiently they follow an instructed route, yet these metrics provide little information about collision behavior. Moreover, continuous VLN evaluation commonly uses a sliding regime, where an agent can slide along obstacles after a collision, potentially masking the effect of unsafe actions on navigation performance. We introduce VLN-Safe, a benchmark for analyzing collision safety under two complementary evaluation regimes: sliding, reflecting common VLN evaluation practice, and blocking, where collision-inducing actions prevent forward motion and more directly expose their consequences. Across six published agents on R2R-CE, 38.5-56.8% of successful episodes contain contact under sliding, while switching to blocking decreases success rate by 11.7-36.1 percentage points. We further analyze signals for anticipating imminent collisions and find that depth geometry and recent movement history are more informative than policy-derived signals, while safe alternatives near the policy's preferred action are rarely available. Motivated by these findings, we introduce a training-free collision guard that vetoes risky forward actions, executes an escape maneuver, and returns control to the original policy. Under blocking, the guard reduces collision steps by 62-98% and improves success rate by 6.9-29.3 percentage points across five recurrent, topological, and vision-language-action agents. Under sliding, it leaves success rate unchanged while raising the share of collision-free successes by 4.1-12.4 points. These results reveal a substantial gap between task success and collision-safe navigation and show that lightweight collision-aware intervention can improve both safety and navigation performance.
Figures & tables
Figure 1. Overview of the VLN settings and contact outcomes. (a) Graph-based VLN moves between predefined viewpoints. (b) Continuous VLN-CE executes low-level movements that can contact obstacles. (c,d) Successful navigation can be collision-free or include contact along the route. Four panels compare navigation settings and outcomes. (a) A route follows edges between fixed viewpoints. (b) A continuous route contacts furniture. (c) A successful route reaches the goal without obstacle contact. (d) A successful route reaches the goal after several contacts.
Figure 2. The collision guard. (a) Every forward primitive, from the policy or from an escape, is screened by the stall memory B and the depth clearance test c(Dt,xt)<τ ; turns and stops pass unchanged, and two counters ( q consecutive and n per-episode depth vetoes) let a flagged step through rather than trap the agent. (b) A veto executes a pending stop , or turns to the nearest heading whose two-step path is clear and outside B , submits one escape forward back to (a), and returns control to the policy. A two-panel flowchart. The first panel screens policy and escape primitives using stall memory, depth clearance, and two veto counters. A veto enters the second panel, which handles STOP, selects an escape heading, and returns an escape forward to the first panel for another check.
Success rate ↑
N-C rate ↓
Dirty success ↓
Stuck ↓
Proxy miss ↓
Agent
Tier
slide
block
slide
block
slide
block
slide
block
slide
R2R-CE, 1,839 episodes
ETP-R1
t2
64.6
50.5 −14.1
4.0
13.9 +9.9
41.7
42.3 +0.6
5.4
44.4 +39.0
77.7
ETPNav
t2
56.9
41.5 −15.4
4.9
21.0 +16.1
40.2
31.8 −8.4
4.2
43.8 +39.6
85.0
OmniNav
t1
67.5
31.4 −36.1
4.9
30.5 +25.6
56.8
48.4 −8.4
5.3
70.5 +65.2
87.6
StreamVLN
t1
56.0
27.7 −28.3
6.8
41.4 +34.6
38.5
17.6 −20.9
5.7
43.4 +37.7
80.3
Table 1. Benchmark results under slide and block (%). Small colored values beside block report block − slide changes: red is worse under block and green is better, following the arrows. Contact uses κρ ; DSR is Pr[contact∣success] , stuck denotes any proxy stuck event, and proxy miss is measured under slide . Bold marks the primary P1 and predicate statistics.
Figure 3. R2R-CE under slide . (a) Episode outcome and contact status; the dirty-success rate (DSR) is printed at right. (b) Position of the first collision step within each dirty success, as a fraction of the episode (boxes: quartiles; whiskers: 10th-90th percentiles; bar: median). Two panels sharing one row per agent. Left: horizontal stacked bars of clean success, dirty success, clean failure and dirty failure shares for six agents, with dirty-success rates between 38 and 57 percent. Right: box plots showing that first contact in a dirty success typically happens between one quarter and two thirds of the way through the episode, with medians between 0.32 and 0.46.
External ↑
Policy outputs ↑
Coverage ↑
Agent
Depth
Hist.
Ent.
Stop
Marg.
Hist. rec.
Swap
R2R-CE
ETP-R1
.811
.891
.673
.416
.637
85.3
4.0
ETPNav
.759
.916
.564
.524
.492
91.1
0.7
OmniNav ∗
.826
.862
–
.493
–
82.3
–
StreamVLN ∗
.832
.981
.539
.358
.518
98.6
0.4
Table 2. Collision-prediction signals under block . Columns report AUROC, movement-history recall, and the share of risky decisions with a geometrically safe alternative within 10 score points (Swap); the latter two are percentages. The largest AUROC per row is bold; ∗ marks t1 agents, whose alternatives are primitives. – : not exposed (OmniNav emits one waypoint, with no scored alternatives).
SR ↑
SPL ↑
nDTW ↑
Clean ↑
Coll. steps ↓
Events ↓
Immobile ↓
Own
Agent
No
Guard
No
Guard
No
Guard
No
Guard
No
Guard
No
Guard
No
Guard
R2R-CE, block
ETP-R1
50.5
57.4 +6.9∗
43.3
49.7 ∗
60.9
64.3 ∗
29.1
39.2 +10.1∗
18.1
3.2 −83%
2.60
2.65
29.0
15.3
8.4
ETPNav
41.5
52.2 +10.7∗
36.4
45.7 ∗
57.9
62.0 ∗
28.3
37.9 +9.6∗
21.1
3.0 −86%
2.15
2.51
33.0
16.3
9.8
OmniNav
31.4
60.7 +29.3∗
30.0
56.2 ∗
58.7
69.2 ∗
16.2
27.0 +10.8∗
18.7
7.1 −62%
3.36
6.27
13.2
12.3
20.1
StreamVLN
27.7
50.1 +22.4∗
25.2
44.0 ∗
56.6
60.4 ∗
22.8
29.7 +6.9∗
201.5
6.8 −97%
46.78
6.29
54.6
10.1
18.4
Table 3. The collision guard, paired by episode with the unguarded run of the same agent (R2R-CE: 1,839 episodes; RxR-CE: 1,223). Each metric reports No guard (left) and Guard (right); small values give Guard − No guard, green better and red worse. Clean: success without any collision step. Immobile: episodes with a 40-primitive window of net displacement below 0.5 m, a stall measure the stall veto cannot satisfy by construction. Own: share of executed primitives chosen by the guard. ∗ : paired 95% bootstrap interval excludes zero. SR, SPL, nDTW, Clean, Immobile and Own in %; steps and events per episode.
Agent
Flips F → S/S → F
Stuck gain (%)
Timeout No/Guard (%)
Budget No/Guard (%)
R2R-CE
ETP-R1
173/46
93
0.1/ 0.1
38.4/ 28.6
ETPNav
233/37
94
0.0/ 0.0
41.2/ 26.8
OmniNav
601/62
95
0.0/ 0.0
– / –
StreamVLN
430/18
79
52.9/ 1.1
– / –
CMA
251/20
100
62.8/ 11.1
– / –
Table 4. Guard mechanisms under block . Flips: failures → successes / successes → failures. Stuck: share of net success gain from episodes stuck without the guard. Timeout (500 steps) and budget exhaustion show No guard / Guard rates (%); guarded values are bold. Budget applies to ETP agents only.
Condition
SR ↑
SPL ↑
Clean ↑
Steps ↓
Events ↓
No guard
50.5
43.3
29.1
18.1
2.60
Guard (ours)
57.4 +6.9
49.7 +6.4
39.2 +10.1
3.2 −14.9
2.65 +0.05
stall only
57.6 +0.2
49.5 −0.2
29.1 −10.1
4.4 +1.2
3.51 +0.86
depth only
52.9 −4.5
45.5 −4.2
39.2
13.2 +10.0
2.05 −0.60
8-frame depth
55.2 −2.2
47.1 −2.6
44.1 +4.9
2.1 −1.1
1.81 −0.84
τ=0.10 m
59.1 +1.7
51.1 +1.4
47.0 +7.8
2.3 −0.9
1.92 −0.73
Table 5. Guard components on ETP-R1, R2R-CE, block . Small values compare Guard with No guard, and indented variants with Guard ( green better, red worse). Stall only and depth only remove the depth and the stall veto; 8-frame depth accumulates eight frames; the oracle replaces both predictors with the navigation mesh itself. SR, SPL, Clean in %; steps and events per episode.
Safe robot navigation develops methods for reaching goals while avoiding collisions and managing uncertainty, dynamic obstacles, and environmental hazards.
Continuous vision-and-language navigation (VLN) is typically evaluated by goal-reaching success, assuming it reflects an agent's ability to execute language-grounded behavior. We show that this assumption can fail: permissive contact models in current simulators may mask physically infeasible actions, creating a physical executability gap between semantic competence and embodied realization. To bridge this gap, we introduce Visual-Semantic Traversability Alignment (VeSTA), a framework that aligns VLN policies with physically executable futures. We define visual-semantic traversability as the ability to predict instruction-conditioned trajectories that are semantically valid, geometrically feasible, and physically robust. VeSTA addresses this misalignment through two complementary mechanisms: improving executable support in trajectory generation and risk-aware selection among multimodal futures. Specifically, Trajectory Relative Group Policy Optimization uses strict-contact feedback to reshape the candidate distribution toward executable regions, while the Traversability Risk Critic learns a risk-adjusted scalar utility for candidate-wise future selection. Experiments on R2R and RxR reveal substantial dependence of existing VLN agents on simulator contact semantics. Under strict-contact evaluation, VeSTA substantially reduces collision-induced failures while preserving or improving goal-reaching performance, and controlled real-world robot experiments provide evidence that the learned preference transfers beyond simulation.
Vision-language-action (VLA) benchmarks measure whether a policy completes a requested manipulation task, but binary success can hide safety violations along the trajectory: a policy may reach the goal while applying excessive contact, disturbing bystander objects, destabilizing a held object, or entering robot self-contact. We present SafeVLA-Bench, a post-hoc safety-evaluation framework for existing simulator-based VLA benchmarks that reveals violations missed by success-only evaluation. It encodes task-aware safety requirements as Signal Temporal Logic (STL) invariants with quantitative robustness semantics. Alongside native success, it reports the safety rate and the success-but-unsafe rate (SBU) used in prior safety evaluations, and introduces the Violation Severity Index (VSI), a bounded worst-violation depth score. We instantiate SafeVLA-Bench on LIBERO and RoboCasa-365, evaluating twenty-seven policy-benchmark entries across tabletop and kitchen manipulation tasks. High task success does not imply safe execution: the fifteen tabletop policies above 90% mean success still have 18-28% unsafe-episode rates, and 38-56% of successful RoboCasa-365 rollouts violate at least one active safety clause. A post-training case study further shows that SafeVLA-Bench can be used to improve policy safety. Project page: https://safevla.org
Jialiang Fan, Weizhe Xu, Zijun Wang +3
University of Notre Dame · University of Pennsylvania
Safe reinforcement learning seeks policies that maximise task performance while satisfying safety constraints. In driving benchmarks, however, collision costs typically appear only at the time of collision, providing no advance warning of an approaching hazard. Frozen vision--language models can provide dense semantic feedback, yet it remains unclear whether their scores anticipate collisions and which component drives an observed safety improvement. Episodic cost can also favour policies that make little task progress. To address these gaps, we propose VLM-Safe-RL, a framework that integrates frozen CLIP signals into PPO-Lagrangian through reward shaping and an augmented multiplier update. On MetaDrive Hard, which combines the densest traffic with the largest map, the catastrophe rate falls from 31.6% to 19.4%. FormulaOne-L2 analysis finds no evidence that the CLIP signals anticipate collisions and shows that the VLM term has a negligible effect on the Lagrange multiplier. These findings show a conditional reduction in observed catastrophe rate without evidence of collision anticipation.