Pre-trained visual world models generate multi-step action plans from current and goal observations, yet terminal-proximity objectives fail to prevent forbidden-zone traversal or rigid-body collision sliding. Purely visual interfaces may lack analytical dynamics, structured states, or explicit collision geometry, requiring internal prediction of action-conditioned process risk. We propose Risk-Triggered Search (RTS) for frozen Joint-Embedding Predictive Architecture (JEPA) world models. Rather than relying on static state features that cannot distinguish candidate plans sharing an observation, RTS unrolls action-conditioned future latent prefixes in the frozen world model and trains a lightweight risk head with process-violation supervision. To intervene selectively while preserving safe nominal behavior, RTS uses the same predicted prefix-risk criterion as both the search trigger gate and hard feasibility gate: safe nominal plans execute directly, whereas alarms trigger local Cross-Entropy Method (CEM) search to minimize nominal deviation within the feasible set. To prevent isotropic Euclidean minimal modification from shrinking obstacle-directed actions into zero-amplitude stalling, we introduce an amplitude-preserving metric dAP that retains tangential progression while maintaining a unique zero at the nominal plan. Under unified gating and this unique-zero condition, bypassing safe-state search yields closed-loop trajectories strictly identical to every-step same-kernel search. Evaluated under a process-safety benchmark and Safe Arrival Rate (SAR) metric across five tasks, RTS improves SAR over Direct by 12.67--41.67 percentage points on navigation and articulated control and +2.08/+4.17 pp on Cube/PushT boundary cohorts, maintaining 100% action identity with every-step search while cutting controller time by 17.71%--70.41% across 1,968 paired trajectories.
Figure 2: Overall RTS architecture. Frozen policy π generates nominal plan UD , and world model F unrolls predicted latent prefixes z^0:H−1 scored by risk head Rϕ . Safe plans ( bt=0 ) pass directly; alarms ( bt=1 ) launch 3-round CEM search using the same risk criterion to define feasibility Ft , where amplitude-preserving selection minimizes dAP .
Table 1: Closed-loop evaluation summary across five benchmarks. Panel A: navigation and articulated control; Panel B: full-test contact manipulation (Cube boundary confirmation in Appendix A , Table A2 ). Task-adapted RTS uses RTS-AP in Panel A and RTS-Repair in Panel B. ΔSAR reports percentage-point gains over Direct and Recovery-MPC. Column headers report total evaluated trajectories n for 3-seed stochastic methods (deterministic Direct runs once per task; see Table A9 ).
Risk input
TwoRoom
Reacher
State
0.500
0.500
State + action
0.547 [0.476,0.622]
0.527 [0.421,0.630]
Predicted rollout
0.791 [0.733,0.844]
0.791 [0.700,0.874]
Rollout − action
+0.243 [0.145,0.337]
+0.263 [0.119,0.405]
Table 2: Risk representation and spatial addressing ablations. (A) Within-context AUROC [95% CI] for equal-capacity heads (327,425 parameters; three training seeds; 128/64/128 fitting/selection/audit states per task, 17 plans/state). (B) PointMaze spatial addressing on 64 development tasks under 9,600 matched training steps (implementation and audit details in Appendix C.1 ).
Figure 3: Five-task risk discrimination and controller overhead. (a) Within-context candidate risk ranking AUROC. (b) Candidate hazard recall and safe false-alarm rate. (c) Episode controller time ratios ( <1.0 : savings). Timing cohorts and execution settings are specified in Appendix A.3 .
TwoRoom
Reacher
PointMaze
G
F
SAR
VR
Zero
SAR
VR
Zero
SAR
VR
Zero
MSE
MSE
83.67
3.83
0.000
47.33
30.33
0.482
72.14
7.55
2.133
MSE
AP
83.67
4.17
0.000
47.17
31.83
0.188
75.78
9.11
0.734
AP
MSE
84.33
4.17
0.000
48.17
27.83
0.445
73.96
5.73
2.125
AP
AP
84.33
4.50
0.000
48.83
28.17
0.185
76.82
6.51
0.719
Table 3: Matched 2×2 ablation of candidate generation G and final selection F (protocol in Appendix C.2 ). Execution intervals are fixed at 25/25/5 steps for TwoRoom/Reacher/PointMaze; 6,336 trajectories in total. Zero denotes non-fallback zero-action decisions per episode.
TwoRoom
Reacher
PointMaze
Scheduler
SAR
VR
Time
SAR
VR
Time
SAR
VR
Time
Periodic
83.17
12.33
179.64
32.00
56.50
665.20
77.86
21.35
1313.20
Random
77.33
18.17
127.35
25.00
65.83
646.15
72.66
26.30
1321.16
Risk
83.17
11.33
131.33
49.00
39.33
674.61
83.85
15.89
1458.65
Table 4: Search scheduling under a shared 50% search-call cap and the same AP kernel (Appendix H.3 ). Time is controller ms/episode. Risk-based scheduling improves SAR on Reacher and PointMaze; on TwoRoom it matches periodic scheduling with lower controller time.
Table A2 : Independent confirmation and same-kernel efficiency verification on Cube with frozen configurations. Ordinary and boundary cohorts each contain 32 held-out tasks. Panel A reports percentages (bold indicates best in column); Direct runs once per task (32 episodes) and search methods run 3 seeds (96 episodes). Panel B pairs RTS-Repair with every-step same-kernel Always-Repair. Boundary SAR gain over Direct is +2.08 pp (10,000-sample task-cluster bootstrap 95% CI [0.00,6.25]).
Task / Cohort
Paired Traj.
Time Ratio [95% CI]
Time ms (RTS / Always)
TwoRoom
600
0.636 [0.591,0.678]
196.98 / 197.38 (decision p95)
Reacher
600
0.651 [0.620,0.683]
200.47 / 201.27 (decision p95)
SimplePointMaze
384
0.823 [0.765,0.874]
196.20 / 172.11 (decision p95)
PushT (Test Subset)
192
0.296 [0.273,0.322]
169.22 / 571.99 (episode mean)
Cube Ordinary
96
0.390 [0.329,0.453]
237.46 / 346.60 (decision p95)
Cube Boundary
96
0.462 [0.379,0.551]
278.48 / 325.45 (decision p95)
Appendix
Table A3 : Same-kernel paired efficiency (RTS vs. Always-RTS). Ratio denotes mean episode controller time of RTS divided by Always-RTS; confidence intervals are clustered by task/map.
Task
Exec. Steps
RTS (ms)
Guarded A (ms)
Time Ratio
TwoRoom
15
135.02
154.97
0.871
Reacher
15
438.60
684.91
0.640
SimplePointMaze
10
804.22
1368.91
0.587
PushT
15
196.13
300.91
0.652
Cube
15
170.42
209.71
0.813
Appendix
Table A4 : Measured controller time against Guarded A in Figure 3 (c). Both methods use the identical execution interval per task (ms/episode). Protocols are detailed in Section A.3 .
Figure A1 : Contact manipulation extension (RTS-Repair) cohort evaluation and representative case studies on Cube and PushT. (a–b) Safe Arrival Rate (SAR ↑ ) and Process Violation Rate (VR ↓ ) on development cohorts. (c) Cube 223 safe detour rescue trajectory. (d) Cube 220 tail-block false rejection stalling caused by a fixed 5-block audit horizon exceeding the 4 remaining steps to goal completion. (e) PushT safe nominal trajectory intercepted by a false-positive alarm.
Gating Configuration
SR ↑
SAR ↑
VR ↓
Safe Timeout
Direct-15
89.06
83.59
10.16
6.25
Baseline-15
88.28
82.81
10.68
6.51
Calib-15
85.42
81.25
8.85
9.90
Decoupled-00
85.68
81.25
9.11
9.64
Decoupled-10
85.68
81.25
9.11
9.64
Decoupled-01
85.16
80.99
8.85
10.16
Appendix
Table A5 : Closed-loop gating intervention results on the PushT confirmation cohort (%). Safe Timeout denotes episodes without violations that fail to reach the goal within Tmax .
Pre-Specified Comparison
Δ SAR (pp)
99% CI (Bonferroni)
Calib-15 − Baseline-15
−1.56
[ −5.21,0.52 ]
Decoupled-00 − Calib-15
0.00
[0.00,0.00]
Decoupled-11 − Decoupled-00
−0.26
[ −2.08,1.04 ]
ShortExec-5 − Baseline-15
−3.91
[ −8.59,0.00 ]
Aligned-15 − Baseline-15
0.00
[0.00,0.00]
Appendix
Table A6 : Pre-specified paired SAR differences on the PushT confirmation cohort (percentage points, Bonferroni-corrected 99% CI).
Task
Audited Process Quantities and Scope
Safety Upper Bound
Tolerance ε
TwoRoom
Max single-step commanded disk sweep ct & state overlap ot+1 / Cumulative ∑tΔt / Max consecutive steps
0.5+ε px / 1.0+ε px / 2 steps
10−3 px
Reacher
Max penetration depth of link capsules and fingertip into forbidden zone across all native physics substeps
ε
10−6 m
SimplePointMaze
Max commanded disk sweep penetration depth (including maze walls) across all primitive control steps
ε
10−6 env. units
PushT
Max forbidden-zone penetration of T-block/pusher and workspace escape depth across all physics substeps
ε
10−3 px
Cube
Max forbidden-zone penetration of object OBB/grasp proxy sphere and workspace escape depth across substeps
ε
10−6 m
Appendix
Table A7 : Process safety criteria and numerical thresholds for SAR. ε is the geometric numerical tolerance, independent of the learned threshold τ .
Safe Control Paradigm
FZ
Pure-Vis
Trigger Mechanism
Selection Criterion
Pass
Eqv.
Anti-Stall
Predictive Safety Filter [ 34 , 36 ]
✓
✗
State / violation event
Euclidean projection ℓ2
✓
✗
✗
Recovery RL [ 32 ]
✗
✓
Qrisk threshold
Standalone risk recovery policy
✓
✗
✗
DreamAvoid [ 10 ]
✓
✓
Critical-phase classifier
Value-model reranking
✗
✗
—
JEPA Shield [ 38 ]
✓
✗
Every-step shield audit
Analytical shield filtering
✗
✗
✗
RTS (Ours)
✓
✓
Latent prefix risk
Shared hard gate + dAP
✓
✓
✓
Appendix
Table A8 : Six-dimensional mechanism comparison between RTS and prior safe control paradigms. FZ: frozen backbone; Pure-Vis: purely visual closed-loop (no external analytical shield); Pass: exact pass-through of safe nominal plans; Eqv.: strict equivalence between bypassed search and every-step same-kernel search (Prop. 1); Anti-Stall: anti-orthogonal-shrinkage stalling.
Figure A2 : TwoRoom process-safety diagnostic: arrival under default sliding (Strict SR · slide), safe arrival under sliding (Process SAR · slide), and arrival when remaining displacement stops at first contact (Strict SR · stop).
Method / Evaluation Scale
TwoRoom
Reacher
PointMaze
Cube
PushT
Direct / Guarded A / RTS
15
15
10
25
15
RTS-MSE
25
25
5
25
15
Recovery-MPC / LeWM / Fast-LeWM
25
25
5
25
25
BC / RiskFT
25
25
5
25
15
Independent Tasks
200
200
128
997
1,538
3-Seed Method Trajectories ( n )
600
600
384
2,991
4,614
Appendix
Table A9 : Main-table receding-horizon execution intervals and independent evaluation scales. Cube uses the 25-step configuration.
Task / Risk Tag
Nominal TP / Haz.
Nominal Recall (%)
Nominal FP / Safe
Nominal FPR (%)
Candidate TP / Haz.
TwoRoom
44 / 46
95.65
5 / 82
6.10
727 / 795
Reacher (continuous 25-step)
18 / 18
100.00
3 / 110
2.73
313 / 330
SimplePointMaze
74 / 78
94.87
1 / 50
2.00
5,771 / 6,013
Cube (signed-clearance)
2 / 2
100.00
5 / 43
11.63
78 / 99
PushT (Short-5, q95)
4 / 4
100.00
8 / 242
3.31
1,401 / 1,453
Appendix
Table A10 : Nominal and candidate hazard detection at the Figure 3 (b) operating points. Recall denotes TP / Hazardous; False-Alarm / False-Rejection Rate (FPR) denotes FP / Safe.
Task
Nmix
Within-Context AUROC
Cand. TP / Haz.
Recall (%)
Cand. FP / Safe
FPR (%)
TwoRoom
56
0.901
727 / 795
91.45
83 / 1,381
6.01
Reacher
19
0.962
313 / 330
94.85
46 / 1,846
2.49
SimplePointMaze
128
0.982
5,771 / 6,013
95.98
257 / 2,307
11.14
PushT (q95)
21
0.985
1,401 / 1,453
96.42
2,600 / 93,257
2.79
Cube
17
0.926
78 / 99
78.79
125 / 1,324
9.44
Appendix
Table A11 : Five-task candidate risk ranking AUROC and operating point statistics. Nmix denotes audit contexts containing both safe and hazardous candidates.
Figure A3 : Six-point receding-horizon feedback scan under fixed risk audit windows. The horizontal axis denotes execution/observation interval k∈{1,2,5,10,15,25} ; columns show strict Safe Arrival Rate (SAR), mean episode controller time, and single-decision latency (PushT evaluated on 128 validation tasks).
Task
k=1
k=2
k=5
k=10
k=15
k=25
TwoRoom
70.50
69.33
71.33
86.83
87.83
83.67
Reacher
38.33
43.67
39.33
46.83
48.67
47.17
SimplePointMaze
71.61
74.74
75.78
77.34
72.40
69.79
Cube
94.92
97.02
97.69
97.63
97.12
97.89
PushT (val)
79.69
81.25
81.25
82.03
82.03
78.91
Appendix
Table A12 : Closed-loop SAR (%) of RTS across receding-horizon execution intervals k under fixed risk windows. Cube in Table 1 uses k=25 .
Figure A4 : Six-point receding-horizon feedback scan with risk audit coverage scaled to match the execution window (Horizon-Aligned Scan). Columns show SAR, mean episode controller time, and single-decision latency (PushT evaluated on 128 validation tasks).
Figure A5 : Full efficiency and constrained-budget (K-of-N) scheduling curves. Panel (a) shows same-kernel episode controller time; panels (b–e) compare risk-triggered, periodic, and uniform random scheduling under 25% (circle), 50% (square), and 100% (triangle) search call caps.
Task (Exec. Steps)
Direct SAR
BC SAR
RiskFT SAR
RiskFT − Direct [95% CI]
TwoRoom (25)
74.50
73.83
76.67
+2.17 [ −1.00,5.33 ]
Reacher (25)
9.50
7.17
10.33
+0.83 [ −3.67,5.50 ]
SimplePointMaze (5)
50.00
47.92
50.00
0.00 [ −5.99,5.99 ]
Cube (25)
97.49
97.96
99.00
+1.50 [0.60,2.41]
PushT (15)
75.62
71.43
71.33
−4.29 [ −5.94,−2.67 ]
Appendix
Table A13 : Matched-interval learning baseline comparisons (Direct runs once per task; BC and RiskFT run 3 training seeds; SAR in %).
Task
Model
zone SAR
clear SAR
clear SR
clear VR
SAR Diff [95% CI]
TwoRoom
LeWM / Fast
34.50 / 34.83
34.50 / 34.83
78.17 / 85.00
65.33 / 64.50
0.00 / 0.00
Reacher
LeWM
2.17
2.67
25.67
64.33
+0.50 [ −1.33,2.50 ]
Fast-LeWM
2.17
5.17
41.50
60.50
+3.00 [0.50,5.67]
PushT
LeWM
57.56
67.64
82.42
25.18
+10.08 [8.24,11.94]
Fast-LeWM
62.74
70.22
87.17
25.18
+7.48 [5.77,9.23]
Cube
LeWM
52.96
62.62
64.79
8.69
+9.66 [7.79,11.57]
Appendix
Table A14 : Paired comparison of external baselines with (zone) and without (clear) visual hazard overlays (SAR/SR/VR in %).
Setting
LeWM
Fast-LeWM
TwoRoom (25/50)
88 / 87
96 / 98
Reacher
76 / 86
86 / 88
TwoRoom (100/150)
10 / 87
38 / 98
PushT
96 / 96
98 / 96
Cube
70 / 74
78 / 80
Appendix
Table A15 : Native unconstrained success rates: measured / published reference (%) from [ 11 ] . TwoRoom settings specify the goal interval and step budget used in our evaluations; the same published reference rate is shown for both settings.
Vision-language-action policies generalize broadly but provide no execution-time guarantees; classical model-based planners respect kinematic and geometric constraints but generalize poorly. We study whether an action-conditioned Joint-Embedding Predictive Architecture (JEPA) world model can predict, before execution, both task progress and physical risk for candidate action chunks, and whether coupling these predictions to an embodiment-specific model-based safety shield yields a deployable pipeline for heterogeneous robots. We propose a receding-horizon decision pipeline: (1) a proposer produces K candidate action chunks; (2) an action-conditioned JEPA rolls each candidate forward in a frozen-encoder latent space conditioned on an embodiment embedding; (3) calibrated risk and progress heads score each rollout and report uncertainty; (4) a deterministic per-embodiment safety shield filters inadmissible candidates; (5) a fallback ladder handles empty-admissible-set cases. The learned ranking only reorders admissible candidates; enforcement guarantees come from the deterministic shield and fallback ladder. We evaluate with a pre-registered protocol in simulation (LIBERO-Long). In 600-episode configurations the full framework improved success over a shield-only baseline and reduced collision false negatives at matched recall. Deployment-efficiency measurements on target on-robot and edge accelerators are included. Real-robot experiments and an offline reranking significance test remain future work; see the paper for disclosures.
Kaiming Zhong, Tianhua Liu, Yue Wang
Guangdong Bifang Intelligent Control Technology Co., Ltd.
Embodied visual tracking requires a robot to follow a language-specified person under occlusion, look-alike distractors, and ambiguous instructions. We show that improving tracking persistence can simultaneously increase safety violations. Using a compact 0.6B vision-language-action (VLA) tracker with a frozen backbone, we update only the planner head with proximal policy optimization (PPO). PPO improves success and following rates across all three EVT-Bench settings, but increases the collision rate on distracted tracking because episodes that previously ended in target loss can instead end in contact. We introduce TRACE, an execution-time target-state risk gate that preserves the learned policy and intervenes selectively. A lightweight risk head processes 84 cues, including candidate boxes, identity margins, a learned range-and-bearing probe, and the policy's commands. When the calibrated risk exceeds a threshold, TRACE rescales the translational component of the next waypoint while leaving yaw unchanged. The executed action is identical to PPO's on more than 98% of steps. In paired evaluations on identical episodes, TRACE reduces the distracted-tracking collision rate by 3.96 percentage points, while further improving success and following rates. Matched slowdown controls without target-state timing do not reproduce this improvement. Concurrent evaluation reproduces a collision reduction of 3.77 points, compared with 0.29 points for identical-arm null pairs, indicating that the effect exceeds evaluation variability. These results show that target-state-aware execution can recover the safety cost of policy post-training with minimal intervention.
Mingyang Lyu, Gongce Cao, Yiyang Jia +7
Institute of Automation, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · Zhongguancun Academy +2
Recent vision-language-action (VLA) models are promising for general-purpose manipulation, but long-horizon execution remains fragile. Small state-estimation or control errors can lead to irreversible failures (e.g., collisions and object drops). Avoiding these risks requires a proactive safety mechanism capable of anticipating hazards. In this paper, we introduce SafeLoop, a non-invasive external wrapper that adds hazard prediction and rollback-based recovery to a VLA model without changing its parameters. SafeLoop trains a risk predictor from vision and proprioception to output four values: the probability and time-to-hazard for body collisions and for object failures. A lightweight controller then chooses one of three actions based on the predicted risk: continue execution (noop), save a safety checkpoint (record), or retreat in joint space (rollback). Rollback moves the robot back to a recent safe waypoint and queries the base policy again, which may yield an alternative continuation. Across 24 LIBERO tasks (16 random seeds each) and three real-robot tasks (25 rollouts each), SafeLoop achieves a stronger overall safety-success trade-off than alternative methods, reducing hazard cases by roughly 70% while preserving task success and the base-policy control rate. Project code is available at https://github.com/Loule0-0/SafeLoop/tree/release/safeloop.
Zeyu Lou, Tianran Zhang, Xinquan Yue +2
Nanjing University, Nanjing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangdong, China · Beijing University of Technology, Beijing, China