TRACE: Target-State Risk-Aligned Conditional Execution for Embodied Tracking
Organizations: Institute of Automation, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · Zhongguancun Academy · University of Science and Technology Beijing ·
Abstract
Embodied visual tracking requires a robot to follow a language-specified person under occlusion, look-alike distractors, and ambiguous instructions. We show that improving tracking persistence can simultaneously increase safety violations. Using a compact 0.6B vision-language-action (VLA) tracker with a frozen backbone, we update only the planner head with proximal policy optimization (PPO). PPO improves success and following rates across all three EVT-Bench settings, but increases the collision rate on distracted tracking because episodes that previously ended in target loss can instead end in contact. We introduce TRACE, an execution-time target-state risk gate that preserves the learned policy and intervenes selectively. A lightweight risk head processes 84 cues, including candidate boxes, identity margins, a learned range-and-bearing probe, and the policy's commands. When the calibrated risk exceeds a threshold, TRACE rescales the translational component of the next waypoint while leaving yaw unchanged. The executed action is identical to PPO's on more than 98% of steps. In paired evaluations on identical episodes, TRACE reduces the distracted-tracking collision rate by 3.96 percentage points, while further improving success and following rates. Matched slowdown controls without target-state timing do not reproduce this improvement. Concurrent evaluation reproduces a collision reduction of 3.77 points, compared with 0.29 points for identical-arm null pairs, indicating that the effect exceeds evaluation variability. These results show that target-state-aware execution can recover the safety cost of policy post-training with minimal intervention.
Figures & tables
| Method | STT | DT | AT |
|---|---|---|---|
| SR / FR / CR | SR / FR / CR | SR / FR / CR | |
| (a) Reported by the original authors under their own protocols; not compared with (b). | |||
| IBVS ( Gupta et al., 2017 ) | 42.9 / 56.2 / 3.75 | 10.6 / 28.4 / 6.14 | 15.2 / 39.5 / 4.90 |
| EVT ( Zhong et al., 2024 ) | 32.5 / 49.9 / 40.5 | 15.7 / 35.7 / 53.3 | 18.3 / 21.0 / 44.9 |
| Uni-NaVid (Vicuna-7B) ( Zhang et al., 2024a ) | 25.7 / 39.5 / 41.9 | 11.3 / 27.4 / 43.5 | 8.26 / 28.6 / 43.7 |
| TrackVLA (Vicuna-7B) ( Wang et al., 2025 ) | 85.1 / 78.6 / 1.65 | 57.6 / 63.2 / 5.80 | 50.2 / 63.7 / 17.1 |
| Setting | Intervention | SR [95% CI] | FR [95% CI] | CR [95% CI] | (coll.) |
|---|---|---|---|---|---|
| DT | risk gate | ||||
| AT | AT-calibrated gate, orig. eval. | ||||
| AT | concurrent re-eval. | ||||
| STT | model selection, full set | ||||
| STT | independent subset only |
| Base PPO ending | normal | lost | collision | base total |
|---|---|---|---|---|
| normal | 642 | 74 | 69 | 785 |
| lost | 109 | 307 | 30 | 446 |
| collision | 33 | 15 | 101 | 149 |
| PPO total | 784 | 396 | 200 | 1380 |
| DT, five runs | SR | FR | CR | Intervention rate |
|---|---|---|---|---|
| Planner-head PPO | – | |||
| Average attenuation | ||||
| Random-trigger attenuation | ||||
| Lagged-risk attenuation (37 steps) | ||||
| TRACE risk gate |
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
| Module | Hyperparameter | Value |
| PPO planner-head | Learning rate (planner / value) | |
| Weight decay | ||
| Clip | ||
| Discount | ||
| GAE | ||
| Value-loss coefficient |
| AT, five runs | SR | FR | CR | Intervention rate |
|---|---|---|---|---|
| Planner-head PPO | – | |||
| Average attenuation | ||||
| Random-trigger attenuation | – | |||
| Lagged-risk attenuation (37 steps) | – | |||
| TRACE calibrated gate | – |
| Run | PPO | Average | Random | Lagged | Calibrated gate | Gate rate |
|---|---|---|---|---|---|---|
| 42 | ||||||
| 43 | ||||||
| 44 | ||||||
| 45 | ||||||
| 46 |
| Run | PPO, same code | Learned-detector gate | SR / FR / CR | Trigger (steps / ep.) |
|---|---|---|---|---|
| 42 | / / | / | ||
| 43 | / / | / | ||
| 44 | / / | / | ||
| 45 | / / | / | ||
| 46 | / / | / | ||
| pooled | / / | / |
| Pair (concurrent arms) | Run | Arm A | Arm B | SR / FR / CR |
| gate-off vs learned gate | 42 | / / | ||
| gate-off vs learned gate | 43 | / / | ||
| gate-off vs learned gate | 44 | / / | ||
| gate-off vs learned gate | 45 | / / | ||
| gate-off vs learned gate | 46 | / / | ||
| pooled, five runs | / / |
| Run | Gate off | Risk gate ( ) | SR / FR / CR |
|---|---|---|---|
| 42 | / / | ||
| 43 | / / | ||
| 44 | / / | ||
| 45 | / / | ||
| 46 | / / | ||
| pooled | / / |
| Backbone | SFT corpus | DT SR / FR / CR | STT SR / FR / CR |
|---|---|---|---|
| Qwen3-0.6B (main model) | original | ||
| Qwen2.5-0.5B | distilled, k | – | |
| Qwen3-0.6B (control) | distilled, k | – | |
| Qwen2.5-1.5B | distilled, k | – | |
| Qwen2.5-0.5B + PPO | distilled, k | ||
| Qwen2.5-0.5B | expert rollouts, k |
| Label window | Positive rate (count) | ROC-AUC | AP | ECE (raw) | Brier (raw) |
|---|---|---|---|---|---|
| ( ) | |||||
| ( ) | |||||
| ( ) |
| Set | Source | Episodes | Rows / steps | Used for |
|---|---|---|---|---|
| head training | 5 DT runs, of episode ids by hash | rows | fitting | |
| head validation | same runs, remaining | rows | , | |
| deployed evaluation | 5 PPO runs all episodes | steps | Tables 1 – 2 |
| AT operating point | SR | FR | CR | Role |
|---|---|---|---|---|
| Planner-head PPO | Ref. | |||
| TRACE calibrated gate | Main | |||
| TRACE margin gate | FR-oriented |
| 0.80 | 0.84 | 0.85 | 0.86 | 0.868 | 0.88 | 0.89 | 0.90 | 0.92 | |
|---|---|---|---|---|---|---|---|---|---|
| step-trigger rate (%) | 4.37 | 2.53 | 2.18 | 1.81 | 1.59 | 1.27 | 1.02 | 0.81 | 0.48 |
| Robot–human proximity | Wall clearance (five runs) | ||||||||
| Condition | med. | p10 | TTH med. | TTH p10 | p10 | p25 | p50 | p75 | p90 |
| (m) | (m) | (s) | (s) | (m) | (m) | (m) | (m) | (m) | |
| Planner-head PPO | 1.520 | 1.014 | 8.558 | 3.268 | 0.027 | 0.093 | 0.221 | 0.450 | 0.715 |
| TRACE (non-triggered) | 1.537 | 1.038 | 8.639 | 3.320 | 0.024 | 0.086 | 0.212 | 0.437 | 0.701 |
| TRACE (triggered) | 0.861 | 0.622 | 11.716 | 1.278 | 0.014 | 0.043 | 0.142 | 0.254 | 0.355 |
| Metric (DT seed 42, 600-episode sample) | PPO | Gate (non-trig.) | Gate (trig.) |
|---|---|---|---|
| steps | 56,292 | 58,472 | 871 (1.47%) |
| median (m) | 1.520 | 1.537 | 0.861 |
| p10 (m) | 1.014 | 1.038 | 0.622 |
| TTH median (s) | 8.558 | 8.639 | 11.716 |
| TTH p10 (s) | 3.268 | 3.320 | 1.278 |
| step trigger rate (%) | SR | FR | CR | |
|---|---|---|---|---|
| 0.83 | 2.95 | 44.44 | 55.84 | 12.52 |
| 0.85 | 2.18 | 44.12 | 56.24 | 12.97 |
| 0.868 (calibrated) | 1.59 | 46.02 | 57.58 | 9.83 |
| 0.88 | 1.27 | 44.41 | 55.57 | 12.09 |
| 0.90 | 0.81 | 44.16 | 55.87 | 12.45 |
| percentile (m) | PPO | Gate (all) | Gate (trig.) | Gate (non-trig.) |
|---|---|---|---|---|
| steps | 136,012 | 141,798 | 443 | 141,355 |
| p10 | 0.027 | 0.024 | 0.014 | 0.024 |
| p25 | 0.093 | 0.086 | 0.043 | 0.086 |
| p50 | 0.221 | 0.212 | 0.142 | 0.212 |
| p75 | 0.450 | 0.436 | 0.254 | 0.437 |
| p90 | 0.715 | 0.700 | 0.355 | 0.701 |
| Alternative | Result and takeaway |
|---|---|
| REINFORCE (critic-free policy gradient) | Development phase (training-rollout statistics). Collapses within the -episode budget: the second round’s on-policy episodes end at SR and CR ; not evaluated with the corrected evaluator. Takeaway: a learned critic is necessary in this long-horizon setting. |
| PPO budget | Corrected evaluator, full DT, run 42 (re-evaluated). against for planner-head PPO under the same code ( SR, CR): more persistence, and more contact. Takeaway: additional RL raises success and collisions together. |
| Collision penalty | Corrected evaluator, full DT, run 42 (re-evaluated). with a lower learning rate and a -episode budget: against for PPO under the same code, i.e. CR points at equal SR. Takeaway: scalar collision retuning produces aggressive following, not safe control. |
| Imitation / preference (RWR, RAFT, DPO, GRPO, FTSFT) | Development phase (pre-correction evaluator). No stable gain, or regression, under matched conditions. Takeaway: imitation-style objectives distill rollout errors when the reference is the deployed policy. |
| Distance-only learned filter | Corrected evaluator, full DT, run 42. against PPO . Takeaway: distance is necessary but not sufficient as a risk feature. |
| Heuristic / geometry filtering | Corrected evaluator, -episode subset. Navmesh-radius sweeps fail the CR-drop / SR-retention criterion. Takeaway: safety needs richer target-state conditioning. |
| Control | Run | SR | FR | CR | SR vs PPO | FR vs PPO | CR vs PPO | Step rate |
|---|---|---|---|---|---|---|---|---|
| average | 42 | 42.75 | 55.01 | 13.84 | -1.82 | -1.40 | +1.09 | 100.00% |
| average | 43 | 43.26 | 55.10 | 15.29 | +0.00 | +0.49 | +0.07 | 100.00% |
| average | 44 | 43.55 | 55.83 | 13.77 | -0.73 | +0.19 | -1.45 | 100.00% |
| average | 45 | 44.78 | 55.53 | 11.96 | +0.72 | +0.28 | -0.79 | 100.00% |
| average | 46 | 42.61 | 54.21 | 13.48 | -1.16 | -1.07 | +0.51 | 100.00% |
| random | 42 | 42.83 | 55.04 | 14.06 | -1.74 | -1.37 | +1.31 | 1.36% |
| Run | -rate : SR / FR / CR | -rate : SR / FR / CR |
|---|---|---|
| 42 | / / | / / |
| 43 | / / | / / |
| 44 | / / | / / |
| 45 | / / | / / |
| 46 | / / | / / |
| mean std | / / | / / |
| Attempt | Scope | SR / FR / CR | Decision |
|---|---|---|---|
| Planner-head PPO | 5 runs | Main STT reference. | |
| Micro-gate on planner | 5 runs | Essentially neutral; no tracking gain. | |
| Gated planner | 5 runs | SR and FR improve, but CR worsens. | |
| Collision-anchored continuation | 5 runs | Strongest tracking among single continuations; CR remains above PPO. | |
| Gate-repair sweep | selection subset | Best selection-subset row among four gate settings, all of which trigger on of episodes and steps, so the shift comes from the planner. | |
| Weak-run gate repair | runs 44/45 | not aggregated | CR rises on both diagnostic runs while the gate still triggers on of episodes and steps. |
| Run | Two-level pick | Five-level pick | |||||
|---|---|---|---|---|---|---|---|
| 42 | |||||||
| 43 | |||||||
| 44 | |||||||
| 45 | |||||||
| 46 |
| DT ending diagnostic | Compact baseline | + Planner-head PPO (run 42) | Change |
|---|---|---|---|
| SR / FR / CR | 41.01 / 53.37 / 10.80 | 42.17 / 54.45 / 14.49 | +1.16 / +1.08 / +3.69 |
| Episode status: normal / lost / collision | 785 / 446 / 149 | 784 / 396 / 200 | / / +51 |
| too-far / no-recovery after following | 44.0% (358) | 36.7% (293) | failures |
| too-far / never recovered | 21.1% (172) | 21.2% (169) | failures |
| collision while following | 14.3% (116) | 20.4% (163) | failures |
| lost after following / switch / occlusion proxy | 10.6% (86) | 10.8% (86) | unchanged |
| Run | SR | FR | CR | Normal | Lost | Collision |
|---|---|---|---|---|---|---|
| 42 | ||||||
| 43 | ||||||
| 44 | ||||||
| 45 | ||||||
| 46 | ||||||
| Mean |
Visual Object Tracking
Visual object tracking follows objects across video frames, maintaining their locations and identities despite motion, occlusion, and changing appearance.
See all 182 papers