VP-EVT: Active Visibility Preservation for Embodied Visual Tracking
Authors: Yanci Wen, Tianle Zeng, Jianwei Peng, Weixi Situ, Hanjing Ye, Hong Zhang
Organizations: Shenzhen Key Laboratory of Robotics and Computer Vision, Southern University of Science and Technology, Shenzhen, China. · Southern University of Science and Technology, Shenzhen, China. · Tencent Robotics X, Shenzhen, China.
Embodied Visual Tracking (EVT) is an essential capability for intelligent robots in human-robot collaboration, requiring an agent to follow a designated target person in dynamic environments relying solely on egocentric visual perception. However, persistent pedestrian occlusions in dynamic scenes pose critical challenges to existing EVT methods. Lacking proactive visibility preservation, reactive tracking methods frequently experience prolonged target loss under severe occlusions. Therefore, we propose VP-EVT, a framework that treats target visibility as an explicit objective of motion control rather than a by-product of following. Acting on this objective requires understanding both what obstructs the target and how robot motion can change that obstruction. VP-EVT therefore uses language-guided perception to extract separate target and pedestrian observations. When occlusion makes current observations incomplete, VP-EVT combines visual history with ego-motion to maintain estimates of their relative positions and motion. Using these estimates and the current following waypoints, VP-EVT predicts whether continued following will sustain the obstruction. When the predicted risk is low, the original following motion is retained; otherwise, a recurrent controller combines updated estimates with maneuver context from preceding decisions to coordinate viewpoint adjustment and the return to stable following. Moreover, we introduce OccEVT-Bench for closed-loop evaluation of persistent occlusion and collect expert demonstrations for training. On its Risk split, VP-EVT improves success rate by 121.4% relative to a competitive baseline and reduces visibility deficit and longest target-loss duration by 78.8% and 74.4%, respectively.
Figures & tables
Fig. 1: Standard following and VP-EVT under dynamic pedestrian occlusion. Continuing behind an occluding pedestrian can prolong target loss, whereas visibility-preserving motion changes the robot’s viewpoint to reduce the obstruction while continuing to follow the target. The upper and lower panels provide perspective and bird’s-eye illustrations, respectively.
Fig. 2: Overview of VP-EVT. (a) A causal RGB history and a language prompt produce Current Entity Observations : ztT for the designated target ( T ) and ztP for a potential occluding pedestrian ( P ). (b) Visual observations and robot ego-motion ΔEt update a persistent target belief BtT and a short-term pedestrian belief BtP . Uncertainty increases when visual evidence becomes unavailable. (c) The target and pedestrian beliefs, together with following waypoints Wt , condition persistent-occlusion risk prediction and recurrent visibility-preserving control. The curves schematically illustrate visibility degradation, while the motion example contrasts continued following with a visibility-preserving maneuver. The controller produces linear and angular velocity commands [v,ω] , and the resulting ego-motion ΔEt+1 informs the next belief update.
Fig. 3: Overview of OccEVT-Bench. Episodes span indoor and outdoor scenes, diverse target appearances and motions, and dynamic pedestrian interactions. Risk episodes assess persistent-occlusion handling, while complementary Safe episodes assess continued following without unnecessary intervention. Easy, Medium, and Hard levels cover different interaction difficulties.
Benchmark
RGB-CL
Occ.
LTL
Attr.
Safety
Scene
HOOT [ 27 ]
✗
✓
✗
✗
✗
Video
TCOW [ 28 ]
✗
✓
✗
✗
✗
Video
TPT-Bench [ 29 ]
✗
✓
✗
✗
✗
Video
EVT-Bench [ 4 ]
✓
✗
✗
✗
✓
Mesh
Follow-Bench [ 31 ]
✗
✓
✗
✗
✓
2D
OccEVT-Bench
✓
✓
✓
✓
✓
3DGS
TABLE I: Comparison of tracking benchmarks.
Risk episodes
Safe episodes
Method
SR ↑
VD ↓
LTL ↓
POL ↓
OOF ↓
CFR ↑
SR ↑
VD ↓
LTL ↓
OOF ↓
CFR ↑
Human Teleoperation
0.925
3.324
1.213
1.221
0.988
0.950
0.980
0.132
0.113
0.116
0.980
Adap-RPF † [ 23 ]
0.270
20.879
19.109
1.991
18.815
0.920
0.865
1.336
1.289
1.336
0.920
Mono-RPF [ 6 ]
0.005
24.646
24.704
24.334
2.419
0.985
0.950
2.134
1.128
1.136
1.000
OA-VAT [ 10 ]
0.350
19.986
13.432
3.755
16.041
0.690
0.730
0.542
0.521
0.521
1.000
MiniCPM-RobotTrack
0.020
21.735
17.388
17.536
8.337
0.880
0.870
1.980
1.653
1.920
0.960
TABLE II: Closed-loop results on OccEVT-Bench. SR and CFR are fractions; duration metrics are in seconds.
Variant
Hist.
Beliefs
Prop.
Plan
Mem.
SR ↑
VD ↓
LTL ↓
POL ↓
OOF ↓
CFR ↑
Single-frame observation
✗
✓
✗
✓
✓
0.435
8.901
6.611
3.176
5.647
0.630
w/o target–pedestrian beliefs
✓
✗
–
✓
✓
0.110
20.184
18.762
12.974
8.431
0.685
w/o occlusion-time belief propagation
✓
✓
✗
✓
✓
0.475
5.200
3.988
2.725
2.421
0.545
w/o following-waypoint conditioning
✓
✓
✓
✗
✓
0.425
4.547
4.198
3.567
1.929
0.485
w/o recurrent maneuver state
✓
✓
✓
✓
✗
0.695
5.046
4.044
2.999
1.985
0.755
w/o predictive risk (reactive triggering)
✓
✓
✓
✓
✓
0.685
9.675
6.246
2.975
3.747
0.725
TABLE III: Ablation study on 200 Risk episodes. All variants predict and execute four controls per decision. SR and CFR are fractions; duration metrics are in seconds.
Fig. 4: Temporal visibility comparison. We compare target visibility and cumulative visibility deficit across methods on selected Risk episodes. The dashed line marks the target-loss threshold τloss=0.35 . Higher visibility and lower cumulative deficit indicate better visibility preservation.
Fig. 5: Real-world deployment of VP-EVT. The robot adjusts its viewpoint around the pedestrian in the patterned shirt to reduce occlusion and resume following the target in black. Gray dotted paths indicate the following reference, while green dotted paths illustrate visibility-preserving motion. Frames are ordered chronologically from left to right, top to bottom.