PathLift: Depth-Conditioned Metric Lifting for Visual-Navigation Priors
Organizations: Interdisciplinary Program in Artificial Intelligence, Seoul National University · Department of Mechanical Engineering, Seoul National University
Abstract
Generalist visual-navigation policies tell a robot which direction to go but not how far. A separate lifting step is needed at deployment to convert their unitless trajectory into a metric path that a controller can track; existing scalar lifters apply uniform scaling that misses per-waypoint corrections, while in-policy approaches that inject monocular depth into the policy inherit the prediction errors of the depth model near obstacles and at close range. We present PathLift, a per-waypoint depth-conditioned adapter that lifts a pre-trained visual-navigation policy by consuming an explicit, sensor-grade depth source (RGB-D, stereo, or projected LiDAR) and fitting per-waypoint magnitude and lateral residuals to a canonicalized prior. PathLift consistently improves closed-loop navigation across diverse visual-navigation policies in simulation and on a real wheeled robot under both stereo and LiDAR depth. By turning the policy's unitless intent into a depth-feasible metric path in a single feed-forward pass, the adapter lifts real-world success without test-time optimization.
Figures & tables
| Base prior | Adapter | SR % ( ) | SPL % ( ) | TR % ( ) | Adapter (ms) | Total (ms) |
| GNM | – | 17.40 | 17.14 | 39.03 | — | 2.6 |
| GNM | PathLift | 25.60 ( ) | 24.54 ( ) | 49.39 ( ) | 5.3 | 8.2 |
| ViNT | – | 25.00 | 24.57 | 48.34 | — | 5.9 |
| ViNT | PathLift | 33.70 ( ) | 32.61 ( ) | 55.96 ( ) | 5.2 | 11.1 |
| NoMaD | – | 17.45 | 17.25 | 44.00 | — | 24.9 |
| NoMaD | MetricNet | 28.95 ( ) | 28.05 ( ) | 53.42 ( ) | 10.7 | 35.6 |
| E1 | E2 | E3 | Total | ||||
| Method | SR | #Coll | SR | #Coll | SR | #Coll | SR |
| GNM | |||||||
| GNM PathLift (SD) | |||||||
| ViNT | |||||||
| ViNT PathLift (SD) | |||||||
| NoMaD | |||||||
| Method | mean dist. (m) | max dist. (m) | mean dur. (s) | max dur. (s) |
| NoMaD | ||||
| NoMaD PathLift |
| Variant | SR (%) |
| PathLift | |
| depth input | |
| Plücker tokenizer (CNN encoder) | |
| Radial-only (per-waypoint) | |
| supervision (direct regression) | |
| Bare NoMaD (fixed-scale lift) |
| Method | SR (%) | SPL (%) | Stage (ms) |
| PathLift | |||
| DWA | |||
| MPPI | |||
| PathLift DWA | |||
| PathLift MPPI |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Sub-family | Shape parameter | Count | |
| identity | — | ||
| radial | — | ||
| smooth radial | or quadratic basis with | ||
| progress | truncation or gamma | ||
| rotation | |||
| rotation smooth radial | , |
| Sub-family | Habitat pin. | Habitat fish. | CODa | OpenLORIS |
| T 0 identity | 48.5 | 55.6 | 95.4 | 60.5 |
| T 1 radial | 44.0 | 49.4 | 94.0 | 52.4 |
| T 2 smooth radial | 47.6 | 53.0 | 94.7 | 55.6 |
| T 3 progress | 43.0 | 47.4 | 88.7 | 50.9 |
| T 4 rotation | 40.6 | 44.9 | 81.9 | 46.1 |
| T 5 rotation radial | 45.1 | 49.9 | 84.9 | 50.1 |
| change rate (%) | (m) | ||||||||
| Source | |||||||||
| Habitat pinhole | 48.5 | 31.0 | 10.9 | 6.9 | 16.9 | 18.6 | 0.616 | 0.573 | 0.553 |
| Habitat fisheye | 43.1 | 29.0 | 13.9 | 6.6 | 15.1 | 17.2 | 0.644 | 0.611 | 0.595 |
| CODa | 62.4 | 29.4 | 17.1 | 36.0 | 41.9 | 56.4 | 2.947 | 2.776 | 2.593 |
| Term | Weight | Role |
| Squared error to supervision target | ||
| Clearance hinge against | ||
| Magnitude equivariance regularizer | ||
| Tangent alignment to | ||
| Auxiliary log-magnitude regression | ||
| Second-difference smoothness |
| Component | Specification |
| Base | indoor wheeled differential-drive mobile robot |
| Forward RGB camera | stereo pair, left lens HFOV , per channel |
| LiDAR | MID-360; top-mounted, downward-facing scan plane |
| Stereo depth model | Fast-FoundationStereo [ 33 ] |
| Onboard compute | NVIDIA Jetson AGX Orin (64 GB) |
| Controller | pure-pursuit tracker at Hz |
| Variant | Enc. params | SR (%) |
| PathLift (Plücker, per-wp) | k | |
| Depth ablation | ||
| depth input | — | |
| Encoder ablation (input tokenization) | ||
| Plücker tokenization (CNN encoder) | k | |
| Output-head ablation | ||
| Invalid (%) | |||||||||
| ADE (m) | |||||||||
| tube (m) | |||||||||
| (%) |
Robot Navigation
Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.
See all 557 papers