PathLift: Depth-Conditioned Metric Lifting for Visual-Navigation Priors

Gwanjun Shin; Hyeongseok Suh; Hongmo Jung; Jongwoo Lim

cs.ROOct 12, 2026

PathLift: Depth-Conditioned Metric Lifting for Visual-Navigation Priors

Authors: Gwanjun Shin, Hyeongseok Suh, Hongmo Jung, Jongwoo Lim

Organizations: Interdisciplinary Program in Artificial Intelligence, Seoul National University · Department of Mechanical Engineering, Seoul National University

Abstract

Generalist visual-navigation policies tell a robot which direction to go but not how far. A separate lifting step is needed at deployment to convert their unitless trajectory into a metric path that a controller can track; existing scalar lifters apply uniform scaling that misses per-waypoint corrections, while in-policy approaches that inject monocular depth into the policy inherit the prediction errors of the depth model near obstacles and at close range. We present PathLift, a per-waypoint depth-conditioned adapter that lifts a pre-trained visual-navigation policy by consuming an explicit, sensor-grade depth source (RGB-D, stereo, or projected LiDAR) and fitting per-waypoint magnitude and lateral residuals to a canonicalized prior. PathLift consistently improves closed-loop navigation across diverse visual-navigation policies in simulation and on a real wheeled robot under both stereo and LiDAR depth. By turning the policy's unitless intent into a depth-feasible metric path in a single feed-forward pass, the adapter lifts real-world success without test-time optimization.

Figures & tables

Figure 1: PathLift in action. The raw NoMaD output (red) is the unitless prior. Lifted by a const-scalar baseline (orange, $\times$ ), the trajectory drives the robot into structural obstacles; lifted by PathLift (blue, $\checkmark$ ), it routes around them. Each pair shows the fisheye RGB and the corresponding BEV with occupied cells (black) and depth points (brown).

Figure 2: PathLift inference pipeline. An off-the-shelf visual-navigation policy emits a unitless policy path; the canonicalized prior and a depth observation (RGB-D, stereo, or projected LiDAR) reprojected into the policy’s camera frame feed PathLiftNet, which outputs a metric trajectory in body coordinates.

Figure 3: Supervision target on Habitat (top) and OpenLORIS (bottom). Col. 1: camera RGB. Col. 2: all candidates in $\mathcal{C}_{\mathrm{lift}}(\hat{\mathbf{p}})$ from the NoMaD prior (faded). Col. 3: four lowest-ADE admissibles; supervision target in solid blue. The NoMaD $\times$ const scale (red) and occupied BEV cells (black) overlaid in both BEV columns.

Base prior	Adapter	SR % ( $\Delta$ )	SPL % ( $\Delta$ )	TR % ( $\Delta$ )	Adapter (ms)	Total (ms)
GNM	–	17.40	17.14	39.03	—	2.6
GNM	PathLift	25.60 ( $+47.1\,\%$ )	24.54 ( $+43.2\,\%$ )	49.39 ( $+26.5\,\%$ )	5.3	8.2
ViNT	–	25.00	24.57	48.34	—	5.9
ViNT	PathLift	33.70 ( $+34.8\,\%$ )	32.61 ( $+32.7\,\%$ )	55.96 ( $+15.8\,\%$ )	5.2	11.1
NoMaD	–	17.45	17.25	44.00	—	24.9
NoMaD	MetricNet	28.95 ( $+65.9\,\%$ )	28.05 ( $+62.6\,\%$ )	53.42 ( $+21.4\,\%$ )	10.7	35.6

Table 1: Closed-loop simulation benchmark on 50 HM3D scenes (2,000 trials per row).

Panel of Figure 4 — Figure 4: Real-robot experimental environments. (a) Image-goal navigation in E1/E2 ( $20$ m corridors) and E3 ( $10$ m corridor), shown left to right (Sec. 4.2.1 ). (b) Goal-free exploration in a $12\!\times\!10$ m indoor area without a pre-built map (Sec. 4.2.2 ).

	E1		E2		E3		Total
Method	SR $\uparrow$	#Coll $\downarrow$	SR $\uparrow$	#Coll $\downarrow$	SR $\uparrow$	#Coll $\downarrow$	SR $\uparrow$
GNM	$20$	$1.6$	$80$	$2.2$	$40$	$2.0$	$46.67$
GNM $+$ PathLift (SD)	$\mathbf{80}$	$\mathbf{1.0}$	$\mathbf{100}$	$\mathbf{1.0}$	$\mathbf{80}$	$\mathbf{1.0}$	$\mathbf{86.67}$
ViNT	$80$	$1.4$	$60$	$2.2$	$60$	$1.6$	$66.67$
ViNT $+$ PathLift (SD)	$\mathbf{100}$	$\mathbf{0.4}$	$\mathbf{100}$	$\mathbf{0.6}$	$\mathbf{80}$	$\mathbf{0.8}$	$\mathbf{93.33}$
NoMaD	$40$	$1.6$	$20$	$1.2$	$40$	$1.4$	$33.33$

Table 2: Per-policy image-goal results across three environments (E1, E2, E3);

5

trials per environment cell,

15

trials per row in total. SR in %, #Coll is mean collisions per trial.

Figure 5: Results on different depth sources: success rate (top) and collisions per trial (bottom).

Method	mean dist. $\uparrow$ (m)	max dist. $\uparrow$ (m)	mean dur. $\uparrow$ (s)	max dur. $\uparrow$ (s)
NoMaD	$7.86$	$13.38$	$29.79$	$47.57$
NoMaD $+$ PathLift	$\mathbf{11.67}$	$\mathbf{22.50}$	$\mathbf{70.66}$	$\mathbf{111.87}$

Table 3: Real-robot goal-free exploration; NoMaD in no-goal mode,

10

trials per method, run until first collision. Distances and durations are measured from the start pose to the contact event.

Variant	SR (%)
PathLift	$\mathbf{29.62}$
$-$ depth input	$6.50$
$-$ Plücker tokenizer (CNN encoder)	$14.25$
$-$ Radial-only (per-waypoint)	$27.62$
$-$ $\mathcal{C}_{\mathrm{lift}}$ supervision (direct $y^{\star}$ regression)	$14.88$
Bare NoMaD (fixed-scale lift)	$19.88$

Table 4: Ablations on NoMaD over an

800

-trial HM3D subset. Each variant removes one component from PathLift’s default configuration. Full results in Appendix F .

Method	SR (%)	SPL (%)	Stage (ms)
PathLift	$29.62$	$28.20$	$\mathbf{5.6}$
DWA	$26.75$	$24.47$	$13.8$
MPPI	$34.12$	$29.11$	$21.3$
PathLift $\,\rightarrow\,$ DWA	$34.62$	$31.89$	$19.4$
PathLift $\,\rightarrow\,$ MPPI	$\mathbf{38.00}$	$\mathbf{34.57}$	$26.9$

Table 5: Comparison with local planners on the 800-trial NoMaD subset of Tab. 4 (bare NoMaD:

19.88\%

SR). DWA and MPPI take their metric goal from the fixed-scale lift, and the composition variants take it from PathLift’s lifted endpoint. Runtime excludes the prior forward pass.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Sub-family	$c_{t}(\hat{\mathbf{p}};R,\theta)$	Shape parameter $\theta$	Count
$T_{0}$ identity	$\hat{\mathbf{p}}$	—	$1$
$T_{1}$ radial	$R\,\hat{\mathbf{p}}$	—	$\|\mathcal{R}\|=19$
$T_{2}$ smooth radial	$R\,f(\mathrm{arc})\,\hat{\mathbf{p}}$	$f\in\{\,\mathrm{slow,\ fast,\ sat_{0.7},\ sat_{0.5}}\,\}$ or quadratic basis with $(a_{1},a_{2})\!\in\!\{-0.4,0,0.4\}^{2}$	$40+45=85$
$T_{3}$ progress	$R\,\hat{\mathbf{p}}\!\circ\!w(\mathrm{arc})$	truncation $\tau\!\in\!\{0.4,0.55,0.7,0.85,1.0\}$ or gamma $\gamma\!\in\!\{0.5,0.75,1,1.5,2\}$	$50+50=100$
$T_{4}$ rotation	$R\,\mathbf{R}_{z}(\theta)\,\hat{\mathbf{p}}$	$\theta\!\in\!\pm\{1.5,3,5,8,12,20,30\}^{\circ}$	$14\times 7=98$
$T_{5}$ rotation $\times$ smooth radial	$R\,f(\mathrm{arc})\,\mathbf{R}_{z}(\theta)\,\hat{\mathbf{p}}$	$\theta\!\in\!\{-30,-12,-5,-1.5,3,8,20\}^{\circ}$ , $f\!\in\!\{\mathrm{slow,fast}\}$	$7\times 2\times 5=70$

Appendix

Table 6: Candidate sub-family parameterizations.

\mathbf{R}_{z}(\theta)

is the

2

-D rotation around the body origin;

f(\mathrm{arc})

are radial schedules;

b

is the Bézier lateral envelope

b(\mathrm{arc};\alpha_{m},\alpha_{e})=2\alpha_{m}\,\mathrm{arc}(1-\mathrm{arc})+\alpha_{e}\,\mathrm{arc}^{2}

;

\rho

and

e

are cubic smoothstep envelopes that anchor the lateral component at the body origin.

T_{7}

is the rescue sub-family with an enlarged lateral envelope; the admissibility filter prunes it whenever a standard-envelope candidate is admissible.

Sub-family	Habitat pin.	Habitat fish.	CODa	OpenLORIS
T 0 identity	48.5	55.6	95.4	60.5
T 1 radial	44.0	49.4	94.0	52.4
T 2 smooth radial	47.6	53.0	94.7	55.6
T 3 progress	43.0	47.4	88.7	50.9
T 4 rotation	40.6	44.9	81.9	46.1
T 5 rotation $\times$ radial	45.1	49.9	84.9	50.1

Appendix

Table 7: Admissibility pass rate (%) per sub-family, averaged within each source. Standard sub-families fall in a uniform

35

–

56\%

indoor range; T 7 rescue passes

0.1

–

2.3\%

, consistent with its role as a fallback pruned whenever a standard-envelope candidate is admissible.

	$c^{\star}$ change rate (%)						$\mathrm{ADE}(c^{\star},y^{\star})$ (m)
Source	$0.15$	$0.20$	$0.25$	$0.35$	$0.40$	$0.50$	$0.15$	$0.30$	$0.50$
Habitat pinhole	48.5	31.0	10.9	6.9	16.9	18.6	0.616	0.573	0.553
Habitat fisheye	43.1	29.0	13.9	6.6	15.1	17.2	0.644	0.611	0.595
CODa	62.4	29.4	17.1	36.0	41.9	56.4	2.947	2.776	2.593

Appendix

Table 8: Sensitivity to the intent-preservation threshold

\tau_{\mathrm{tube}}

, with the other admissibility thresholds fixed. The left block reports the fraction of samples whose selected

c^{\star}

changes by more than

0.05

m relative to

\tau_{\mathrm{tube}}=0.30

. The right block reports

\mathrm{ADE}(c^{\star},y^{\star})

. OpenLORIS is omitted because its expert paths often exceed the per-source candidate range

R_{\max}

(App. A ), making the selection insensitive to this threshold.

Term	Weight	Role
$\mathcal{L}_{\mathrm{lift}}$	$1$	Squared error to supervision target $c^{\star}$
$\mathcal{L}_{\mathrm{clr}}$	$0.2$	Clearance hinge against $\Omega$
$\mathcal{L}_{\mathrm{equiv}}$	$0.1$	Magnitude equivariance regularizer
$\mathcal{L}_{\mathrm{tan{-}prior}}$	$0.5$	Tangent alignment to $\hat{\mathbf{p}}$
$\mathcal{L}_{r\text{-aux}}$	$0.1$	Auxiliary log-magnitude regression
$\mathcal{L}_{\mathrm{curv}}$	$0.05$	Second-difference smoothness

Appendix

Table 9: Loss-term weights. All values are fixed across experiments and visual-navigation policies.

Component	Specification
Base	indoor wheeled differential-drive mobile robot
Forward RGB camera	stereo pair, left lens HFOV $=90^{\circ}$ , $640\!\times\!480$ per channel
LiDAR	MID-360; top-mounted, downward-facing scan plane
Stereo depth model	Fast-FoundationStereo [ 33 ]
Onboard compute	NVIDIA Jetson AGX Orin (64 GB)
Controller	pure-pursuit tracker at $4$ Hz

Appendix

Table 10: Real-robot deployment platform.

Variant	Enc. params	SR (%)
PathLift (Plücker, per-wp)	$2.3$ k	$\mathbf{29.62}$
Depth ablation
$-$ depth input	—	$6.50$
Encoder ablation (input tokenization)
$-$ Plücker tokenization (CNN encoder)	$37$ k	$14.25$
Output-head ablation

Appendix

Table 11: Full ablation results on the

800

-trial NoMaD subset. Each block removes one component from PathLift while keeping the rest fixed; setup details are in the corresponding paragraphs.

Invalid (%)	$0$	$10$	$25$	$50$	$75$	$90$	$95$	$99$	$100$
ADE (m)	$0.232$	$0.232$	$0.232$	$0.232$	$0.232$	$0.233$	$0.233$	$0.239$	$0.413$
tube (m)	$0.079$	$0.079$	$0.079$	$0.079$	$0.079$	$0.079$	$0.079$	$0.072$	$0.030$
$\mathcal{C}_{\mathrm{adm}}$ (%)	$68.5$	$68.8$	$68.9$	$68.9$	$69.0$	$68.6$	$69.1$	$68.4$	$64.4$

Appendix

Table 12: Partial-invalid depth sweep on

800

Habitat-pinhole samples with the NoMaD prior. Random pixels are replaced by the training-time invalid value

-1

, using nested masks across invalid fractions.

Robot Navigation

Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.

See all 557 papers

Sensor-Layout-Agnostic Navigation via Geometric Observation Canonicalization
Oct 6, 2026Welf Rehberg, Kostas AlexisRobot Navigation UAV Navigation
Norwegian Institute of Science and Technology, Norway
OpenFrontier: General Navigation with Visual-Language Grounded Frontiers
Mar 5, 2026Esteban Padilla-Cerdio, Boyang Sun, Marc Pollefeys +1Robot Navigation Vision-Language Navigation
1ETH Zurich · 2Microsoft Spatial AI Lab · University of Bonn
NavCMPO: Critic-Guided MeanFlow Policy Optimization for Adaptive Navigation
Jul 16, 2026Junjie An, Yi Wu, Xiao Liu +5Flow Matching Robot Policy Adaptation
Zhejiang University, Hangzhou, China · Shandong University

Oct 6, 2026J/K move · Enter open · S save

Sensor-Layout-Agnostic Navigation via Geometric Observation Canonicalization

Welf Rehberg, Kostas Alexis

Norwegian Institute of Science and Technology, Norway

Robot Navigation UAV Navigation

Open paper AskPDF arXiv

Existing visual navigation policies are inherently bound to fixed camera configurations, creating a fundamental barrier to zero-shot deployment across heterogeneous robot sensor layouts. To overcome this limitation, we present an embodiment-informed navigation policy capable of generalizing across diverse depth sensor configurations on a specific aerial platform. Instead of implicitly learning spatial alignments, our approach explicitly unprojects depth measurements from arbitrary depth sensor payloads, varying in sensor count, mounting extrinsics, and intrinsics, into a shared robot-centric frame, stitching them into a unified spherical range image and a binary validity mask. This mask allows the downstream policy to explicitly distinguish covered space from unobserved blind spots. Trained via reinforcement learning with aggressive camera randomization, our policy generalizes zero-shot to unseen layouts featuring up to seven cameras, scaling success rates from 78% to 95% as total spatial sensing coverage increases. Finally, real-world flight trials on a physical quadrotor, conducted in an obstacle-filled corridor and an outdoor forest, validate the policy's zero-shot transfer across camera configurations and its resilience to sudden online sensor dropouts.

Abstract

Robot Navigation

Explore similar work

Sensor-Layout-Agnostic Navigation via Geometric Observation Canonicalization

OpenFrontier: General Navigation with Visual-Language Grounded Frontiers

NavCMPO: Critic-Guided MeanFlow Policy Optimization for Adaptive Navigation