PosePlaner: Denoising Any Feed-Forward Pose Predictor from Pairwise Planar Geometry

Yihang Chen; Bin Tan; Yujun Shen; Yinghao Xu; Nan Xue

cs.CVOct 26, 2026

PosePlaner: Denoising Any Feed-Forward Pose Predictor from Pairwise Planar Geometry

Authors: Yihang Chen, Bin Tan, Yujun Shen, Yinghao Xu, Nan Xue

Organizations: HKUST, Hong Kong, China · Robbyant, Ant Group, China

Abstract

Recent multi-view Transformers (e.g., DUSt3R and VGGT) have advanced 3D reconstruction by scaling training data and model capacity from two views to many views and long sequences, subsuming the classical SfM/SLAM pipeline of feature extraction, matching, and pose estimation in a single feed-forward pass. Yet this subsumption hides the interfaces between pipeline stages---correspondences, geometry, and pose are entangled in a single forward pass, leaving supervised fine-tuning as the major path to improvement and offering no mechanism to correct predictions from geometric evidence at test time. We propose PosePlaner to close this gap by reviving the classical coupling between correspondences and pose via flow matching, treating pose refinement as a conditional generation problem that takes coarse geometric matches and initial pose as input condition and fuses the resulting pose hypotheses into a single refined estimate, providing a continuous, data-driven analog of RANSAC voting. Our method is trained efficiently on two-view data, yet serves as a periodic refiner for streaming n-view pose estimators with no retraining. Empirically, it corrects failures of feed-forward predictors, substantially improves correspondence-based solvers, and reduces accumulated drift in long streaming sequences.

Figures & tables

Figure 1: Our proposed PosePlaner refines any feed-forward pose in one Euler step of a plane-embedded diffusion adapter, and extends to $n$ -view streaming via per-edge view-graph refinement aggregated by rotation averaging and a Glomap -style positioner. Left: $T_{\text{init}}$ ( blue ) drifts from GT ( black ). Middle: conditioned on per-view planar primitives, the refiner yields $T_{\text{refine}}=T_{\text{init}}\cdot\Delta T_{\theta}(T_{\text{init}}\mid\text{planes})$ ( red ) in one step. Right: refined trajectory ( red ), raw feed-forward ( blue ), GT ( black ).

Figure 2: PosePlaner pipeline. A per-view local-frame plane head produces $M$ geometry tokens $\{\bm{g}_{m}\}$ (paired plane primitives across the two views) and a global pose token $\bm{c}$ that encodes the initial relative pose $T_{\text{init}}$ . The multi-query diffusion Transformer denoises $T_{\text{init}}$ in a single Euler step under these conditioning tokens, yielding $M$ hypothesis frustums $h_{1},\dots,h_{M}$ . A score head softly fuses the hypotheses into the final refined pose $T_{\text{refined}}$ , used to align the two-view reconstruction.

Method	ScanNet1500					7Scenes $\dagger$
Method	R@5	R@10	t@5	t@10	Valid	R@5	R@10	t@5	t@10	Valid
Correspondence-based solvers
SuperGlue ( Sarlin et al., 2020 )	43.9	62.4	21.3	39.6	$1427/1500$	42.3	65.9	18.9	37.6	$4014/4070$
SuperGlue + NopeSAC head	34.9	58.7	7.4	19.9		34.3	61.5	7.1	19.9
SuperGlue + PosePlaner	48.9	68.9	21.5	40.4		45.8	69.7	19.0	37.6
LoFTR ( Sun et al., 2021 )	11.5	25.0	2.1	5.6	$1028/1500$	17.7	34.2	3.7	10.1	$3585/4070$

Table 1: Two-view pose refinement on ScanNet1500 ( Dai et al., 2017 ) and 7Scenes ( Shotton et al., 2013 ) . R/t: rotation/translation-direction AUC at

5^{\circ}

and

10^{\circ}

(%, higher is better). Valid gives the common evaluation subset within each block. PosePlaner uses MASt3R conditioning throughout (Tab. 9 ). Best per block in bold .

\dagger

Tested zero-shot.

Initialization / refinement	ScanNet1500			7Scenes
Initialization / refinement	R@5	R@10	R@20	R@5	R@10	R@20
LoFTR (pre-BA)	11.5	25.2	44.1	17.8	34.3	53.4
+ BA	13.4	26.5	44.9	24.4	39.6	56.7
+ PosePlaner	29.5	53.1	70.7	31.2	56.8	75.4
MASt3R (pre-BA)	66.0	82.1	90.9	49.6	72.7	86.1
+ BA	66.5	82.4	91.1	50.6	73.3	86.4

Table 2: Controlled comparison with bundle adjustment. BA and PosePlaner use the same initial poses and MASt3R correspondences. R@5, R@10, and R@20 are rotation AUC at

5^{\circ}

10^{\circ}

, and

20^{\circ}

(%, higher is better). Best results in each initialization group are bold . Table 1 ’s correspondence-based initializations have already undergone BA.

Panel of Figure 3 — Figure 3: Planar primitives as a stand-alone match filter. Three 7Scenes pairs, LoFTR ( Sun et al., 2021 ) upstream. Top: raw LoFTR matches. Bottom: matches surviving the cut $\lVert\Delta\bm{n}\rVert+\lvert\Delta d\rvert\!\leq\!1$ . Green / red mark RANSAC inliers/outliers; top-left numbers are inlier ratio and pose error.

Method	$N{=}200$				$N{=}500$				$N{=}2000$
Method	R@5	R@20	t@5	t@20	R@5	R@20	t@5	t@20	R@5	R@20	t@5	t@20
ScanNet
CUT3R	23.3	65.0	4.1	24.0	16.5	55.7	2.2	19.0	17.3	53.4	1.0	13.2
CUT3R + PosePlaner	40.5	80.3	7.7	38.0	37.3	78.9	6.3	38.4	37.3	79.8	3.4	31.2
TTT3R	46.2	83.1	11.1	42.7	36.1	79.3	8.0	38.7	27.5	72.5	3.4	28.6
TTT3R + PosePlaner	50.7	85.6	10.8	45.9	47.1	84.3	10.8	47.3	48.2	85.0	10.4	41.8

Table 3:

n

-view pose accuracy on streaming ScanNet ( Dai et al., 2017 ) and 7Scenes ( Shotton et al., 2013 ) . Pairwise R and t AUC at

5^{\circ}

and

20^{\circ}

over the

\binom{100}{2}

pairs in the refined tail. Higher is better.

\dagger

Tested zero-shot.

Panel of Figure 4 — Figure 4: Streaming pose refinement on a representative ScanNet ( Dai et al., 2017 ) sequence (Scene0757_00). Left: sample frames. Right three panels: estimated camera trajectories under CUT3R ( Wang et al., 2025b ) , TTT3R ( Chen et al., 2025 ) , and WinT3R ( Li et al., 2025 ) . Black : ground-truth; blue : raw baseline; red : after PosePlaner refinement. $\bullet$ start, $\times$ end.

Method	$N=200$				$N=500$				$N=2000$
Method	R@5	R@20	t@5	t@20	R@5	R@20	t@5	t@20	R@5	R@20	t@5	t@20
Oxford Spires $\dagger$
CUT3R	59.1	85.1	28.9	66.2	35.5	75.7	8.1	52.5	34.0	70.1	11.6	46.5
CUT3R + PosePlaner	65.8	90.8	33.5	70.5	50.4	82.8	22.4	64.1	53.3	85.8	22.7	67.4
TTT3R	73.6	92.7	45.8	77.3	59.7	88.5	27.5	74.9	37.2	76.5	11.4	53.2
TTT3R + PosePlaner	72.3	92.5	42.0	74.9	62.9	89.2	29.8	74.0	51.3	85.6	19.9	67.9

Table 4: Zero-shot outdoor multi-view refinement. R@5/R@20 and t@5/t@20 are rotation and translation-direction AUC at

5^{\circ}

20^{\circ}

(%,

\uparrow

). Better results within each pair are bold .

\dagger

Tested zero-shot.

Variant	Init: SuperGlue						Init: LoFTR
Variant	R@5	R@10	R@20	t@5	t@10	t@20	R@5	R@10	R@20	t@5	t@10	t@20
Full (ours)	48.9	68.9	82.1	21.5	40.4	59.5	29.4	52.9	70.6	3.5	9.0	18.4
w/o $\{\bm{g}_{m}\}$	43.0	61.7	76.0	21.3	39.6	58.0	11.5	25.1	44.2	2.1	5.6	13.0
w/o $\bm{c}$	17.4	36.2	54.5	2.1	5.0	10.9	17.4	36.2	54.5	2.1	5.0	10.9
w/o score head	48.0	68.1	81.6	21.4	40.4	59.4	26.5	50.4	69.0	3.2	8.7	17.9
w/o translation	40.5	61.1	77.3	21.0	39.7	58.7	16.4	36.1	58.4	2.3	6.7	15.9

Table 5: Ablation study on ScanNet1500 with SuperGlue and LoFTR upstream. R and t AUC at

\tau\!\in\!\{5,10,20\}^{\circ}

; higher is better, best per column in bold . Removing the pose token

\bm{c}

forces the model to estimate pose without the upstream init, so the SuperGlue and LoFTR blocks of that row are identical by construction.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Refinement collapses the plane-warping residuals. (a) ScanNet1500 and (b) 7Scenes, LoFTR upstream. Each sub-figure: $\|\Delta\bm{n}\|_{2}$ (left) and $|\Delta d|$ (right, scene-normalized) over all matched plane pairs; blue = raw, red = after PosePlaner ; dashed lines mark means.

Dataset	Type	# Pairs
ScanNetV2 ( Dai et al., 2017 )	Indoor / Real	$610$ K
ScanNet++ ( Yeshwanth et al., 2023 )	Indoor / Real	$810$ K
ARKitScenes ( Dehghan et al., 2021 )	Indoor / Real	$2{,}400$ K
Habitat ( Savva et al., 2019 )	Indoor / Synthetic	$120$ K
Total		$\bm{\sim\!3{,}940}$ K

Appendix

Table 6: Training data and sample sizes. Same mixture used by Plana3r ( Liu et al., 2025a ) .

Panel of Figure 6 — Figure 6: Predicted plane primitives and matched patch correspondences. Three 7Scenes pairs (columns). Top : input RGB (view 1 $|$ view 2). Bottom : PlanarSplatting -rasterized normal from our plane head with the $24\!\times\!32$ patch grid (white); yellow lines connect MASt3R ’s matched patch centers, which form $\{\bm{g}_{m}\}_{m=1}^{M}$ in Eq. ( 3 ).

Method	LoFTR ( Sun et al., 2021 )				SuperGlue ( Sarlin et al., 2020 )
Method	R@5	R@10	R@20	Valid	R@5	R@10	R@20	Valid
init	10.7	18.8	30.6	$4385/7996$	38.3	53.5	66.7	$6853/7996$
+ PosePlaner	20.2	36.0	52.8		40.8	57.2	70.6

Appendix

Table 7: Two-view rotation refinement on Matterport3D. Evaluated zero-shot on the

7996

-pair test set of SparsePlanes ( Jin et al., 2021 ) . Rotation AUC@

\tau^{\circ}

for two upstream matchers in two configurations: init and + PosePlaner . The Valid column counts pairs with a valid upstream pose; AUC is computed only on this subset. Higher is better; best per block in bold .

Method	$N=200$				$N=500$
Method	R@5	R@20	t@5	t@20	R@5	R@20	t@5	t@20
CUT3R	20.5	55.2	22.6	64.6	11.7	42.7	5.1	45.0
CUT3R + PosePlaner	20.3	60.9	24.6	68.4	12.1	44.5	5.7	48.1
TTT3R	26.0	69.6	33.3	73.5	20.5	70.9	24.8	73.6
TTT3R + PosePlaner	26.2	70.0	29.6	65.9	25.1	73.0	28.2	75.0

Appendix

Table 8: Zero-shot multi-view refinement on CrossLoc drone sequences. R@5/R@20 and t@5/t@20: rotation/translation-direction AUC at

5^{\circ}

20^{\circ}

(%,

\uparrow

). Better results within each pair are bold .

Conditioning match	Upstream: SuperGlue			Upstream: LoFTR
Conditioning match	R@5	R@10	R@20	R@5	R@10	R@20
init (no refinement)	43.9	62.4	76.4	11.5	25.0	44.0
+ PosePlaner , SuperGlue ( Sarlin et al., 2020 ) match	45.1	64.1	78.0	17.8	38.8	59.5
+ PosePlaner , LoFTR ( Sun et al., 2021 ) match	43.5	62.6	76.8	14.5	31.1	50.5
+ PosePlaner , ROMA ( Edstedt et al., 2024 ) match	49.8	69.6	82.5	32.9	56.3	72.8
+ PosePlaner , MASt3R ( Leroy et al., 2024 ) match	48.9	68.9	82.1	29.4	52.9	70.6

Appendix

Table 9: Sensitivity to the external match source. Two-view rotation AUC on ScanNet1500 with the upstream pose fixed to SuperGlue or LoFTR , while the matcher used to produce patch correspondences for PosePlaner ’s conditioning is varied across four options. The init row reports each upstream’s raw pose accuracy from Tab. 1 . Higher is better.

Panel of Figure 7 — Figure 7: Additional streaming trajectories on ScanNet. Each row is one $200$ -frame sequence refined by PosePlaner in the same $100$ -frame sliding-window scheme as Fig. 4 ( $50$ -frame overlap). Right three columns plot the estimated camera trajectory under CUT3R , TTT3R , and WinT3R ; black is ground truth, blue is the raw baseline, and red is after PosePlaner refinement.

Method	R@5	R@10	R@20	t@5	t@10	t@20
LoFTR init	11.5	25.0	44.0	2.1	5.6	13.0
Direct- $x_{0}$	14.8	30.3	49.2	2.3	6.6	14.5
DDPM pred- $x_{0}$	16.7	33.2	51.8	2.2	6.5	14.6
Flow matching	29.4	52.9	70.6	3.5	9.0	18.4

Appendix

Table 10: Training objective comparison on ScanNet1500. LoFTR initialization,

1028

valid pairs. R@

\theta

/t@

\theta

: rotation/translation-direction AUC at

\theta^{\circ}

(%,

\uparrow

). Best results are bold .

Plane source	R@5	R@10	R@20
LoFTR init (no refinement)	11.5	25.0	44.0
MoGe + RANSAC	27.1	50.4	69.2
Learned planes (ours)	29.4	52.9	70.6

Appendix

Table 11: Alternative plane source on ScanNet1500. Rotation AUC at

5^{\circ}/10^{\circ}/20^{\circ}

(%,

\uparrow

) on the same

1028

valid LoFTR pairs. Best results are bold .

Relative Pose Estimation

Relative pose estimation recovers the rotation and translation between cameras, objects, or robots from visual and other sensor measurements.

See all 66 papers

PnP-Transformer: A Neural Pose Solver for Zero-Shot 6D Object Pose Estimation
Oct 19, 2026Emirkan Burak Yilmaz, Yakup GencObject Pose Estimation
Computer Engineering Dept., Gebze Technical University, Kocaeli, Türkiye
Graph3R: Unifying View Graph Initialization with Feed-Forward Reconstruction
Oct 12, 2026Jingwen Yu, Hengli Wang, Jianhao Jiao +33D Reconstruction Multi-View 3D Reconstruction
Hong Kong University of Science and Technology · Southern University of Science and Technology · China Merchants Group, LionRock AI Lab +1
NoDrift3R: Raymap-Guided Coupling for Drift-Robust Unposed Feed-Forward 3D Reconstruction
Jul 8, 2026Xiangyu Sun, Liu Liu, Seungkwon Yang +43D Gaussian Splatting 3D Reconstruction
Sungkyunkwan University, South Korea · ⋆Intern at Horizon Robotics · Horizon Robotics, China +1

Oct 19, 2026J/K move · Enter open · S save

PnP-Transformer: A Neural Pose Solver for Zero-Shot 6D Object Pose Estimation

Emirkan Burak Yilmaz, Yakup Genc

Computer Engineering Dept., Gebze Technical University, Kocaeli, Türkiye

Object Pose Estimation

Open paper AskPDF arXiv

We present PnP-Transformer, a neural pose solver designed as an iterative-free, differentiable alternative to classical PnP-RANSAC algorithms. PnP-Transformer regresses the 6D object pose by processing predicted semi-dense 2D-3D correspondence fields alongside confidence scores, while mapping geometric inputs into a continuous latent space. To evaluate its practical utility, we integrate PnP-Transformer into a zero-shot 6D pose estimation pipeline, where the correspondence module and pose solver are trained separately. PnP-Transformer achieves a 41.2% mean AR score with low latency (0.46 s/obj) on seven core BOP benchmark datasets, surpassing all prior baselines (such as RayPose with 39.1% AR) but one (Co-op with 58.4% AR). When the pose solver is replaced with EPnP-RANSAC (similar to Co-op) while keeping the correspondence module unchanged, the RANSAC alternative outperforms PnP-Transformer on six of the seven datasets by reaching 52.2% mean AR at a cost of an additional 60 ms per object, revealing an accuracy-latency trade-off. Code and pretrained models are available at https://github.com/ebylmz/pnp-transformer.

Abstract

Relative Pose Estimation

Explore similar work

PnP-Transformer: A Neural Pose Solver for Zero-Shot 6D Object Pose Estimation

Graph3R: Unifying View Graph Initialization with Feed-Forward Reconstruction

NoDrift3R: Raymap-Guided Coupling for Drift-Robust Unposed Feed-Forward 3D Reconstruction