GRC-Pose: Generation-Reconstruction Correspondence for Prior-Free 6D Object Pose Tracking

Shiyang Liu; Weiquan Lin; Luping Xiao; Jiadong Tang; Yi Yang; Yu Gao; Xingyu Chen

cs.CVSep 30, 2026

GRC-Pose: Generation-Reconstruction Correspondence for Prior-Free 6D Object Pose Tracking

Authors: Shiyang Liu, Weiquan Lin, Luping Xiao, Jiadong Tang, Yi Yang, Yu Gao, Xingyu Chen

Organizations: School of Automation, Beijing Institute of Technology · Zhongguancun Academy · Artificial Intelligence Academy, Xidian University · School of Information and Communication Engineering, Beijing University of Posts and Telecommunications

Abstract

Prior-free 6D object pose tracking seeks to recover the trajectory of an unseen object from a single RGB video without object-specific CAD models, posed reference images, or pose annotations. Geometric foundation models provide complementary object-centric and scene-centric cues, yet SAM3D CAD is indexed by an arbitrary object-local surface parameterization, whereas reconstructed evidence is expressed in a sequence-specific world frame with partial surface coverage. To exploit this complementarity, we formulate tracking as generation-reconstruction correspondence and introduce GRC-Pose, a correspondence-based framework that combines learned correspondence prediction with robust pose estimation. Concretely, GeoCorr-Matcher estimates weighted object-scene correspondences and per-match uncertainty for each pose candidate. FGH-Solver integrates these matches through multiple robust geometric estimators and sequence-level posterior inference, while a posterior-gated memory retains only inlier-supported observations through occlusion and viewpoint change. Extensive evaluation shows that with SAM3D CAD, GRC-Pose achieves state-of-the-art Average Recall and motion retention on HOT3D, improving the latter by 58% over prior art. On classical benchmarks including YCBInEOAT and LINEMOD, it remains highly competitive.

Figures & tables

Figure 2: Overview of GRC-Pose. The top panel maps input video $I_{1:T}$ , SAM3D CAD geometry, and scene-level reconstructions to 6D object trajectories in the world frame. The bottom panel details pose-conditioned correspondences with match-level uncertainty, while FGH-Solver (right) resolves factorized pose candidates, performs temporal filtering with rotation refinement, and guides surface-memory updates.

Figure 3: Qualitative comparison across tracking benchmarks. Pose tracking results on HOT3D, YCBInEOAT and LINEMOD with SAM3D CAD.

Figure 4: Qualitative temporal tracking on HOT3D. Images are cropped around the target object with yellow 3D bounding boxes and RGB object-coordinate axes visualizing the recovered 6D pose. Columns follow chronological order.

Method	PF	HOT3D			YCBInEOAT		LINEMOD
Method	PF	Avg. AR $\uparrow$	AR MSPD $\uparrow$	Motion $\uparrow$	ADD $\uparrow$	ADD-S $\uparrow$	ADD $\uparrow$	ADD-S $\uparrow$
FoundationPose ‡	$\times$	3.4	7.0	2.4	43.1	67.2	38.2	84.1
RGBTrack	$\times$	3.4	9.2	1.3	5.6	21.5	0.4	2.1
GigaPose	$\times$	31.4	69.2	11.9	16.1	31.3	10.4	37.5
FreePose	$\times$	1.9	5.6	0.0	0.0	5.7	1.7	18.3
Dynhor	$\times$	31.9	76.8	17.0	2.0	25.5	59.9	85.5

Table 1: Main quantitative comparison with SAM3D CAD. PF denotes prior-free; all values are percentages, and best and second-best values are bold and underlined. HOT3D uses RGB-derived VGGT-

\Omega

proxy depth where a method consumes scene geometry. Full BOP components, Metric CAD controls, and complete per-asset/per-object results are in the supplementary material.

\ddagger

FoundationPose uses independent register on every fixed evaluation key.

(a) Pose-conditioned correspondence
Variant	Inlier@ $0.1d$ $\uparrow$	3D residual $/d$ $\downarrow$	Duplicate assignment $\downarrow$
w/o pose cond.	13.94	58.42	37.04
+ pose cond.	19.35	54.78	34.32
+ contrastive	20.34	49.99	30.98

Table 2: Component ablations. Both panels share the same HOT3D recording–asset streams under SAM3D CAD, adaptation seeds, and model checkpoints. All entries except Supports/frame are percentages; residuals use the shared correspondence field before and after the shared posterior decoder.

Figure 5: Qualitative visualization of pose-conditioned correspondences. Blue and orange links denote $0.1d$ inliers and outliers.

Figure 6: Qualitative comparison of geometric solving strategies. Branch-specific supports retain distinct pose modes, whereas direct averaging produces an intermediate pose. This illustration accompanies the solver mechanism diagnostic in Tab. 2 .

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

	Algorithm 1: Recording-local 6D pose tracking
1	Input: $\mathcal{O},\{I_{t},M_{t},\mathbf{K}_{t},\mathbf{p}^{w}_{t,j},\mathbf{u}_{t,j},d_{t,j},v_{t,j},\mathbf{f}^{s}_{t,j}\}_{t=1}^{T},\{\widetilde{\mathbf{x}}_{i},\mathbf{f}_{i}^{o}\}_{i=1}^{N_{o}}$ .
2	$\mathbf{x}_{i}\leftarrow s_{\mathrm{on}}R_{\mathrm{on}}\widetilde{\mathbf{x}}_{i}+\mathbf{t}_{\mathrm{on}}$ .
3	for $t=1,\ldots,T$ do
4	$\mathcal{H}_{t}^{\mathrm{retr}}\leftarrow\operatorname{Top}_{240}\!\bigl(\operatorname{DINO}(I_{t},\{\mathbf{f}_{i}^{o}\})\bigr)$ .
5	$\mathbf{L}_{t,h}\leftarrow\mathbf{Q}_{t,h}(\mathbf{A}_{t}^{s})^{\top}/\sqrt{d}+\mathbf{B}_{t,h}^{\mathrm{geo}}$ ; $\mathbf{C}_{t,h}\leftarrow\operatorname{softmax}_{j}(\mathbf{L}_{t,h})\odot\operatorname{softmax}_{i}(\mathbf{L}_{t,h})$ .
6	$(\bar{\mathbf{u}}_{t,h,i},\bar{\mathbf{p}}_{t,h,i}^{w},w_{t,h,i})\leftarrow\operatorname{GeoCorr}(\mathbf{C}_{t,h},\mathbf{L}_{t,h})$ ; $\mathcal{H}_{t}^{\mathrm{2D\text{-}3D}}\leftarrow\operatorname{RANSAC\text{-}PnP}(\mathbf{x},\bar{\mathbf{u}},w)$ .

Appendix

Table A1: Algorithmic procedure of GRC-Pose. Candidate support is fixed before adaptation; only

\Theta

and bounded local pose variables are optimized. Causal and cross-fit controls restrict the frames used in lines 12–14.

HOT3D Object CAD Method AR

\uparrow

{}_{\mathrm{MSPD}}\uparrow

{}_{\mathrm{MSSD}}\uparrow

{}_{\mathrm{VSD}}\uparrow

Motion

\uparrow

SAM3D CAD FoundationPose 3.4 7.0 3.0 0.0 2.4 RGBTrack 3.4 9.2 0.9 0.1 1.3 GigaPose 31.4 69.2 22.6 2.4 11.9 FreePose 1.9 5.6 0.0 0.0 0.0 Dynhor 31.9 76.8 17.9 1.0 17.0 BIT 1.6 4.0 0.7 0.1 0.0 Ours 34.0 79.0 20.9 2.1 26.9 Metric CAD FoundationPose 2.5 5.7 1.7 0.0 0.4 RGBTrack 4.3 9.1 2.4 1.4 1.5 GigaPose 48.5 74.6 44.2 26.7 22.1 FreePose 1.6 4.9 0.0 0.0 0.0 Dynhor 38.8 80.2 31.8 4.3 27.8 BIT 2.4 6.7 0.5 0.0 0.0 Ours 43.1 81.1 34.6 13.5 36.9

YCBInEOAT Object CAD Method AR

\uparrow

{}_{\mathrm{MSPD}}\uparrow

{}_{\mathrm{MSSD}}\uparrow

{}_{\mathrm{VSD}}\uparrow

ADD AUC

\uparrow

ADD-S AUC

\uparrow

ADD-

0.1d\uparrow

Rot. (deg.)

\downarrow

Trans. (cm)

\downarrow

SAM3D CAD FoundationPose 46.3 61.4 53.3 24.1 43.1 67.2 41.0 60.9 7.5 RGBTrack 17.2 42.6 8.7 0.2 5.6 21.5 0.3 109.9 42.4 GigaPose 20.7 39.8 19.7 2.6 16.1 31.3 6.3 53.1 49.1 FreePose 13.1 37.7 1.5 0.0 0.0 5.7 0.0 111.6 67.0 Dynhor 15.3 33.6 11.2 1.0 2.0 25.5 0.0 153.0 18.5 BIT 7.2 18.3 3.0 0.3 2.3 7.8 0.3 98.8 36.5 Ours 45.4 56.5 53.1 26.6 41.2 67.0 40.7 71.8 6.7 Metric CAD FoundationPose 88.7 98.5 96.8 70.9 84.5 95.9 85.2 27.9 0.7 RGBTrack 55.1 91.1 50.1 24.2 46.4 56.2 35.7 40.0 49.4 GigaPose 27.2 51.7 22.9 7.2 9.1 33.3 1.9 118.8 75.9 FreePose 14.8 44.5 0.0 0.0 0.0 0.0 0.0 144.3 106.0 Dynhor 38.1 82.0 30.2 2.2 20.4 57.5 4.8 88.2 9.4 BIT 27.9 48.8 29.3 5.5 26.9 42.0 7.7 54.9 23.3 Ours 87.4 98.0 96.4 67.9 92.5 96.2 95.8 6.4 0.5

LINEMOD Object CAD Method AR

\uparrow

{}_{\mathrm{MSPD}}\uparrow

{}_{\mathrm{MSSD}}\uparrow

{}_{\mathrm{VSD}}\uparrow

ADD AUC

\uparrow

ADD-S AUC

\uparrow

ADD-

0.1d\uparrow

Rot. (deg.)

\downarrow

Trans. (cm)

\downarrow

SAM3D CAD FoundationPose 43.1 64.0 61.7 3.6 38.2 84.1 5.6 94.1 3.3 RGBTrack 2.3 6.1 0.8 0.1 0.4 2.1 0.1 115.4 63.6 GigaPose 24.0 53.0 18.8 0.1 10.4 37.5 1.0 90.1 17.1 FreePose 14.4 36.1 7.0 0.0 1.7 18.3 0.0 125.9 26.4 Dynhor 66.5 94.5 73.6 31.5 59.9 85.5 47.2 47.1 3.0 BIT 5.3 14.7 1.1 0.1 0.6 1.7 0.1 120.4 2345.3 Ours 42.8 62.1 63.6 2.7 44.8 86.1 6.2 83.8 2.6 Metric CAD FoundationPose 2.4 3.6 2.2 1.4 1.8 3.4 1.7 111.2 83.7 RGBTrack 3.8 9.4 1.5 0.4 1.0 3.3 0.6 106.6 82.9 GigaPose 76.0 95.8 86.0 46.3 75.1 89.6 70.6 18.5 3.0 FreePose 15.0 44.0 0.9 0.0 0.2 4.0 0.0 125.2 33.7 Dynhor 66.5 94.5 73.6 31.5 59.9 85.5 47.2 47.1 3.0 BIT 8.1 21.5 1.7 1.2 1.6 1.9 1.6 111.2 1240.1 Ours 85.6 96.1 96.4 64.2 86.7 96.4 88.0 14.7 0.6

Appendix

Table A2: Complete aggregate results across all three benchmarks and both CAD conditions. All values except rotation and translation are percentages; translation is reported in centimeters; best values are bold.

Figure A1: Qualitative comparison on HOT3D with SAM3D CAD. Each row compares the shared RGB observation, ground truth, GRC-Pose, and the external methods for one target object. Yellow boxes identify target regions; rendered SAM3D CAD overlays visualize the recovered 6D poses.

Asset	Method	$N$	AR $\uparrow$	AR MSPD $\uparrow$	AR MSSD $\uparrow$	AR VSD $\uparrow$	Motion $\uparrow$
Assets 1–6
aria	FoundationPose	4	4.2	12.5	0.0	0.0	0.0
	RGBTrack		0.0	0.0	0.0	0.0	0.0
	GigaPose		4.2	12.5	0.0	0.0	0.0
	FreePose		0.0	0.0	0.0	0.0	0.0
	Dynhor		14.2	22.5	20.0	0.0	0.0

Appendix

Table A4: Complete HOT3D per-asset results with SAM3D CAD. The two panels jointly report all twelve assets and all seven methods. All values except the frame count

N

are percentages; Motion is correctness-gated consecutive-pair retention at the reported

50

-pixel/

0.5d

gate. Best per-asset values are bold.

Figure A2: Qualitative comparison on object-centric benchmarks with SAM3D CAD. Each row compares the shared RGB observation, ground truth, GRC-Pose, and the external methods for one target object. Rendered SAM3D CAD overlays visualize the recovered 6D poses.

Object ( $N$ )	Method	AR $\uparrow$	ADD AUC $\uparrow$	ADD-S AUC $\uparrow$	ADD- $0.1d$ $\uparrow$	Rot. (deg.) $\downarrow$	Trans. (cm) $\downarrow$
SAM3D CAD
bleach (57)	FoundationPose	59.2	56.1	83.9	49.1	55.5	3.8
	RGBTrack	26.8	3.8	26.9	0.0	131.6	48.1
	GigaPose	25.9	10.7	34.6	1.8	56.0	16.1
	FreePose	12.2	0.0	0.0	0.0	116.9	84.4
	Dynhor	28.5	3.5	40.8	0.0	135.5	15.2

Appendix

Table A5: Complete YCBInEOAT per-object results for SAM3D CAD and Metric CAD. AR is the BOP average recall; AUC and threshold values are percentages; translation is reported in centimeters; best per-object values are bold.

Object	Method	$N$	AR $\uparrow$	ADD AUC $\uparrow$	ADD-S AUC $\uparrow$	ADD- $0.1d\uparrow$	Rot. (deg.) $\downarrow$	Trans. (cm) $\downarrow$
SAM3D CAD
ape	FoundationPose	62	45.1	45.7	88.5	0.0	149.6	2.1
	RGBTrack		1.8	1.0	4.6	0.0	122.8	30.9
	GigaPose		22.3	1.9	11.1	0.0	151.2	27.4
	FreePose		13.2	0.7	5.5	0.0	132.9	24.0
	Dynhor		59.3	53.5	82.1	22.6	80.7	3.3

Appendix

Table A6: Complete LINEMOD per-object results for SAM3D CAD and Metric CAD. All metrics except rotation and translation are percentages;

N

is the number of scored frames. Best per-object values are bold.

Configuration	AR $\uparrow$	ADD AUC $\uparrow$	ADD-S AUC $\uparrow$	Rot. (deg.) $\downarrow$	Trans. (m) $\downarrow$
GeoCorr-Matcher + FGH-Solver	.6475	37.1	79.3	109.12	.0842
$+$ posterior-gated surface memory	.8174	83.4	92.8	21.55	.0110
$+$ temporal Bayesian filtering	.8436	90.6	94.9	7.49	.0076
$+$ dense silhouette/RGB measurement	.8693	92.3	96.1	6.56	.0055
$+$ trust-region rotation refinement	.8738	92.5	96.2	6.45	.0054

Appendix

Table A10: Component controls on the fixed YCBInEOAT Metric CAD protocol. Panel (a) gives the sequential system assembly; panels (b) and (c) separate solver support and posterior selection from frame-level emission and sequence-specific self-adaptation. ADD and ADD-S are AUC percentages, translation is in meters, and all rows use identical frame keys.

Memory state	Write prob.	Observability	Pose update
No memory	.0000	.2508	$4.178^{\circ}$ / .01121 m
Gated memory	.3351	.3365	$4.072^{\circ}$ / .01072 m

Appendix

Table A11: Posterior-gated surface memory and sequence-specific self-adaptation on the fixed YCBInEOAT Metric CAD protocol. Observability is the learned reliability weight used in temporal filtering; memory values are scene means over three matched seeds.

Support	Raw bank	AR $\uparrow$	AR MSSD $\uparrow$	Adapt. (s) $\downarrow$
Geometric parent	–	.3399	.2086	–
$K=16$	7.2%	.3398	.2086	29.31
$K=32$	14.3%	.3400	.2086	29.25
$K=64$	28.6%	.3401	.2090	29.80

Appendix

Table A12: Candidate budget and temporal posterior on HOT3D with SAM3D CAD. Panel (a) reports median sequence-specific self-adaptation time over 23 recording–assets. Panel (b) reports effective posterior support over 195 frames and three adaptation seeds with

K=64

Mode	AR $\uparrow$	AR MSPD $\uparrow$	AR MSSD $\uparrow$
Geometric parent	.5551	.8262	.2841
All-frame adaptation	.5554	.8262	.2846
Prefix-causal adaptation	.5551	.8262	.2841

Appendix

Table A13: Causal execution and trust-region rotation refinement. Panel (a) uses the same candidate bank and scored HOT3D keys. Panel (b) uses YCBInEOAT with Metric CAD; all rows reuse the same fixed posterior update and differ only in the refined pose components.

Explore similar work

Prior-free relative 6D pose estimation of multiple object instances
Sep 8, 2026Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini +1Category-Level Object Pose Estimation Objects
Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy
Robust 6-DoF Object Pose Tracking with Built-In Recovery under Occlusions and Rapid Object Motions
Jul 26, 2026Balázs Opra, Léo Ghafari, Thomas Stewart +1Category-Level Object Pose Estimation Multi-Object Tracking
Woven by Toyota, Inc. · University of Bonn, Germany. · University of Bonn, Center for Robotics, and the Lamarr Institute for Machine Learning and Artificial Intelligence.
RRTrack: Robust and Recoverable Object 6D Pose Tracking for Dynamic Scenes
Jul 26, 2026Junyue Li, Ye Zheng, Yifan Chen +2Category-Level Object Pose Estimation Multi-Object Tracking
College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Institute of Artificial Intelligence (TeleAI), China Telecom, China · College of Future Information Technology, Fudan University, Shanghai 200433, China

Sep 8, 2026, cs.CVJ/K move · Enter open · S save

Prior-free relative 6D pose estimation of multiple object instances

Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini, Fabio Poiesi

Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy

Category-Level Object Pose Estimation Objects Multimodal Dataset Prior Knowledge Integration

Open paper AskPDF arXiv

Object 6D pose estimation formulations have progressively reduced reliance on object-specific priors, evolving from explicit 3D models to multi-view object captures to single reference images. We take this progression to its extreme by introducing prior-free relative 6D pose estimation, which lifts the assumption of knowing which object is to be posed within the scene. This novel setting aims to estimate the relative poses of multiple instances of an unknown object within the same image, without requiring CAD models, templates, or reference images. We solve this by formulating a novel method (PROSE) that finds coarse correspondences between object instances using multimodal foundation features, thus requiring no training. We refine these correspondences by imposing cycle consistency across tuples of instances, and leverage the resulting globally consistent correspondences to estimate the relative 6D pose between any pair of instances. To enable systematic evaluation, we design a novel benchmark (PRENCH) built from three multi-instance BOP datasets and enriched with task-specific metadata. PROSE consistently outperforms baselines obtained by adapting state-of-the-art single-image methods to the proposed setting, while requiring neither task-specific supervision nor additional learned components. Project website: https://tev-fbk.github.io/PROSE/

Abstract

Explore similar work

Prior-free relative 6D pose estimation of multiple object instances

Robust 6-DoF Object Pose Tracking with Built-In Recovery under Occlusions and Rapid Object Motions

RRTrack: Robust and Recoverable Object 6D Pose Tracking for Dynamic Scenes