Back to the Feature: Zero-Shot 6DoF Pose Estimation via Dense Local Features

Ali Rafiaei; Michael Greenspan

cs.CVSep 6, 2026

Back to the Feature: Zero-Shot 6DoF Pose Estimation via Dense Local Features

Authors: Ali Rafiaei, Michael Greenspan

Organizations: Dept. Electrical & Computer Engineering, Ingenuity Labs Queen’s University Kingston, ON, Canada

Abstract

We present B2TFPose, a training-free zero-shot method for 6DoF pose estimation of unseen objects from RGB images. Using a single frozen DINOv3 vision transformer as its only pretrained component within the pose estimation pipeline, B2TFPose extracts dense patch-level features that generalize across the synthetic-to-real domain gap without any task-specific fine-tuning, revisiting the classical local feature matching paradigm through the lens of large-scale self-supervised foundation models. Three contributions advance the training-free state of the art. A geodesic non-maximum suppression strategy retrieves a viewpoint-diverse template set for coarse-to-fine correspondence matching. Render-guided Re-Correspondence (RRC) synthesizes object-specific views at the estimated pose and re-establishes dense 2D-3D correspondences to sharpen the initial estimate without additional learned parameters. A multi-mask hypothesis selection strategy jointly scores competing segmentation candidates to resolve segmentation ambiguity. On the seven core datasets of the BOP Benchmark, B2TFPose achieves 40.7 mean AR without refinement and 56.4 with refinement, establishing state-of-the-art performance among training-free RGB methods and outperforming trained counterparts including GigaPose and GenFlow, at competitive inference speed.

Figures & tables

Figure 1: Overview of B2TFPose. Offline stage (top-left): the 3D object model is rendered from 162 uniformly distributed viewpoints and augmented with 12 in-plane rotations to yield 1944 templates, each encoded by a frozen DINOv3 backbone into a global 1024-D descriptor for coarse retrieval and a $32{\times}32$ grid of local patch descriptors for dense matching; both are stored in a template database. Online stage (bottom): the input RGB image is segmented and encoded by the same DINOv3 backbone; global feature similarity retrieves the most viewpoint-diverse candidate templates from the database, and mutual nearest-neighbor patch matching establishes 2D-3D correspondences that are passed to RANSAC+PnP for an initial 6DoF pose estimate. Render-guided Re-Correspondence (top-right): ten views are rendered at orientation variants around the initial estimate, re-encoded by DINOv3, and re-matched against the query patch features to produce a second, tighter set of 2D-3D correspondences; a final RANSAC+PnP yields the refined pose.

Training-free methods (no task-specific training)
Method	Refinement	Dataset							Mean
Method	Refinement	LM-O	T-LESS	TUD-L	IC-BIN	ITODD	HB	YCB-V	Mean
FoundPose [ 26 ]	-	39.6	33.8	46.7	23.9	20.4	50.8	45.2	37.2
B2TFPose (Ours)	-	38.9	34.6	58.1	22.8	23.4	58.3	48.7	40.7
FoundPose [ 26 ]	Feat + MegaPose	55.7	51.0	63.3	43.3	35.7	69.7	66.1	55.0
B2TFPose (Ours)	RRC + MegaPose	58.6	52.0	69.6	40.0	35.8	73.1	66.1	56.4

Table 1: Comparison with state-of-the-art zero-shot RGB pose estimation methods on the seven BOP core datasets. Values show the BOP Average Recall. Methods are grouped by whether they require training on object-specific or task-specific data. “Refinement” indicates use of a dedicated pose refiner. Dashed lines separate the with and without refinement sub-groups within each category. All methods use CNOS [ 24 ] for instance segmentation. All results are taken either from the original papers or the BOP leaderboard.

Figure 2: Speed-accuracy trade-off across the seven BOP datasets. Orange markers denote performance without pose refinement; blue markers denote performance with pose refinement. Dashed lines connect the two conditions for the same method. The star (

\bigstar

) marks our method (B2TFPose). Inference time is measured per object per image on TUD-L. The hollow marker for FoundPose (with refinement) indicates that the reported time is taken from the BOP leaderboard and was measured on different hardware, as its full-pipeline code is not publicly released. Co-op [ 40 ] and GenFlow [ 39 ] are excluded as their code is not publicly released. Upper-left is better.

#	RRC	MegaPose	Dataset							Mean
#	RRC	MegaPose	LM-O	T-LESS	TUD-L	IC-BIN	ITODD	HB	YCB-V	Mean
1	✗	✗	38.9	34.6	58.1	22.8	23.4	58.3	48.7	40.7
2	✓	✗	43.2	37.0	57.3	22.5	25.2	57.0	49.5	41.7
3	✗	✓	56.2	50.0	69.6	40.5	33.6	73.7	65.1	55.5
4	✓	✓	58.6	52.0	69.6	40.0	35.8	73.1	66.1	56.4

Table 2: Ablation of the RRC and MegaPose refinement stages. Row 1 is the base pipeline with no refinement; rows 2-4 add RRC alone, MegaPose alone, and both in combination. AR (%) is reported per dataset and as the mean across all seven BOP core datasets.

Figure 3: Effect of three key hyperparameters on TUD-L AR. (a) Number of retrieved templates

k

: performance peaks at

k\!=\!125

; using all 1944 templates is marginally worse as the geodesic diversity constraint loses force for large

k

. (b) PnP reprojection threshold: the optimal is 4 px; tighter thresholds reject valid correspondences while looser ones admit outlier correspondences. (c) Number of patch neighbors: accuracy saturates at 3 neighbors and remains stable beyond; 3 is adopted as the default as values of 5 and above reach only 0.1 AR higher at greater retrieval cost.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 4 — Figure 4: B2TFPose qualitative results. The 3D object model is overlaid at the estimated pose. The method achieves accurate pose estimates across diverse object geometries and scene conditions; when the initial estimate is slightly off, refinement recovers the correct pose.

Panel of Figure 5 — Figure 5: B2TFPose failure cases. B2TFPose degrades under rotational symmetry, severe occlusion, textureless appearance, and inter-object ambiguity in bin-picking scenarios; inherent limitations of appearance-based matching without explicit depth cues.

Panel of Figure 6 — Figure 6: RRC visualization for three object instances. Top row of each case: query image and pose overlays at four pipeline stages. Bottom row of each case: the ten pose variants rendered at the initial estimate and used by RRC for a second round of 2D-3D correspondences.

Explore similar work

TinyDETR-Pose: Towards End-to-End Real-Time Single-Stage 6DoF Object Pose Estimation with Lightweight Transformers
Aug 15, 2026Paul Julius Kühn, Duc Anh Nguyen, Saptarshi Neil Sinha +2Object Pose Estimation Edge Inference
Fraunhofer IGD 64283 Darmstadt, Germany · TU Darmstadt, 64289 Darmstadt, Germany · TU Delft 2628 CD Delft, Netherlands
PIXIE: A Zero-Shot texture-invariant 6D pose estimation framework for unseen objects with assembly defects
Jul 17, 2026Leon Jungemeyer, Alejandro Magaña, Gautham Mohan +2Object Pose Estimation Robotic Perception
Carl Zeiss AG, Oberkochen, Germany · Carl Zeiss Digital Innovation, Munich, Germany
Prior-free relative 6D pose estimation of multiple object instances
Sep 8, 2026Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini +1Relative Pose Estimation Object Pose Estimation
Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy

Aug 15, 2026, cs.CVJ/K move · Enter open · S save

TinyDETR-Pose: Towards End-to-End Real-Time Single-Stage 6DoF Object Pose Estimation with Lightweight Transformers

Paul Julius Kühn, Duc Anh Nguyen, Saptarshi Neil Sinha, Michael Weinmann+1

Fraunhofer IGD 64283 Darmstadt, Germany · TU Darmstadt, 64289 Darmstadt, Germany · TU Delft 2628 CD Delft, Netherlands

Object Pose Estimation Edge Inference Detection Transformer

Open paper AskPDF arXiv

Real-time 6DoF object pose estimation on resource-constrained hardware remains challenging, as accurate correspondence-based and refinement pipelines typically rely on non-differentiable PnP/RANSAC stages or costly iterative refinement, while recent foundation-model-based approaches incur inference costs that are prohibitive for edge deployment. We present TinyDETR-Pose, a lightweight, end-to-end, single-stage framework that jointly detects objects and regresses their full 6D pose in a single forward pass. Built on the efficient LW-DETR architecture, TinyDETR-Pose formulates detection and pose estimation as a set-prediction problem and attaches dedicated MLP heads for rotation, monocular depth, and projected object center regression to each decoder query, eliminating the need for PnP, NMS (non-maximum suppression), or iterative pose refinement. Object symmetries are handled through a ADD-S loss applied uniformly to all objects, without the need for object-specific loss schedules or separate geodesic/ADD supervision. In addition, predictions are assigned to ground truth using a symmetry-safe Hungarian matcher based on class and 2D spatial cues, yielding stable assignment under symmetry and depth ambiguity. On YCB-V, TinyDETR-Pose achieves a comparable ADD-S AUC of 85.9, while requiring up to 72.7% fewer parameters than other DETR-based single-stage pose-estimation approaches. Due to its compact design, TinyDETR-Pose runs in real time and achieves an inference latency of only ~4.5 ms per frame on an NVIDIA Jetson Nano using TensorRT, demonstrating that accurate end-to-end transformer-based 6D pose estimation can be made practical for edge deployment.

Abstract

Explore similar work

TinyDETR-Pose: Towards End-to-End Real-Time Single-Stage 6DoF Object Pose Estimation with Lightweight Transformers

PIXIE: A Zero-Shot texture-invariant 6D pose estimation framework for unseen objects with assembly defects

Prior-free relative 6D pose estimation of multiple object instances