Real-time 6DoF object pose estimation on resource-constrained hardware remains challenging, as accurate correspondence-based and refinement pipelines typically rely on non-differentiable PnP/RANSAC stages or costly iterative refinement, while recent foundation-model-based approaches incur inference costs that are prohibitive for edge deployment. We present TinyDETR-Pose, a lightweight, end-to-end, single-stage framework that jointly detects objects and regresses their full 6D pose in a single forward pass. Built on the efficient LW-DETR architecture, TinyDETR-Pose formulates detection and pose estimation as a set-prediction problem and attaches dedicated MLP heads for rotation, monocular depth, and projected object center regression to each decoder query, eliminating the need for PnP, NMS (non-maximum suppression), or iterative pose refinement. Object symmetries are handled through a ADD-S loss applied uniformly to all objects, without the need for object-specific loss schedules or separate geodesic/ADD supervision. In addition, predictions are assigned to ground truth using a symmetry-safe Hungarian matcher based on class and 2D spatial cues, yielding stable assignment under symmetry and depth ambiguity. On YCB-V, TinyDETR-Pose achieves a comparable ADD-S AUC of 85.9, while requiring up to 72.7% fewer parameters than other DETR-based single-stage pose-estimation approaches. Due to its compact design, TinyDETR-Pose runs in real time and achieves an inference latency of only ~4.5 ms per frame on an NVIDIA Jetson Nano using TensorRT, demonstrating that accurate end-to-end transformer-based 6D pose estimation can be made practical for edge deployment.
Figures & tables
Figure 1: Overview of TinyDETR-Pose. Given a single RGB image, our model jointly detects all objects and regresses their full 6D pose in one forward pass via set prediction.
Figure 2: Architecture of TinyDETR-Pose. A CAEv2-pretrained ViT backbone [ 33 ] feeds multi-scale features into a 3-layer transformer decoder operating on N learnable object queries. Hungarian matching [ 34 ] assigns predictions to ground truth during training, using our projected object-center cost (Sec. 3.1 ). Each matched query is decoded by shared, parallel MLP heads for class, box, rotation, and translation, see Sec. 3.2 for details on the translation decomposition and rotation parameterization. ★ differs from T6D-Direct [ 18 ] / RACE-6D [ 32 ] .
Figure 3: Pose estimation head. Rotation is predicted in 6D [ 24 ] and orthogonalized to a 3×3 matrix. Translation follows [ 35 ] : the object center is predicted in normalized UV space for Hungarian matching, while tz is estimated as depth. Back-projecting UV and tz through K recovers the translation; combined with the predicted rotation, this yields the full 6DoF pose.
Method
RGB-only
Refinement
Real-time
Params (M)
Num.
AUC of
AUC of
Decoders
ADD(-S) ↑
ADD-S ↑
Refinement-based
CosyPose (ECCV’20) [ 25 ]
✓
✓
✗
–
–
84.5
89.8
DeepIM (ECCV’18) [ 47 ]
✓
✓
✗
–
–
81.9
88.1
Direct regression
GDR-Net (CVPR’21) [ 9 ]
✓
✗
✓
–
–
84.4
89.1
Table 1: Comparison on YCB-V. ⋄ marks single-stage DETR-based methods; decoder layers and parameter counts are reported where available. TinyDETR-Pose achieves competitive ADD-S AUC with 14.5M parameters, 70.2–72.7% fewer than the DETR-based baselines with reported model sizes.
Method
AR
ARVSD
ARMSSD
ARMSPD
ms
CosyPose (ECCV’20) [ 25 ]
65.5
–
–
–
395
YOLO-6D-Pose (s) (3DV’24) [ 16 ]
70.8
65.9
72.3
74.2
16.9
RACE-6D (CVPR’26) [ 32 ]
76.7
70.3
79.9
79.8
12
TinyDETR-Pose ∗ (Ours)
34.4
41.2
35.3
26.6
7
Table 2: Comparison on the YCB-V BOP benchmark using the official BOP metrics (Average Recall %). Higher is better for all AR metrics, while lower is better for inference time. Runtime values are reported by the respective methods. ∗ Results: https://bop.felk.cvut.cz/sub_info/41155/
Figure 4: Qualitative results of TinyDETR-Pose on YCB-V [ 8 ] . Each row shows two successful predictions followed by one representative failure case. Failure cases: Green denotes ground-truth poses, and red denotes predictions from our model. The failure cases are mainly caused by strong occlusion.
Setting
AUC ofADD(-S) ↑
AUC ofADD-S ↑
(a) λcntrC
0.0
63.1
84.5
1.0
62.9
84.9
2.5
63.3
85.3
5.0
64.6
85.9
(b) λADD-SL
10.0
57.1
79.5
20.0
58.5
81.0
Table 3: Sensitivity of pose accuracy to the matching-cost (a) and loss weights (b,c) on YCB-V. One factor is varied at a time; all remaining weights are kept at the configuration described in Sec. 3 .
Method
Backbone
Num. Decoders
ms/frame ↓
AUC of ADD-S ↑
CosyPose † [ 25 ]
EfficientNet-B3
–
395
89.8
GDR-Net ‡ [ 9 ]
ResNet-34
–
65
89.1
YOLO-6D-Pose (s) ∗ [ 16 ]
CSPDarkNet-s
–
16.9
–
T6D-Direct [ 18 ]
ResNet-50
6
17
86.2
YOLOPoseV2 [ 19 ]
ResNet-50
6
17
90.1
YOLOPoseV2-A [ 19 ]
ResNet-50
6
22
91.2
Table 4: Runtime comparison on YCB-V. † : refinement-based, ‡ : direct-regression, ∗ : NMS post-processing; all others are end-to-end. If reported for DETR-based methods, decoder layers are reported. Latency is given in ms/frame. TinyDETR-Pose maintains comparable AUC of ADD-S with substantially lower latency, further improved by TensorRT deployment on the NVIDIA Jetson Nano.
Metric
AUC of ADD-S
AUC of ADD(-S)
Object
TinyDETR- Pose (ours)
PoET [ 17 ]
YOLO Pose [ 19 ]
YOLO Pose-A [ 19 ]
T6D [ 18 ]
TinyDETR- Pose (ours)
PoET [ 17 ]
YOLO PoseV2 [ 19 ]
YOLO PoseV2-A [ 19 ]
T6D [ 18 ]
master chef can
90.3
88.4
91.3
91.7
91.1
64.6
-
64.0
71.3
61.5
cracker box
78.5
80.5
86.8
92.0
86.6
6.5
-
77.9
83.3
76.3
sugar box
83.6
92.4
92.6
91.5
90.3
67.1
-
87.3
83.6
81.8
tomato soup can
89.7
91.4
90.5
87.8
88.9
64.7
-
77.8
72.9
72.0
mustard bottle
88.8
91.7
93.6
96.6
94.7
62.8
-
87.9
93.4
85.7
Table 5: Per-object results on YCB-V. ∗ marks symmetric objects. We compare single-stage end-to-end DETR-based methods using AUC of ADD-S and ADD(-S), where available. Results within 3.0 AUC points of the state-of-the-art are highlighted in blue .
Multi-person pose estimation (MPPE), which involves detecting body joint positions (keypoints) for every person in an image, is a fundamental task in computer vision. Despite recent advances, no transformer-based model currently achieves real-time performance. This work addresses the latency challenge by introducing DETRPose, the first family of real-time, end-to-end transformer models for multi-person 2D pose estimation. DETRPose significantly enhances the GroupPose decoder, enabling real-time inference. For training, a novel denoising keypoint technique is proposed to accelerate convergence. The varifocal loss is also extended for keypoints, termed Keypoint Similarity VariFocal loss, to improve query quality. Extensive evaluation demonstrates that DETRPose models achieve accuracy comparable to or exceeding that of leading alternatives while requiring five to ten times fewer training epochs. DETRPose-S matches the accuracy of YOLOv8-Pose-X and YOLO11-Pose-X on the COCO dataset (67.0 vs 67.3 and 67.2 in AP) with 81% fewer parameters (11.5M vs 69.4M and 58.8M) and 52% faster inference speed (2.39ms vs 5.23ms and 4.93ms). On the CrowdPose dataset, DETRPose-X has 13.1× fewer FLOPs (232.3G vs 3048.1G) and only 2 fewer precision (75.1 vs 76.6 in AP) than ED-Pose-SwinL-5S. On the OCHuman dataset, DETRPose-S surpasses all previous models, showing the robustness of DETRPose on out-of-distribution datasets. Code is available at https://github.com/SebastianJanampa/DETRPose
We present B2TFPose, a training-free zero-shot method for 6DoF pose estimation of unseen objects from RGB images. Using a single frozen DINOv3 vision transformer as its only pretrained component within the pose estimation pipeline, B2TFPose extracts dense patch-level features that generalize across the synthetic-to-real domain gap without any task-specific fine-tuning, revisiting the classical local feature matching paradigm through the lens of large-scale self-supervised foundation models. Three contributions advance the training-free state of the art. A geodesic non-maximum suppression strategy retrieves a viewpoint-diverse template set for coarse-to-fine correspondence matching. Render-guided Re-Correspondence (RRC) synthesizes object-specific views at the estimated pose and re-establishes dense 2D-3D correspondences to sharpen the initial estimate without additional learned parameters. A multi-mask hypothesis selection strategy jointly scores competing segmentation candidates to resolve segmentation ambiguity. On the seven core datasets of the BOP Benchmark, B2TFPose achieves 40.7 mean AR without refinement and 56.4 with refinement, establishing state-of-the-art performance among training-free RGB methods and outperforming trained counterparts including GigaPose and GenFlow, at competitive inference speed.
Ali Rafiaei, Michael Greenspan
Dept. Electrical & Computer Engineering, Ingenuity Labs Queen’s University Kingston, ON, Canada
Category-level object pose estimation predicts the rotation, translation, and metric size of unseen instances within known categories. Many accurate RGB-D methods rely on external instance segmentation and crop-based pose estimation, introducing separate stages and object-dependent processing costs that hinder real-time inference. To bring accurate pose estimation into real time, we present \ours{}, an end-to-end trainable query-based RGB-D set predictor. It jointly detects and segments objects and estimates their \mbox{9-DoF} poses without explicit CAD-derived shape priors or a separately trained instance segmentor. Shared image and scene encoding avoids repeated per-object crop encoding. A query-conditioned geometry pathway associates observed 3D points and RGB features with object queries and incorporates shared scene context. Object-centric refinement uses the resulting point descriptors to update an explicit pose state through pose-conditioned cross-attention and recurrent residual corrections. On NOCS, \ours{} substantially improves on published RGB-D joint detection and pose estimation results. It achieves competitive performance compared with two-stage methods under all-object evaluation on REAL275 and HouseCat6D, while enabling real-time full-frame pose estimation at 31.8 FPS on an RTX~A6000. Project page: https://yopo-series.github.io/RYOPO-project-page/.
Hakjin Lee, Junghoon Seo, Jaehoon Sim
PIT IN Co., Anyang-si, Gyeonggi-do, Republic of Korea.