ASM-6D: Fast Certified Pose-Shape Inference with Complete Correspondence Flow Maps

Ho Jin Choi; Minku Kim; Nadia Figueroa

cs.CVOct 11, 2026

ASM-6D: Fast Certified Pose-Shape Inference with Complete Correspondence Flow Maps

Authors: Ho Jin Choi, Minku Kim, Nadia Figueroa

Organizations: University of Pennsylvania, Philadelphia, PA, USA · Oregon State University, Corvallis, OR, USA

Abstract

Category-level 6D pose and shape estimation requires recovering an object's pose and geometry from partial observations without an exact instance model. We present ASM-6D, which couples generative complete-geometry prediction with explicit category-level pose-shape inference through shared correspondences. Given an RGB-D image, ASM-6D generates alternative complete 3D correspondence hypotheses whose point identities are shared with a category-level Active Shape Model (ASM), including occluded points. Each hypothesis is fitted directly for rotation, translation, metric scale, and instance shape without a separate correspondence-assignment stage. We analytically eliminate translation and shape coefficients, reducing fitting to a nine-variable scaled-rotation optimization problem that enables efficient batched evaluation of multiple hypotheses in real time. ASM-6D achieves state-of-the-art performance on ROPE and HouseCat6D benchmarks. Ablations demonstrate the importance of generative correspondence prediction and active shape variation. The representation also supports shape completion, shape-dependent pose ambiguity, tracking, and robotic grasping at 6-38 Hz, depending on the inference budget.

Figures & tables

Fig. 1: ASM-6D generates alternative complete indexed 3D correspondence hypotheses, whose shared point identities enable direct ASM fitting for pose, scale, and instance shape with a computation rate of 6–38 Hz.

Fig. 2: Overview of ASM-6D. Standardized training shapes define category-level Active Shape Models (Sec III ). Given an RGB-D crop, the model generates multiple hypotheses of the complete indexed correspondence set (Sec IV ). Each hypothesis is reconstructed in metric coordinates and fitted to the known ASM or, when the category is unknown, to a finite library of candidate priors. A visibility-weighted fitting residual selects the final pose, metric scale, shape, and, when applicable, category (Sec V ). Online computation time of 33–147 ms from input RGB-D to output shape, pose, and class prediction.

Fig. 3: Shape standardization. A common 642-vertex icosphere template is deformed onto each normalized object surface while preserving vertex indices.

Method	AUC $\uparrow$			VUS $\uparrow$				Mean error $\downarrow$
Method	IoU 25	IoU 50	IoU 75	$5^{\circ}$ /2 cm	$5^{\circ}$ /5 cm	$10^{\circ}$ /2 cm	$10^{\circ}$ /5 cm	Rot. ( ∘ )	Trans. (cm)
Prior work
NOCS [ 1 ]	0.0	0.0	0.0	0.0	0.0	0.0	0.0	–	–
SGPA [ 31 ]	10.5	2.0	0.0	4.3	6.7	9.3	15.0	60.2	2.81
IST-Net [ 32 ]	28.7	10.6	0.5	2.0	3.4	5.3	8.8	78.4	3.40
HS-Pose [ 33 ]	31.6	13.6	1.1	3.5	5.3	8.4	12.7	63.3	3.02

TABLE I: Category-level pose estimation on ROPE and ASM-6D adaptation and distillation variants.

Fig. 4: Qualitative comparison on Omni6DPose ROPE . From top to bottom: ground truth, GenPose++ (GP++), and ASM-6D. Predicted 3D bounding boxes and coordinate frames visualize the pose, while ASM-6D additionally recovers the fitted object shape from its generated correspondences.

Factor	Setting	IoU 50 AUC $\uparrow$	$10^{\circ}$ /5 cm VUS $\uparrow$
Prediction	Regression ( $H=1$ )	11.58	16.98
Prediction	Flow matching ( $H=1$ )	25.91	29.24
Hypotheses	$H=1$	30.07	37.73
	$H=4$	30.59	38.65
	$H=40$	30.95	39.14
Shape model	Category mean	19.82	34.63

TABLE II: Core design ablations on ROPE.

$H$	Known single category				All $156$ priors
$H$	3 iter.	5 iter.	10 iter.	20 iter.	3 iter.	5 iter.	10 iter.	20 iter.
1	38.85	38.14	37.19	34.04	28.66	28.37	27.05	25.11
4	31.50	31.32	30.51	30.24	22.93	23.03	22.04	20.65
8	26.26	26.04	26.14	25.73	18.97	18.88	18.33	17.24
16	18.51	18.75	18.64	18.54	13.30	13.14	12.88	12.43
32	12.28	12.29	12.22	12.16	8.43	8.34	8.26	8.02

TABLE III: Full-pipeline throughput across hypothesis and ADMM iteration budgets in FPS.

Method	Shape	IoU 25 $\uparrow$	IoU 50 $\uparrow$	$5^{\circ}$ /2 $\uparrow$	$5^{\circ}$ /5 $\uparrow$	$10^{\circ}$ /2 $\uparrow$	$10^{\circ}$ /5 $\uparrow$
AG-Pose [ 34 ]	–	88.1	76.9	21.3	22.1	51.3	54.3
SpotPose [ 35 ]	–	89.1	77.0	23.8	24.5	52.3	54.8
GCE-Pose [ 36 ]	✓	–	79.2	24.8	25.7	55.4	58.4
ComPose [ 4 ]	✓	90.3	80.6	25.8	27.6	57.8	61.5
ASM-6D (HC6D)	✓	91.0	85.7	30.2	32.7	64.3	71.1
Additional SOPE pose-task pretraining

TABLE IV: Category-level pose estimation on HouseCat6D.

Fig. 5: ASM solver latency versus keypoint count with $40$ hypotheses, $32$ PCA modes, and $20$ ADMM iterations. Columns compare CPU and CUDA, while rows compare one-prior fitting with exhaustive fitting over all $156$ priors. Timings include problem construction, optimization, scaled-rotation projection, and parameter reconstruction, but exclude correspondence inference and JIT compilation.

			SOPE test, stride 3 ( $n=339{,}840$ )			RealNOCS real_test ( $n=14{,}077$ )
Method	Input	Prior	F1@1% $\uparrow$	CD-L1(m) $\downarrow$	HD95(m) $\downarrow$	F1@1% $\uparrow$	CD-L1(m) $\downarrow$	HD95(m) $\downarrow$
PoinTr [ 14 ]	M-PC	–	0.0521	0.1181	0.2416	0.0277	0.0452	0.0809
SnowflakeNet [ 15 ]	M-PC	–	0.0498	0.0782	0.1721	0.0215	0.0509	0.1014
AdaPoinTr [ 16 ]	M-PC	–	0.0445	0.1045	0.1763	0.0146	0.0543	0.0867
DR-Pose † [ 37 ]	M-PC	mean	–	–	–	0.0524	0.0195	0.0232
ASM-6D w/o prior	ROI	–	0.1749	0.0114	0.0146	0.0516	0.0185	0.0199

TABLE V: Shape-completion results. M-PC : instance-mask segmented depth point cloud; ROI : RGB-D crop and box.

Fig. 6: Full-pipeline RealNOCS completion at the native $642$ -point metric budget. All outputs are shown in a common ground-truth camera frame and scene scale. Point-cloud methods use the released Mask R-CNN partial cloud, whereas ASM-6D uses the object ROI without mask pixels. RGB-D colors are used only for visualization.

Fig. 7: Shape-conditioned pose ambiguity for one cup under three visibility conditions . Each point represents an independently sampled correspondence set after ASM fitting. Mollweide position and color encode rotation, while columns separate geometry-derived handle-free and handle-bearing completions.

Dataset / class	Top-1	Correct / total
RealNOCS (all)	73.7%	11884/16117
Bottle	47.0%	1295/2757
Bowl	89.0%	2109/2370
Camera	64.8%	1729/2669
Can	43.0%	1174/2728
Laptop	100.0%	2754/2754

TABLE VI: Exact canonical-class selection through exhaustive ASM fitting.

Panel of Figure 14 — Fig. 8: Representative exact-class errors. Left: RealNOCS camera $\rightarrow$ handbag. Right: SOPE knife $\rightarrow$ fork. Each view shows the isolated RGB-D observation and selected incorrect ASM fit; the inset shows the ground-truth mesh for context.

Fig. 9: Tracking on the high-motion mug sequence. Rows show ground truth, FoundationPose (FP) with periodic re-registration, and ASM-6D over time. The comparison illustrates the temporal behavior of the evaluated tracking configurations under rapid object motion (mean translation speed: $0.197~\mathrm{m\,s^{-1}}$ , mean rotation speed: $88.2^{\circ}\,\mathrm{s^{-1}}$ ).

Fig. 10: Qualitative object grasping experiments. In the tabletop scene, ASM-6D estimates the pose and shape of target objects among multiple surrounding objects, enabling grasping of the bottle and bowl.

Method	Trans. (cm) $\downarrow$		Rot. ( ∘ ) $\downarrow$		ADD (cm) $\downarrow$
	Med.	P90	Med.	P90	Med.
ASM-6D	0.46	2.60	4.2	53.9	0.57
FP (native)	0.11	0.24	1.9	7.2	0.20
FP (30f reinit.)	0.13	1.16	2.7	179.6	0.23

TABLE VII: Tracking accuracy on HO3D, pooled over scored frames.

Method	Trans. (cm) $\downarrow$		Rot. ( ∘ ) $\downarrow$		ADD (cm) $\downarrow$	FPS $\uparrow$
	Med.	P90	Med.	P90	Med.	Steady / Amort.
ASM-6D	1.45	2.74	12.3	35.4	1.56	28.3 / 26.4
FP (native)	63.60	83.80	144.4	165.2	62.12	64.0 / 55.3
FP (30f reinit.)	3.03	7.80	64.8	163.7	4.29	55.3 / 14.5
FP (60f reinit.)	3.06	11.14	72.8	163.1	4.47	55.0 / 22.5
FP (120f reinit.)	3.30	40.00	48.7	161.4	4.16	55.1 / 30.2

TABLE VIII: Tracking accuracy and throughput on the high-motion mug sequence. ∗ FP is FoundationPose [ 39 ]

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	AUC $\uparrow$			VUS $\uparrow$
Configuration	IoU 25	IoU 50	IoU 75	$5^{\circ}$ /2cm	$5^{\circ}$ /5cm	$10^{\circ}$ /2cm	$10^{\circ}$ /5cm
A) Regression vs. flow matching, frozen $H=1$
Regression	27.87	11.58	0.93	4.38	6.81	10.88	16.98
Flow Matching	43.91	25.91	5.10	10.29	14.45	20.56	29.24
B) Hypothesis count, distilled model
$H=1$	47.94	30.07	7.11	14.67	20.54	26.70	37.73

Appendix

TABLE IX: Ablation study of ASM-6D.

Fig. 11: RGB-only input ablation. Top: representative paired RGB-D and RGB-only predictions. Bottom: predicted global depth across $50$ objects compared with observed metric depth.

Object Pose Estimation

Object pose estimation recovers an object's 3D position and orientation from images, depth, point clouds, or contact sensing.

See all 187 papers

LEGAU: Learning Semantic Gaussian Priors for Scalable Category-level Pose Estimation
Sep 28, 2026Hongli Xu, Zhaowei Lu, Junwen Huang +5Object Pose Estimation 3D Representation Learning
Technical University of Munich Munich, Germany · Munich Center for Machine Learning Munich, Germany · Siemens AG Munich, Germany +1
RYOPO: Bringing End-to-End Category-Level Object Pose Estimation into Real Time
Oct 2, 2026Hakjin Lee, Junghoon Seo, Jaehoon SimObject Pose Estimation 3D Instance Segmentation
PIT IN Co., Anyang-si, Gyeonggi-do, Republic of Korea.
Flow6D: Discrete-to-Continuous Flow Matching for Efficient and Accurate Category-Level 6D Pose Estimation
Jun 22, 2026Mingyu Mei, Li Zhang, Zibo Dai +4Flow Matching Object Pose Estimation
Zhejiang University · University of Science and Technology of China · Shanghai Jiao Tong University

Sep 28, 2026J/K move · Enter open · S save

LEGAU: Learning Semantic Gaussian Priors for Scalable Category-level Pose Estimation

Hongli Xu, Zhaowei Lu, Junwen Huang, Jiaqi Hu+4

Technical University of Munich Munich, Germany · Munich Center for Machine Learning Munich, Germany · Siemens AG Munich, Germany+1

Object Pose Estimation 3D Representation Learning

Open paper AskPDF arXiv

Category-level 6D pose estimation from a single RGB-D observation is inherently under-constrained, since partial visible geometry must be interpreted together with a canonical object structure before a stable pose can be determined. We present LEGAU, a unified framework that jointly predicts NOCS correspondence, object pose and size, and a canonical Semantic Gaussian Field. Rather than treating reconstruction as a detached auxiliary task, LEGAU uses the Gaussian field as a category-conditioned structural prior that participates in multimodal feature fusion and provides global guidance for local pose reasoning. Conditioned on a categorical text embedding, LEGAU processes RGB-D observations through a transformer-based fusion module that integrates visual, geometric, and category-level cues, decoding the NOCS map, pose and size information and the Gaussian-based object representation. Extensive experiments on synthetic and real-world benchmarks show that this coupled pose-shape formulation achieves strong performance in a single-model multi-category setting, with up to 22% on SOPE and competitive transfer to real-world data. These results highlight the benefit of jointly learning canonical correspondence, object shape, and pose alignment within a unified representation.

Abstract

Object Pose Estimation

Explore similar work

LEGAU: Learning Semantic Gaussian Priors for Scalable Category-level Pose Estimation

RYOPO: Bringing End-to-End Category-Level Object Pose Estimation into Real Time

Flow6D: Discrete-to-Continuous Flow Matching for Efficient and Accurate Category-Level 6D Pose Estimation