CLON: Cue-Calibrated Linguistic Object Onboarding for Zero-Shot 6D Pose Front-Ends

Seojin Ji; Yoojin Kwon; Hyung-Sin Kim

cs.CVSep 4, 2026

CLON: Cue-Calibrated Linguistic Object Onboarding for Zero-Shot 6D Pose Front-Ends

Authors: Seojin Ji, Yoojin Kwon, Hyung-Sin Kim

Organizations: Seoul National University

Abstract

Novel-object 6D pose pipelines depend on front-end hypotheses that retain target instances under clutter and occlusion without overwhelming downstream pose estimation. We introduce Cue-Calibrated Linguistic Object Onboarding (CLON), a training-free proposal-and-matching front-end. CLON converts multi-view CAD renderings into compact linguistic memories that provide object-specific conditioning to a frozen proposal generator. Constructed offline and reused across scenes, these memories augment generic prompts before segmentation; subsequent template verification assigns object identity. As a complementary onboarding step, CLON derives fixed cue weights from template statistics for semantic, appearance, and geometric verification. On seven BOP-Classic-Core datasets, CLON improves detection AP by 8.2 percentage points (pp), segmentation AP by 6.2 pp, and downstream pose AR by up to 4.1 pp over the evaluated CNOS and SAM-6D front-ends. In a real-world localization-to-grasp study under clutter and occlusion with 100 trials per method, CLON's improved localization doubles grasp success relative to the strongest evaluated baseline. Controlled ablations evaluate linguistic conditioning, multi-view construction, and verification. Analysis of a smaller candidate set with higher class-agnostic recall on five datasets.

Figures & tables

Fig. 1: Method Overview. Offline onboarding builds linguistic object memories for target-aware proposal generation and object-set cue weights for calibrated proposal-object scoring.

Panel of Figure 2 — Fig. 2: Summary of the fixed prompt used to generate linguistic object memories with Qwen-2.5-VL [ 15 ] .

Method	LM-O [ 24 ]	T-LESS [ 25 ]	TUD-L [ 26 ]	IC-BIN [ 27 ]	ITODD [ 28 ]	HB [ 29 ]	YCB-V [ 30 ]	$AP_{\text{mean}}$
Detection Results
CNOS (SAM) [ 7 ]	0.392	0.329	0.376	0.204	0.314	0.424	0.495	0.362
CNOS (FastSAM) [ 7 ]	0.430	0.397	0.540	0.225	0.322	0.518	0.573	0.429
SAM-6D (SAM) [ 2 ]	0.465	0.437	0.536	0.261	0.393	0.530	0.517	0.448
SAM-6D (FastSAM) [ 2 ]	0.464	0.458	0.573	0.245	0.419	0.551	0.589	0.471
CLON	0.529	0.552	0.665	0.358	0.465	0.628	0.671	0.553

TABLE I: 2D detection and segmentation results on the seven core BOP datasets. The best and second-best results within each dataset are shown in bold and underline , respectively.

Method	Detection/ Segmentation	LM-O	T-LESS	TUD-L	IC-BIN	ITODD	HB	YCB-V	$AR_{\text{mean}}$
GigaPose $\dagger$ [ 3 ]	CNOS (SAM) [ 7 ]	0.524	0.547	0.472	0.456	0.373	0.600	0.640	0.516
	CNOS (FastSAM) [ 7 ]	0.599	0.571	0.635	0.472	0.397	0.721	0.664	0.580
	SAM-6D (SAM) [ 2 ]	0.607	0.578	0.651	0.521	0.409	0.728	0.656	0.593
	SAM-6D (FastSAM) [ 2 ]	0.600	0.576	0.647	0.477	0.407	0.721	0.667	0.585
	CLON	0.620	0.673	0.685	0.567	0.414	0.778	0.699	0.634
SAM-6D [ 2 ]	CNOS (SAM) [ 7 ]	0.607	0.475	0.650	0.498	0.505	0.619	0.788	0.592

TABLE II: 6D pose estimation results using different detection/segmentation and pose estimation methods on the seven core BOP datasets. (

\dagger

denotes the use of MegaPose refinement [ 1 ] .) The best and second-best results within each dataset are shown in bold and underline , respectively.

Dataset (# targets)	Detection	# Proposal	Det Memory (GB)	Det Time (s)	Pose Est Time (s)	Total Time (s)
LM-O (8 targets)	CNOS	73	5.20	3.31	1.07	4.38
	SAM-6D	112	6.55	3.26	1.75	5.01
	CLON	22	5.86	1.02	0.44	1.46
T-LESS (30 targets)	CNOS	62	5.20	2.98	0.80	3.78
	SAM-6D	79	6.74	4.08	1.18	5.27
	CLON	12	6.37	3.25	0.24	3.49

TABLE III: Latency and memory consumption per image across BOP datasets, using SAM-6D as the pose solver. The best and second-best results within each dataset are shown in bold and underline , respectively.

Method	Metric	LM-O	T-LESS	TUD-L	IC-BIN	ITODD	HB	YCB-V
SAM-6D	Count	174	107	95	108	36	89	80
(SAM)	$AR_{\rm mean}$	0.650	0.769	0.763	0.495	–	–	0.772
CLON	Count	23	14	5	16	21	14	9
(SAM 3 + LM)	$AR_{\rm mean}$	0.654	0.818	0.778	0.511	–	–	0.837

TABLE IV: Proposal count and class-agnostic mask recall after proposal consolidation and before proposal–object verification. Proposal count is reported for all seven datasets, while

AR_{\rm mean}

is reported only for the five datasets with available ground-truth annotations and averages recall over mask IoU thresholds

[0.50:0.05:0.95]

using all retained proposals and all ground-truth instances. Dashes denote unavailable recall measurements for ITODD and HB.

(A) Proposal Generation & Prompt ( CLON verifier fixed)			(B) Verification Policy (SAM 3 + LM + three cues fixed)
Proposal Generator	Prompt	$AP_{\text{mean}}$	Verification	$AP_{\text{mean}}$
SAM	–	0.440	SAM-6D	0.540
G-DINO	Object	0.312	SAM-6D-All	0.535
G-DINO	Multi-view LM	0.449	CLON	0.553
SAM 3	Object	0.412	–	–
SAM 3	Single-view LM	0.536	–	–

TABLE V: Ablation of proposal generation, linguistic memory, view aggregation, and verification. (A) fixes the descriptor, CLON verifier, cue weights, and postprocessing while varying the proposal generator and prompt construction. (B) fixes SAM 3 and LM and compares semantic-first verification (SAM-6D), an all-object reference (SAM-6D-All) and CLON . SAM-6D-All and CLON share the proposal–object cue scores, assignment rule, and postprocessing; only their cue weights differ. G-DINO denotes Grounding DINO followed by SAM.

(A) Linguistic Prompt	$AP_{\text{mean}}$	(B) Prompt Modality	$AP_{\text{mean}}$
Object	0.412	Image	0.008
Object + Color	0.488	Image + LM	0.553
Object + Top-1	0.525	LM ( CLON )	0.553
Object + Top-1 + Color	0.544	–	–
LM ( CLON )	0.553	–	–

TABLE VI: Ablation study of linguistic prompt composition and prompt modality for CLON . The left block evaluates different compositions of linguistic prompts, while the right block compares linguistic and image-based prompt modalities. For each ablation, all other components (SAM 3 and Cue Weight) are fixed to isolate the effect of the prompt. Here, Object denotes the generic prompt set

T_{\rm gen}

Fig. 3: Qualitative results of LM on T-LESS. (a) The template image of target object (b) The LM generated for each object (c)-(e) SAM 3 segmentation with the generic prompt set, generic + color prompts, and our full LM, respectively.

Fig. 4: Real-world robot deployment under clutter and occlusion. (a) Experimental setup, (b) example test image, and (c) quantitative object detection and grasping success rate.

Fig. 5: Qualitative results of real-world robot deployment. (a) Test image, (b) target object, (c–e) detection results of CNOS, SAM-6D, and CLON , and (f) robot grasping sequence using the CLON detection result.

Object Pose Estimation

Object pose estimation recovers an object's 3D position and orientation from images, depth, point clouds, or contact sensing.

See all 187 papers

Prior-free relative 6D pose estimation of multiple object instances
Sep 8, 2026Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini +1Relative Pose Estimation Object Pose Estimation
Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy
MAPRPose: Mask-Aware Proposal and Amodal Refinement for Multi-Object 6D Pose Estimation
Apr 22, 2026Yang Luo, Yan Gong, Yongsheng Gao +2Object Pose Estimation Amodal Segmentation
aState Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin, 150001, China · bSchool of Civil Engineering, Harbin Institute of Technology, Harbin, 150001, China
Back to the Feature: Zero-Shot 6DoF Pose Estimation via Dense Local Features
Sep 6, 2026Ali Rafiaei, Michael GreenspanObject Pose Estimation Local Feature Matching
Dept. Electrical & Computer Engineering, Ingenuity Labs Queen’s University Kingston, ON, Canada

Sep 8, 2026J/K move · Enter open · S save

Prior-free relative 6D pose estimation of multiple object instances

Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini, Fabio Poiesi

Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy

Relative Pose Estimation Object Pose Estimation Cycle Consistency

Open paper AskPDF arXiv

Object 6D pose estimation formulations have progressively reduced reliance on object-specific priors, evolving from explicit 3D models to multi-view object captures to single reference images. We take this progression to its extreme by introducing prior-free relative 6D pose estimation, which lifts the assumption of knowing which object is to be posed within the scene. This novel setting aims to estimate the relative poses of multiple instances of an unknown object within the same image, without requiring CAD models, templates, or reference images. We solve this by formulating a novel method (PROSE) that finds coarse correspondences between object instances using multimodal foundation features, thus requiring no training. We refine these correspondences by imposing cycle consistency across tuples of instances, and leverage the resulting globally consistent correspondences to estimate the relative 6D pose between any pair of instances. To enable systematic evaluation, we design a novel benchmark (PRENCH) built from three multi-instance BOP datasets and enriched with task-specific metadata. PROSE consistently outperforms baselines obtained by adapting state-of-the-art single-image methods to the proposed setting, while requiring neither task-specific supervision nor additional learned components. Project website: https://tev-fbk.github.io/PROSE/

Abstract

Object Pose Estimation

Explore similar work

Prior-free relative 6D pose estimation of multiple object instances

MAPRPose: Mask-Aware Proposal and Amodal Refinement for Multi-Object 6D Pose Estimation

Back to the Feature: Zero-Shot 6DoF Pose Estimation via Dense Local Features