Foundational feature fusion for conditional flow matching in 6D pose estimation

Amir Hamza; Davide Boscaini; Fabio Poiesi

cs.CVAug 29, 2026

Foundational feature fusion for conditional flow matching in 6D pose estimation

Authors: Amir Hamza, Davide Boscaini, Fabio Poiesi

Organizations: Fondazione Bruno Kessler Trento, Italy · University of Trento Trento, Italy

Abstract

Conditional flow matching has enabled a step forward in object 6D pose estimation, achieving state-of-the-art performance by progressively denoising and registering object representations to observed scenes. Existing methods require training task-specific encoders supervised on object-scene overlap and rely on trivial feature fusion strategies to resolve pose ambiguities. We present FunFlow6D, a novel flow matching-based formulation that leverages features from geometric and appearance foundation models for pose estimation, eliminating the need for task-specific encoder training. We also introduce a cross attention-based fusion mechanism that dynamically combines geometric and appearance features to provide richer conditioning for the flow matching module. Experiments on four datasets from the BOP benchmark show that FunFlow6D outperforms the previous state of the art while reducing supervision requirements and memory overhead. Extensive ablations validate the contribution of each proposed component. Project website: https://tev-fbk.github.io/FunFlow6D/.

Figures & tables

Figure 1: \acronym estimates object 6D poses from input data (image + 3D model) via flow matching over multimodal foundational features (left). It advances over previous state-of-the-art on two fronts. First, it removes the need to train a task-specific encoder with object-scene overlap supervision (right, pink). Second, instead of relying on predefined fusion schemes such as point-wise summation, it learns how to optimally fuse multimodal features from frozen foundation models during flow matching (right, azure). As a result, \acronym achieves higher accuracy in a single training phase while requiring less supervision.

Figure 2: Overview of \acronym . Given the canonical query point cloud $\mathcal{Q}$ and an RGBD image I as input (top left), \acronym recovers the 6D object pose $(\textbf{R},\textbf{t})$ (bottom right) through three stages: feature extraction, generative denoising, and pose estimation. Throughout, $\mathcal{Q}$ acts as a static anchor and the flow operates only on the target $\mathcal{T}$ , lifted from the masked depth of I . Feature extraction: A frozen geometric encoder $\Phi$ (dGeDi) and a frozen semantic encoder $\Theta$ (DINOv2, reduced via UMAP) produce per-point descriptors that are fused by a bidirectional gated cross-attention module (right inset) into $\textbf{F}^{\mathcal{Q}},\textbf{F}^{\mathcal{T}}$ . Colors encode feature similarity: corresponding regions across $\mathcal{Q}$ and $\mathcal{T}$ share similar colors. Generative denoising: A conditional flow-matching network $\Psi_{\Omega}$ (DiT), conditioned on $\textbf{F}^{\mathcal{Q}},\textbf{F}^{\mathcal{T}}$ , learns a velocity field that canonicalizes $\mathcal{T}$ into $\hat{\mathcal{T}}$ while the anchor velocities are zeroed. Pose estimation: The flow-induced correspondences between $\mathcal{T}$ and $\hat{\mathcal{T}}$ yield the camera-to-canonical transform via a RANSAC-based orthogonal Procrustes solver followed by ICP, and the pose $(\textbf{R},\textbf{t})$ . denote frozen modules, while indicate trainable ones.

Figure 3: Average number of inliers obtained with different appearance encoders (left), geometric encoders (center), and dimensionality reduction techniques (right).

	Method	Models	Params	LM-O	TUD-L	IC-BIN	YCB-V	Avg
1	HccePose(BF) [ 45 ]	34	41.5M	80.5	94.4	72.4	91.1	84.6
2	GDRNPP (BOP23) [ 26 ]	4	106.8M	79.4	96.4	73.7	92.8	85.6
3	Flose [ 10 ]	4	89.2M	86.1	98.8	74.8	92.9	88.2
4	\acronym (ours)	4	43.1M	87.1	98.9	75.3	93.6	88.7
5	Improvement wrt row 3	-	-	+1.0	+0.1	+0.5	+0.7	+0.5

Table 1: 6D pose accuracy on the BOP benchmark, evaluated in terms of AR.

Panel of Figure 4 — Figure 4: \acronym (right) vs. Flose [ 10 ] (center) on LM-O (a) and YCB-V (b). \acronym predicts more accurate poses under severe occlusions and symmetry ambiguities.

Panel of Figure 5 — Figure 5: Inference time analysis on LM-O. (a) Average per-image runtime against competing methods. Object instance segmentation is excluded for all methods. (b) Per-image breakdown of \acronym . The Euler integration (50 steps) dominates the pipeline.

	Geom. Enc.	App. Enc.	Dim. Red.	Feat. fusion	Pose estim.	Pose refin.	AR
1	overlap-aware	-	-	-	SVD	ICP	83.5
2	overlap-aware	DINOv2	PCA	Point-level sum	RANSAC	ICP	86.1
3	dGeDi	DINOv2	PCA	Point-level sum	RANSAC	ICP	81.8
4	dGeDi	DINOv2	UMAP	Point-level sum	RANSAC	ICP	83.6
5	dGeDi	DINOv2	UMAP	No attention	RANSAC	ICP	85.7
6	dGeDi	DINOv2	UMAP	Cross attention	RANSAC	ICP	86.3

Table 2: Ablation study on LM-O [ 3 ] . Key: = trained, = frozen, - = not used. The default configuration of \acronym is highlighted .

Object Pose Estimation

Object pose estimation recovers an object's 3D position and orientation from images, depth, point clouds, or contact sensing.

See all 184 papers

Generative 6D Pose Estimation via Conditional Flow Matching
Feb 23, 2026Amir Hamza, Davide Boscaini, Weihang Li +2Object Pose Estimation Conditional Flow Matching
Fondazione Bruno Kessler · University of Trento · Technical University of Munich +1
Flow6D: Discrete-to-Continuous Flow Matching for Efficient and Accurate Category-Level 6D Pose Estimation
Jun 22, 2026Mingyu Mei, Li Zhang, Zibo Dai +4Flow Matching Object Pose Estimation
Zhejiang University · University of Science and Technology of China · Shanghai Jiao Tong University
Prior-free relative 6D pose estimation of multiple object instances
Sep 8, 2026Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini +1Relative Pose Estimation Object Pose Estimation
Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy

Feb 23, 2026J/K move · Enter open · S save

Generative 6D Pose Estimation via Conditional Flow Matching

Amir Hamza, Davide Boscaini, Weihang Li, Benjamin Busam+1

Fondazione Bruno Kessler · University of Trento · Technical University of Munich+1

Object Pose Estimation Conditional Flow Matching

Open paper AskPDF arXiv

Existing methods for instance-level 6D pose estimation typically rely on neural networks that either directly regress the pose in

\mathrm{SE}(3)

or estimate it indirectly via local feature matching. The former struggle with object symmetries, while the latter fail in the absence of distinctive local features. To overcome these limitations, we propose a novel formulation of 6D pose estimation as a conditional flow matching problem in

\mathbb{R}^3

. We introduce Flose, a generative method that infers object poses via a denoising process conditioned on local features. While prior approaches based on conditional flow matching perform denoising solely based on geometric guidance, Flose integrates appearance-based semantic features to mitigate ambiguities caused by object symmetries. We further incorporate RANSAC-based registration to handle outliers. We validate Flose on five datasets from the established BOP benchmark. Flose outperforms prior methods with an average improvement of +4.5 Average Recall. Project Website : https://tev-fbk.github.io/Flose/

Abstract

Object Pose Estimation

Explore similar work

Generative 6D Pose Estimation via Conditional Flow Matching

Flow6D: Discrete-to-Continuous Flow Matching for Efficient and Accurate Category-Level 6D Pose Estimation

Prior-free relative 6D pose estimation of multiple object instances