Conditional flow matching has enabled a step forward in object 6D pose estimation, achieving state-of-the-art performance by progressively denoising and registering object representations to observed scenes. Existing methods require training task-specific encoders supervised on object-scene overlap and rely on trivial feature fusion strategies to resolve pose ambiguities. We present FunFlow6D, a novel flow matching-based formulation that leverages features from geometric and appearance foundation models for pose estimation, eliminating the need for task-specific encoder training. We also introduce a cross attention-based fusion mechanism that dynamically combines geometric and appearance features to provide richer conditioning for the flow matching module. Experiments on four datasets from the BOP benchmark show that FunFlow6D outperforms the previous state of the art while reducing supervision requirements and memory overhead. Extensive ablations validate the contribution of each proposed component. Project website: https://tev-fbk.github.io/FunFlow6D/.
Figures & tables
Figure 1: \acronym estimates object 6D poses from input data (image + 3D model) via flow matching over multimodal foundational features (left). It advances over previous state-of-the-art on two fronts. First, it removes the need to train a task-specific encoder with object-scene overlap supervision (right, pink). Second, instead of relying on predefined fusion schemes such as point-wise summation, it learns how to optimally fuse multimodal features from frozen foundation models during flow matching (right, azure). As a result, \acronym achieves higher accuracy in a single training phase while requiring less supervision.
Figure 2: Overview of \acronym . Given the canonical query point cloud Q and an RGBD image I as input (top left), \acronym recovers the 6D object pose (R,t) (bottom right) through three stages: feature extraction, generative denoising, and pose estimation. Throughout, Q acts as a static anchor and the flow operates only on the target T , lifted from the masked depth of I . Feature extraction: A frozen geometric encoder Φ (dGeDi) and a frozen semantic encoder Θ (DINOv2, reduced via UMAP) produce per-point descriptors that are fused by a bidirectional gated cross-attention module (right inset) into FQ,FT . Colors encode feature similarity: corresponding regions across Q and T share similar colors. Generative denoising: A conditional flow-matching network ΨΩ (DiT), conditioned on FQ,FT , learns a velocity field that canonicalizes T into T^ while the anchor velocities are zeroed. Pose estimation: The flow-induced correspondences between T and T^ yield the camera-to-canonical transform via a RANSAC-based orthogonal Procrustes solver followed by ICP, and the pose (R,t) . denote frozen modules, while indicate trainable ones.
Figure 3: Average number of inliers obtained with different appearance encoders (left), geometric encoders (center), and dimensionality reduction techniques (right).
Method
Models
Params
LM-O
TUD-L
IC-BIN
YCB-V
Avg
1
HccePose(BF) [ 45 ]
34
41.5M
80.5
94.4
72.4
91.1
84.6
2
GDRNPP (BOP23) [ 26 ]
4
106.8M
79.4
96.4
73.7
92.8
85.6
3
Flose [ 10 ]
4
89.2M
86.1
98.8
74.8
92.9
88.2
4
\acronym (ours)
4
43.1M
87.1
98.9
75.3
93.6
88.7
5
Improvement wrt row 3
-
-
+1.0
+0.1
+0.5
+0.7
+0.5
Table 1: 6D pose accuracy on the BOP benchmark, evaluated in terms of AR.
Figure 4: \acronym (right) vs. Flose [ 10 ] (center) on LM-O (a) and YCB-V (b). \acronym predicts more accurate poses under severe occlusions and symmetry ambiguities.
Figure 5: Inference time analysis on LM-O. (a) Average per-image runtime against competing methods. Object instance segmentation is excluded for all methods. (b) Per-image breakdown of \acronym . The Euler integration (50 steps) dominates the pipeline.
Geom. Enc.
App. Enc.
Dim. Red.
Feat. fusion
Pose estim.
Pose refin.
AR
1
overlap-aware
-
-
-
SVD
ICP
83.5
2
overlap-aware
DINOv2
PCA
Point-level sum
RANSAC
ICP
86.1
3
dGeDi
DINOv2
PCA
Point-level sum
RANSAC
ICP
81.8
4
dGeDi
DINOv2
UMAP
Point-level sum
RANSAC
ICP
83.6
5
dGeDi
DINOv2
UMAP
No attention
RANSAC
ICP
85.7
6
dGeDi
DINOv2
UMAP
Cross attention
RANSAC
ICP
86.3
Table 2: Ablation study on LM-O [ 3 ] . Key: = trained, = frozen, - = not used. The default configuration of \acronym is highlighted .
Existing methods for instance-level 6D pose estimation typically rely on neural networks that either directly regress the pose in SE(3) or estimate it indirectly via local feature matching. The former struggle with object symmetries, while the latter fail in the absence of distinctive local features. To overcome these limitations, we propose a novel formulation of 6D pose estimation as a conditional flow matching problem in R3. We introduce Flose, a generative method that infers object poses via a denoising process conditioned on local features. While prior approaches based on conditional flow matching perform denoising solely based on geometric guidance, Flose integrates appearance-based semantic features to mitigate ambiguities caused by object symmetries. We further incorporate RANSAC-based registration to handle outliers. We validate Flose on five datasets from the established BOP benchmark. Flose outperforms prior methods with an average improvement of +4.5 Average Recall. Project Website : https://tev-fbk.github.io/Flose/
Amir Hamza, Davide Boscaini, Weihang Li +2
Fondazione Bruno Kessler · University of Trento · Technical University of Munich +1
6D pose estimation is a key task in computer vision and embodied AI, widely used in robotic manipulation, augmented reality, etc. Existing methods directly regress in a high-dimensional continuous space, facing two key challenges in category-level pose estimation: limited accuracy due to noise and local optima, and inefficient search over an infinite space that hinders real-time performance. This paper proposes Flow6D, a hierarchical flow matching framework with a two-stage discrete latent space localization-continuous pose regression strategy. Rotation and translation parameters are first discretized into bins, with a discrete flow matching model locking the latent space around the true pose to reduce search complexity. Then, by sampling in the latent space, a continuous flow matching model predicts local pose residuals to optimize the estimate and regress to an accurate pose. The framework also naturally extends to articulated objects, outperforming state-of-the-art methods on synthetic and real datasets with real-time inference at 70 FPS. Project website: https://flow6d.github.io/.
Mingyu Mei, Li Zhang, Zibo Dai +4
Zhejiang University · University of Science and Technology of China · Shanghai Jiao Tong University
Object 6D pose estimation formulations have progressively reduced reliance on object-specific priors, evolving from explicit 3D models to multi-view object captures to single reference images. We take this progression to its extreme by introducing prior-free relative 6D pose estimation, which lifts the assumption of knowing which object is to be posed within the scene. This novel setting aims to estimate the relative poses of multiple instances of an unknown object within the same image, without requiring CAD models, templates, or reference images. We solve this by formulating a novel method (PROSE) that finds coarse correspondences between object instances using multimodal foundation features, thus requiring no training. We refine these correspondences by imposing cycle consistency across tuples of instances, and leverage the resulting globally consistent correspondences to estimate the relative 6D pose between any pair of instances. To enable systematic evaluation, we design a novel benchmark (PRENCH) built from three multi-instance BOP datasets and enriched with task-specific metadata. PROSE consistently outperforms baselines obtained by adapting state-of-the-art single-image methods to the proposed setting, while requiring neither task-specific supervision nor additional learned components. Project website: https://tev-fbk.github.io/PROSE/
Behdad Khodabandehloo, Andrea Caraffa, Davide Boscaini +1
Fondazione Bruno Kessler, Trento, Italy · University of Trento, Trento, Italy