DMM-Align: Closed-Loop Optimization for 2D-3D Registration with Dual-Role Diffusion

Chongjian Wang; Junjie Gao

cs.CVAug 17, 2026

DMM-Align: Closed-Loop Optimization for 2D-3D Registration with Dual-Role Diffusion

Authors: Chongjian Wang, Junjie Gao

Organizations: Shandong Women’s University Jinan, China · Shandong University of Science and Technology Qingdao, China

Abstract

2D-3D registration remains brittle in challenging scenarios such as low overlap, occlusion, repetitive structures, and severe cross-modal ambiguity. A key reason is that existing methods improve representation learning, correspondence estimation, or pose computation in isolation, while the dominant failure mode is inherently cross-level, where errors propagate between features, correspondences, and pose. To address this limitation, we propose DMM-Align: Diffusion-based Matching Matrix Alignment, a closed-loop framework that couples correspondence refinement, pose estimation, and representation learning through a shared differentiable geometric state. Our method leverages diffusion in two coordinated roles: a geometry-aware diffusion process refines the soft matching matrix for robust correspondence estimation, while a geometry-conditioned diffusion teacher injects pose-induced supervision back into feature learning. These processes are connected via a differentiable geometric hinge that converts correspondences into a global pose and exposes geometric inconsistency to upstream modules. Extensive experiments on 7-Scenes and RGB-D Scenes V2 demonstrate that DMM-Align consistently outperforms strong baselines, especially under low-overlap and heavy-occlusion conditions, highlighting the effectiveness of closed-loop geometric feedback for robust 2D-3D registration.

Figures & tables

Figure 1. Comparison of correspondence confidence maps for a query 3D point. We select a point in a structurally repetitive region and visualize its matching confidence over the image as predicted by Diff2I2P and our method. Our method produces a response that is more concentrated near the ground-truth correspondence.

Figure 2. Overview of DMM-Align. Given a query image and a scene point cloud, a detection-free coarse-to-fine backbone extracts multi-scale features and constructs an initial soft 2D–3D matching matrix. A geometry-aware diffusion module refines the correspondence structure within local candidate subspaces. The refined matches are converted into a global pose via a differentiable geometric hinge, which also produces pose-induced cues such as depth, visibility, and reprojection consistency. These cues are fed into a geometry-conditioned diffusion teacher to distill pose-level supervision back to the feature space, forming a closed-loop optimization across features, correspondences, and pose. During inference, the distillation branch is removed, and the pipeline reduces to feature extraction, matching refinement, and pose estimation.

Model	Chess	Fire	Heads	Office	Pump	Kitchen	Stairs	Mean
Inlier Ratio $\uparrow$
FCGF ( Choy2019FullyCG )	34.2	32.8	14.8	26.0	23.3	22.5	6.0	22.8
P2Net ( Wang2021P2NetJD )	55.2	46.7	13.0	36.2	32.0	32.8	5.8	31.7
Predator ( Huang2020PREDATORRO )	34.7	33.8	16.6	25.9	23.1	22.2	7.5	23.4
MATR ( Li20232D3DMATR2M )	72.1	66.0	31.3	60.7	50.2	52.5	18.1	50.1
Diff 2 I2P ( mu2025diff2i2p )	74.1	68.8	39.2	65.6	52.1	54.2	18.1	53.2

Table 1. Registration results of DMM-Align and baselines on 7-Scenes. The best result for each metric is shown in bold .

Figure 3. Qualitative results on the 7-Scenes dataset. The red lines indicate erroneous correspondences (3D distance greater than 5 cm), while the green lines represent correct correspondences.

Method	PIR (%) $\uparrow$	IR (%) $\uparrow$	FMR (%) $\uparrow$	RR (%) $\uparrow$
FCGF ( Choy2019FullyCG )	20.1	10.3	29.2	32.5
P2-Net ( Wang2021P2NetJD )	30.4	14.5	63.7	41.7
Predator ( Huang2020PREDATORRO )	32.6	15.8	68.1	33.6
MATR ( Li20232D3DMATR2M )	57.6	36.3	76.0	56.9
Diff 2 I2P ( mu2025diff2i2p )	60.8	36.9	77.1	60.5
DMM-Align	62.4	39.8	78.4	63.7

Table 2. Registration results of DMM-Align and baselines on RGB-D Scenes V2. The best result for each metric is shown in bold .

Model	IR $\uparrow$	FMR $\uparrow$	RR $\uparrow$
Matrix diffusion only	52.7	92.1	81.6
Distillation only	53.3	92.3	83.3
Coexistence without coupling	53.9	92.4	84.7
DMM-Align (full)	54.5	92.5	86.2

Table 3. Overall effect of DMM-Align on 7-Scenes. DMM-Align consistently outperforms reduced variants, showing the benefit of explicit closed-loop coupling.

Matching formulation	IR $\uparrow$	FMR $\uparrow$	RR $\uparrow$
Strict doubly stochastic	52.6	92.0	82.4
Partial doubly stochastic	53.1	92.1	83.6
Partial DS + dustbin	53.8	92.3	84.7
Partial DS + dustbin + geometry-aware diffusion	54.5	92.5	86.2

Table 4. Effect of matching matrix modeling on 7-Scenes. More flexible correspondence modeling and geometry-aware diffusion consistently improve performance.

Figure 4. Robustness under local point cloud degradation. From left to right, the occlusion ratio increases from $0$ to $0.6$ . (a) Input image and degraded point cloud. (b) Diff 2 I2P. (c) DMM-Align. Green and red lines denote correct and incorrect matches, respectively, and IR is reported for each case. DMM-Align consistently achieves higher IR under severe degradation.

Hinge variant	IR $\uparrow$	FMR $\uparrow$	RR $\uparrow$
Without hinge	53.2	92.1	83.8
+ 3D offset only	53.7	92.2	84.7
+ 3D offset + 2D offset	54.1	92.4	85.5
+ 3D offset + 2D offset + uncertainty weighting	54.5	92.5	86.2

Table 5. Effect of the differentiable geometric hinge on 7-Scenes. Residual correction and uncertainty-aware weighting consistently improve performance.

Distillation variant	IR $\uparrow$	FMR $\uparrow$	RR $\uparrow$
Depth only + binary mask	52.5	92.0	82.4
Depth + reprojection + soft mask	53.3	92.2	83.9
Full geometry bundle + soft mask	54.0	92.4	85.2
Full geometry bundle + uncertainty-aware soft mask	54.5	92.5	86.2

Table 6. Effect of geometry-conditioned distillation on 7-Scenes. Richer geometric cues and uncertainty-aware masking consistently improve performance.

2D-3D Image Registration

See all 21 papers

Ordered Diffusion for 3D Human Registration
Aug 6, 2026Mattia Masiero, Ilya A. Petrov, Daniel Cremers +2Point Cloud Registration 3D Diffusion Models
University of Tübingen, Germany · Tübingen AI Center, Germany · Technical University of Munich, Germany +2
Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration
Sep 16, 2026Zhixin Cheng, Jiacheng Deng, Xiaotian Yin +3Masked Autoencoders Point Cloud Registration
School of Computer Science and Technology, Hefei University of Technology, Hefei 230601, China · School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China · Institute of Advanced Technology, University of Science and Technology of China, Hefei 230027, China
UniCorrn: Unified Correspondence Transformer Across 2D and 3D
May 5, 2026Prajnan Goswami, Tianye Ding, Feng Liu +1Image Matching Point Cloud Registration
Northeastern University · Adobe Research

Aug 6, 2026J/K move · Enter open · S save

Ordered Diffusion for 3D Human Registration

Mattia Masiero, Ilya A. Petrov, Daniel Cremers, Gerard Pons-Moll+1

University of Tübingen, Germany · Tübingen AI Center, Germany · Technical University of Munich, Germany+2

Point Cloud Registration 3D Diffusion Models

Open paper AskPDF arXiv

3D human registration has historically been treated as a regression task, assuming a unique ground-truth alignment exists between the template and an input point cloud. In reality, acquisition noise, occlusions, and unknown soft tissue dynamics introduce inherent ambiguity into human scans. Regression-based methods consequently converge to an average prediction, often failing to represent a plausible geometry. In our work, we embrace such uncertainty by modeling the registration as a distribution of alignments. We propose ODin, which formulates registration as a 3D diffusion process that generates a point cloud aligned with the target geometry while preserving template semantics through consistent point ordering. To achieve this, ODin relies on global, local, and positional conditioning, guiding each point to its correct location. Our experiments demonstrate that such a generative formulation not only outperforms its regression-based baseline, but also establishes a new state of the art, surpassing highly engineered methods while reducing the registration time by two-thirds. Pre-trained models and code are available at https://riccardomarin.github.io/odin/.

Abstract

2D-3D Image Registration

Explore similar work

Ordered Diffusion for 3D Human Registration

Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration

UniCorrn: Unified Correspondence Transformer Across 2D and 3D