DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis

Xinglong Luo; Ao Luo; Zhengning Wang; Yueqi Yang; Chaoyu Feng; Lei Lei; Bing Zeng; Shuaicheng Liu

cs.CVFeb 26, 2026

DMAligner: Enhancing Image Alignment via Diffusion Model Based View Synthesis

Authors: Xinglong Luo, Ao Luo, Zhengning Wang, Yueqi Yang, Chaoyu Feng, Lei Lei, Bing Zeng, Shuaicheng Liu

Organizations: University of Electronic Science and Technology of China · Qianyuan Laboratory, Hangzhou, China · Southwest Jiaotong University · Independent Researcher

Abstract

Image alignment is a fundamental task in computer vision with broad applications. Existing methods predominantly employ optical flow-based image warping. However, this technique is susceptible to common challenges such as occlusions and illumination variations, leading to degraded alignment visual quality and compromised accuracy in downstream tasks. In this paper, we present DMAligner, a diffusion-based framework for image alignment through alignment-oriented view synthesis. DMAligner is crafted to tackle the challenges in image alignment from a new perspective, employing a generation-based solution that showcases strong capabilities and avoids the problems associated with flow-based image warping. Specifically, we propose a Dynamics-aware Diffusion Training approach for learning conditional image generation, synthesizing a novel view for image alignment. This incorporates a Dynamics-aware Mask Producing (DMP) module to adaptively distinguish dynamic foreground regions from static backgrounds, enabling the diffusion model to more effectively handle challenges that classical methods struggle to solve. Furthermore, we develop the Dynamic Scene Image Alignment (DSIA) dataset using Blender, which includes 1,033 indoor and outdoor scenes with over 30K image pairs tailored for image alignment. Extensive experimental results demonstrate the superiority of the proposed approach on DSIA benchmarks, as well as on a series of widely-used video datasets for qualitative comparisons. Our code is available at https://github.com/boomluo02/DMAligner.

Figures & tables

Figure 1: (a) Conventional image alignment based on optical flow and image warping, resulting in ghosting artifacts and occlusion. (b) Our DMAligner directly generates the complete alignment image via diffusion-based view synthesis.

Figure 2: Overview of DSIA dataset generation. The ground truth image is rendered by setting the time to $t_{2}$ and the camera pose to $P_{1}$ , where the background statics mirror those in $I_{1}$ , and the dynamics are akin to $I_{2}$ , with slight visualization variances caused by pose changes. As a result, $I_{gt}$ can be regarded as the alignment-oriented view synthesis with reference from $I_{2}$ to $I_{1}$ ( i.e., ${I_{2}}^{\prime}$ for $I_{2}$ to $I_{1}$ alignment).

Figure 3: Overview of our DMAligner. Instead of using the discriminative learning paradigm for optical flow estimation and image warping, our framework employs a generative approach to achieve image alignment with a diffusion model. Dynamics-aware Mask Producing (DMP) module is crucial for providing dynamic information, essential for performing the Dynamics-aware Diffusion Training process in this task.

Figure 4: Qualitative comparisons between DPFlow [ 34 ] , COGS [ 14 ] , GenWarp [ 41 ] and our DMAligner on our DSIA dataset.

Method	Input Structure
DPFlow [ 34 ]	2 images ( $I_{1},I_{2}$ )
COGS [ 14 ]	2 images ( $I_{1},I_{2}$ ) +
COGS [ 14 ]	Depth ( $D_{1},D_{2}$ ) + Mask ( $M_{1},M_{2}$ )
AccidentalGS [ 33 ]	2 images ( $I_{1},I_{2}$ ) +
AccidentalGS [ 33 ]	Depth ( $D_{1},D_{2}$ ) + Camera Intrinsics ( $K$ )
GenWarp [ 41 ]	Single image ( $I_{2}$ ) + Camera Intrinsics ( $K$ ) +

Table 1: Comparison of network inputs among DPFlow [ 34 ] , COGS [ 14 ] , AccidentalGS [ 33 ] , GenWarp [ 41 ] and our DMAligner. Other optical flow networks (PWCNet [ 44 ] , RAFT [ 45 ] , FlowFormer++ [ 42 ] , FlowDiffuser [ 27 ] ) have the same input structure as DPFlow.

Methods based on Flow Warping
Methods	LcLf		LcSf		ScLf		ScSf		Avg.
Methods	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
PWCNet [ 44 ]	16.53	0.74	16.40	0.73	20.53	0.74	21.01	0.76	18.62	0.74
RAFT [ 45 ]	18.22	0.74	18.34	0.74	22.46	0.77	22.95	0.76	20.49	0.75
FlowFormer++ [ 42 ]	21.02	0.75	20.97	0.76	22.84	0.77	23.94	0.79	22.19	0.77
FlowDiffuser [ 27 ]	20.18	0.75	20.33	0.75	23.39	0.78	23.91	0.79	21.95	0.77

Table 2: Quantitative comparison on our DSIA dataset across four subsets (LcLf, LcSf, ScLf, and ScSf) using PSNR and SSIM metrics, higher scores indicate better performance . The average (“Avg”) across all subsets summarizes overall results. † indicates that occlusion and ghost regions are excluded during metric computation, as determined by forward-backward consistency check [ 50 ] .

Figure 5: Qualitative comparisons between DPFlow [ 34 ] , GenWarp [ 41 ] and our DMAligner on Sintel dataset.

Figure 6: Qualitative comparisons between DPFlow [ 34 ] , GenWarp [ 41 ] and our DMAligner on the real-world DAVIS dataset.

Figure 7: Comparison of HDR results using flow-based method HDRFlow [ 49 ] , DPFlow [ 34 ] and our DMAligner. Our DMAligner significantly reduces ghosting and background distortions, especially around occluded regions and moving objects.

Methods based on Flow Warping
Methods	Sintel (Clean)		Sintel (Final)		DAVIS		Avg.
Methods	LPIPS $\downarrow$	DreamSim $\downarrow$	LPIPS $\downarrow$	DreamSim $\downarrow$	LPIPS $\downarrow$	DreamSim $\downarrow$	LPIPS $\downarrow$	DreamSim $\downarrow$
PWCNet [ 44 ]	0.251	0.146	0.177	0.109	0.217	0.102	0.215	0.119
RAFT [ 45 ]	0.245	0.141	0.171	0.102	0.219	0.104	0.212	0.116
FlowFormer++ [ 42 ]	0.264	0.150	0.184	0.113	0.224	0.106	0.224	0.123
FlowDiffuser [ 27 ]	0.258	0.141	0.179	0.103	0.211	0.090	0.216	0.111

Table 3: Quantitative comparison on Sintel and DAVIS dataset. Due to the lack of ground truth for precise evaluation, we compute LPIPS and DreamSim [ 9 ] between

I_{pred}

and

I_{1}

. Lower scores indicate better performance .

Exp.	Predict	DMP	Avg
Exp.	Targets	DMP	PSNR	SSIM
(a)	pred $\hat{\epsilon}_{t}$	w/o	24.82	0.77
(b)	pred $x_{0}$	w/o	25.12	0.79
(c)	rectified flow	w/	25.98	0.80
(d)	v-prediction	w/	26.16	0.81
(e)	pred $x_{0}$	w/	26.67	0.81

Table 4: Ablation studies of predict targets and our DMP.

Figure 8: Stress test: performance vs. motion magnitude.

Sep 21, 2026

AlignMorph: Tuning-Free Diffusion Image Morphing via Explicit Semantic Transport

Image morphing aims to produce a smooth and semantically consistent transition between two input images. Existing diffusion-based morphing methods either require expensive per-pair optimization or rely on implicit spatial alignment, which easily fails under large layout discrepancies. To address these limitations, we propose AlignMorph, a novel tuning-free diffusion framework guided by the principle of transport-then-denoise. We explicitly decouple geometric alignment from generative denoising to avoid structural entanglement. Our framework consists of two core components. (1) Global Semantic Transport, which achieves diffusion-compatible semantic alignment via entropic optimal transport and reliability-aware latent warping; and (2) Coordinate-Aligned Generation, which uses a symmetric bi-phase attention handoff to maintain consistent spatial coordinates throughout denoising. Without any tuning, AlignMorph effectively eliminates ghosting and achieves superior structural coherence and temporal smoothness on morphing benchmarks. Code is available at https://github.com/51xOne/Alignmorph.

Wuyi Liu, Xu Han, Yuren Chen +3

Huazhong University of Science and Technology · Beijing Jiaotong University · Beihang University +1

Diffusion-Based Image Generation

Ask PDF

Oct 14, 2026

ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis

Generating 3D-consistent novel views from 2D diffusion models remains a fundamental challenge in computer vision. In this work, we introduce a framework that enhances geometric alignment for monocular view synthesis under drastic camera changes by jointly generating a target view and its corresponding depth map. By conditioning the diffusion pipeline on the single source image, source depth, and target camera rays, our approach forces the model to inherently reason in 3D space even under extreme viewpoint shifts. To guarantee spatial fidelity, we propose a novel per-pixel depth consistency constraint, which verifies and enforces 3D consistency across generated views. Our framework establishes a robust pipeline for geometrically consistent view synthesis, yielding superior visual accuracy and spatial alignment compared to current state-of-the-art methods. With this work, we pave the road towards robust 3D world models capable of understanding complex environments from a single view image.

Ge Ya Luo, Daniel Severo, Brian Karrer +3

FAIR at Meta · Mila – Québec AI Institute · Université de Montréal +3

Novel View Synthesis Multi-View Image Synthesis Multi-View Consistency

Ask PDF

Apr 3, 2026

Few-Shot Distribution-Aligned Flow Matching for Data Synthesis in Medical Image Segmentation

Data heterogeneity hinders clinical deployment of medical image analysis models, and generative data augmentation helps mitigate this issue. However, recent diffusion-based methods that synthesize image-mask pairs often ignore distribution shifts between generated and real images across scenarios, and such mismatches can markedly degrade downstream performance. To address this issue, we propose AlignFlow, a flow matching model that aligns with the target reference image distribution via differentiable reward fine-tuning, and remains effective even when only a small number of reference images are provided. Specifically, we divide the training of the flow matching model into two stages: in the first stage, the model fits the training data to generate plausible images; Then, we introduce a distribution alignment mechanism and employ differentiable reward to steer the generated images toward the distribution of the given samples from the target domain. In addition, to enhance the diversity of generated masks, we also design a flow matching based mask generation to complement the diversity in regions of interest. Extensive experiments demonstrate the effectiveness of our approach, i.e., performance improvement by 3.5-4.0% in mDice and 3.5-5.6% in mIoU across a variety of datasets and scenarios.

Jie Yang, Ziqi Ye, Aihua Ke +3

Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, Wuhan, China · Shanghai Innovation Institute, Shanghai, China · Shanghai AI Laboratory, Shanghai, China

Synthetic Data Augmentation Medical Image Synthesis Distribution Matching Flow Matching Medical Image Segmentation

Ask PDF

Abstract

Explore similar work

AlignMorph: Tuning-Free Diffusion Image Morphing via Explicit Semantic Transport

ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis

Few-Shot Distribution-Aligned Flow Matching for Data Synthesis in Medical Image Segmentation