CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration

Xuecong Liu; Mengzhu Ding; Zixuan Sun; Zhang Li; Xichao Teng

cs.CVApr 7, 2026

CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration

Authors: Xuecong Liu, Mengzhu Ding, Zixuan Sun, Zhang Li, Xichao Teng

Organizations: Northeastern University, China · National University of Defense Technology, China

Abstract

We present Consistent-Recurrent Feature Flow Transformer (CRFT), a unified coarse-to-fine framework based on feature flow learning for robust cross-modal image registration. CRFT learns a modality-independent feature flow representation within a transformer-based architecture that jointly performs feature alignment and flow estimation. The coarse stage establishes global correspondences through multi-scale feature correlation, while the fine stage refines local details via hierarchical feature fusion and adaptive spatial reasoning. To enhance geometric adaptability, an iterative discrepancy-guided attention mechanism with a Spatial Geometric Transform (SGT) recurrently refines the flow field, progressively capturing subtle spatial inconsistencies and enforcing feature-level consistency. This design enables accurate alignment under large affine and scale variations while maintaining structural coherence across modalities. Extensive experiments on diverse cross-modal datasets demonstrate that CRFT consistently outperforms state-of-the-art registration methods in both accuracy and robustness. Beyond registration, CRFT provides a generalizable paradigm for multimodal spatial correspondence, offering broad applicability to remote sensing, autonomous navigation, and medical imaging. Code and datasets are publicly available at https://github.com/NEU-Liuxuecong/CRFT.

Figures & tables

Figure 1: Overview of the proposed CRFT framework. CRFT follows a unified coarse-to-fine pipeline for robust cross-modal image registration. (a) Multi-Scale Feature Extraction: A CNN encoder extracts modality-independent features at $\{1/2,1/4,1/8\}$ resolutions from the input image pair $(I^{A},I^{B})$ . (b) Coarse-Scale Flow Estimation: At 1/8 resolution, transformer-style self-attention (SA) and cross-attention (CA) operate on low-resolution features to compute global correlation and generate an initial coarse flow field. (c) Fine-Scale Flow Refinement: The 1/2 and 1/4 features are fused through window-based self-attention and cross-attention to inject fine-grained spatial details and locally refine coarse correspondences. (d) Iterative Discrepancy-Guided Flow Optimization: Within $N$ refinement iterations, fine-scale features undergo Fine-Scale Feature Transformation (FSFT), aligned via a Spatial Geometric Transform (SGT), and compared to compute feature discrepancies. These discrepancies guide the Discrepancy-Guided Flow Optimization (DGFO), followed by a Residual Update (RU) and a Confidence Estimation Network (CENet) for smoothing, producing the final high-precision dense flow.

Figure 2: Iterative discrepancy-guided flow refinement. At each iteration, fine-scale features from both modalities are first mapped into a shared feature space (FSFT). The current flow then drives a SGT to align the local window of $F_{5\times 5}^{B}$ with $F_{5\times 5}^{A}$ . Feature discrepancies between the aligned windows are computed and used to weight an attention-based refinement module, which predicts a residual flow added to the current estimate. A confidence network further smooths the updated flow before the next iteration. This geometry-aware, discrepancy-driven process progressively improves cross-modal alignment.

Figure 3: Visual comparison between the predicted flow and ground-truth flow fields. The proposed CRFT achieves geometrically consistent and dense alignment across challenging optical-SAR and optical-infrared image pairs, demonstrating strong robustness to nonlinear radiation and geometric variations.

Figure 4: Quantitative comparison with state-of-the-art methods. For each image pair, 5000 candidate correspondences are uniformly sampled, and only the matches with registration error below 2 pixels are visualized. The density and spatial consistency of the displayed inliers reflect the alignment accuracy of different methods across both OSdataset and RoadScene. CRFT produces the largest number of geometrically consistent inliers, demonstrating clear advantages in cross-modal and affine-deformed scenarios.

Panel of Figure 5 — Figure 5: CMR curves under varying thresholds on (a) OSdataset and (b) RoadScene. We compare CRFT with representative handcrafted, sparse, and (semi-)dense matching methods by evaluating the CMR across a range of pixel thresholds. CRFT consistently achieves the highest CMR across a wide range of thresholds.

Methods	AEPE $\downarrow$	CMR $\uparrow$ (%)
Methods	AEPE $\downarrow$	$@3px$	$@1px$	$@0.7px$
HOWP [ 64 ] ISPRS’23	26.91	14.2	0.0	0.0
LNIFT [ 22 ] TGRS’22	48.18	22.9	0.0	0.0
MSG [ 66 ] TGRS’25	44.38	28.1	0.0	0.0
RIFT2 [ 20 ] TIP’20	23.61	22.9	0.0	0.0
GMFlow [ 58 ] TPAMI’23	11.91	17.0	0.0	0.0

Table 1: Quantitative comparison on the OSdataset . AEPE and CMR at thresholds

t=\{3,1,0.7\}

. Best results are in bold , and second-best are underlined .

Methods	AEPE $\downarrow$	CMR $\uparrow$ (%)
Methods	AEPE $\downarrow$	$@3px$	$@1px$	$@0.7px$
HOWP [ 64 ] ISPRS’23	22.01	31.8	0.0	0.0
LNIFT [ 22 ] TGRS’22	24.24	30.7	0.0	0.0
MSG [ 66 ] TGRS’25	50.87	45.5	0.0	0.0
RIFT2 [ 20 ] TIP’20	17.27	36.4	0.0	0.0
GMFlow [ 58 ] TPAMI’23	12.02	18.2	0.0	0.0

Table 2: Quantitative comparison on the RoadScene . AEPE and CMR at thresholds

t=\{3,1,0.7\}

Figure 6: Ablation study on OSdataset. We evaluate the contribution of each component in CRFT by progressively adding the flow estimation (FE), iterative discrepancy-guided optimization (IDGO), and the Iterative Loss (IL) to the XoFTR baseline. The CMR curves across varying thresholds demonstrate that each module brings a clear performance gain, while integrating all modules (X+FE+IDGO+IL) yields the most significant improvement, especially under strict pixel thresholds.

Method	AEPE $\downarrow$	CMR $\uparrow$ (%)
Method	AEPE $\downarrow$	$@3px$	$@1px$	$@0.7px$
(1) X	3.76	23.6	0.0	0.0
(2) X+L1	0.51	80.9	73.3	66.7
(3) X+FE+L1	0.63	99.0	93.4	73.3
(4) X+FE+IDGO+L1	0.43	99.7	97.2	88.9
(5) X+FE+IDGO+IL (CRFT)	0.40	99.7	97.2	93.1

Table 3: Ablation experiments on OSdataset . AEPE and CMR at thresholds

t=\{3,1,0.7\}

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Flow prediction visualization on OSdataset. For each pair, we visualize the predicted dense flow, the ground-truth flow, and the corresponding difference map. CRFT produces smooth and geometrically consistent flow fields across optical-SAR modality gap, indicating accurate geometric correspondence and robustness to strong appearance variations

Figure 8: Flow prediction visualization on RoadScene. For each pair, we visualize the predicted dense flow, the ground-truth flow, and the corresponding difference map. CRFT maintains stable and consistent flow behavior across heterogeneous modalities, demonstrating robustness to illumination changes, noise patterns, and texture inconsistencies in complex driving environments.

Figure 9: Qualitative comparison on the OSdataset (top) and RoadScene (Bottom). The red quadrilateral denotes the ground-truth alignment, while the yellow quadrilateral shows the predicted registration result. CRFT achieves more accurate and geometrically consistent registration under large geometric deformation and modality gaps compared with existing optical-SAR registration baselines.

Figure 10: Checkerboard registration on OSdataset (left) and RoadScene (right). CRFT yields geometrically coherent checkerboard fusion across both optical-SAR and optical-infrared modalities. Zoomed-in regions demonstrate precise alignment of boundaries and textures, confirming the model’s ability to recover fine-scale geometric correspondence under large cross-modal appearance differences.

Explore similar work

RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration
Jun 29, 2026Mengzhu Ding, Xin Song, Xiaoke Ding +2Optical Flow Bayesian Inference
Northeastern University, China
Disentangle-then-Align: Non-Iterative Hybrid Multimodal Image Registration via Cross-Scale Feature Disentanglement
Mar 20, 2026Chunlei Zhang, Jiahao Xia, Yun Xiao +2Cross-Modal Image Registration Deformable Image Registration
Faculty of Engineering and IT, University of Technology Sydney, Australia · School of Artificial Intelligence, Anhui University, Hefei, China · School of Computer Science and Technology, Anhui University, Hefei, China
Geometry-Aligned Semantic Matching for Cross-Modal Planar Image Registration
Oct 2, 2026Zhiwei Wang, Defeng He, Yuxing Li +2Image Matching Cross-Modal Image Registration

Jun 29, 2026J/K move · Enter open · S save

RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration

Mengzhu Ding, Xin Song, Xiaoke Ding, Hongwei Ding+1

Northeastern University, China

Optical Flow Bayesian Inference Cross-Modal Image Registration

Open paper AskPDF arXiv

Cross-modal image registration is essential for multi-sensor perception but remains fundamentally challenging due to severe non-linear radiometric discrepancies and geometric distortions. Existing deterministic matching methods lack uncertainty awareness, struggling to navigate the resulting highly non-convex optimization landscape and frequently accumulating errors in ambiguous regions. In this paper, we propose RBE-Flow, a novel framework that reformulates dense cross-modal flow estimation as a closed-loop recurrent Bayesian estimation problem on learned feature manifolds. Diverging from standard feed-forward regression, RBE-Flow establishes a robust self-correcting mechanism by deeply coupling feature-metric non-linear optimization with probabilistic state updates. Specifically, a Recurrent Manifold Optimization (RMO) block iteratively generates flow observations and their associated uncertainties, which are then optimally assimilated into the prior state via an Uncertainty-Adaptive Probabilistic Update (UAPU) using deterministic sigma-point projection. Crucially, the resulting calibrated posterior covariance is fed back to adaptively regularize the damping of subsequent optimization steps, allowing the system to modulate its convergence based on predictive confidence. To ensure stable probabilistic training, we introduce a hybrid supervision scheme featuring a geometry-aware rectified NLL loss that structurally prevents variance collapse. Extensive experiments on challenging OSdataset, WHU-OPT-SAR, and RoadScene benchmarks demonstrate that RBE-Flow consistently achieves state-of-the-art performance, outperforming existing methods by a significant margin, particularly under strict sub-pixel criteria. Project page: https://github.com/NEU-Liuxuecong/RBE-Flow

Abstract

Explore similar work

RBE-Flow: Recurrent Bayesian Estimation on Feature Manifolds for Cross-Modal Registration

Disentangle-then-Align: Non-Iterative Hybrid Multimodal Image Registration via Cross-Scale Feature Disentanglement

Geometry-Aligned Semantic Matching for Cross-Modal Planar Image Registration