Modality-Invariant Coarse-to-Fine Retinal Image Registration

Bo Wen; Nehal Nailesh Mehta; Melanie Tran; Dirk-Uwe Bartsch; William Freeman; Truong Nguyen

eess.IVAug 14, 2026

Modality-Invariant Coarse-to-Fine Retinal Image Registration

Authors: Bo Wen, Nehal Nailesh Mehta, Melanie Tran, Dirk-Uwe Bartsch, William Freeman, Truong Nguyen

Abstract

Retinal image registration is essential for ophthalmic diagnosis, longitudinal disease monitoring, and multimodal retinal image analysis. Existing retinal registration methods are typically modality-dependent: they are designed or optimized either for a single imaging modality in mono-modal registration or for a fixed pair of modalities in cross-modal registration. This limits their flexibility and applicability in practical scenarios involving diverse retinal imaging modalities and different combinations of them. In this work, we propose a generalizable two-stage, modality-invariant framework for retinal image registration. First, we introduce a sparse feature-matching model driven by a universal retinal vessel segmentation to achieve robust coarse global alignment across modalities. Second, we develop a modality-invariant optical flow estimation network, termed MI-RAFT, to refine the alignment through dense local registration. Extensive experiments demonstrate that the proposed method can handle diverse combinations of commonly used retinal imaging modalities, exhibiting strong modality invariance while outperforming state-of-the-art modality-dependent registration methods.

Figures & tables

Fig. 1: Modality-invariant coarse (global, linear)-to-fine (local, deformable) retinal image registration (best zoom in to view and viewed in color).

Fig. 2: Pipeline of the proposed modality-invariant coarse/global retinal image registration.

Fig. 3: Proposed Modality Invariant (MI)-RAFT model architecture (top); Dual-pair synthetic training strategy with correlation lookup feature alignment (bottom left) and inference procedure of the proposed MI-RAFT model/fine alignment (bottom right).

Fig. 4: Two toy cross-modal retinal image pairs for Table I .

	Measures	RAFT	DINO	Fused	Fused-CLFA
Toy Pair 1	$M_{HN}$	0.5808	0.0218	0.5288	0.5367
(Fig. 4 L)	cos-sim	0.1281	0.8707	0.2303	0.3003
Toy Pair 2	$M_{HN}$	0.5041	0.0232	0.4865	0.4743
(Fig. 4 R)	cos-sim	0.1189	0.8951	0.2707	0.3053

TABLE I: Toy examples on two real image pairs illustrating how feature characteristics evolve within MI-RAFT to improve both modality invariance and local discriminability of the correlation landscape.

M_{\mathrm{HN}}

denotes the hard-negative margin, and cos-sim denotes cosine similarity. RAFT and DINO denote the raw outputs of the RAFT feature encoder and DINO feature extractor, respectively; Fused denotes the adapted fused features used to construct the correlation volume; and Fused-CLFA denotes the corresponding features from the model trained with the proposed Correlation-Lookup Feature Alignment (CLFA) strategy.

Method/Dataset	FIRE (CF-CF)		CF-IR		CF-FA		FAF-MC
Method/Dataset	$Dice$	$Dice_{s}$	$Dice$	$Dice_{s}$	$Dice$	$Dice_{s}$	$Dice$	$Dice_{s}$
SuperPoint [ 34 ]	0.2593( $\pm$ 0.1663)	0.2915	0.0397( $\pm$ 0.0304)	0.0602	0.1121( $\pm$ 0.0373)	0.2139	0.0967( $\pm$ 0.0458)	0.1314
SuperPoint+SG [ 42 ]	0.6190( $\pm$ 0.1926)	0.6317	0.1653( $\pm$ 0.1801)	0.2026	0.2997( $\pm$ 0.1665)	0.3960	0.4347( $\pm$ 0.1793)	0.4690
SuperPoint+ORN [ 34 , 35 ]	0.6542( $\pm$ 0.1906)	0.6611	0.4442( $\pm$ 0.2644)	0.4626	0.4071( $\pm$ 0.2502)	0.4916	0.5239( $\pm$ 0.1401)	0.5499
Y. Wang et al. [ 22 ]	0.6200( $\pm$ 0.1564)	0.6332	0.5724( $\pm$ 0.1837)	0.6091	0.4257( $\pm$ 0.1665)	0.5091	0.3715( $\pm$ 0.1963)	0.4057
SuperRetina [ 3 ]	0.4745( $\pm$ 0.2534)	0.4957	0.3691( $\pm$ 0.2474)	0.3985	0.0871( $\pm$ 0.0542)	0.1647	0.3417( $\pm$ 0.1892)	0.3779

TABLE II: Quantitative comparison of coarse (sparse-feature-based) alignment with state-of-the-art methods. ORN denotes the same pretrained outlier-rejection network used in our method [ 22 , 35 ] , and SG denotes SuperGlue [ 42 ] . Unless otherwise specified, each baseline uses the default feature matcher/outlier rejection from its original implementation. Results are reported as mean

\pm

standard deviation.

Fig. 5: Qualitative comparison with state-of-the-art methods in coarse global modality-invariant retinal image registration. On the vessel overlap maps, yellow shows overlay-ed vessels, red and green shows non-overlayed vessels from source and target images, respectively.

Method/Dataset	FIRE (CF-CF)		CF-IR		CF-FA		FAF-MC
Method/Dataset	$Dice$	$Dice_{s}$	$Dice$	$Dice_{s}$	$Dice$	$Dice_{s}$	$Dice$	$Dice_{s}$
Coarse Registration	0.6688( $\pm$ 0.1943)	0.6866	0.6005( $\pm$ 0.1667)	0.6324	0.6228( $\pm$ 0.0928)	0.7103	0.5379( $\pm$ 0.1249)	0.5634
MI-B-Splines [ 45 , 46 ]	0.7022( $\pm$ 0.1047)	0.7110	0.6153( $\pm$ 0.1712)	0.6345	0.6529( $\pm$ 0.1087)	0.7294	0.5578( $\pm$ 0.1300)	0.5865
RAFT [ 36 ]	0.7559( $\pm$ 0.1189)	0.7657	0.6162( $\pm$ 0.1554)	0.6406	0.6373( $\pm$ 0.1012)	0.7166	0.5700( $\pm$ 0.1100)	0.6093
CrossRAFT [ 26 ]	0.7246( $\pm$ 0.1481)	0.7342	0.6225( $\pm$ 0.1692)	0.6421	0.6395( $\pm$ 0.1076)	0.7242	0.5697( $\pm$ 0.1118)	0.6079
Seg-DeformNet [ 5 ]	0.6915( $\pm$ 0.1588)	0.7067	0.6531( $\pm$ 0.1726)	0.6638	0.4162( $\pm$ 0.0907)	0.5220	0.5503( $\pm$ 0.1158)	0.5824

TABLE III: Quantitative comparison of fine (dense-feature/optical-flow-based) alignment with state-of-the-art methods.

Fig. 6: Qualitative comparison of fine (dense-correspondence- and optical-flow-based) alignment with state-of-the-art methods and comparison with the proposed coarse alignment. All methods are initialized from the same coarse alignment produced by our proposed coarse-registration method, where $I_{\mathrm{src}}^{\mathrm{coarse}}$ denotes the resulting coarsely aligned source image.

Fig. 7: Ablation study of additional training strategies for MI-RAFT. DA+ denotes the low-frequency illumination and focus/resolution perturbations, while CLFA denotes the proposed Correlation-Lookup Feature Alignment. Results are reported as mean Dice scores.

Fig. 8: Ablation study of the effect of the CLFA loss weight $\lambda_{\mathrm{CLFA}}$ on final registration performance.

Dice	FIRE	CF-IR	CF-FA	FAF-MC
Vanilla MI-RAFT	0.7715	0.6604	0.6616	0.6364
GMM Clusering [ 16 ]	0.7592	0.6531	0.6546	0.6216
Pre-Corr Alignment	0.7632	0.6559	0.6486	0.6203
CLFA	0.7761	0.6642	0.6701	0.6443

TABLE IV: Comparison of alternative feature-alignment strategies against the proposed CLFA method using DINOv3 as the modality-invariant feature extractor.

Dice	FIRE	CF-IR	CF-FA	FAF-MC
Proposed Design	0.7715	0.6604	0.6616	0.6364
ACBP	0.7696	0.6583	0.6550	0.6284
Corr. Lookup	0.7730	0.6601	0.6608	0.6353

TABLE V: Ablation study of where to fuse the RAFT and modality-invariant (MI) features. ACBP denotes fusion after computing the top-level correlation maps but before downsampling them to construct the full correlation pyramid.

Dice	FIRE	CF-IR	CF-FA	FAF-MC
Pretrained CLe [ 22 , 35 ]	0.6688	0.6005	0.6228	0.5379
Finetuned CLe [ 22 , 35 ]	0.6689	0.5990	0.6210	0.5367
Finetuned SuperGlue [ 42 ]	0.6510	0.5813	0.6037	0.5155

TABLE VI: Ablation study on different choices of outlier rejection method in the proposed modality-invariant coarse registration model.

		Inference Time (s)
Coarse	Segmentation (URVSM)	0.0238( $\pm$ 0.0162)
Coarse	Registration	0.0308( $\pm$ 0.0195)
Fine Registration (MI-RAFT-DINO)		0.1095( $\pm$ 0.0147)
Fine Registration (MI-RAFT-DIFT)		0.2034( $\pm$ 0.0180)

TABLE VII: Inference time of the proposed modality-invariant coarse and fine retinal image registration algorithms.

Medical Image Registration

Medical image registration aligns scans across time, patients, or imaging modalities to track anatomy and support diagnosis, treatment, and image-guided procedures.

See all 95 papers

CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration
Apr 7, 2026Xuecong Liu, Mengzhu Ding, Zixuan Sun +2Cross-Modal Image Registration Cross-Modal Feature Matching
Northeastern University, China · National University of Defense Technology, China
Adapting Frozen Mono-modal Backbones for Multi-modal Registration via Contrast-Agnostic Instance Optimization
Mar 27, 2026Yi Zhang, Yidong Zhao, Qian TaoDeformable Image Registration Cross-Modal Image Registration
Department of Imaging Physics, Delft University of Technology, The Netherlands
Search-MIND: Training-Free Multi-Modal Medical Image Registration
Apr 10, 2026Boya Wang, Ruizhe Li, Chao Chen +1Medical Image Registration Cross-Modal Image Registration
Intelligent Modelling & Analysis Group (IMA), School of Computer Science, University of Nottingham, UK · Lab for Uncertainty in Data and Decision Making (LUCID), Nottingham Biomedical Research Centre (BRC), School of Medicine, University of Nottingham, UK

Apr 7, 2026J/K move · Enter open · S save

CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration

Xuecong Liu, Mengzhu Ding, Zixuan Sun, Zhang Li+1

Northeastern University, China · National University of Defense Technology, China

Cross-Modal Image Registration Cross-Modal Feature Matching

Open paper AskPDF arXiv

We present Consistent-Recurrent Feature Flow Transformer (CRFT), a unified coarse-to-fine framework based on feature flow learning for robust cross-modal image registration. CRFT learns a modality-independent feature flow representation within a transformer-based architecture that jointly performs feature alignment and flow estimation. The coarse stage establishes global correspondences through multi-scale feature correlation, while the fine stage refines local details via hierarchical feature fusion and adaptive spatial reasoning. To enhance geometric adaptability, an iterative discrepancy-guided attention mechanism with a Spatial Geometric Transform (SGT) recurrently refines the flow field, progressively capturing subtle spatial inconsistencies and enforcing feature-level consistency. This design enables accurate alignment under large affine and scale variations while maintaining structural coherence across modalities. Extensive experiments on diverse cross-modal datasets demonstrate that CRFT consistently outperforms state-of-the-art registration methods in both accuracy and robustness. Beyond registration, CRFT provides a generalizable paradigm for multimodal spatial correspondence, offering broad applicability to remote sensing, autonomous navigation, and medical imaging. Code and datasets are publicly available at https://github.com/NEU-Liuxuecong/CRFT.

Abstract

Medical Image Registration

Explore similar work

CRFT: Consistent-Recurrent Feature Flow Transformer for Cross-Modal Image Registration

Adapting Frozen Mono-modal Backbones for Multi-modal Registration via Contrast-Agnostic Instance Optimization

Search-MIND: Training-Free Multi-Modal Medical Image Registration