Radiation, Rotation and Scale Invariant Feature Descriptor for Multimodal Image Matching

Yuanxin Ye; Tengfeng Tang; Tao Peng; Zhiqiang Han; Jiayuan Li; Mi Wang

cs.CVSep 6, 2026

Radiation, Rotation and Scale Invariant Feature Descriptor for Multimodal Image Matching

Authors: Yuanxin Ye, Tengfeng Tang, Tao Peng, Zhiqiang Han, Jiayuan Li, Mi Wang

Organizations: Faculty of Geosciences and Engineering, Southwest Jiaotong University, Chengdu, China · School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing,Wuhan University, Wuhan, China

Abstract

Multimodal image matching is a fundamental task for multi-source information fusion. However, geometric distortions and nonlinear radiometric differences (NRD) severely limit performance, especially under radiometric, rotation, and scale variations. To address this issue, we propose a radiation, rotation, and scale invariant (RRSI) feature descriptor. First, a dual-head regional sampling (DHRS) module simultaneously performs Cartesian and Log-Polar sampling on keypoint neighborhoods, retaining spatial structural properties while enhancing robustness to rotation and scale variations. We then jointly encode geometric and radiometric relations between multimodal images in a unified deep feature space, enabling feature encoding, interaction, and fusion across intra-modal, dual-head sampled, and inter-modal regions. Furthermore, we introduce a bidirectional cross-modal generative reconstruction constraint during training. By decoding implicit features into structural patches of the counterpart modality, this mechanism anchors modality-invariant geometric topologies without additional inference overhead. Experiments on optical-infrared and optical-SAR datasets demonstrate highly competitive matching performance and strong robustness to rotation and scale variations. RRSI supports the full rotation range from 0 to 360 degrees and scale factors up to four. Its generalization ability is further validated on multimodal images from computer vision, remote sensing, and medical imaging. The implementation will be made publicly available at https://github.com/yeyuanxin110/RRSI .

Figures & tables

Figure 1: Multimodal image examples from different fields

Figure 2: Overall framework of multimodal image matching based on the RRSI descriptor

Figure 3: Flowchart of keypoint detection incorporating scale pyramid, gradient magnitude maps, and FAST detector

Figure 4: Local region sampling (Cartesian vs. Log-Polar) under different sampling ratios

Figure 5: The architecture of self attention and cross attention

Figure 6: Training-phase supervision mechanism of the cross-modal generative reconstruction decoders. The two symmetric branches illustrate cross-modal pseudo-patch generation and the source-side skip connections used to regularize the joint representation space through $L_{Rec}$ .

Panel of Figure 7 — Figure 7: Examples of Multimodal Images from Experimental Datasets

Methods	LLVIP				DroneVehicle				OSdataset
	RMSE	AUC/(%)			RMSE	AUC/(%)			RMSE	AUC/(%)
	/px	@3px	@5px	@10px	/px	@3px	@5px	@10px	/px	@3px	@5px	@10px
RIFT [ 12 ]	5.01	3.27	9.38	24.14	5.04	3.46	8.19	19.37	4.74	4.05	9.15	20.99
WSSF [ 30 ]	5.36	2.87	7.42	21.39	5.21	4.74	9.52	21.30	5.09	3.81	10.54	26.57
LoFTR [ 32 ]	3.17	18.65	39.99	67.03	3.40	20.66	37.23	59.99	3.72	11.12	29.06	55.40
LightGlue [ 18 ]	3.37	17.27	36.99	66.36	2.86	25.08	45.27	69.02	3.53	14.07	35.26	64.53

Table 1: Quantitative Results of Image Matching on the LLVIP, DroneVehicle, and OSdataset ( Bold Indicates the Optimal Value, and Underline Indicates the Sub-optimal Value)

Panel of Figure 8 — Figure 8: SR- $T$ curve: image matching success rate at different reprojection error thresholds

Panel of Figure 9 — Figure 9: Visualization of matching results by different methods

Panel of Figure 10 — Figure 10: Checkerboard visualization of the RRSI’s registration results

Figure 11: RMSE under different rotational angles

Panel of Figure 12 — Figure 12: Visualization of matching results under different rotational angles

Figure 13: RMSE under different scale factors, where values exceeding 10 pixels are clipped to 10 pixels

Panel of Figure 14 — Figure 14: Visualization of matching results under different scale factors $s$ and rotational angles $\theta$

Ablation Setting	RMSE/px	AUC/(%)
		@3px	@5px	@10px
Cartesian Only	3.60	16.51	37.82	64.81
Log-Polar Only	3.72	15.34	36.19	62.97
Cartesian & Log-Polar	3.17	17.97	40.05	67.83
(the complete RRSI)

Table 2: Ablation Results of the DHRS Module on the OSdataset (Optical-SAR)

Ablation Setting	RMSE	AUC/(%)
Ablation Setting	/px	@3px	@5px	@10px
Baseline	3.48	16.87	38.27	65.36
Intra-Mod. Att.	3.41	17.42	39.07	66.26
Inter-Mod. Att.	3.25	17.74	39.45	66.90
Intra & Inter-Mod. Att.	3.17	17.97	40.05	67.83
(the complete RRSI)	3.17	17.97	40.05	67.83

Table 3: Ablation Results for the Joint Encoding of Geometric and Radiometric Relations on the OSdataset (Optical-SAR)

Ablation Setting	RMSE	AUC/(%)
	/px	@3px	@5px	@10px
w/o Reconstruction	3.31	16.42	37.32	65.11
with Reconstruction	3.17	17.97	40.05	67.83
(the complete RRSI)

Table 4: Ablation Results of the Generative Reconstruction Constraint on the OSdataset (Optical-SAR)

Domain	RMSE	AUC/(%)
Domain	/px	@3px	@5px	@10px
Remote Sensing	1.27	37.78	54.00	76.33
Medical	2.67	17.78	39.33	68.33
Computer Vision	1.42	30.00	49.33	74.33

Table 5: Generalization Results for the RRSI Method on the MIMD dataset

Panel of Figure 15 — Figure 15: Visual matching of the RRSI on the MIMD dataset

Cross-Modal Feature Matching

Cross-modal feature matching finds corresponding features across different data types or sensors despite differences in appearance, geometry, and viewpoint.

See all 66 papers

Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network
Jun 2, 2026Zhikang Li, Yan Wu, Xin Hu +2Multi-Scale Feature Fusion Cross-Modal Feature Matching
Remote Sensing Image Processing and Fusion Group, School of Electronic Engineering, Xidian University, Xi’an 710071, China · National Key Laboratory of Radar Signal Processing, Xidian University, Xi’an 710071, China
CrossFeat: Bridging Imaging Modalities in Feature Descriptor Space
Aug 31, 2026Paul Schneider, Nazim HaouchineCross-Modal Representation Learning Multimodal Disentangled Representation Learning
Harvard Medical School, Brigham and Women’s Hospital, Boston, MA, USA · Technical University of Munich, Munich, Germany
Disentangle-then-Align: Non-Iterative Hybrid Multimodal Image Registration via Cross-Scale Feature Disentanglement
Mar 20, 2026Chunlei Zhang, Jiahao Xia, Yun Xiao +2Cross-Modal Image Registration Deformable Image Registration
Faculty of Engineering and IT, University of Technology Sydney, Australia · School of Artificial Intelligence, Anhui University, Hefei, China · School of Computer Science and Technology, Anhui University, Hefei, China

Jun 2, 2026J/K move · Enter open · S save

Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network

Zhikang Li, Yan Wu, Xin Hu, Yi Dai+1

Remote Sensing Image Processing and Fusion Group, School of Electronic Engineering, Xidian University, Xi’an 710071, China · National Key Laboratory of Radar Signal Processing, Xidian University, Xi’an 710071, China

Multi-Scale Feature Fusion Cross-Modal Feature Matching Cross-Modal Image Registration Multimodal Fusion Feature Fusion

Open paper AskPDF arXiv

In multi-modal image registration, the primary challenge lies in shared structural information extraction. Compared to Transformers, Structured State Space Duality (SSD) offers greater global structural feature extraction with higher efficiency during training and inference. Inspired by these advantages, we propose a novel algorithm for multi-modal image registration, named RegNetMamba-2. Our algorithm incorporates SSD into coarse-to-fine matching process to extract local and global structural features effectively. Firstly, SSD is applied in three different scales for multi-modal feature extraction in our network. To strengthen local representation, we pay more attention on foreground edge and structural information by feature scaling function of SSD. Secondly, for shared feature extraction of input images and multi-modal feature fusion in all scales, we propose cross-modality feature fusion model based on SSD, consisting of Cross-Modality feature Interaction (CMI) module and Multi-Scale feature Fusion (MSF) module. CMI module is designed for cross-modality feature extraction of each scale by SSD in cross form. MSF module is designed to employ a progressive upward fusion in feature-level to obtain fine features, consisting of multi-modal features in all scales. Following coarse-to-fine, the features in 1/8 scale from CMI and 1/2 scale from MSF are collected to calculate matching probability scores. Then we respectively establish matching process by correspondences of pixel-wise. Extensive experiments demonstrate that comparing with state-of-the-art deep-learning based algorithms, RegNetMamba-2 has achieved good effects in both performance and efficiency for multi-modal image registration on the following datasets: VIS-SAR (OSDataset), VIS-IR (LGHD/RoadSence) and VIS-NIR (RGB-NIR sense).

Abstract

Cross-Modal Feature Matching

Explore similar work

Cross-Modality Feature Fusion Based on Structured State Space Duality for Multimodal Image Registration Network

CrossFeat: Bridging Imaging Modalities in Feature Descriptor Space

Disentangle-then-Align: Non-Iterative Hybrid Multimodal Image Registration via Cross-Scale Feature Disentanglement