Towards Generalized Multimodal Homography Estimation

Jinkun You; Jiaxin Cheng; Jie Zhang; Yicong Zhou

cs.CVMar 4, 2026

Towards Generalized Multimodal Homography Estimation

Authors: Jinkun You, Jiaxin Cheng, Jie Zhang, Yicong Zhou

Organizations: Department of Computer and Information Science, University of Macau Macau, China

Abstract

Supervised and unsupervised homography estimation methods depend on image pairs tailored to specific modalities to achieve high accuracy. However, their performance deteriorates substantially when applied to unseen modalities. To address this issue, we propose a training data synthesis method that generates unaligned image pairs with ground-truth offsets from a single input image. Our approach renders the image pairs with diverse textures and colors while preserving their structural information. These synthetic data empower the trained model to achieve greater robustness and improved generalization across various domains. Additionally, we design a network to fully leverage cross-scale information and decouple color information from feature representations, thus improving estimation accuracy. Extensive experiments show that our training data synthesis method improves generalization performance. The results also confirm the effectiveness of the proposed network.

Figures & tables

Figure 1: Estimation results. The rows present the results of the supervised MCNet [ 49 ] and the unsupervised SSHNet [ 41 ] . The first two columns represent within-dataset results, while the remaining columns depict cross-dataset performance. Greater similarity between the green and red quadrilaterals indicates higher accuracy.

Figure 2: Illustration of the training data synthesis and the homography estimation network. (a) Training Data Synthesis can be applied to a public RGB dataset for zero-shot homography estimation or integrated with existing datasets to enhance generalization. (b) Cross-Scale and Color-Invariant Network (CCNet) integrates cross-scale information into the extracted features while decoupling color information from the feature representations. (c) Color Decoupling. (d) Iterative Homography Estimation. (e) Multiscale Feature Extractor.

Figure 3: Examples of synthetic data. The first row presents the synthesis results with various content weights. The second row shows the results with different template images. The third row displays the examples with different smoothing weights. The weights become smaller and greater from left to right for the first and third rows, respectively.

Method	GoogleEarth	RGB-NIR	PDSCOCO
DHN [ 12 ]	27.653	26.605	29.856
$\textrm{DHN}^{*}$	22.933	13.864	14.173
MHN [ 18 ]	39.474	30.372	33.490
$\textrm{MHN}^{*}$	3.110	7.549	4.251
IHN [ 5 ]	3.038	12.491	5.352
$\textrm{IHN}^{+}$	2.770	7.456	4.481

Table 1: Cross-dataset evaluation for GoogleMap.

``*"

means that the model is trained on our synthetic data.

``+"

indicates that the data synthesis method is applied to GoogleMap for training.

Method	GoogleMap	RGB-NIR	PDSCOCO
DHN [ 12 ]	24.604	21.937	21.348
$\textrm{DHN}^{*}$	11.321	13.864	14.173
MHN [ 18 ]	26.806	20.581	11.518
$\textrm{MHN}^{*}$	11.218	7.549	4.251
IHN [ 5 ]	14.962	12.616	5.278
$\textrm{IHN}^{+}$	7.223	5.647	1.684

Table 2: Cross-dataset evaluation for GoogleEarth.

``*"

means that the model is trained on our synthetic data.

``+"

indicates that the data synthesis method is applied to GoogleEarth for training.

Method	GoogleMap	GoogleEarth	PDSCOCO
DHN [ 12 ]	15.188	18.288	19.962
$\textrm{DHN}^{*}$	11.321	22.933	14.173
MHN [ 18 ]	16.931	4.265	4.927
$\textrm{MHN}^{*}$	11.218	3.110	4.251
IHN [ 5 ]	10.629	2.095	2.873
$\textrm{IHN}^{+}$	3.768	1.894	1.901

Table 3: Cross-dataset evaluation for RGB-NIR.

``*"

means that the model is trained on our synthetic data.

``+"

indicates that the data synthesis method is applied to RGB-NIR for training.

Method	GoogleMap	GoogleEarth	RGB-NIR
DHN [ 12 ]	18.315	11.961	12.669
$\textrm{DHN}^{*}$	11.321	22.933	13.864
MHN [ 18 ]	22.822	5.694	11.272
$\textrm{MHN}^{*}$	11.218	3.110	7.549
IHN [ 5 ]	21.727	2.156	10.745
$\textrm{IHN}^{*}$	5.303	1.853	5.647

Table 4: Cross-dataset evaluation for PDSCOCO.

``*"

means that the model is trained on our synthetic data.

``+"

indicates that the data synthesis method is applied to PDSCOCO for training.

Type	Method	GoogleMap	GoogleEarth	RGB-NIR	PDSCOCO
Supervised	DHN [ 12 ]	3.556	7.735	12.855	11.937
	MHN [ 18 ]	1.561	2.061	7.078	5.142
	IHN [ 5 ]	1.025	1.147	4.588	1.445
	MCNet [ 49 ]	0.261	0.577	3.226	1.062
	CCNet (Ours)	0.184	0.526	2.992	1.001
	$\textrm{IHN}^{+}$	1.515	1.236	3.552	1.498

Table 5: Within-dataset and zero-shot evaluation results (MACE

\downarrow

) of our training data synthesis method and the homography estimation network. The best result is written in bold while the second-best one is underlined. “+” indicates that the models are trained on the dataset augmented by our synthesis method. “Zero-shot” means that the models are trained on our synthetic method.

Figure 4: Visualization results of within-dataset evaluation. The first row presents the estimation results from GoogleEarth, while the second row features results from RGB-NIR. The first column displays the source image to be warped, while the other columns demonstrate the target images. Greater similarity between the red and green quadrilaterals indicates higher accuracy in the estimation.

Figure 5: Visualization results of zero-shot evaluation using testing images from GoogleMap. The first column displays the source image to be warped. A greater similarity between the red and green quadrilaterals indicates higher accuracy in estimation.

DHN [ 12 ]	MHN [ 18 ]	IHN [ 5 ]	MCNet [ 49 ]	CCNet
11.21	10.92	26.14	31.01	32.73
34.20	2.57	0.76	0.85	1.21

Table 6: Computational costs of supervised methods and our network. The first row displays the runtime (in milliseconds) while the second row presents the model size (in megabytes).

Multimodal Domain Generalization

See all 14 papers

SA-Homo: Scale Adaptive Homography Estimation for Scale Variation Scenarios
Jun 29, 2026Shangxuan Xie, Haifeng Wu, Yuhang Wang +2Multi-View Geometry
University of Electronic Science and Technology of China (UESTC), Chengdu, China · University of Chinese Academy of Sciences, Beijing, China · Beijing Institute of Technology, Beijing, China
AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images
Jun 30, 2026Meng Yang, Zizhuo Li, Linfeng Tang +2Synthetic Data Augmentation Cross-Modal Feature Matching
Electronic Information School1, Wuhan University, Wuhan 430072, China · School of Robotics, Wuhan University2, Wuhan 430072, China
AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling
May 28, 2026Yiheng Li, Zhuo Li, Ruibing Hou +4Cross-Modal Learning Unified Multimodal Models
Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, China · Independent Author · University of Chinese Academy of Sciences, China

Jun 29, 2026J/K move · Enter open · S save

SA-Homo: Scale Adaptive Homography Estimation for Scale Variation Scenarios

Shangxuan Xie, Haifeng Wu, Yuhang Wang, Huarong Jia+1

University of Electronic Science and Technology of China (UESTC), Chengdu, China · University of Chinese Academy of Sciences, Beijing, China · Beijing Institute of Technology, Beijing, China

Multi-View Geometry

Open paper AskPDF arXiv

Homography estimation, as one of the fundamental problems in computer vision, remains challenged by scale variation scenarios where image pairs potentially exhibit significant scale discrepancies. Existing deep learning frameworks frequently suffer from a significant performance degradation in such cases, as they rely on limited displacement assumptions and local feature consistency that might not hold under large scale gaps. In this paper, we propose SA-Homo, a novel scale-adaptive homography estimation framework designed to achieve robust alignment across a wide range of scale discrepancy ratios. We adopt a hierarchical scale alignment strategy that transitions from the global perspective with a heavy module to a local perspective with a light module. Specifically, we introduce the Scale-aware Discrepancy Bridging Module (SDBM) for initial alignment, which utilizes a Multi-scale Linear Attention Cascade (MLAC) to capture long-range dependencies and mitigate feature inconsistencies, along with a global Cross-scale Similarity Matrix Block (CSMB) for scale robust correlation representation. Once the initial scale gap is bridged, a lightweight Iterative Homography Estimation Refinement Module (IHERM) progressively polishes the result using local correlations. To facilitate this research, we contribute the HMSA dataset, a high-resolution, multi-modal satellite benchmark specifically tailored for scale-variant challenges. Extensive experiments demonstrate that SA-Homo maintains high precision even under 8

\times

scale discrepancies, outperforming state-of-the-art methods in both conventional scale-similar scenarios and challenging scale variation scenarios. Code and collected datasets are available at https://github.com/shangxuanx330/SA_Homo

Abstract

Multimodal Domain Generalization

Explore similar work

SA-Homo: Scale Adaptive Homography Estimation for Scale Variation Scenarios

AnyMatch: Supercharging Universal Multi-Modal Image Matching with Large-Scale Single-View Images

AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling