UniStitch: Unifying Semantic and Geometric Features for Image Stitching

Yuan Mei; Lang Nie; Kang Liao; Yunqiu Xu; Chunyu Lin; Bin Xiao

cs.CVMar 11, 2026

UniStitch: Unifying Semantic and Geometric Features for Image Stitching

Authors: Yuan Mei, Lang Nie, Kang Liao, Yunqiu Xu, Chunyu Lin, Bin Xiao

Organizations: CQUPT · PolyU · NTU · NUS ·

Abstract

Traditional image stitching methods estimate warps from hand-crafted geometric features, whereas recent learning-based solutions leverage semantic features from neural networks instead. These two lines of research have largely diverged along separate evolution, with virtually no meaningful convergence to date. In this paper, we take a pioneering step to bridge this gap by unifying semantic and geometric features with UniStitch, a unified image stitching framework from multimodal features. To align discrete geometric features (i.e., keypoint) with continuous semantic feature maps, we present a Neural Point Transformer (NPT) module, which transforms unordered, sparse 1D geometric keypoints into ordered, dense 2D semantic maps. Then, to integrate the advantages of both representations, an Adaptive Mixture of Experts (AMoE) module is designed to fuse geometric and semantic representations. It dynamically shifts focus toward more reliable features during the fusion process, allowing the model to handle complex scenes, especially when either modality might be compromised. The fused representation can be adopted into common deep stitching pipelines, delivering significant performance gains over any single feature. Experiments show that UniStitch outperforms existing state-of-the-art methods with a large margin, paving the way for a unified paradigm between traditional and learning-based image stitching.

Figures & tables

Figure 1: Existing image stitching solutions vs. UniStitch (ours). The proposed UniStitch integrates the traditional geometric feature and learning-based semantic feature into a unified representation. It effectively eliminates artifacts that persist when either feature modality is used alone, delivering a clear improvement.

Figure 2: The overall structure of UniStitch. It takes original images and their keypoints as input, and follows a three-stage pipeline: multimodal feature alignment, multimodal feature fusion, and global-to-local warp.

Figure 3: The details of the neural point transformer module.

	Method	mPSNR $\uparrow$				mSSIM $\uparrow$
	Method	Easy	Moderate	Hard	Average	Easy	Moderate	Hard	Average
1	APAP [ 50 ]	26.77	22.88	18.75	22.39	0.868	0.770	0.587	0.726
2	SPW [ 28 ]	25.82	21.49	15.85	20.52	0.844	0.693	0.434	0.634
3	LPC [ 13 ]	25.01	21.27	17.34	20.82	0.815	0.673	0.485	0.640
4	UDIS [ 36 ]	23.53	19.73	17.42	19.94	0.761	0.545	0.376	0.542
5	UDIS++ [ 37 ]	27.58	23.75	20.04	23.41	0.880	0.792	0.632	0.755

Table 1: Quantitative comparison on the UDIS-D dataset.

	Method	mPSNR $\uparrow$				mSSIM $\uparrow$
	Method	Easy	Moderate	Hard	Average	Easy	Moderate	Hard	Average
1	APAP [ 50 ]	24.33	19.48	14.47	18.92	0.818	0.687	0.442	0.628
2	SPW [ 28 ]	23.83	19.16	14.66	18.75	0.793	0.647	0.445	0.610
3	LPC [ 13 ]	23.19	18.67	13.90	18.11	0.761	0.600	0.412	0.573
4	UDIS [ 36 ]	20.58	16.27	13.37	16.40	0.660	0.471	0.271	0.454
5	UDIS++ [ 37 ]	21.93	17.53	13.83	17.36	0.709	0.526	0.325	0.500

Table 2: Quantitative comparison on classical datasets.

*

denotes post processing with the iterative adaptation [ 37 ] .

Figure 4: Qualitative comparison on the UDIS-D dataset.

Figure 5: Qualitative comparison on the classical dataset.

	Multimodal Alignment		Multimodal Fusion		UDIS-D	Classic
	Point	Image	AMoE	MR	UDIS-D	Classic
1	✓				24.22/0.784	18.02/0.553
2		✓			24.63/0.797	17.91/0.534
3	✓	✓	✓		24.94/0.808	18.56/0.581
4	✓	✓	✓	✓	25.07 / 0.813	18.80 / 0.596

Table 3: Ablation study of network component.

	Backbone	UDIS-D	Classic
1	PointNet [ 41 ]	24.47 / 0.793	17.89/0.542
2	PointNet++ [ 42 ]	24.27/0.786	18.00/0.551
3	PointTransformer v3 [ 48 ]	23.96/0.778	17.90/0.546
4	PointNeXt [ 43 ] (ours)	24.22/0.784	18.02 / 0.553

Table 4: Ablation study of different backbones for point feature extraction in the point-only configuration.

	Point Configuration	UDIS-D	Classic
	Training with Raw Points
1	– Testing in Fixed Order	24.78/0.803	17.90/0.549
2	– Testing in Random Order	24.76/0.802	17.90/0.549
	Training with Matched Points
3	– Testing in Fixed Order	24.91 / 0.808	18.48 / 0.577
4	– Testing in Random Order	24.91 / 0.808	18.48 /0.576

Table 5: Ablation study of different point configurations, including the point order and point quality.

	Fusion Strategy	UDIS-D	Classic
1	Add	24.42/0.790	17.22/0.510
2	Concat	24.61/0.795	17.74/0.538
3	CSPFusion	24.90/0.808	18.52/0.576
4	AMoE	24.94/0.808	18.56/0.581
5	AMoE + MR (ours)	25.07 / 0.813	18.80 / 0.596

Table 6: Ablation Study of multimodal feature fusion methods. Refer to Table 3 for the results obtained with single-modality features.

Resolution	Warping	Time (s) $\downarrow$	Peak GPU(GB)	mPSNR $\uparrow$	mSSIM $\uparrow$
$566\times 800$	Vanilla TPS	0.141	4.81	16.85	0.776
$566\times 800$	FFD-based TPS	0.132	2.76	16.86	0.776
$1329\times 2000$	Vanilla TPS	0.202	13.99	17.14	0.475
$1329\times 2000$	FFD-based TPS	0.177	8.57	17.12	0.474
$2448\times 3264$	Vanilla TPS	-	OOM	-	-
$2448\times 3264$	FFD-based TPS	0.386	22.62	22.71	0.627

Table 7: Ablation Study of FFD-based TPS warping strategy. Compared with vanilla TPS, our method offers higher efficiency and lower memory usage without sacrificing alignment accuracy.

Figure 6: Ablation study of different geometric features. We draw the matched points on the stitched results to illustrate the keypoint quality. Several local regions are zoomed in to demonstrate the benefit of extra geometric features using our method.

	Feature Configuration	UDIS-D	Classic
	Baseline
1	– Image-only Configuration	24.63/0.797	17.91/0.534
	Tranditional
2	– SIFT [ 32 ] + KNN	24.84/0.805	18.17/0.560
3	– SURF [ 1 ] + KNN	24.81/0.804	18.14/0.561
4	– ORB [ 44 ] + KNN	24.85/0.806	18.22/0.560

Table 8: The impact of various geometric features. ‡ denotes experiments that use only keypoint coordinates, without their corresponding descriptors.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Ablation study on different multimodal feature fusion methods.

Pooling Strategy	UDIS-D	Classic
Mean Pooling	24.89/ 0.808	18.45/0.576
Sum Pooling	24.92 /0.807	18.49 / 0.577
Max Pooling (ours)	24.91/ 0.808	18.48/ 0.577

Appendix

Table 9: Ablation study of different pooling strategies.

Configuration	UDIS-D	Classic
w/o $L_{reg}$	24.91/0.807	18.42/0.579
w/ $L_{reg}^{1/8}$	24.91/ 0.808	18.48/0.577
w/ $L_{reg}^{1/16}$ (ours)	24.94 / 0.808	18.56 / 0.581
w/ $L_{reg}^{1/8}$ + $L_{reg}^{1/16}$	24.93/ 0.808	18.39/0.568

Appendix

Table 10: Ablation study of balance loss.

Dataset	Cat [ 20 ]	Reception [ 13 ]	Construction site [ 50 ]
Resolution	400 $\times$ 600	675 $\times$ 1200	1329 $\times$ 2000
UDIS [ 36 ]	0.0991	0.1339	0.1901
UDIS++ [ 37 ]	0.0548	0.0869	0.1784
DunhuangStitch [ 33 ]	0.0468	0.1092	0.1271
StabStitch++ [ 38 ]	0.0466	0.1262	0.1933
RopStitch (w/o search) [ 40 ]	0.0389	0.0868	0.1566

Appendix

Table 11: Comparisons on inference time (/s). These methods are tested with Intel(R) Core(TM) i7-14700KF CPU @ 3.40GHz and NVIDIA RTX 4090 D GPU.

Figure 8: More comparative results on the UDIS-D dataset.

Figure 9: More comparative results on the classical dataset.

Feature Fusion

Feature fusion combines complementary representations across modalities, scales, layers, or feature types to improve prediction and perception.

See all 119 papers

Pano360: Perspective to Panoramic Vision with Geometric Consistency
Mar 12, 2026Zhengdong Zhu, Weiyi Xue, Zuyuan Yang +2Multi-View Consistency Multi-View Geometry
South China University of Technology · Tongji University · Guangdong University of Technology
Revisiting Model Stitching In the Foundation Model Era
Mar 12, 2026Zheda Mai, Ke Zhang, Fu-En Wang +6Vision Foundation Models Multimodal Foundation Models
1The Ohio State University · 3Amazon · 2Boston University
UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decomposition
May 8, 2026Shuai Zhang, Zhecheng Shi, Zhuxiao Li +4Disentangled Representation Learning Multimodal Semantic Segmentation
1HKUST(GZ) · 2HKUST

Mar 12, 2026J/K move · Enter open · S save

Pano360: Perspective to Panoramic Vision with Geometric Consistency

Zhengdong Zhu, Weiyi Xue, Zuyuan Yang, Wenlve Zhou+1

South China University of Technology · Tongji University · Guangdong University of Technology

Multi-View Consistency Multi-View Geometry

Open paper AskPDF arXiv

Prior panorama stitching approaches heavily rely on pairwise feature correspondences and are unable to leverage geometric consistency across multiple views. This leads to severe distortion and misalignment, especially in challenging scenes with weak textures, large parallax, and repetitive patterns. Given that multi-view geometric correspondences can be directly constructed in 3D space, making them more accurate and globally consistent, we extend the 2D alignment task to the 3D photogrammetric space. We adopt a novel transformer-based architecture to achieve 3D awareness and aggregate global information across all views. It directly utilizes camera poses to guide image warping for global alignment in 3D space and employs a multi-feature joint optimization strategy to compute the seams. Additionally, to establish an evaluation benchmark and train our network, we constructed a large-scale dataset of real-world scenes. Extensive experiments show that our method significantly outperforms existing alternatives in alignment accuracy and perceptual quality.

Abstract

Feature Fusion

Explore similar work

Pano360: Perspective to Panoramic Vision with Geometric Consistency

Revisiting Model Stitching In the Foundation Model Era

UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decomposition