We address geometric video alignment for the same scene captured by multiple freely moving cameras. Existing methods, which are based on image stitching or video stabilization, are typically limited to two cameras or must assume the restrictive setting of a fixed linear ordering of the cameras. We propose MViS, a lightweight test-time optimization framework for joint multi-view alignment with temporal consistency. MViS uses a spatial-transformer-inspired net to jointly process frames from all views within a short temporal window and predict per-view, per-time homographies. Optimization proceeds in two stages: geometry-guided alignment followed by appearance-based refinement, with both stages enforcing multi-view and temporal consistency across the sequence. The method requires no scene-specific hyperparameter tuning, handles arbitrary and time-varying view-intersection graphs, and remains robust under rapid camera motion. On a standard two-view video benchmark, MViS matches or surpasses the state of the art. To facilitate evaluation in more complex scenarios, we introduce an annotated dataset with more than two freely moving cameras and establish the first benchmark for this setting. Project page: https://bgu-cs-vil.github.io/MViS-project/ .
Figures & tables
Figure 1: MViS aligns N moving-camera videos, without assuming a known/fixed topology of the view-intersection graph and requires no scene-specific hyperparameter tuning.
Figure 2: An overview of the proposed method, Multiple Video Stitching (MViS).
Figure 3: The alignment network for N cameras. The weights of the CNNs are shared.
Figure 4: Stage 1’s ORT-guided Geometric Loss , Lgeo . Illustration using a sliding window St with N=3 . Top row: the inter-view pairs. Bottom: the temporal pairs. In each plot, the grid warped by the predicted transformation is overlaid on the grid warped by the ORT estimate. Each image corresponds to the target frame of the transformation ( e.g . , top-left: I1t→I2t , visualized over I2t ).
Figure 5: Stage 2’s Appearance Loss Lrgb . In this figure, N=3 .
Method
Regular
Low-Light
Low-Texture
Fast-Moving
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
LCP Jia et al. (2021)
24.22
.812
-
-
-
-
23.88
.813
UDIS ++ Nie et al. (2023)
23.19
.785
31.09
.936
29.98
.906
21.56
.756
StabStitch Nie et al. (2024)
24.64
.832
34.51
.958
33.63
.927
23.36
.787
StabStitch ++ Nie et al. (2025)
25.51
.837
35.10
.958
34.23
.928
25.64
.830
MViS (Ours)
25.47
.846
36.51
.960
30.80
.932
25.76
.850
Table 1: N=2 . Comparison across the 4 categories in the StabStitch-D dataset Nie et al. (2024) . The PSNR (↑) and SSIM (↑) values were averaged over all of the frames in the category.
Method
N=3
N=4
N=5
N=6
GV
Stab.
CMSA
Succ.
GV
Stab.
CMSA
Succ.
GV
Stab.
CMSA
Succ.
GV
Stab.
CMSA
Succ.
MIS
AutoStitch Brown and Lowe (2007)
3.3
.847
635
14/22
2.4
.901
217
6/10
3.1
.834
>1k
9/11
3.3
.924
111
5/10
APAP Zaragoza et al. (2013)
2.9
.850
>1k
19/22
.837
>1k
9/10
2.4
.846
>1k
11/11
4.0
.875
>1k
8/10
GSP Chen and Chuang (2016)
2.8
.648
>1k
19/22
2.5
.649
>1k
9/10
2.1
.625
>1k
11/11
3.0
.672
>1k
9/10
SPW Liao and Li (2020)
4.8
.807
>1k
22/22
10.4
.716
>1k
10/10
2.5
.835
>1k
10/11
14.8
.745
>1k
10/10
GES-GSP Du et al. (2022)
2.9
.618
>1k
19/22
2.5
.668
>1k
9/10
2.4
.614
>1k
10/11
3.2
.670
>1k
9/10
Table 2: The Multi-Camera Benchmark. Results on the MMCD Dataset. N is the # of videos per scene. The values shown, Stab. (↑), GV (↓) and CMSA (↓), are averaged over the scenes (on which the respective method succeeded without collapsing) with the same N . StabStitch++’s high GV (27.8) on N=3 reflects severe misalignment in scenes where the assumed camera ordering is violated. StabStitch++’s code does not support N>3 .
Figure 6: Visual comparisons. Top row: N=2 cases from the StabStitch-D dataset Nie et al. (2024) . StabStitch++’s severe misalignment makes the traffic sign vanish ( 6(a) ) and causes ghosting of the buildings ( 6(b) ). Middle: N=3 cases from the MMCD dataset. Bottom: MViS results on MMCD cases with N=4,5,6,8 . The N=8 is an extended scene from the N=6 category.
Figure 7: Dynamically changing view-intersection graph, MViS (Ours). Camera outlines are color-coded to highlight overlaps. Each camera is assigned a consistent color throughout the video.
Ablation
N=2
N=3
N=4
N=5
N=6
PSNR
SSIM
GV
Stab.
CMSA
GV
Stab.
CMSA
GV
Stab.
CMSA
GV
Stab.
CMSA
No stage 1
25.22
.794
879
.886
>1k
>1k
.888
>1k
>1k
.872
>1k
>1k
.909
>1k
No stage 2
28.93
.897
3.5
.915
>1k
12.5
.928
275
2.6
.873
264
3.3
.930
10.8
No temporal loss
30.13
.903
3.1
.911
44.9
2.8
.917
>1k
2.4
.870
48.4
3.0
.926
31.3
Both stages simultaneously
25.40
.779
12.9
.905
42.0
11.7
.920
131
4.6
.871
12.7
11.0
.930
0.8
Unnormalized RGB
29.18
.891
3.1
.914
5.8
13.9
.907
31.9
2.6
.874
7.0
3.1
.930
2.5
Table 3: Ablation Study. See text for details.
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Visual comparison between StabStitch++ [ 30 ] and MViS scenes with N=3 cameras (from the MMCD dataset).
Figure 9: Visual comparison between MIS methods and MViS on scenes with N>2 cameras from the MMCD dataset. We show a sequence of consecutive timestamps to illustrate that, although MIS methods often produce high-quality panoramas at individual frames, they lack temporal consistency.
Figure 10: Visual comparison between MIS methods and MViS on scenes with N>3 cameras from the MMCD dataset, continued.
Loss
N=2
N=3
N=4
N=5
N=6
Lgeo (w/o Net)
39.9
>1k
>1k
485.3
18.0
Lrgb (w/o Net)
>1k
>1k
>1k
>1k
>1k
w/o Net
118.7
35.8
86.4
11.6
21.3
MViS
6.8
2.5
14.2
2.4
Appendix
Table 4: Optimization without a net. Measured CMSA on the frame’s corners. In N=2 we used StabStitch-D dataset, and for N>2 the MMCD dataset.
R
α
N=2
N=3
N=4
PSNR ↑
SSIM ↑
GV ↓
Stab. ↑
CMSA ↓
GV ↓
Stab. ↑
CMSA ↓
R1
0.5
30.24
.904
3.1
.917
2.2
2.8
.925
35.1
1.0
30.17
.904
3.1
.917
2.8
2.8
.925
14.2
5.0
29.97
.901
3.1
.916
1.9
2.8
.924
15.6
R2
0.5
29.02
.890
7.2
.913
21.7
6.7
.925
16.6
1.0
25.12
.819
14.3
.909
15.9
24.3
.894
10.6
Appendix
Table 5: Regularization experiments on StabStitch-D dataset ( N=2 ) and the MMCD dataset ( N=3,4 ).
Ablation
N=2
N=3
N=4
N=5
N=6
PSNR
SSIM
GV
Stab.
CMSA
GV
Stab.
CMSA
GV
Stab.
CMSA
GV
Stab.
CMSA
MViS (LoFTR)
30.19
.905
3.1
.917
3.5
2.8
.924
12.4
2.3
.874
8.7
3.0
.930
4.3
MViS (RoMa)
30.24
.904
3.1
.917
2.5
2.8
.925
14.2
2.3
.874
11.0
3.0
.930
2.4
Appendix
Table 6: Image Matchers Comparison.
Group
CameraMotion range
# of scenes
GV ↓
Stab. ↑
CMSA ↓
Low
0.17–0.46
17
2.3 [1.7–3.0]
.927 [.913–.958]
0.1 [0.1–0.5]
Mid
0.48–0.77
16
3.1 [2.1–3.4]
.927 [.913–.940]
1.0 [0.3–7.1]
High
0.78–1.50
20
2.4 [1.6–3.2]
.896 [.852–.928]
5.3 [1.1–14.2]
Appendix
Table 7: Effect of camera motion on MViS
Group
OverlapRatio range
Noise (%)
GV ↓
Stab. ↑
CMSA ↓
# of scenes
Low
0.27 – 0.52
0
2.9
.914
10.7
18
10
2.9
.913
7.2
20
2.9
.912
11.0
30
3.0
.913
10.1
Mid
0.52 – 0.66
0
2.7
.914
6.6
16
10
2.7
.912
6.4
Appendix
Table 8: Results by overlap-ratio group and noise level.
Layer (type:depth-idx)
Output Shape
#Params
Alignment Net
[ 2B , 8N ]
–
In_Norm (BatchNorm2d): 1-1
[ 2BN , 3, 256, 256]
6
CNN: 1-2
[ 2BN , 5, 8, 8]
–
Conv2d: 2-1
[ 2BN , 10, 250, 250]
1,480
BatchNorm2d: 2-2
[ 2BN , 10, 250, 250]
20
AdaptiveMaxPool2d: 2-3
[ 2BN , 10, 32, 32]
–
Appendix
Table 9: Alignment Network Summary.
Method
N=3
N=4
N=5
N=6
GV
Stab.
CMSA
Succ.
GV
Stab.
CMSA
Succ.
GV
Stab.
CMSA
Succ.
GV
Stab.
CMSA
Succ.
w=0.5
GSP + TR
2.9
.821
381
19/22
2.5
.872
165
9/10
2.4
.785
297
11/11
3.0
.839
204
9/10
GES-GSP + TR
3.0
.797
393
19/22
2.6
.857
170
9/10
2.9
.768
318
11/11
3.3
.824
204
9/10
OBJ-GSP + TR
3.0
.803
415
19/22
2.6
.842
244
9/10
2.7
.773
339
11/11
3.1
.833
213
9/10
w=1.0
GSP + TR
3.1
.863
62.9
19/22
2.8
.898
52.2
9/10
3.2
.817
93.6
11/11
3.5
.879
38.1
9/10
GES-GSP + TR
3.3
.849
69.4
19/22
3.1
.884
57.2
9/10
3.8
.800
100
11/11
3.9
.866
39.5
9/10
Appendix
Table 10: Additional hyperparameter experiments for the MIS extensions on the MMCD dataset (the Multi-Camera Benchmark). N denotes the # of videos per scene. The values shown, GV (↓), Stab. (↑) and CMSA (↓), are averaged over the scenes (on which the respective method succeeded without collapsing) with the same N .
Overlap ratio
0.0–0.1
0.1–0.2
0.2–0.3
0.3–0.4
0.4–0.5
0.5–0.6
0.6–0.7
0.7–0.8
0.8–0.9
0.9–1.0
Total
# of scenes
0
0
3
4
11
12
8
13
2
0
53
Percentage
0.0%
0.0%
5.7%
7.5%
20.8%
22.6%
15.1%
24.5%
3.8%
0.0%
100%
Appendix
Table 11: Distribution of MMCD scenes across Overlap ratio intervals.
Data
AutoStitch
APAP
GSP
SPW
GES-GSP
OBJ-GSP
MViS(Ours)
N=3
scene_ 1
✗
1.7
1.7
1.8
1.7
1.7
2.0
scene_ 2
2.9
2.0
1.9
2.2
1.9
1.9
3.0
scene_ 4
11.4
7.0
7.1
7.5
7.1
7.1
7.8
scene_ 6
1.6
1.5
1.5
1.5
1.5
1.5
1.5
scene_ 7
✗
12.1
10.6
10.5
11.1
11.2
13.0
scene_ 8
3.3
2.8
2.8
2.8
2.8
2.8
2.8
Appendix
Table 12: Per-scene GV( ↓ ), the Multi-Camera Benchmark.
Data
AutoStitch
APAP
GSP
SPW
GES-GSP
OBJ-GSP
MViS(Ours)
N=3
scene_ 1
✗
.889
.736
.889
.717
.730
.910
scene_ 2
.870
.901
.855
.889
.704
.643
.911
scene_ 4
.641
.898
.726
.857
.725
.615
.926
scene_ 6
.937
.926
.775
.928
.772
.712
.932
scene_ 7
✗
.709
.165
.646
.211
.214
.938
scene_ 8
.906
.946
.690
.944
.731
.635
.964
Appendix
Table 13: Per-scene Stability score ( ↑ ), proposed by [ 31 ] , the Multi-Camera Benchmark.
Data
AutoStitch
APAP
GSP
SPW
GES-GSP
OBJ-GSP
MViS(Ours)
N=3
scene_ 1
✗
109
>1k
148
>1k
>1k
0.1
scene_ 2
425
109
531
>1k
507
>1k
0.2
scene_ 4
>1k
177
>1k
>1k
>1k
>1k
0.1
scene_ 6
3.8
97.8
>1k
147
>1k
>1k
0.1
scene_ 7
✗
728
>1k
>1k
>1k
>1k
0.1
scene_ 8
13.2
13.7
>1k
>1k
>1k
>1k
0.0
Appendix
Table 14: Per-scene CMSA ( ↓ ), the Multi-Camera Benchmark.
Data
MIS+TR
MIS+PS
GSP
GES-GSP
OBJ-GSP
GSP
GES-GSP
OBJ-GSP
Stab Stitch++
MViS (Ours)
N=3
scene_ 1
2.0
2.0
2.0
1.7
1.7
1.7
6.7
2.0
scene_ 2
2.3
2.3
2.3
1.9
1.9
1.9
5.1
3.0
scene_ 4
6.7
6.8
6.8
7.1
7.1
7.1
9.1
7.8
scene_ 6
1.7
1.7
1.7
1.5
1.5
1.5
5.1
1.5
scene_ 7
10.2
10.7
10.9
10.3
10.9
11.0
12.7
13.0
Appendix
Table 15: Per-scene GV( ↓ ) comparison on the Multi-Camera Benchmark, Continued.
Data
MIS+TR
MIS+PS
GSP
GES-GSP
OBJ-GSP
GSP
GES-GSP
OBJ-GSP
Stab Stitch++
MViS (Ours)
N=3
scene_ 1
.920
.919
.916
.852
.851
.873
.870
.910
scene_ 2
.904
.889
.873
.903
.878
.840
.792
.911
scene_ 4
.931
.921
.925
.876
.899
.853
.854
.926
scene_ 6
.954
.953
.951
.904
.911
.875
.858
.932
scene_ 7
.638
.621
.621
.435
.494
.501
.881
.938
Appendix
Table 16: Per-scene Stab.( ↑ ) comparison on the Multi-Camera Benchmark, Continued.
Data
MIS+TR
MIS+PS
GSP
GES-GSP
OBJ-GSP
GSP
GES-GSP
OBJ-GSP
Stab Stitch++
MViS (Ours)
N=3
scene_ 1
4.1
4.0
6.2
47.3
46.9
38.8
1.4
0.1
scene_ 2
7.4
7.9
8.7
17.2
17.7
49.1
5.6
0.2
scene_ 4
17.8
15.9
16.2
42.0
28.7
59.2
1.3
0.1
scene_ 6
2.6
2.7
4.3
93.2
55.0
75.3
1.1
0.1
scene_ 7
536
550
533
842
555
427
1.0
0.1
Appendix
Table 17: Per-scene CMSA( ↓ ) comparison on the Multi-Camera Benchmark, Continued.
Computing accurate geometry from multi-view images is a fundamental problem in computer vision. Recent feed-forward (FF) models jointly estimate 3D geometry and camera parameters, but they typically suffer from geometry distortion caused by reconstruction ambiguity, even when ground-truth camera parameters are supplied. In this paper, we study the multi-view stereo (MVS) problem with known camera parameters and propose a novel approach that bridges conventional MVS and FF methods. Rather than casting MVS as a sequence-to-one mapping that predicts depth only for a single reference view, we reformulate it as a sequence-to-sequence task, akin to FF models, that jointly predicts geometry for all input views. We introduce a global transformer-based architecture with two components that explicitly exploit camera-induced priors: ray-map embeddings that inject camera parameters into image patch tokens, making the transformer camera-aware, and a unified global cost volume that replaces conventional per-view cost volumes to jointly capture 3D structure across all views. Extensive experiments on multiple public benchmarks show our approach achieves state-of-the-art performance, surpassing both MVS and FF reconstruction baselines.
Prior panorama stitching approaches heavily rely on pairwise feature correspondences and are unable to leverage geometric consistency across multiple views. This leads to severe distortion and misalignment, especially in challenging scenes with weak textures, large parallax, and repetitive patterns. Given that multi-view geometric correspondences can be directly constructed in 3D space, making them more accurate and globally consistent, we extend the 2D alignment task to the 3D photogrammetric space. We adopt a novel transformer-based architecture to achieve 3D awareness and aggregate global information across all views. It directly utilizes camera poses to guide image warping for global alignment in 3D space and employs a multi-feature joint optimization strategy to compute the seams. Additionally, to establish an evaluation benchmark and train our network, we constructed a large-scale dataset of real-world scenes. Extensive experiments show that our method significantly outperforms existing alternatives in alignment accuracy and perceptual quality.
Zhengdong Zhu, Weiyi Xue, Zuyuan Yang +2
South China University of Technology · Tongji University · Guangdong University of Technology
Accurately recovering the camera's calibration and metric poses for any unconstrained video would unlock large-scale training data for navigation and scene understanding. The dominant approaches to this problem are severely limited: Simultaneous Localization and Mapping (SLAM) is sensitive to initialization and transient failures due to its causal, incremental nature; it is often over-optimized for real-time operation and generally requires known camera calibration; while Structure-from-Motion (SfM) typically forgoes any image ordering, enabling optimal initialization and global optimization, but lacks robustness to visual symmetries and extreme motions. To bridge this gap, we introduce a system that combines the strong sequential constraints of SLAM with the flexibility and global optimization of offline SfM, enabling the metric reconstruction of arbitrary, long, uncalibrated videos. This system leverages recent advances in wide-baseline dense image matching, treats temporal ordering as a first-class citizen for reliable loop closure, and augments global optimization with metric monocular depth priors. As a result, thorough evaluations on diverse, challenging datasets that exhibit extreme motion and visual symmetries reveal that our approach is significantly more robust and accurate than both state-of-the-art SLAM and SfM, classical or learned, with given or unknown camera calibration. The code is publicly available at https://github.com/cvg/vidmap.