Video Stitching from Multiple Moving Cameras
Organizations: The Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Israel
Abstract
We address geometric video alignment for the same scene captured by multiple freely moving cameras. Existing methods, which are based on image stitching or video stabilization, are typically limited to two cameras or must assume the restrictive setting of a fixed linear ordering of the cameras. We propose MViS, a lightweight test-time optimization framework for joint multi-view alignment with temporal consistency. MViS uses a spatial-transformer-inspired net to jointly process frames from all views within a short temporal window and predict per-view, per-time homographies. Optimization proceeds in two stages: geometry-guided alignment followed by appearance-based refinement, with both stages enforcing multi-view and temporal consistency across the sequence. The method requires no scene-specific hyperparameter tuning, handles arbitrary and time-varying view-intersection graphs, and remains robust under rapid camera motion. On a standard two-view video benchmark, MViS matches or surpasses the state of the art. To facilitate evaluation in more complex scenarios, we introduce an annotated dataset with more than two freely moving cameras and establish the first benchmark for this setting. Project page: https://bgu-cs-vil.github.io/MViS-project/ .
Figures & tables
| Method | Regular | Low-Light | Low-Texture | Fast-Moving | ||||
|---|---|---|---|---|---|---|---|---|
| PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | |
| LCP Jia et al. (2021) | 24.22 | .812 | - | - | - | - | 23.88 | .813 |
| UDIS ++ Nie et al. (2023) | 23.19 | .785 | 31.09 | .936 | 29.98 | .906 | 21.56 | .756 |
| StabStitch Nie et al. (2024) | 24.64 | .832 | 34.51 | .958 | 33.63 | .927 | 23.36 | .787 |
| StabStitch ++ Nie et al. (2025) | 25.51 | .837 | 35.10 | .958 | 34.23 | .928 | 25.64 | .830 |
| MViS (Ours) | 25.47 | .846 | 36.51 | .960 | 30.80 | .932 | 25.76 | .850 |
| Method | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GV | Stab. | CMSA | Succ. | GV | Stab. | CMSA | Succ. | GV | Stab. | CMSA | Succ. | GV | Stab. | CMSA | Succ. | ||
| MIS | AutoStitch Brown and Lowe (2007) | 3.3 | .847 | 635 | 14/22 | 2.4 | .901 | 217 | 6/10 | 3.1 | .834 | >1k | 9/11 | 3.3 | .924 | 111 | 5/10 |
| APAP Zaragoza et al. (2013) | 2.9 | .850 | >1k | 19/22 | .837 | >1k | 9/10 | 2.4 | .846 | >1k | 11/11 | 4.0 | .875 | >1k | 8/10 | ||
| GSP Chen and Chuang (2016) | 2.8 | .648 | >1k | 19/22 | 2.5 | .649 | >1k | 9/10 | 2.1 | .625 | >1k | 11/11 | 3.0 | .672 | >1k | 9/10 | |
| SPW Liao and Li (2020) | 4.8 | .807 | >1k | 22/22 | 10.4 | .716 | >1k | 10/10 | 2.5 | .835 | >1k | 10/11 | 14.8 | .745 | >1k | 10/10 | |
| GES-GSP Du et al. (2022) | 2.9 | .618 | >1k | 19/22 | 2.5 | .668 | >1k | 9/10 | 2.4 | .614 | >1k | 10/11 | 3.2 | .670 | >1k | 9/10 | |
| Ablation | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PSNR | SSIM | GV | Stab. | CMSA | GV | Stab. | CMSA | GV | Stab. | CMSA | GV | Stab. | CMSA | |
| No stage 1 | 25.22 | .794 | 879 | .886 | >1k | >1k | .888 | >1k | >1k | .872 | >1k | >1k | .909 | >1k |
| No stage 2 | 28.93 | .897 | 3.5 | .915 | >1k | 12.5 | .928 | 275 | 2.6 | .873 | 264 | 3.3 | .930 | 10.8 |
| No temporal loss | 30.13 | .903 | 3.1 | .911 | 44.9 | 2.8 | .917 | >1k | 2.4 | .870 | 48.4 | 3.0 | .926 | 31.3 |
| Both stages simultaneously | 25.40 | .779 | 12.9 | .905 | 42.0 | 11.7 | .920 | 131 | 4.6 | .871 | 12.7 | 11.0 | .930 | 0.8 |
| Unnormalized RGB | 29.18 | .891 | 3.1 | .914 | 5.8 | 13.9 | .907 | 31.9 | 2.6 | .874 | 7.0 | 3.1 | .930 | 2.5 |
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Loss | |||||
|---|---|---|---|---|---|
| (w/o Net) | 39.9 | >1k | >1k | 485.3 | 18.0 |
| (w/o Net) | >1k | >1k | >1k | >1k | >1k |
| w/o Net | 118.7 | 35.8 | 86.4 | 11.6 | 21.3 |
| MViS | 6.8 | 2.5 | 14.2 | 2.4 |
| PSNR | SSIM | GV | Stab. | CMSA | GV | Stab. | CMSA | ||
| 0.5 | 30.24 | .904 | 3.1 | .917 | 2.2 | 2.8 | .925 | 35.1 | |
| 1.0 | 30.17 | .904 | 3.1 | .917 | 2.8 | 2.8 | .925 | 14.2 | |
| 5.0 | 29.97 | .901 | 3.1 | .916 | 1.9 | 2.8 | .924 | 15.6 | |
| 0.5 | 29.02 | .890 | 7.2 | .913 | 21.7 | 6.7 | .925 | 16.6 | |
| 1.0 | 25.12 | .819 | 14.3 | .909 | 15.9 | 24.3 | .894 | 10.6 | |
| Ablation | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PSNR | SSIM | GV | Stab. | CMSA | GV | Stab. | CMSA | GV | Stab. | CMSA | GV | Stab. | CMSA | |
| MViS (LoFTR) | 30.19 | .905 | 3.1 | .917 | 3.5 | 2.8 | .924 | 12.4 | 2.3 | .874 | 8.7 | 3.0 | .930 | 4.3 |
| MViS (RoMa) | 30.24 | .904 | 3.1 | .917 | 2.5 | 2.8 | .925 | 14.2 | 2.3 | .874 | 11.0 | 3.0 | .930 | 2.4 |
| Group | CameraMotion range | # of scenes | GV | Stab. | CMSA |
|---|---|---|---|---|---|
| Low | 0.17–0.46 | 17 | 2.3 [1.7–3.0] | .927 [.913–.958] | 0.1 [0.1–0.5] |
| Mid | 0.48–0.77 | 16 | 3.1 [2.1–3.4] | .927 [.913–.940] | 1.0 [0.3–7.1] |
| High | 0.78–1.50 | 20 | 2.4 [1.6–3.2] | .896 [.852–.928] | 5.3 [1.1–14.2] |
| Group | OverlapRatio range | Noise (%) | GV | Stab. | CMSA | # of scenes |
|---|---|---|---|---|---|---|
| Low | 0.27 – 0.52 | 0 | 2.9 | .914 | 10.7 | 18 |
| 10 | 2.9 | .913 | 7.2 | |||
| 20 | 2.9 | .912 | 11.0 | |||
| 30 | 3.0 | .913 | 10.1 | |||
| Mid | 0.52 – 0.66 | 0 | 2.7 | .914 | 6.6 | 16 |
| 10 | 2.7 | .912 | 6.4 |
| Layer (type:depth-idx) | Output Shape | #Params |
|---|---|---|
| Alignment Net | [ , ] | – |
| In_Norm (BatchNorm2d): 1-1 | [ , 3, 256, 256] | 6 |
| CNN: 1-2 | [ , 5, 8, 8] | – |
| Conv2d: 2-1 | [ , 10, 250, 250] | 1,480 |
| BatchNorm2d: 2-2 | [ , 10, 250, 250] | 20 |
| AdaptiveMaxPool2d: 2-3 | [ , 10, 32, 32] | – |
| Method | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GV | Stab. | CMSA | Succ. | GV | Stab. | CMSA | Succ. | GV | Stab. | CMSA | Succ. | GV | Stab. | CMSA | Succ. | ||
| GSP + TR | 2.9 | .821 | 381 | 19/22 | 2.5 | .872 | 165 | 9/10 | 2.4 | .785 | 297 | 11/11 | 3.0 | .839 | 204 | 9/10 | |
| GES-GSP + TR | 3.0 | .797 | 393 | 19/22 | 2.6 | .857 | 170 | 9/10 | 2.9 | .768 | 318 | 11/11 | 3.3 | .824 | 204 | 9/10 | |
| OBJ-GSP + TR | 3.0 | .803 | 415 | 19/22 | 2.6 | .842 | 244 | 9/10 | 2.7 | .773 | 339 | 11/11 | 3.1 | .833 | 213 | 9/10 | |
| GSP + TR | 3.1 | .863 | 62.9 | 19/22 | 2.8 | .898 | 52.2 | 9/10 | 3.2 | .817 | 93.6 | 11/11 | 3.5 | .879 | 38.1 | 9/10 | |
| GES-GSP + TR | 3.3 | .849 | 69.4 | 19/22 | 3.1 | .884 | 57.2 | 9/10 | 3.8 | .800 | 100 | 11/11 | 3.9 | .866 | 39.5 | 9/10 | |
| Overlap ratio | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 | Total |
|---|---|---|---|---|---|---|---|---|---|---|---|
| # of scenes | 0 | 0 | 3 | 4 | 11 | 12 | 8 | 13 | 2 | 0 | 53 |
| Percentage | 0.0% | 0.0% | 5.7% | 7.5% | 20.8% | 22.6% | 15.1% | 24.5% | 3.8% | 0.0% | 100% |
| Data | AutoStitch | APAP | GSP | SPW | GES-GSP | OBJ-GSP | MViS(Ours) | |
|---|---|---|---|---|---|---|---|---|
| scene_ | ✗ | 1.7 | 1.7 | 1.8 | 1.7 | 1.7 | 2.0 | |
| scene_ | 2.9 | 2.0 | 1.9 | 2.2 | 1.9 | 1.9 | 3.0 | |
| scene_ | 11.4 | 7.0 | 7.1 | 7.5 | 7.1 | 7.1 | 7.8 | |
| scene_ | 1.6 | 1.5 | 1.5 | 1.5 | 1.5 | 1.5 | 1.5 | |
| scene_ | ✗ | 12.1 | 10.6 | 10.5 | 11.1 | 11.2 | 13.0 | |
| scene_ | 3.3 | 2.8 | 2.8 | 2.8 | 2.8 | 2.8 | 2.8 |
| Data | AutoStitch | APAP | GSP | SPW | GES-GSP | OBJ-GSP | MViS(Ours) | |
|---|---|---|---|---|---|---|---|---|
| scene_ | ✗ | .889 | .736 | .889 | .717 | .730 | .910 | |
| scene_ | .870 | .901 | .855 | .889 | .704 | .643 | .911 | |
| scene_ | .641 | .898 | .726 | .857 | .725 | .615 | .926 | |
| scene_ | .937 | .926 | .775 | .928 | .772 | .712 | .932 | |
| scene_ | ✗ | .709 | .165 | .646 | .211 | .214 | .938 | |
| scene_ | .906 | .946 | .690 | .944 | .731 | .635 | .964 |
| Data | AutoStitch | APAP | GSP | SPW | GES-GSP | OBJ-GSP | MViS(Ours) | |
|---|---|---|---|---|---|---|---|---|
| scene_ | ✗ | 109 | >1k | 148 | >1k | >1k | 0.1 | |
| scene_ | 425 | 109 | 531 | >1k | 507 | >1k | 0.2 | |
| scene_ | >1k | 177 | >1k | >1k | >1k | >1k | 0.1 | |
| scene_ | 3.8 | 97.8 | >1k | 147 | >1k | >1k | 0.1 | |
| scene_ | ✗ | 728 | >1k | >1k | >1k | >1k | 0.1 | |
| scene_ | 13.2 | 13.7 | >1k | >1k | >1k | >1k | 0.0 |
| Data | MIS+TR | MIS+PS | |||||||
|---|---|---|---|---|---|---|---|---|---|
| GSP | GES-GSP | OBJ-GSP | GSP | GES-GSP | OBJ-GSP | Stab Stitch++ | MViS (Ours) | ||
| scene_ | 2.0 | 2.0 | 2.0 | 1.7 | 1.7 | 1.7 | 6.7 | 2.0 | |
| scene_ | 2.3 | 2.3 | 2.3 | 1.9 | 1.9 | 1.9 | 5.1 | 3.0 | |
| scene_ | 6.7 | 6.8 | 6.8 | 7.1 | 7.1 | 7.1 | 9.1 | 7.8 | |
| scene_ | 1.7 | 1.7 | 1.7 | 1.5 | 1.5 | 1.5 | 5.1 | 1.5 | |
| scene_ | 10.2 | 10.7 | 10.9 | 10.3 | 10.9 | 11.0 | 12.7 | 13.0 | |
| Data | MIS+TR | MIS+PS | |||||||
|---|---|---|---|---|---|---|---|---|---|
| GSP | GES-GSP | OBJ-GSP | GSP | GES-GSP | OBJ-GSP | Stab Stitch++ | MViS (Ours) | ||
| scene_ | .920 | .919 | .916 | .852 | .851 | .873 | .870 | .910 | |
| scene_ | .904 | .889 | .873 | .903 | .878 | .840 | .792 | .911 | |
| scene_ | .931 | .921 | .925 | .876 | .899 | .853 | .854 | .926 | |
| scene_ | .954 | .953 | .951 | .904 | .911 | .875 | .858 | .932 | |
| scene_ | .638 | .621 | .621 | .435 | .494 | .501 | .881 | .938 | |
| Data | MIS+TR | MIS+PS | |||||||
|---|---|---|---|---|---|---|---|---|---|
| GSP | GES-GSP | OBJ-GSP | GSP | GES-GSP | OBJ-GSP | Stab Stitch++ | MViS (Ours) | ||
| scene_ | 4.1 | 4.0 | 6.2 | 47.3 | 46.9 | 38.8 | 1.4 | 0.1 | |
| scene_ | 7.4 | 7.9 | 8.7 | 17.2 | 17.7 | 49.1 | 5.6 | 0.2 | |
| scene_ | 17.8 | 15.9 | 16.2 | 42.0 | 28.7 | 59.2 | 1.3 | 0.1 | |
| scene_ | 2.6 | 2.7 | 4.3 | 93.2 | 55.0 | 75.3 | 1.1 | 0.1 | |
| scene_ | 536 | 550 | 533 | 842 | 555 | 427 | 1.0 | 0.1 | |
| Symbol | Description |
|---|---|
| Number of input videos (cameras). | |
| Number of frames in each video. | |
| -th input video, . | |
| Frame of video . | |
| Preprocessing resolution, frames resized to . | |
| Resized and RGB-normalized version of . |
Multi-View Geometry
Multi-view geometry studies how images from different viewpoints reveal camera motion, scene structure, and consistent 3D measurements.
See all 112 papers