Stereo video inpainting, which aims to fill the occluded regions of warped videos with visually coherent content while maintaining temporal consistency, remains a challenging open problem. The regions to be filled are scattered along object boundaries and occupy only a small fraction of each frame, leading to two key challenges. First, existing approaches perform poorly on such tasks due to the scarcity of high-quality stereo inpainting datasets, which limits their ability to learn effective inpainting priors. Second, these methods apply equal processing to all regions of the frame, even though most pixels require no modification, resulting in substantial redundant computation. To address these issues, we introduce three interconnected components. We first propose Gradient-Aware Parallax Warping (GAPW), which leverages backward warping and the gradient of the coordinate mapping function to obtain continuous edges and smooth occlusion regions. Then, a Parallax-Based Dual Projection (PBDP) strategy is introduced, which incorporates GAPW to produce geometrically consistent stereo inpainting pairs and accurate occlusion masks without requiring stereo videos. Finally, we present Sparsity-Aware Stereo Inpainting (SASI), which reduces over 70% of redundant tokens, achieving a 10.7x speedup during diffusion inference and delivering results comparable to its full-computation counterpart, enabling real-time processing of HD (768 x 1280) videos at 25 FPS on a single A100 GPU.
Figures & tables
Figure 1: Comparison among different methods at a resolution of 768×1280 : our method achieves a PSNR of 30.5 dB at 1 NFE, with an inference speed of 24.9 fps.
Figure 2: Comparison of forward warping and our GAPW in pixel mapping and occlusion mask generation.
Figure 3: ( a ) Illustration of the Parallax-Based Dual Projection, which utilizes Gradient-Aware Parallax Warping for reprojection to obtain the occlusion mask under input view. ( b ) Our proposed Sparsity-Aware Stereo Inpainting utilizes Mask-Based Token Selection to reduce the redundancy of visual tokens.
Figure 4: Qualitative comparison of data construction between TrajectoryCrafter [ 39 ] and ours.
Figure 5: Qualitative comparison of stereo video inpainting on the HD-100 test set ( 768×1280 ). Our inpainting results are much closer to the ground truth, showing sharper details and clearer structures.
Figure 6: Qualitative comparison of 2D-to-3D conversion. Each group shows the generated right view , zoomed-in patches , stereo-matched disparity , and anaglyph visualization from left to right. Our method produces sharp, high-quality views with accurate and detailed disparity.
Table 2: Quantitative comparison of 2D-to-3D methods on the Dynamic Replica valid set and the SVD AVP test set. † End-to-end approaches.
Dataset Strategy
PSNR ↑
SSIM ↑
LPIPS ↓
Random Mask
26.64
0.906
0.092
TrajectoryCrafter
31.14
0.923
0.047
Ours
32.48
0.933
0.049
Table 3: Ablation of inpainting performance on dataset strategy.
Dilate
Stride
r (%)
Latency ‡ ↓
PSNR ↑
SSIM ↑
LPIPS ↓
baseline
100.0
380.9
32.48
0.933
0.049
—
—
13.7
18.51
29.84
0.916
0.063
3
—
25.6
35.7
32.01
0.931
0.051
3
20
29.4
44.8
32.10
0.931
0.050
3
10
33.2
54.8
32.16
0.931
0.050
5
—
36.1
58.4
32.31
0.932
0.050
Table 4: HD-100 ( 576×1024 ) ablation with fixed NFE =4 , varying dilation kernel and temporal stride. Retention rate r is the fraction of tokens kept for DiT computation; ‡ Latency denotes DiT-only inference time in ms/frame. All settings achieve a substantial speedup with negligible quality loss. The light-gray row ( dilate=5 ) indicates the optimal setting adopted in all other experiments.
Figure 7: Ablation on max disparity. Our method consistently delivers high-quality, geometrically accurate results even under large disparity ranges.
Max Disparity
PSNR ↑
SSIM ↑
LPIPS ↓
0.02
32.18
0.915
0.043
0.05
32.16
0.905
0.049
0.07
30.48
0.900
0.053
0.08
29.91
0.895
0.058
Table 5: HD-100 ( 768×1280 ) ablation with fixed NFE =1 under wide-baseline settings. The light-gray row ( 0.07 ) corresponds to the setting adopted for all stereo inpainting experiments on the HD-100 test set. Larger disparities enlarge occluded areas, causing a slight expected drop in metrics, yet our method remains robust and effective.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Inference pipeline of 2D-to-3D conversion.
Figure 9: Module-wise latency breakdown and reduction on 768×1280 HD videos.
VAE
PSNR ↑
SSIM ↑
LPIPS ↓
WanVAE
30.59
0.906
0.040
Distilled (Ours)
30.48
0.900
0.053
Appendix
Table 6: Ablation on stereo inpainting quality on HD-100 ( 768×1280 ). Results are evaluated under the same final setting as in the main paper (see Tab. 1 ), using different VAEs. The distilled VAE achieves nearly identical quality to the original WanVAE.
VAE
Params
Encoder
Decoder
Total
WanVAE
126.8 M
47.7 ms
75.0 ms
122.7 ms
Distilled (Ours)
80.2 M
9.3 ms
18.5 ms
27.8 ms
Appendix
Table 7: Ablation on VAE parameter efficiency and latency at 1024×1024 resolution.
Train-time token filtering
PSNR ↑
SSIM ↑
LPIPS ↓
No (dense training)
32.48
0.933
0.049
Yes (sparse training)
32.31
0.932
0.050
Appendix
Table 8: Ablation on applying sparsity during training.
Figure 10: Qualitative comparison on HD-100 ( 768×1280 ). This figure complements Fig. 5 by including ZeroStereo [ 30 ] , a training-free stereo inpainting method that often produces random, spatially inconsistent fillings and distorted textures.
Metrics
Deep3D
Depthify.ai
Owl3D
StereoCrafter
ZeroStereo
Ours
AbsRel. ↓
0.029
0.035
0.042
0.039
0.018
0.019
δ<1.05↑
0.847
0.770
0.716
0.751
0.929
0.914
Appendix
Table 9: Geometry-consistency comparison on Dynamic Replica.
Figure 11: Comparison with SpatialDreamer [ 15 ] . Highlighted regions show truncation artifacts in SpatialDreamer, whereas our results preserve complete and consistent stereo geometry.
Figure 12: Comparison with M2SViD [ 26 ] . Our results show sharper details, larger disparities, and better color consistency, while M2SViD suffers from color drift and structural degradation.
Figure 13: Qualitative comparison on the SVD AVP test set ( 768×768 ). The second row shows per-pixel MSE maps, where blue denotes small errors and red large deviations from the ground truth. Our results exhibit the smallest errors and best overall consistency.
Figure 14: Qualitative comparison on the Dynamic Replica valid set( 720×1280 ). Our method yields the most accurate and artifact-free stereo completions, consistent with quantitative improvements in Tab. 2 .
Figure 15: Representative failure cases on transparent, reflective, and highly textured scenes from the SVD AVP test set.