Diffusion-based video matting has recently emerged as a leading approach, leveraging video diffusion priors from internet-scale pretraining to achieve strong zero-shot generalization from limited synthetic data. However, the heavy compute of these diffusion backbones limits inference to a short chunk of just 6 to 24 frames, so even ordinary videos span multiple chunks. Because each chunk is matted independently from coarse mask guidance, existing methods fail to maintain consistency across chunk boundaries, producing flickering edges and discontinuous transparency. We address this with a mask-guided diffusion-matting framework built on two ideas. The RGB and mask conditioning are represented as separate token streams that interact through full spatio-temporal self-attention, letting the model adaptively combine fine RGB texture with coarse mask layout rather than committing to a fixed channel-wise fusion. On top of this, we propose a novel pair of learnable embeddings, the preserve token and the refine token, that act as a per-frame conditioning interface and enable a sliding-window inference scheme in which already-generated mattes are propagated across chunk boundaries for local temporal consistency, complemented by a globally shared reference anchor at every chunk to prevent error accumulation in long sequences without any additional training. Across standard video matting benchmarks and in-the-wild long footage, our framework outperforms prior diffusion-based methods in both fine-detail accuracy and temporal consistency, and uniquely maintains consistency across chunk boundaries on long videos.
Figures & tables
Figure 1: We introduce VideoMaMa++ , a mask-guided video matting framework that recovers fine-grained details (e.g., hair strands) and scales reliably to long videos. (a) Per-frame qualitative results on diverse in-the-wild videos. (b) Long-video matting results: VideoMaMa Lim et al. (2026) , which processes videos in chunks, fails to maintain consistency across chunk boundaries (red boxes), whereas VideoMaMa++ produces mattes that are both finely detailed and temporally consistent.
Figure 2: Overview of the VideoMaMa++ training and inference. (Top) During training, RGB frames are paired with either a binary mask or the ground-truth alpha matte as guide input, encoded into separate token streams by a frozen VAE encoder E , and tagged with a refine token ( R ) or preserve token ( P ). The DiT processes both modes jointly within a single clip, supervised against the ground-truth matte. (Bottom) At inference, long videos are processed in a sliding-window manner. Each chunk receives the refine token on its non-overlapping frames and the preserve token on its overlap frames, which carry the previous chunk’s predicted matte. A globally selected reference anchor frame is additionally supplied as preserve input to every chunk, providing a stable anchor that prevents error accumulation across the sequence.
Figure 3: Long-video temporal consistency. Qualitative comparison between VideoMaMa++ and other methods. The comparison methods exhibits visible flickering at chunk boundaries or struggles to capture fine-grained details, while VideoMaMa++ produces smooth transitions across chunks while still capturing fine-grained details that emerge in later chunks.
Table 6: Ablation studies on video matting design choices. We evaluate the impact of our proposed components across different configurations. The ablations analyze (a) the advantage of incorporating SAM logits, and (b) channel-wise versus token-wise concatenation. All evaluations are performed on the V-Him60 (Hard) benchmark. The best results are highlighted.
Steps
MAD ↓
MAD-T ↓
MSE ↓
GRAD ↓
CONN ↓
DTSSD ↓
1
1.6680
41.7354
0.4027
2.9674
2.6769
1.9546
2
1.7221
42.0193
0.4138
2.9708
2.7389
1.9479
3
2.7264
50.3829
0.5093
3.8081
3.3320
2.1253
5
10.7571
74.6095
2.0941
5.1494
17.5623
2.6155
10
16.6343
101.5099
3.9117
6.1175
31.1540
3.1936
Table 7: Ablation on denoising steps.
Chunk
MAD ↓
MAD-T ↓
GRAD ↓
CONN ↓
DTSSD ↓
3
2.1206
46.2131
3.5778
3.5776
2.2120
6
1.6942
41.2279
2.9196
2.7194
1.9796
12
1.7906
41.6290
3.0313
2.8858
2.0693
Table 8: Ablation on chunk size.
Rank
MAD ↓
MAD-T ↓
MSE ↓
GRAD ↓
CONN ↓
DTSSD ↓
64
2.4294
56.5292
0.8798
4.2180
4.2180
2.6284
128
2.3018
53.3100
0.8108
4.7875
3.9483
2.5732
512
1.6942
41.2279
0.4487
2.9196
2.7194
1.9796
Table 9: Ablation on LoRA rank.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Datasets
Image
DVM Sun et al. (2021) , AM-2k Li et al. (2022) , DAViD Saleh et al. (2025) , P3M-10k Li et al. (2021) , RefMatte Li et al. (2023)
Video
CRGNN Wang et al. (2021) , VideoMatte240K Lin et al. (2021) , DVM Sun et al. (2021)
Background
VideoMatting108 Zhang et al. (2021) , DVM Sun et al. (2021)
Appendix
Table 10: Datasets used to train VideoMaMa and SAM2-Matte.
Iter.
MAD ↓
MAD-T ↓
GRAD ↓
CONN ↓
DTSSD ↓
MSE ↓
1
1.6924
41.9167
2.9813
2.7228
1.9758
0.4234
2
1.6688
41.8064
2.9677
2.6761
1.9589
0.3992
3
1.6634
41.7927
2.9633
2.6655
1.9545
0.3949
Appendix
Table 11: Ablation on interactive refinement.
Method
P3M-500-NP
AM-2k
MAD ↓
MSE ↓
Grad ↓
Conn ↓
MAD ↓
MAD-T ↓
Grad ↓
Mask free methods
RVM
10.89
6.57
14.97
18.81
-
-
-
SparseMat
11.14
6.36
13.47
18.13
-
-
-
GVM
8.39
4.89
12.89
14.21
-
-
-
Mask guided methods
Appendix
Table 12: Quantitative evaluation on P3M-500 and AM-2k datasets.
Figure 6: Annotation noise in AM-2K Li et al. (2022) . The ground-truth alpha mattes in AM-2K often miss fine-grained internal details, that VideoMaMa++ correctly recovers. Because MAD averages pixel-wise error over the entire image, these missing details penalize accurate predictions and inflate the reported error. MAD-T, computed only within the transition region, isolates the matting quality that actually matters and avoids the bias introduced by such annotation gaps.
Figure 7: Re-initialization of SAM3 on YouTubeMatte static_0017. Initializing SAM3 Carion et al. (2025) on the first frame fails to track a sheet of paper that the subject begins holding mid-sequence, since the object is not yet visible at initialization. Initializing on a later frame where both are simultaneously visible recovers the correct mask track.
Metric
Init.
MGM
MGM-IW
MaGGIe
MaGGIe-FT
VideoMaMa
Ours
MAD
First-frame
3.87
2.87
2.29
2.30
1.88
1.61
Re-init
3.59
2.41
1.67
1.52
1.41
1.15
MSE
First-frame
0.78
0.83
0.93
0.95
0.76
0.69
Re-init
0.53
0.38
0.50
0.37
0.30
0.23
Grad
First-frame
4.37
4.37
7.02
6.42
4.20
3.31
Re-init
4.27
4.25
5.11
5.36
4.08
3.27
Appendix
Table 13: Effect of re-initializing YouTubeMatte static_0017. “Re-init”: SAM3 re-initialized at frame 60 for static_0017, as used in Table 1 . “First-frame”: SAM3 initialized on the first frame for every sequence. All metrics ↓ .
YouTubeMatte
In-the-wild (no GT)
Seam-elevated clips
Autocorr.
Seam-elevated
Model
MAX
fg-mean
lag 4
lag 8
clips
VideoMaMa Lim et al. (2026)
62/64
63/64
+0.35
+0.34
14/16
p=1.1×10−16
p=3.5×10−18
p=5×10−3
p=6×10−3
p=2.1×10−3
GT (no chunking)
34/64
32/64
–
–
–
p=0.35
p=0.55
Appendix
Table 14: Annotation-free seam test. Both models run at identical chunk seams. Left: ground-truth validation on YouTubeMatte (64 clips; 24 seam vs. 75 interior transitions per clip). Entries count clips whose mean seam transition exceeds their mean interior transition (chance: 32/64). Right: 16 in-the-wild clips (fg-mean), for which no ground truth exists. Higher autocorrelation at the seam lag and more seam-elevated clips indicate a stronger chunk-boundary discontinuity. p-values for seam-elevated clips are from an exact one-sided sign test against chance.
FPS ↑
Peak memory (GB) ↓
TFLOPs ↓
Method
256
512
768
1K
2K
256
512
768
1K
2K
256
512
768
1K
2K
MGMatting
66.71
66.23
63.56
60.44
18.05
0.58
0.84
1.37
1.98
5.41
0.036
0.115
0.240
0.411
1.55
MaGGIe
23.2
19.3
20.0
15.0
7.5
0.52
1.18
2.52
4.88
16.01
0.021
0.083
0.187
0.332
1.33
MatAnyone
43.6
42.0
38.9
34.6
7.5
0.33
0.67
1.32
3.43
22.40
0.030
0.124
0.289
0.542
2.94
VideoMaMa
27.56
13.64
6.06
3.41
0.71
4.75
5.36
6.41
7.88
17.05
1.60
6.55
15.4
28.9
159
VideoMaMa++
16.4
6.9
2.6
1.17
0.11
3.73
5.17
7.86
11.33
35.59
4.25
24.8
85.4
225
2920
Appendix
Table 15: Computational cost on a single H100 at resolutions 2562 to 20482 (1K = 10242 , 2K = 20482 ), excluding mask generation for all methods. For VideoMaMa++, the cost of each 6-frame window is amortized over its 4 new frames, so FPS accounts for the recomputed overlap frames. MatAnyone/MatAnyone 2 and MGM/MGM-ITW share architectures and are listed once.
Res.
SAM3
VAE Enc.
DiT (1-step)
VAE Dec.
Matting total
Matting FPS
E2E FPS
256 2
45.4
24.9
20.1
15.9
60.9
16.4
9.4
512 2
45.4
37.9
79.2
27.5
144.6
6.9
5.3
768 2
45.4
78.3
254.3
55.8
388.4
2.6
2.3
1024 2
45.4
139.0
616.1
97.7
852.8
1.17
1.11
2048 2
45.4
681.0
7602.1
448.6
8731.7
0.11
0.11
Appendix
Table 16: Per-output-frame cost of VideoMaMa++ (ms) on a single H100, including SAM3 mask propagation. Matting-stage costs are measured per 6-frame window and divided by its 4 new frames. E2E FPS =1000/ (SAM3 + matting total).
Despite impressive advances in image matting, video matting remains challenging due to the inherent gap between high-level tracking, which requires frame-wise understanding, and low-level matting, which focuses on extremely fine-grained details. Existing methods attempt this with expensive and narrowly-scoped video matting datasets, which may limit out-of-domain generalization and compromise tracking robustness. We rethink the paradigm with SAM2Matting, a tracker-to-matting framework that advances VOS trackers to high-fidelity video matting. Specifically, it decouples the task by enhancing a foundational tracker (e.g., SAM2, SAM3) with a region-proposal bridge and dedicated matting heads, enabling the uncompromised tracker to handle temporal consistency while the matting components resolve fine-grained details. Notably, despite being trained only on images, SAM2Matting establishes new state-of-the-art performance on video matting, supports diverse prompt types, maintains strong temporal consistency, and demonstrates robust generalization across both human-centric and in-the-wild scenarios.
Ruiqi Shen, Guangquan Jie, Chang Liu +1
Fudan University · Shanghai University of Finance and Economics
Diffusion-based text-to-video models are increasingly capable, but mask-based editing over hundreds of frames remains challenging: naïve long-video generation suffers from memory blow-up, window seams, and temporal drift, while existing editors often require specialized modules or heavy fine-tuning. We present Overlapping High-Order Co-Denoising, a lightweight framework that turns a single pre-trained text-to-video model into a unified inpainting-outpainting editor. We train only LoRA adapters using mixed interior and border masks together with a dual-region loss that improves synthesis inside the mask while explicitly preserving known content. At inference, we denoise long latent sequences using overlapping windows, apply second-order Heun sampling within each window, and fuse overlaps with Hamming-weighted blending to reduce boundary artifacts and improve temporal coherence. On InpaintBench (30 real-world videos, 81--300 frames), our method outperforms Wan 2.1 variants and VACE in background faithfulness (SSIM/LPIPS), temporal consistency (tLPIPS), and text alignment (CLIP), and scales to long horizons, demonstrated up to 800 frames, with memory bounded by the chosen window size.
Shuangquan Lyu, Jian Mao, Yue Ma
Carnegie Mellon University · Jilin University · Tsinghua University
High-fidelity video generation remains challenging for diffusion models due to the difficulty of modeling complex spatio-temporal dynamics efficiently. Recent video diffusion methods typically represent a video as a sequence of spatio-temporal tokens which can be modeled using Diffusion Transformers (DiTs). However, this approach faces a trade-off between the strong but expensive Full 3D Attention and the efficient but temporally limited Local Factorized Attention. To resolve this trade-off, we propose Matrix Attention, a frame-level temporal attention mechanism that processes an entire frame as a matrix and generates query, key, and value matrices via matrix-native operations. By attending across frames rather than tokens, Matrix Attention effectively preserves global spatio-temporal structure and adapts to significant motion. We build FrameDiT-G, a DiT architecture based on MatrixAttention, and further introduce FrameDiT-H, which integrates Matrix Attention with Local Factorized Attention to capture both large and small motion. Extensive experiments show that FrameDiT-H achieves state-of-the-art results across multiple video generation benchmarks, offering improved temporal coherence and video quality while maintaining efficiency comparable to Local Factorized Attention.
Minh Khoa Le, Kien Do, Duc Thanh Nguyen +1
Applied Artificial Intelligence Initiative, Deakin University, Australia · FPT Smart Cloud, Vietnam · Deakin University, Australia