Wan-Alpha++: Animating a Single RGBA Asset into Controllable Transparent Video Effects
Authors: Haotian Dong, Xin Wang, Wenjing Wang, Chen Li, Jing Lyu, Yan Tina Luximon, Ping Li, Wanchao Su, Ke Xu, Di Lin
Organizations: College of Intelligence and Computing, Tianjin University, China. · Laboratory for Artificial Intelligence in Design Limited (AiDLab), The Hong Kong Polytechnic University, Hong Kong. · independent researcher. · School of Design, The Hong Kong Polytechnic University, Hong Kong. · School of Software, Dalian University of Technology, China and SensiLab, Monash University, Australia. · University of Science and Technology of China, China
RGBA video generation is essential for creating compositable visual assets in filmmaking, game development, and digital content production. However, generating high-quality RGB appearance alongside temporally coherent transparency remains a significant challenge. Existing text-to-RGBA video methods often entangle RGB and alpha distributions and offer limited control over the precise appearance of a target asset. In addition, incorporating image conditioning typically suffers from the emergence-disappearance effects, which fail existing first-frame or first-and-last-frame conditions due to their fully transparent boundary frames. To address these issues, we present Wan-Alpha++, a unified framework that brings controllable RGBA video generation to both text- and image-to-video diffusion backbones through a shared RGBA latent space, extending our conference version Wan-Alpha from the text-conditioned setting to image-conditioned generation with an interior reference frame. Our method first learns shiftable RGB-alpha distributions while preserving the pretrained RGB prior. Specifically, we introduce a transparency-aware bidirectional diffusion objective to produce distinguishable RGBA latent representations, and a transparency-guided mean sampler that shifts the noise distribution to improve transparency generation and enable spatial controllability. Building upon this RGBA generation prior, we introduce temporal phase reparameterization for image-conditioned effect generation. By cyclically transforming a zero-positive-zero alpha pattern into a positive-zero-positive pattern, it relocates visible content to both temporal boundaries, facilitating effective appearance conditioning with limited RGBA data.
Figures & tables
Fig. 1: Training framework of our RGBA VAE. The RGB and alpha channels are encoded separately and fused into a shared latent representation. The transparency-aware bidirectional diffusion objective preserves the pretrained RGB distribution while shifting alpha-related representations. Rendering-based reconstruction objectives further supervise RGB appearance and alpha transparency.
Fig. 2: Training framework of our RGBA video generator. The pretrained RGBA VAE encodes the input video into the shared latent space. Transparency-guided mean sampler provides a transparency prior for DiT adaptation.
Fig. 3: A Gaussian ellipse derived from the alpha channel shifts the noise mean, providing a transparency prior for DiT adaptation. During inference, the mask can be specified by users to control the approximate position and extent of generated content.
Fig. 4: Inference framework of the proposed phase-reparameterized image-to-RGBA video generation. The same input RGBA image conditions both temporal boundaries. Transparent-state guidance applies normalized gradient updates to a user-specified invisible interval. Finally, temporal transformation recovers the desired emergence-disappearance effect.
Fig. 5: Samples from our dataset. Our videos show clear motion and complex edges, such as hair. Each video has short and long captions, plus attribute labels. The dataset also includes various semi‑transparent effects like sheer fabric, smoke, water, and glow.
Fig. 6: The detailed architecture for the feature merging block. RGB and alpha features are concatenated and fused by a causal 3D convolution. Then, we use several causal residual blocks and attention layers. Finally, a causal 3D convolution generates the merged latent Z .
Fig. 7: Qualitative comparison with existing text-to-RGBA video generation methods. Our method produces more realistic RGB content, finer alpha boundaries, and more accurate transparency for challenging objects and semi-transparent effects. Please refer to the supplementary material for video results.
Method
Text Alignment ↑
Aesthetic Quality ↑
Naturalness ↑
Motion Smoothness ↑
Temporal Flickering ↑
LayerFlow (Single) [ 30 ]
2.67
0.535
2.35
0.9837
0.9788
LayerDiffuse [ 5 ] + AnimateDiff [ 6 ]
3.15
0.617
3.03
0.9893
0.9853
TransPixeler (Open) [ 7 ]
3.16
0.570
2.97
0.9821
0.9872
TransPixeler (Close) [ 7 ]
3.45
0.573
3.07
0.9907
0.9822
Ours
4.00
0.649
3.19
0.9949
0.9941
TABLE I: Quantitative comparison with existing text-to-RGBA video generation methods. Our Wan-Alpha consistently achieves superior video generation performance across all five evaluation metrics.
Method
Transparency ↓
Overall ↓
LayerFlow (Single) [ 30 ]
4.29
3.57
LayerDiffuse [ 5 ] + AnimateDiff [ 6 ]
3.40
4.23
TransPixeler (Open) [ 7 ]
2.51
2.71
TransPixeler (Close) [ 7 ]
2.57
3.37
Ours
1.23
1.11
TABLE II: User ranking for text-to-RGBA generation. Lower transparency and overall indicate better performance.
Fig. 8: Qualitative comparison with the two-stage Wan + MatAnyone pipeline. Despite using a white-background prompt and a user-provided initial mask, the matting pipeline leaves residual background content and struggles with semi-transparent effects. Our end-to-end RGBA generation model produces cleaner alpha channels and more accurate transparency.
Method
RGB-A VAE
Text-to-Video Generation
VAE
T2V
PSNR ↑
SSIM ↑
LPIPS ↓
Dist
Text Alignment ↑
Aesthetic Quality ↑
Naturalness ↑
Motion Smoothness ↑
Temporal Flickering ↑
Rendering
TABD
MS
RGB
α
RGB
α
RGB
α
RGB ↓
α↑
40.12
39.98
0.97
0.97
0.043
0.025
0.985
3.56
3.67
0.616
3.17
0.990
0.987
✓
40.88
41.22
0.97
0.98
0.040
0.023
1.015
4.10
3.77
0.619
3.17
0.992
0.984
✓
✓
41.47
42.22
0.97
0.98
0.037
0.022
0.857
4.29
3.86
0.621
3.19
0.992
0.990
✓
✓
✓
41.47
42.22
0.97
0.98
0.037
0.022
0.857
4.29
4.00
0.649
3.19
0.995
0.994
TABLE III: Ablation study on RGB-A video reconstruction and generation performance. Our full model achieves the best performance across all metrics, demonstrating the effectiveness of the proposed framework design.
Fig. 9: Qualitative ablation of the text-to-RGBA generation framework. TABD improves RGB–alpha separation, while MS facilitates clean transparent backgrounds.
Fig. 10: Effect of the mean-shift strength μ in the transparency-guided mean sampler.
Fig. 11: Results under different Gaussian ellipse masks G (shown at the bottom right). Users can use G to control the shape and position of transparent regions.
Fig. 12: Qualitative comparison with the two-stage Wan2.1-I2V + MatAnyone pipeline. Despite using a white-background prompt and a user-provided initial mask, the matting pipeline leaves residual background content and struggles with semi-transparent effects. Our end-to-end RGBA generation model produces cleaner alpha channels and more accurate transparency.
Fig. 13: Qualitative comparison with the two-stage Wan2.1-FLF2V + MatAnyone pipeline under different matting initialization positions. With First initialization, MatAnyone extracts the visible first frame, but fails to produce the emergence-disappearance RGBA effect. The RGB generator also fails to preserve fine appearance details such as text. With Middle initialization, MatAnyone only processes the frames following the condition frame and discards the first half of the video. Moreover, it continues to propagate an object-shaped alpha artifact even after the RGB content has disappeared. In contrast, our method conditions on a user-specified visible frame and generates a complete RGBA effect that gradually emerges and then disappears with clean transparent states.
Method
DINO-I ↑
CLIP-I ↑
Aesthetic Quality ↑
Motion Smoothness ↑
Temporal Flickering ↑
Normal
Wan2.1-I2V + MatAnyone [ 61 ]
0.9033
0.9089
0.6296
0.9912
0.9858
Ours
0.9066
0.9445
0.6305
0.9916
0.9864
Effect
Wan2.1-FLF2V + MatAnyone [ 61 ]
0.6679
0.7969
0.5749
0.9917
0.9926
Ours
0.9227
0.9368
0.5968
0.9941
0.9930
TABLE IV: Quantitative comparison on normal and effect image-to-RGBA video generation. DINO-I and CLIP-I measure reference fidelity, while the remaining metrics evaluate general video quality.
Method
Temporal Effect ↓
Overall ↓
Normal
Wan2.1-I2V + MatAnyone
1.73
1.62
Ours
1.27
1.38
Effect
Wan2.1-FLF2V + MatAnyone
1.88
1.84
Ours
1.12
1.16
TABLE V: User ranking for image-to-RGBA generation.
Reparameterization
Guidance
DINO-I ↑
CLIP-I ↑
Aesthetic Quality ↑
Motion Smoothness ↑
Temporal Flickering ↑
Temporal Effect ↓
0.6315
0.7542
0.5793
0.9919
0.9918
2.76
✓
0.9136
0.9275
0.5924
0.9936
0.9927
2.09
✓
✓
0.9227
0.9368
0.5968
0.9941
0.9930
1.12
TABLE VI: Component ablation of temporal phase reparameterization ( Reparameterization ) and transparent-state guidance ( Guidance ). We report reference fidelity, general video quality, and the user ranking for temporal-effect correctness. Lower is better for Temporal Effect, while higher is better for all other metrics.
Parameter
Value
DINO-I ↑
CLIP-I ↑
Aesthetic Quality ↑
Motion Smoothness ↑
Temporal Flickering ↑
Step size ηs
10
0.9208
0.9335
0.5908
0.9935
0.9925
25
0.9220
0.9352
0.5947
0.9939
0.9928
50
0.9227
0.9368
0.5968
0.9941
0.9930
75
0.9188
0.9310
0.5921
0.9934
0.9924
Guided sampling steps
16
0.9235
0.9372
0.5905
0.9934
0.9924
24
0.9230
0.9369
0.5943
0.9938
0.9927
TABLE VII: Sensitivity to transparent-state guidance hyperparameters.
RGBA videos combine RGB appearance with an alpha channel, enabling animated assets to be applied across arbitrary backgrounds, which are heavily used in gaming industry. However, generating high-quality RGBA animations for games remains challenging for two reasons. First, most existing RGBA video datasets are dominated by photorealistic content, with limited coverage of game assets. Second, the traditional generate-then-matte pipelines estimate alpha only after RGB synthesis, so semi-transparent regions are often blurred by background, resulting in unstable matting outputs. More recently, many methods have begun to model RGB and alpha jointly, but existing approaches are mostly text-conditioned, and still have unresolved issues in efficiency and quality. To address these challenges, we introduce GameAlpha-2.4K, a 2.4K-clip game-style RGBA video dataset built with matte-friendly synthesis, multi-hypothesis alpha recovery, and compositing-based quality gates. Using this dataset, we train a reference-conditioned RGBA video generator that jointly produces RGB frames and alpha mattes in a single pass. To improve efficiency, we propose a visibility router that identifies transparent tokens in an early stage and bypasses their later DiT updates, while x_0-lock guides them along the original flow-matching schedule toward self-predicted endpoints. Our model obtains lower FVD than traditional two-stage pipelines, and the visibility router skips 35% of token evaluations in the final two DiT denoising steps, providing a 1.2x backbone speedup with negligible quality degradation compared to dense inference.
Zhe Li, Honghao Qiao, Zhixin Xu +3
Peking University · ByteDance · Tsinghua University
Existing RGBA generation methods commonly learn a transparency-specific representation through reconstruction, introducing an additional training stage and potentially altering the RGB decoding function inherited from pretrained models. We present AlphaAnything, a lightweight framework that extends pretrained RGB generators to RGBA directly in their original latent space, without RGBA reconstruction pretraining. AlphaAnything inserts LoRA modules into the generative transformer and attaches an alpha branch to the frozen RGB VAE decoder. Unidirectional RGB-to-alpha feature sharing allows opacity supervision to shape the generative latent while guaranteeing that, for any fixed latent, RGB decoding remains identical to that of the pretrained decoder. We further introduce Endpoint-Aware Alpha Loss (EA Loss), which derives opacity supervision from the complementary foreground--background weights used in compositing. It supports exact opacity endpoints and continuous partial transparency while emphasizing localized alpha errors. Experiments on text-to-RGBA and image-to-RGBA tasks across multiple pretrained RGB backbones demonstrate competitive generation quality, and controlled ablations validate the shared-latent training design and EA Loss.
Qijie Wang, Zhixin Xu, Zhe Li +5
Tsinghua University · ByteDance · Peking University
We introduce the first method, to the best of our knowledge, for adapting image-to-video models to layer-aware text (glyph) animation, a capability critical for practical dynamic visual design. Existing approaches predominantly handle the transparency-encoding (alpha channel) as an extra latent dimension appended to the RGB space, necessitating the reconstruction of the underlying RGB-centric variational autoencoder (VAE). However, given the scarcity of high-quality transparent glyph data, retraining the VAE is computationally expensive and may erode the robust semantic priors learned from massive RGB corpora, potentially leading to latent pattern mixing. To mitigate these limitations, we propose TransText, a framework based on a novel Alpha-as-RGB paradigm to jointly model appearance and transparency without modifying the pre-trained generative manifold. TransText embeds the alpha channel as an RGB-compatible visual signal through latent spatial concatenation, explicitly ensuring strict cross-modal (RGB-and-Alpha) consistency while preventing feature entanglement. Our experiments demonstrate that TransText significantly outperforms baselines, generating coherent, high-fidelity transparent animations with diverse, fine-grained effects.
Fei Zhang, Zijian Zhou, Bohao Tang +9
Shanghai Jiao Tong University · Shanghai Innovation Institute (SII) · Meta AI