DualCast: Dual-Image Conditioned Diffusion Transformers for Digital Human E-Commerce Video Generation
Organizations: State Key Lab of CAD&CG, Zhejiang University · Ant Group
Abstract
Digital human e-commerce video generation requires preserving product appearance, human identity, and overall video style during dynamic demonstrations. Although recent models support multiple reference images, their general conditioning schemes do not explicitly distinguish the asymmetric roles of "what is presented" and "who presents it." Consequently, generated videos may favor a single reference, resulting in product distortion, identity drift, or single-reference collapse. To address this issue, we propose DualCast, a role-addressed dual-image conditioning framework for video generation. By explicitly modeling the distinct roles of product and human conditions, DualCast reduces role ambiguity and feature competition between references. First, a dual-spatiotemporal slot injection mechanism VAE-encodes the product and human images separately and places their latent features at the front and rear boundaries of the DiT sequence, enabling sequence-level condition separation and directed injection. Second, a mask-guided region-weighted loss strengthens supervision over product, face, and product-face overlap regions, improving generation fidelity in critical areas. Third, image condition dropout enables a single model to support dual-image, product-only, and human-only generation. In addition, we construct a curated dataset of 10K e-commerce videos for model training. Experimental results demonstrate the strong performance of DualCast.
Figures & tables
| Method | Open | Semantic & Consistency | Perceptual Quality (0–1) | Video Quality (VBench) | Avg Rank | |||||||||
| ViCLIP | S2V-C | ID-Adj | EcomStyle | FaceID | ProdID | Interaction | Motion | 5D Avg. | Dynamic Deg. | Aesthetic | Imaging | |||
| Kaleido [ 34 ] | ✓ | 0.220 | 0.516 | 0.4491 | 0.904 | 0.908 | 0.828 | 0.832 | 0.860 | 0.8664 | 0.500 | 0.499 | 0.726 | 8.14 |
| Phantom [ 17 ] | ✓ | 0.227 | 0.493 | 0.6283 | 0.956 | 0.904 | 0.852 | 0.864 | 0.872 | 0.8896 | 0.650 | 0.510 | 0.751 | 6.95 |
| VACE [ 10 ] | ✓ | 0.210 | 0.596 | 0.5406 | 0.936 | 0.968 | 0.936 | 0.876 | 0.896 | 0.9224 | 0.750 | 0.513 | 0.755 | 5.86 |
| SkyReels-A2 [ 4 ] | ✓ | 0.197 | 0.575 | 0.6409 | 0.876 | 0.992 | 0.860 | 0.808 | 0.768 | 0.8608 | 0.787 | 0.490 | 0.740 | 7.50 |
| Wan2.2-I2V-14B † (Base) [ 26 ] | ✓ | 0.197 | 0.717 | 0.8206 | 0.824 | 0.796 | 0.732 | 0.716 | 0.748 | 0.7632 | 0.775 | 0.512 | 0.709 | 7.86 |
| Model Variant | Condition Fidelity | Visual Quality | FaceDet | Avg. | |||
| ID-Ret | S2V-C | ProdID | Aesthetic | Imaging | |||
| DualCast (Full) | 0.5109 | 0.5886 | 0.868 | 0.52 | 0.77 | 0.7566 | 0.669 |
| w/o Dual-End | 0.0703 | 0.5083 | 0.856 | 0.52 | 0.74 | 0.7297 | 0.571 |
| Gain of Full | 626.7% | 15.8% | 1.4% | 0.0% | 4.1% | 3.7% | 17.2% |
| w/o Dual-End + w/o Region-Loss | 0.0827 | 0.5138 | 0.800 | 0.53 | 0.75 | 0.7734 | 0.575 |
| Gain of Full | 517.8% | 14.6% | 8.5% | 1.9% | 2.7% | 2.2% | 16.4% |
| Training | Inference | ID-Ret | EcomStyle | FaceID | ProdID | Avg. |
|---|---|---|---|---|---|---|
| Product-Centric Comparison | ||||||
| Baseline (Product-Only) | Product-Only | 0.0645 | 0.960 | 0.2120 | 0.838 | 0.519 |
| Baseline (Product-Only) | Dual | 0.4260 | 0.904 | 0.7660 | 0.808 | 0.726 |
| Ours (Joint) | Product-Only | — | 0.980 | 0.9600 | 0.862 | 0.934 † |
| Ours (Joint) | Dual | 0.5109 | 0.980 | 0.9840 | 0.868 | 0.836 |
| vs. Baseline (Dual) | 19.9% | 8.4% | 28.5% | 7.4% | 15.1% | |
| Evaluation Metric | vs. Kaleido | vs. Phantom | vs. VACE | vs. SkyReels-A2 |
|---|---|---|---|---|
| Product Consistency | 63.6% ∗ | 67.2% ∗ | 52.0% ∗ | 67.3% ∗ |
| Identity Preservation | 76.8% ∗ | 79.8% ∗ | 60.2% ∗ | 61.6% ∗ |
| Video Quality | 71.4% ∗ | 72.2% ∗ | 67.7% ∗ | 82.4% ∗ |
| Overall Preference | 70.6% | 73.1% | 60.0% | 70.4% |
| Diagnostic | Result |
|---|---|
| Execution and coverage | |
| Video outputs | 150/150 |
| Completed diagnostic windows | 1050/1050 |
| Decoded-frame coverage | 100% |
| API / file / decoding errors | 0 / 0 / 0 |
| Content and temporal checks | |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Event type | Videos | Events | Frames |
|---|---|---|---|
| Product deformation | 14 | 15 | 114 |
| Ghosting | 13 | 13 | 62 |
| Interpenetration | 3 | 3 | 9 |
| Camera shake | 1 | 1 | 3 |
| Identity drift | 0 | 0 | 0 |
| Reference copying | 0 | 0 | 0 |
| Axis | Mean (0–5) |
|---|---|
| E-commerce style | 4.925 |
| Face identity | 4.794 |
| Product identity | 4.748 |
| Human–product interaction | 4.920 |
| Motion naturalness | 4.008 |
| Segment | Style | Face | Product | Inter. | Motion |
|---|---|---|---|---|---|
| First | 4.928 | 4.876 | 4.766 | 4.923 | 4.029 |
| Middle | 4.887 | 4.597 | 4.680 | 4.901 | 3.725 |
| Last | 4.961 | 4.908 | 4.797 | 4.936 | 4.270 |
| Setting | Value |
|---|---|
| Hardware | 4 NVIDIA A100-SXM4-80GB GPUs |
| Precision | bfloat16 |
| Offloading | None |
| Parameter residency | all_params_gpu_resident |
| Resolution | pixels (as logged) |
| Frames / frame rate | 81 / 15 FPS |
| Statistic | Dual | Human | Product |
|---|---|---|---|
| Cases | 60 | 60 | 60 |
| Methods | 6 | 6 | 6 |
| Mean latency (s) | 867.0 | 869.0 | 865.6 |
| Median latency (s) | 866.3 | 868.9 | 864.7 |
| P10 latency (s) | 865.5 | 867.8 | 863.7 |
| P25 latency (s) | 865.8 | 868.4 | 864.0 |
Video Generation
Video generation studies models that create and edit moving images from text, images, audio, and structured controls.
See all 387 papers