Recent advances in diffusion models have significantly improved text-to-video generation, enabling personalized content creation with fine-grained control over both foreground and background elements. However, precise face-attribute alignment across subjects remains challenging, as existing methods lack explicit mechanisms to ensure intra-group consistency. Addressing this gap requires both explicit modeling strategies and face-attribute-aware data resources. We therefore propose LumosX, a framework that advances both data and model design. On the data side, a tailored collection pipeline orchestrates captions and visual cues from independent videos, while multimodal large language models (MLLMs) infer and assign subject-specific dependencies. These extracted relational priors impose a finer-grained structure that amplifies the expressive control of personalized video generation and enables the construction of a comprehensive benchmark. On the modeling side, Relational Self-Attention and Relational Cross-Attention intertwine position-aware embeddings with refined attention dynamics to inscribe explicit subject-attribute dependencies, enforcing disciplined intra-group cohesion and amplifying the separation between distinct subject clusters. Comprehensive evaluations on our benchmark demonstrate that LumosX achieves state-of-the-art performance in fine-grained, identity-consistent, and semantically aligned personalized multi-subject video generation. Code and models are available at https://jiazheng-xing.github.io/lumosx-home/.
Figures & tables
Figure 1: Lumos X supports flexible personalized multi-subject video generation.
Figure 2: Dataset construction pipeline for personalized multi-subject video generation. We build the training dataset from raw videos in three steps: (1) generate a caption and detect human subjects in extracted frames; (2) retrieve entity words from the caption and match subjects with their attributes; (3) use these entity tags to localize and segment target subjects and objects, producing a clean background image.
Figure 3: Overview of Lumos X . Built on the T2V model Wan2.1 [ 40 ] , our framework encodes all condition images into image tokens via a VAE encoder, concatenates them with denoising video tokens, and feeds the result into DiT [ 31 ] blocks. Within each block, the proposed Relational Self-Attention and Relational Cross-Attention enable causal conditional modeling, enhance visual token representations, and ensure precise face–attribute alignment.
Figure 4: Illustration of Attention Mask Design: Causal Self-Attention Mask (CSAM) and Multilevel Cross-Attention Mask (MCAM). We present a specific customized task case as an example.
Table 5
Methods
Entire Video
Extracted Subjects
Dynamic ↑
ViCLIP-T ↑
ViCLIP-V ↑
CLIP-T↑
CLIP-I ↑
DINO-I↑
ArcSim↑
CurSim↑
SkyReels-A2 [ 8 ]
0.671
0.251
0.839
0.178
0.606
0.192
0.271
0.290
Phantom [ 24 ]
0.661
0.254
0.865
0.185
0.647
0.216
0.444
0.477
Lumos X
0.723
0.260
0.932
0.201
0.692
0.261
0.454
0.483
Table 3: Comparison of different methods for subject-consistent video generation, including evaluation of the entire video and evaluation on the extracted subjects.
Figure 5: Qualitative comparison for identity-consistent and subject-consistent video generation.
Methods
CLIP-T↑
ArcSim ↑
None
0.184
0.316
+R2PE
0.178
0.363
+R2PE + CSAM
0.182
0.363
+R2PE + CSAM + MCAM ( 0.1 )
0.182
0.364
+R2PE + CSAM + MCAM ( 0.5 )
0.186
0.429
+R2PE + CSAM + MCAM ( 1.0 )
0.187
0.384
Table 4: Ablation study for the subject-consistent video generation. The value in the MACM parentheses is r .
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Process of entity words retrieval and subject-attributes matching with four steps: Data Collection, Data Organization, Data Processing, and Data Annotation.
Figure 7: Process of acquiring condition images of subjects, objects, and the background.
Methods
Numbers
Identity Consistency
Prompt Following
ArcSim ↑
CurSim ↑
ViCLIP-T ↑
SkyReels-A2 [ 8 ]
1 face
0.509
0.540
0.258
Phantom [ 24 ]
1 face
0.602
0.637
0.261
Lumos X
1 face
0.542
0.575
0.262
SkyReels-A2 [ 8 ]
≥ 2 faces
0.282
0.292
0.263
Phantom [ 24 ]
≥ 2 faces
0.434
0.457
0.266
Appendix
Table 5: Fine-grained quantitative comparison of identity-consistent video generation across different numbers of faces.
Methods
Numbers
Entire Video
Extracted Subjects
Dynamic ↑
ViCLIP-T ↑
ViCLIP-V ↑
CLIP-T↑
CLIP-I ↑
DINO-I↑
ArcSim↑
CurSim↑
SkyReels-A2
1 subject
0.772 (0.080)
0.252 (0.007)
0.870 (0.064)
0.177 (0.022)
0.604 (0.082)
0.199 (0.069)
0.356 (0.133)
0.380 (0.136)
Phantom
1 subject
0.848 (0.004)
0.254 (0.005)
0.880 (0.054)
0.186 (0.013)
0.646 (0.040)
0.215 (0.053)
0.539 (0.050)
0.581 (0.065)
Lumos X
1 subject
0.852
0.259
0.934
0.199
0.686
0.268
0.489
0.516
SkyReels-A2
2 subjects
0.655 (0.009)
0.251 (0.011)
0.820 (0.110)
0.182 (0.021)
0.620 (0.079)
0.185 (0.064)
0.216 (0.227)
0.233 (0.241)
Phantom
2 subjects
0.526 (0.138)
0.255 (0.007)
0.856 (0.074)
0.188 (0.015)
0.658 (0.041)
0.215 (0.034)
0.396 (0.047)
0.424 (0.050)
Appendix
Table 6: Fine-grained quantitative comparison of subject-consistent video generation across different numbers of subjects. ( ⋅ ) denotes that the respective method yields a lower value than our Lumos X on this metric, whereas ( ⋅ ) indicates better performance. Values in parentheses are recalibrated differences computed as (Method − Lumos X ) for the same setting and metric.
Methods
Entire Video
Extracted Subjects
Dynamic ↑
ViCLIP-T ↑
CLIP-T↑
CLIP-I ↑
DINO-I↑
ArcSim↑
CurSim↑
SkyReels-A2 [ 8 ]
0.839
0.235
0.207
0.614
0.173
0.160
0.166
Phantom [ 24 ]
0.929
0.255
0.226
0.623
0.195
0.271
0.305
Lumos X
0.804
0.247
0.232
0.668
0.336
0.317
0.346
Appendix
Table 7: Comparison of different methods for subject-consistent video generation, including evaluation of the entire video and evaluation on subjects.
Figure 8: Visualization of 10-person T2V results of our baseline model Wan2.1-T2V-1.3B.
Methods
Subject Consistency ↑
Background Consistency ↑
Motion Smoothness ↑
Face Consistency ↑
SkyReels-A2 [ 8 ]
0.654
0.798
0.979
0.739
Phantom [ 24 ]
0.768
0.853
0.986
0.883
Lumos X
0.962
0.946
0.988
0.895
Appendix
Table 8: Quantitative comparison of temporal coherence metrics on subject-consistent video generation.
Method
#Base Model
Text-S ↑
Vid-S ↑
Subj-S ↑
Dync-D ↑
SkyReels-A2 [ 8 ]
Wan2.1-14B-T2V
0.253
0.781
0.554
0.783
Phantom [ 24 ]
Wan2.1-1.3B-T2V
0.270
0.696
0.534
0.461
Lumos X
Wan2.1-1.3B-T2V
0.258
0.707
0.549
0.786
Appendix
Table 9: Quantitative comparison on MSRVTT-personalization [ 5 ] benchmark. boldfacen and underline font indicate the highest and the second highest results.
Figure 9: Visual effectiveness of individual components proposed in our Lumos X .
Figure 11: Qualitative comparison under varying control hyperparameters r in MCAM.
Method
FID ↓
AI Judgement ↑
Stable-Diffusion-2-Inpainting
96.32
36%
FLUX-Inpainting
92.83
64%
Appendix
Table 10: Comparison between Stable-Diffusion-2 [ 33 ] and FLUX [ 21 ] for background inpainting.
Figure 12: Qualitative comparison between Stable-Diffusion-2 and FLUX for background inpainting.
Figure 13: Human study results for multi-subject video customization. (a) Results across the four evaluation dimensions. (b) Average scores computed over all four dimensions.
Methods
Attribute Control
CLIP-T
CLIP-I
DINO-I
SkyReels-A2 [ 8 ]
text-only
0.192
0.643
0.192
text&visual
0.178
0.606
0.192
Phantom [ 24 ]
text-only
0.207
0.687
0.209
text&visual
0.185
0.647
0.216
Lumos X
text-only
0.205
0.684
0.210
text&visual
0.193
0.681
0.265
Appendix
Table 11: Quantitative comparison of text-only vs. text&visual attribute control across methods.
Methods
Entire Video
Extracted Subjects
Dynamic ↑
ViCLIP-T ↑
ViCLIP-V ↑
CLIP-T↑
CLIP-I ↑
DINO-I↑
ArcSim↑
CurSim↑
UNO + Wan2.1-I2V-14B
0.547
0.261
0.880
0.201
0.650
0.197
0.237
0.244
Lumos X
0.723
0.260
0.932
0.201
0.692
0.261
0.454
0.483
Appendix
Table 12: Comparison of image-personalization–based method and Lumos X for subject-consistent video generation, including evaluation of the entire video and evaluation on subjects.
Methods
Self-Attn Latency
Cross-Attn Latency
Latency/step
FLOPs/step
GPU Memory Usage
None
0.1440s
0.0026s
8.66s
195.44 T
21.5G
+R2PE
0.1441s
0.0025s
8.66s
195.44 T
21.5G
None (MagiAttention in Self-Attn)
0.0935s
0.0025s
5.79s
195.44 T
21.5G
+R2PE (MagiAttention in Self-Attn)
0.0936s
0.0026s
5.79s
195.44 T
21.5G
+R2PE+CSAM
0.0966s
0.0026s
5.81s
195.44 T
21.5G
+R2PE+CSAM+MCAM
0.0965s
0.0045s
6.11s
195.46 T
22.7G
Appendix
Table 13: Inference-stage computational overhead and latency statistics for each module under the same video customization case on an H20 GPU.
Figure 14: Qualitative comparison for identity-consistent video generation.
Figure 15: Qualitative comparison for subject-consistent video generation.
Figure 16: Qualitative comparison for subject-consistent video generation.
Subject-driven and multi-element video generation are central to controllable video synthesis, but existing methods still struggle to preserve identity consistency and model complex relationships among multiple subjects. In this paper, we propose Aura, a unified framework for high-fidelity and identity-consistent video generation. To better capture scene dynamics and subject interactions, we introduce AI director-level captions that provide dense and structured descriptions of video content. We further leverage a vision-language model (VLM) with learnable queries to extract multimodal semantic features from textual and visual references, covering both global semantics and fine-grained visual cues. To bridge the representational gap between the VLM and the Diffusion Transformer (DiT), we design a two-stage alignment strategy that progressively maps VLM features into the DiT feature space. For visual conditioning, we adopt token concatenation to inject reference information directly into the generation process. To distinguish heterogeneous subject types and reduce common copy-paste artifacts, we develop a subject-aware RoPE-Shift mechanism. To further differentiate reference images of different categories, we introduce subject-aware learnable tokens. In addition, we introduce Memory Tokens to balance the training signal across examples with different numbers of reference subjects. During inference, Progressive-APG (Adaptive Prompt Guidance) further alleviates oversaturation and improves semantic alignment with user prompts. Finally, we build a high-quality video-subject image dataset through a dedicated data construction pipeline. Extensive experiments show that our method achieves state-of-the-art performance on both single-subject generation and more challenging multi-element scenarios.
Subject-to-video (S2V) generation has made substantial progress in preserving reference subjects across diverse categories, yet generic subject consistency remains insufficient for human-centric generation. A video may appear globally consistent while identity-critical human details still drift across frames, poses, and interactions. This issue becomes more severe in multi-person scenarios, where incorrect identity-role binding leads to subject confusion, attribute swapping, and excessive copying of reference-specific appearance cues. We propose Vera, a unified human-centric S2V framework for single- and multi-person generation. We first construct a million-pair identity-aligned human image-video dataset through person-level cross-clip retrieval, providing explicit identity correspondence and diverse references. Built on this dataset, Vera introduces two complementary designs. Identity-Focal Masked Supervision (IFMS) strengthens identity-aware learning with spatially focused supervision while reducing interference from irrelevant artifacts. Reference-Aware Layer-wise Attention (RALA) regulates how video tokens interact with reference identity cues in the DiT backbone, preserving stable identity anchors and enhancing layer-aware identity readout. Extensive experiments demonstrate that Vera improves human identity consistency, multi-person subject binding, and motion naturalness, while reducing identity confusion and excessive reference-image copying.
Yulong Xu, Xinyue Liu, Shujuan Li +6
Kuaishou Technology · Institute of Automation, Chinese Academy of Sciences · Tsinghua University
Recent progress in video generation has shown impressive visual synthesis capabilities. However, open-domain customized video generation remains limited by the lack of large-scale, annotated datasets capturing diverse identity-specific attributes. To address this, we introduce PexelsCustom-1M, the first publicly available million-scale dataset for identity-preserving video generation, containing one million curated <identity, text, video> triplets across 8,000+ categories. Leveraging this, we propose CustoMDiT, a parameter-efficient framework that adapts a pretrained multimodal Diffusion Transformer into a customized video generator with only 8% additional learnable parameters. Our method surpasses prior state-of-the-art. However, benchmarks such as DreamBooth cover only 100 classes, which is insufficient for real-world applications. To overcome this, we construct OpenCustom, a new benchmark with 1,000+ categories, created via cross-dataset knowledge fusion from ImageNet and MS-COCO. Extensive experiments confirm the advantages of both our dataset and model. We will open-source the entire ecosystem--including dataset, pipeline, benchmark, and implementations--to support further research.
Jingxu Zhang, Yuqian Hong, Daneul Kim +6
University of Science and Technology of China · 2Microsoft Research Asia · *Work done during internship at Microsoft Research Asia. +1