LumosX: Relate Any Identities with Their Attributes for Personalized Video Generation

Jiazheng Xing; Fei Du; Hangjie Yuan; Pengwei Liu; Hongbin Xu; Hai Ci; Ruigang Niu; Weihua Chen

cs.CVMar 20, 2026

LumosX: Relate Any Identities with Their Attributes for Personalized Video Generation

Authors: Jiazheng Xing, Fei Du, Hangjie Yuan, Pengwei Liu, Hongbin Xu, Hai Ci, Ruigang Niu, Weihua Chen,

Organizations: Zhejiang University · National University of Singapore · DAMO Academy, Alibaba Group · Hupan Lab

Abstract

Recent advances in diffusion models have significantly improved text-to-video generation, enabling personalized content creation with fine-grained control over both foreground and background elements. However, precise face-attribute alignment across subjects remains challenging, as existing methods lack explicit mechanisms to ensure intra-group consistency. Addressing this gap requires both explicit modeling strategies and face-attribute-aware data resources. We therefore propose LumosX, a framework that advances both data and model design. On the data side, a tailored collection pipeline orchestrates captions and visual cues from independent videos, while multimodal large language models (MLLMs) infer and assign subject-specific dependencies. These extracted relational priors impose a finer-grained structure that amplifies the expressive control of personalized video generation and enables the construction of a comprehensive benchmark. On the modeling side, Relational Self-Attention and Relational Cross-Attention intertwine position-aware embeddings with refined attention dynamics to inscribe explicit subject-attribute dependencies, enforcing disciplined intra-group cohesion and amplifying the separation between distinct subject clusters. Comprehensive evaluations on our benchmark demonstrate that LumosX achieves state-of-the-art performance in fine-grained, identity-consistent, and semantically aligned personalized multi-subject video generation. Code and models are available at https://jiazheng-xing.github.io/lumosx-home/.

Figures & tables

Figure 1: Lumos $\mathcal{X}$ supports flexible personalized multi-subject video generation.

Figure 2: Dataset construction pipeline for personalized multi-subject video generation. We build the training dataset from raw videos in three steps: (1) generate a caption and detect human subjects in extracted frames; (2) retrieve entity words from the caption and match subjects with their attributes; (3) use these entity tags to localize and segment target subjects and objects, producing a clean background image.

Figure 3: Overview of Lumos $\mathcal{X}$ . Built on the T2V model Wan2.1 [ 40 ] , our framework encodes all condition images into image tokens via a VAE encoder, concatenates them with denoising video tokens, and feeds the result into DiT [ 31 ] blocks. Within each block, the proposed Relational Self-Attention and Relational Cross-Attention enable causal conditional modeling, enhance visual token representations, and ensure precise face–attribute alignment.

Figure 4: Illustration of Attention Mask Design: Causal Self-Attention Mask (CSAM) and Multilevel Cross-Attention Mask (MCAM). We present a specific customized task case as an example.

Table 5

Methods	Entire Video			Extracted Subjects
Methods	Dynamic $\uparrow$	ViCLIP-T $\uparrow$	ViCLIP-V $\uparrow$	$\text{CLIP-T}\uparrow$	CLIP-I $\uparrow$	$\text{DINO-I}\uparrow$	$\text{ArcSim}\uparrow$	$\text{CurSim}\uparrow$
SkyReels-A2 [ 8 ]	0.671	0.251	0.839	0.178	0.606	0.192	0.271	0.290
Phantom [ 24 ]	0.661	0.254	0.865	0.185	0.647	0.216	0.444	0.477
Lumos $\mathcal{X}$	0.723	0.260	0.932	0.201	0.692	0.261	0.454	0.483

Table 3: Comparison of different methods for subject-consistent video generation, including evaluation of the entire video and evaluation on the extracted subjects.

Figure 5: Qualitative comparison for identity-consistent and subject-consistent video generation.

Methods	$\text{CLIP-T}\uparrow$	ArcSim $\uparrow$
None	0.184	0.316
+R2PE	0.178	0.363
+R2PE + CSAM	0.182	0.363
+R2PE + CSAM + MCAM ( $0.1$ )	0.182	0.364
+R2PE + CSAM + MCAM ( $0.5$ )	0.186	0.429
+R2PE + CSAM + MCAM ( $1.0$ )	0.187	0.384

Table 4: Ablation study for the subject-consistent video generation. The value in the MACM parentheses is

r

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Process of entity words retrieval and subject-attributes matching with four steps: Data Collection, Data Organization, Data Processing, and Data Annotation.

Figure 7: Process of acquiring condition images of subjects, objects, and the background.

Methods	Numbers	Identity Consistency		Prompt Following
Methods	Numbers	ArcSim $\uparrow$	CurSim $\uparrow$	ViCLIP-T $\uparrow$
SkyReels-A2 [ 8 ]	1 face	0.509	0.540	0.258
Phantom [ 24 ]	1 face	0.602	0.637	0.261
Lumos $\mathcal{X}$	1 face	0.542	0.575	0.262
SkyReels-A2 [ 8 ]	$\geq$ 2 faces	0.282	0.292	0.263
Phantom [ 24 ]	$\geq$ 2 faces	0.434	0.457	0.266

Appendix

Table 5: Fine-grained quantitative comparison of identity-consistent video generation across different numbers of faces.

Methods	Numbers	Entire Video			Extracted Subjects
Methods	Numbers	Dynamic $\uparrow$	ViCLIP-T $\uparrow$	ViCLIP-V $\uparrow$	$\text{CLIP-T}\uparrow$	CLIP-I $\uparrow$	$\text{DINO-I}\uparrow$	$\text{ArcSim}\uparrow$	$\text{CurSim}\uparrow$
SkyReels-A2	1 subject	0.772 (0.080)	0.252 (0.007)	0.870 (0.064)	0.177 (0.022)	0.604 (0.082)	0.199 (0.069)	0.356 (0.133)	0.380 (0.136)
Phantom	1 subject	0.848 (0.004)	0.254 (0.005)	0.880 (0.054)	0.186 (0.013)	0.646 (0.040)	0.215 (0.053)	0.539 (0.050)	0.581 (0.065)
Lumos $\mathcal{X}$	1 subject	0.852	0.259	0.934	0.199	0.686	0.268	0.489	0.516
SkyReels-A2	2 subjects	0.655 (0.009)	0.251 (0.011)	0.820 (0.110)	0.182 (0.021)	0.620 (0.079)	0.185 (0.064)	0.216 (0.227)	0.233 (0.241)
Phantom	2 subjects	0.526 (0.138)	0.255 (0.007)	0.856 (0.074)	0.188 (0.015)	0.658 (0.041)	0.215 (0.034)	0.396 (0.047)	0.424 (0.050)

Appendix

Table 6: Fine-grained quantitative comparison of subject-consistent video generation across different numbers of subjects. (

\cdot

) denotes that the respective method yields a lower value than our Lumos

\mathcal{X}

on this metric, whereas (

\cdot

) indicates better performance. Values in parentheses are recalibrated differences computed as (Method

-

Lumos

\mathcal{X}

) for the same setting and metric.

Methods	Entire Video		Extracted Subjects
Methods	Dynamic $\uparrow$	ViCLIP-T $\uparrow$	$\text{CLIP-T}\uparrow$	CLIP-I $\uparrow$	$\text{DINO-I}\uparrow$	$\text{ArcSim}\uparrow$	$\text{CurSim}\uparrow$
SkyReels-A2 [ 8 ]	0.839	0.235	0.207	0.614	0.173	0.160	0.166
Phantom [ 24 ]	0.929	0.255	0.226	0.623	0.195	0.271	0.305
Lumos $\mathcal{X}$	0.804	0.247	0.232	0.668	0.336	0.317	0.346

Appendix

Table 7: Comparison of different methods for subject-consistent video generation, including evaluation of the entire video and evaluation on subjects.

Figure 8: Visualization of 10-person T2V results of our baseline model Wan2.1-T2V-1.3B.

Methods	Subject Consistency $\uparrow$	Background Consistency $\uparrow$	Motion Smoothness $\uparrow$	Face Consistency $\uparrow$
SkyReels-A2 [ 8 ]	0.654	0.798	0.979	0.739
Phantom [ 24 ]	0.768	0.853	0.986	0.883
Lumos $\mathcal{X}$	0.962	0.946	0.988	0.895

Appendix

Table 8: Quantitative comparison of temporal coherence metrics on subject-consistent video generation.

Method	#Base Model	Text-S $\uparrow$	Vid-S $\uparrow$	Subj-S $\uparrow$	Dync-D $\uparrow$
SkyReels-A2 [ 8 ]	Wan2.1-14B-T2V	0.253	0.781	0.554	0.783
Phantom [ 24 ]	Wan2.1-1.3B-T2V	0.270	0.696	0.534	0.461
Lumos $\mathcal{X}$	Wan2.1-1.3B-T2V	0.258	0.707	0.549	0.786

Appendix

Table 9: Quantitative comparison on MSRVTT-personalization [ 5 ] benchmark. boldfacen and underline font indicate the highest and the second highest results.

Figure 9: Visual effectiveness of individual components proposed in our Lumos $\mathcal{X}$ .

Figure 10: Visualization of attention maps. (i): Self-Attention similarity score. (ii): Cross-Attention similarity score.

Figure 11: Qualitative comparison under varying control hyperparameters $r$ in MCAM.

Method	FID $\downarrow$	AI Judgement $\uparrow$
Stable-Diffusion-2-Inpainting	96.32	36%
FLUX-Inpainting	92.83	64%

Appendix

Table 10: Comparison between Stable-Diffusion-2 [ 33 ] and FLUX [ 21 ] for background inpainting.

Figure 12: Qualitative comparison between Stable-Diffusion-2 and FLUX for background inpainting.

Figure 13: Human study results for multi-subject video customization. (a) Results across the four evaluation dimensions. (b) Average scores computed over all four dimensions.

Methods	Attribute Control	CLIP-T	CLIP-I	DINO-I
SkyReels-A2 [ 8 ]	text-only	0.192	0.643	0.192
SkyReels-A2 [ 8 ]	text&visual	0.178	0.606	0.192
Phantom [ 24 ]	text-only	0.207	0.687	0.209
Phantom [ 24 ]	text&visual	0.185	0.647	0.216
Lumos $\mathcal{X}$	text-only	0.205	0.684	0.210
Lumos $\mathcal{X}$	text&visual	0.193	0.681	0.265

Appendix

Table 11: Quantitative comparison of text-only vs. text&visual attribute control across methods.

Methods	Entire Video			Extracted Subjects
Methods	Dynamic $\uparrow$	ViCLIP-T $\uparrow$	ViCLIP-V $\uparrow$	$\text{CLIP-T}\uparrow$	CLIP-I $\uparrow$	$\text{DINO-I}\uparrow$	$\text{ArcSim}\uparrow$	$\text{CurSim}\uparrow$
UNO + Wan2.1-I2V-14B	0.547	0.261	0.880	0.201	0.650	0.197	0.237	0.244
Lumos $\mathcal{X}$	0.723	0.260	0.932	0.201	0.692	0.261	0.454	0.483

Appendix

Table 12: Comparison of image-personalization–based method and Lumos

\mathcal{X}

for subject-consistent video generation, including evaluation of the entire video and evaluation on subjects.

Methods	Self-Attn Latency	Cross-Attn Latency	Latency/step	FLOPs/step	GPU Memory Usage
None	0.1440s	0.0026s	8.66s	195.44 T	21.5G
+R2PE	0.1441s	0.0025s	8.66s	195.44 T	21.5G
None (MagiAttention in Self-Attn)	0.0935s	0.0025s	5.79s	195.44 T	21.5G
+R2PE (MagiAttention in Self-Attn)	0.0936s	0.0026s	5.79s	195.44 T	21.5G
+R2PE+CSAM	0.0966s	0.0026s	5.81s	195.44 T	21.5G
+R2PE+CSAM+MCAM	0.0965s	0.0045s	6.11s	195.46 T	22.7G

Appendix

Table 13: Inference-stage computational overhead and latency statistics for each module under the same video customization case on an H20 GPU.

Figure 14: Qualitative comparison for identity-consistent video generation.

Video Generation

Video generation studies models that create and edit moving images from text, images, audio, and structured controls.

See all 364 papers

Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
Jul 5, 2026Zixiang Zhou, Zhentao Yu, Yifeng Ma +9Video Diffusion Models Identity-Preserving Video Generation
Tencent Hunyuan
Vera: Identity-Faithful Human Subject-to-Video Generation
Jul 22, 2026Yulong Xu, Xinyue Liu, Shujuan Li +6Video Diffusion Models Identity-Preserving Video Generation
Kuaishou Technology · Institute of Automation, Chinese Academy of Sciences · Tsinghua University
A Comprehensive Ecosystem for Open-Domain Customized Video Generation
Jun 10, 2026Jingxu Zhang, Yuqian Hong, Daneul Kim +6Video-Reference Video Generation Identity-Preserving Video Generation
University of Science and Technology of China · 2Microsoft Research Asia · *Work done during internship at Microsoft Research Asia. +1

Jul 5, 2026J/K move · Enter open · S save

Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment

Zixiang Zhou, Zhentao Yu, Yifeng Ma, Hongmei Wang+8

Tencent Hunyuan

Video Diffusion Models Identity-Preserving Video Generation Video Generation Vision-Language Alignment

Open paper AskPDF arXiv

Subject-driven and multi-element video generation are central to controllable video synthesis, but existing methods still struggle to preserve identity consistency and model complex relationships among multiple subjects. In this paper, we propose Aura, a unified framework for high-fidelity and identity-consistent video generation. To better capture scene dynamics and subject interactions, we introduce AI director-level captions that provide dense and structured descriptions of video content. We further leverage a vision-language model (VLM) with learnable queries to extract multimodal semantic features from textual and visual references, covering both global semantics and fine-grained visual cues. To bridge the representational gap between the VLM and the Diffusion Transformer (DiT), we design a two-stage alignment strategy that progressively maps VLM features into the DiT feature space. For visual conditioning, we adopt token concatenation to inject reference information directly into the generation process. To distinguish heterogeneous subject types and reduce common copy-paste artifacts, we develop a subject-aware RoPE-Shift mechanism. To further differentiate reference images of different categories, we introduce subject-aware learnable tokens. In addition, we introduce Memory Tokens to balance the training signal across examples with different numbers of reference subjects. During inference, Progressive-APG (Adaptive Prompt Guidance) further alleviates oversaturation and improves semantic alignment with user prompts. Finally, we build a high-quality video-subject image dataset through a dedicated data construction pipeline. Extensive experiments show that our method achieves state-of-the-art performance on both single-subject generation and more challenging multi-element scenarios.

Abstract

Video Generation

Explore similar work

Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment

Vera: Identity-Faithful Human Subject-to-Video Generation

A Comprehensive Ecosystem for Open-Domain Customized Video Generation