Single-view reference-to-video methods often struggle to preserve identity consistency under large facial-angle variations. This limitation naturally motivates the incorporation of multi-view facial references. However, simply introducing additional reference images exacerbates the copy-paste problem, particularly the view-dependent copy-paste artifact, which reduces facial motion naturalness. Although cross-paired data can alleviate this issue, collecting such data is costly. To balance the consistency and naturalness, we propose Mv2ID, a multi-view conditioned framework under in-paired supervision. We introduce a region-masking training strategy to prevent shortcut learning and extract essential identity features by encouraging the model to aggregate complementary identity cues across views. In addition, we design a reference decoupled-RoPE mechanism that assigns distinct positional encoding to video and conditioning tokens for better modeling of their heterogeneous properties. Furthermore, we construct a large-scale dataset with diverse facial-angle variations and propose dedicated evaluation metrics for identity consistency and motion naturalness. Extensive experiments demonstrate that our method significantly improves identity consistency while maintaining motion naturalness, outperforming existing approaches trained with cross-paired data.
Figures & tables
Figure 1: In this work, we leverage multi-view references to improve identity consistency under large facial angle variations, while maintaining motion naturalness (see enlarged view for trajectory details).
Figure 2: Copy-paste : Within certain time intervals, the generated video collapses to a specific reference image, resulting in reduced motion diversity(more detailed analysis can be found in the Observation in Sec. 3 ).
Figure 3: The overview of the training pipeline. We design the region masking training(Sec. 4.3) and reference-decoupled RoPE(Sec. 4.4). For inference, we use the clean condition images.
Figure 4: Qualitative comparison under large facial-angle changes. Our method,Mv 2 ID, better balances the consistency and facial motion naturalness.
Figure 5: Facial trajectory comparison. Each dot represents the normalized 3D facial direction vector of a frame, and the black dot denotes the origin. Colors indicate the temporal progression of frames, normalized to the range [0,1] across 81 frames.
Figure 6: Single-view vs. Multi-view and qualitative comparison of proposed components. R: RD-RoPE. M : Region-Masking Training.
Figure 7: Facial trajectory for ablation.
Figure 8: User-study results.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Overview of our dataset construction pipeline. We progressively filter raw videos through coarse filtering, clip-level processing, and pose-level processing to obtain human-centric video clips with large facial-angle variation.
Dataset
Human-centric
Large-Angle
Pose
OpenS2V-5M [ 51 ]
×
×
×
Phantom-Data [ 3 ]
×
×
×
HuMoSet [ 2 ]
✓
×
×
Ours
✓
✓
✓
Appendix
Table 3: Comparison with representative open-source subject-to-video related datasets. Our dataset explicitly focuses on human-centric samples with large facial-angle variations.
Figure 10: The high-level understanding of VM and RM.
Figure 11: The visualization results of VM and RM.
Figure 12: Qualitative comparison of different masking ratios in Region Masking. A higher masking ratio helps mitigate the copy-paste issue and produces more natural results.
Masking Ratio
NaturalScore
40%
4.13
60%
4.30
Appendix
Table 4: NaturalScore under different masking ratios. Higher is better.
Figure 13: The h/w offset can accelerate the convergence of training. The model with the h/w offset learns to utilize reference information within only 450 training steps.
Figure 14: More visual results of our method.
Figure 15: The facial trajectory results of our method and compared methods.
Current identity-consistent video generation methods struggle to preserve appearance fidelity under large viewpoint changes. While introducing multi-view reference input offers a natural solution, progress remains constrained by the lack of effective frameworks for multi-view inputs and the scarcity of multi-view data. We address these challenges by proposing HarmoView, a robust framework for identity-consistent video generation that effectively integrates multi-view cues through three architectural refinements complemented by a staged training curriculum. Specifically, we first introduce Multi-level Feature Injection to anchor identity fidelity; by injecting raw ViT features from frontal references alongside text tokens via cross-attention, MFI provides persistent low-level appearance anchors that complement the high-level identity features within DiT blocks, leading to enhanced identity preservation. Then, we employ learnable proxy tokens to unify heterogeneous reference layouts across single-/multi-view settings while simultaneously resolving the reference-view mismatch problem. Jump-RoPE is further developed for identity-wise feature isolation to reduce identity crosstalk. To activate these structural capabilities while preserving the original generative priors, we propose the Progressive View Curriculum. This four-stage training strategy employs view dropout to facilitate a stable transition from vanilla T2V generation to high-fidelity, identity-persistent spatial reasoning. Furthermore, we construct a large-scale multi-view dataset to address the issue of data scarcity. Extensive evaluation on our multi-view benchmark, comprising 100 manually-curated cases spanning 52 unique identities, demonstrates that HarmoView significantly outperforms open-source baselines and matches leading closed-source engines, achieving state-of-the-art performance in identity-consistent video generation.
Subject-to-video (S2V) generation has made substantial progress in preserving reference subjects across diverse categories, yet generic subject consistency remains insufficient for human-centric generation. A video may appear globally consistent while identity-critical human details still drift across frames, poses, and interactions. This issue becomes more severe in multi-person scenarios, where incorrect identity-role binding leads to subject confusion, attribute swapping, and excessive copying of reference-specific appearance cues. We propose Vera, a unified human-centric S2V framework for single- and multi-person generation. We first construct a million-pair identity-aligned human image-video dataset through person-level cross-clip retrieval, providing explicit identity correspondence and diverse references. Built on this dataset, Vera introduces two complementary designs. Identity-Focal Masked Supervision (IFMS) strengthens identity-aware learning with spatially focused supervision while reducing interference from irrelevant artifacts. Reference-Aware Layer-wise Attention (RALA) regulates how video tokens interact with reference identity cues in the DiT backbone, preserving stable identity anchors and enhancing layer-aware identity readout. Extensive experiments demonstrate that Vera improves human identity consistency, multi-person subject binding, and motion naturalness, while reducing identity confusion and excessive reference-image copying.
Yulong Xu, Xinyue Liu, Shujuan Li +6
Kuaishou Technology · Institute of Automation, Chinese Academy of Sciences · Tsinghua University
Identity-preserving text-to-video generation (IPT2V) empowers users to produce diverse and imaginative videos with consistent human facial identity. Despite recent progress, existing methods often suffer from significant identity distortion under large facial pose variations or facial occlusions. In this paper, we propose FaithfulFaces, a pose-faithful facial identity preservation learning framework to improve IPT2V in complex dynamic scenes. The key of FaithfulFaces is a pose-shared identity aligner that refines and aligns facial poses across distinct views via a pose-shared dictionary and a pose variation-identity invariance constraint. By mapping single-view inputs into a global facial pose representation with explicit Euler angle embeddings, FaithfulFaces provides a pose-faithful facial prior that guides generative foundations toward robust identity-preserving generation. In particular, we develop a specialized pipeline to curate a high-quality video dataset featuring substantial facial pose diversity. Extensive experiments demonstrate that FaithfulFaces achieves state-of-the-art performance, maintaining superior identity consistency and structural clarity even as pose changes and occlusions occur. Project Page: https://mdswyz.github.io/FaithfulFaces/
Yuanzhi Wang, Xuhua Ren, Jiaxiang Cheng +7
Nanyang Technological University · Tencent Hunyuan · University of Science and Technology of China +1