State-of-the-art models for audio-driven digital human generation have achieved photo-realistic results in talking-head synthesis. However, extending these models to singing-head synthesis remains challenging due to a significant Domain Gap: singing requires more exaggerated expressions, vivid jaw openings, and precise rhythmic synchronization. Current models, primarily trained on speech datasets, often struggle with "rhythmic drift" and constrained dynamics. To address this, we introduce Hi-Singers, the first large-scale, high-quality, in-the-wild video dataset specifically tailored for singing head synthesis. Hi-Singers undergoes a rigorous automated and manual filtering pipeline, ensuring strict thematic adherence, high-resolution rendering, and stable motion, resulting in 29,608 video segments totaling approximately 170 hours. We further establish a dedicated evaluation benchmark balanced across linguistic and musical styles. Extensive experiments across diverse architectures, including 3D-coefficient and diffusion-based models, demonstrate that Hi-Singers consistently and significantly improves performance across all dimensions. Specifically, it enables models to achieve superior visual realism, enhanced lip-sync consistency, and more precise rhythmic dynamics, effectively bridging the domain gap and setting a new performance standard for the singing synthesis task. The dataset is available at https://huggingface.co/datasets/CharlesZhang-USTC/Hi-Singers
Figures & tables
Figure 1. The framework diagram of the data processing workflow illustrates the entire process from raw internet data to the final dataset. This process can be divided into the following three stages: 1) Data collection and preprocessing 2) Automated filtering 3) Manual filtering.
Figure 2. This Sankey diagram illustrates the flow of data during the processing stages. Specifically, it shows the data filtered out and retained at each step of the filtering process.
Figure 3. This diagram displays sample video frames representing the primary music genres within the dataset (pop, classical, hip-hop and rock).
Dataset
Modality / Task
Setting
Res.
Scale
RAVDESS ( Livingstone and Russo, 2018 )
Emotion / Speech
Lab
720p
∼ 1 hr
MusicFace ( Liu et al., 2024 )
3D Mesh / Anim.
Studio
-
∼ 40 hrs
Song2Face ( Iwase et al., 2020 )
3D Animation
Lab
-
∼ 2 hrs
SingingHead ( Gao et al., 2025 )
4D Scan / 3D Anim.
Lab
1024p
∼ 27 hrs
ChorusHead ( Xing et al., 2025 )
3D Head Anim.
In-the-wild
-
∼ 8 hrs
RapVerse ( Chen et al., 2025a )
Body Motion / 3D
In-the-wild
-
∼ 27 hrs
Table 1. Comparison of existing singing-related datasets. Most prior works focus on 3D animation or small-scale lab environments, leaving a gap for large-scale 2D video generation.
Figure 4. This chart presents statistical data on the distribution of languages and music genres within the dataset. It is evident that the dataset contains roughly equal numbers of English and Chinese language entries; the music genres are more diverse, with pop music dominating as the predominant style.
Figure 5. The reference coordinate system for facial Euler angles (yaw, pitch, and roll) is illustrated in the leftmost diagram. The three scenarios requiring filtering are depicted in the subsequent diagrams, each representing excessive angular deviations relative to a face perfectly aligned with the camera within the coordinate system.
Category / Metrics
Calculation
Approach
1. Head Region Processing
Face Height
Key Points
TBF
Lip-sync Alignment
SyncNet Score
TBF
2. Motion Characteristics Filtering
Facial Movement
Mean Displacement
TBF
Facial Rotation
Face Eular Angle
TBF
Table 2. Tables for the filtering stage, metrics, calculation, and filtering approach (TBF or SBF)
Method
FID ↓
FVD ↓
Sync-C ↑
BAS ↑
MOS ↑
Additional SOTA Baselines (Domain Gap Evaluation)
SadTalker ( Zhang et al., 2023 ) (Orig.)
38.54
245.12
6.85
0.052
3.12
V-Express ( Wang et al., 2024 ) (Orig.)
25.68
192.45
6.92
0.076
3.45
Main Results and Ablations
Hallo (Original Weights)
23.21
188.34
7.58
0.139
3.85
Hallo (CelebV-HQ ( Wang et al., 2022 ) )
42.23
385.67
7.12
0.111
3.63
Table 3. Overall performance comparison on singing head generation. Hi-Singers achieves SOTA results across all dimensions. ‘Orig.’ denotes official weights trained on generic talking datasets.
Training Dataset
Metric
Language
Music Style
Chinese
English
Pop
Classical
Rock
Hip-hop
Other
Hallo (Original)
FID ↓
23.30
23.12
23.10
23.25
23.18
23.30
23.22
FVD ↓
190.18
186.50
187.20
189.50
187.80
188.50
188.70
Sync-C ↑
7.50
7.66
7.54
7.55
7.40
7.85
7.56
Sync-D ↓
7.98
7.58
7.90
7.85
8.02
7.25
7.88
BAS ↑
0.136
0.142
0.140
0.135
0.138
0.145
0.137
Table 4. Detailed Cross-Domain Analysis. We analyze the performance of Hallo (Original) and Hallo (Hi-Singers) across different categories. The averages of sub-categories are fitted to the overall performance results. BAS quantifies rhythmic synchronization.
Figure 6. Qualitative Comparison. Top: Baseline trained on generic datasets shows constrained motion and blur. Bottom: Ours trained on Hi-Singers displays vivid jaw openings and sharpness.