Emotional 3D talking head generation aims to synthesize expressive facial animations with accurate lip synchronization. However, existing methods often rely on discrete emotion categories, which fail to capture the continuous evolution of affect. They also overlook the temporal frequency mismatch between audio articulation and emotional expression. In this paper, we propose CETalk, an audio-driven 3D facial animation framework conditioned on continuous Valence--Arousal (VA) representations for fine-grained emotion control. CETalk predicts a sequence of FLAME parameters through three key components: a Dynamic Emotion Modulation Module that adaptively scales emotional intensity using audio-derived cues; a Multi-Scale Temporal Modeling mechanism that employs parallel branches to decouple high-frequency articulatory movements from low-frequency emotional dynamics; and a Dynamic Fusion Mechanism that integrates these multi-scale features via an adaptive gating network. To support training and evaluation, we construct 3D-VA-MEAD, a large-scale dataset with automatically estimated VA annotations and reconstructed 3D facial motions. Extensive experiments demonstrate that CETalk outperforms state-of-the-art methods in both lip-sync accuracy and emotional expressiveness, while enabling smooth and controllable emotion transitions.
Figures & tables
Figure 1: Overview of CETalk. Given input audio and continuous Valence–Arousal (VA) labels, the model synthesizes expressive 3D facial animations with accurate lip synchronization and controllable emotional dynamics.
Figure 2: Valence–Arousal affective space for facial expressions. Emotions are represented in a continuous two-dimensional space defined by valence (positive–negative) and arousal (active–passive). Different regions correspond to distinct emotional states (e.g., happy, surprised, sad), illustrating how facial expressions vary smoothly across the VA plane.
Figure 3: Overview of the CETalk framework. Given input audio A , speaker identity S , and continuous emotion condition Econd in the Valence–Arousal space, the model predicts a sequence of FLAME parameters P . The Dynamic Emotion Modulation module transforms Econd into a dynamic emotion embedding Emod , the Multi-Scale Temporal Modeling module extracts high-frequency articulatory features FHF and low-frequency emotional features FLF , and the Dynamic Fusion Mechanism integrates them to generate expressive 3D facial animations.
Method
MEAD
RAVDESS
HDTF
LVE ↓
MVE ↓
EVE ↓
LVE ↓
MVE ↓
EVE ↓
LVE ↓
MVE ↓
FaceFormer [ 12 ]
19.1505
17.2842
2.5844
16.0241
13.7128
2.3885
10.3196
11.6200
CodeTalker [ 28 ]
19.3656
17.0043
2.5571
15.9058
13.7034
2.5207
11.0166
11.3826
EMOTE [ 8 ]
10.2828
13.4281
3.4535
7.9951
13.4142
5.9953
5.2671
11.9896
ProbTalk3D [ 27 ]
11.5005
12.7797
2.2024
7.7616
12.1496
3.5825
5.6203
10.9151
UniTalker [ 11 ]
9.5945
12.2777
2.1202
10.1716
11.6141
1.8787
7.2121
10.6160
Table 1: Quantitative comparison of facial motion accuracy ( mm ). Lower values ( ↓ ) indicate better performance. Best results are in bold and second-best are underlined .
Method
RMSE ↓
SAGR ↑
Valence
Arousal
Valence
Arousal
EMOTE [ 8 ]
0.2133
0.1219
0.7909
0.9693
ProbTalk3D [ 27 ]
0.1826
0.1059
0.8491
0.9861
UniTalker [ 11 ]
0.3118
0.1537
0.6589
0.9870
DEEPTalk [ 14 ]
0.3156
0.1310
0.7720
0.9833
Ours
0.1223
0.0747
0.9247
0.9906
Table 2: Quantitative evaluation of emotion controllability using RMSE and SAGR.
Figure 4: Qualitative comparison of generated 3D talking head animations.
Figure 5: Continuous emotion controllability via VA-space interpolation. With the same audio clip, we linearly interpolate the input Valence–Arousal condition from (−1,−1) to (1,1) .
Figure 6: Heatmap comparison of the mean and standard deviation : Ground Truth versus animations generated by different methods.
Method
LVE ↓
MVE ↓
EVE ↓
Ours
7.4772
9.9059
1.3256
w/o DEMM
9.6963
12.3722
2.4367
w/o MSTM
8.0398
10.3324
1.4123
w/o DFM
8.1931
10.6743
1.6406
Table 3: Ablation study on the MEAD dataset. LVE, MVE, and EVE are reported to evaluate the contribution of each component.
Media Integration and Communication Center (MICC), University of Florence, Italy · University of Copenhagen · IMT Nord Europe, Institut Mines-Télécom, Centre for Digital Systems +1