cs.CVFeb 25, 2026
SaveFlowPortrait: Reinforcement Learning for Audio-Driven Portrait Video Generation
Organizations: ByteDance Seed · Johns Hopkins University
Abstract
Generating realistic talking-head videos remains challenging due to persistent issues such as imperfect lip synchronization, unnatural motion, and evaluation metrics that correlate poorly with human perception. We propose FlowPortrait, a reinforcement-learning framework for audio-driven portrait animation built on a multimodal backbone for autoregressive audio-to-video generation. FlowPortrait introduces a human-aligned evaluation system based on Multimodal Large Language Models (MLLMs) to assess lip-sync accuracy, expressiveness, and motion quality. These signals are combined with perceptual and temporal consistency regularizers to form a stable composite reward, which is used to post-train the generator via Group Relative Policy Optimization (GRPO). Extensive experiments, including both automatic evaluations and human preference studies, demonstrate that FlowPortrait consistently produces higher-quality talking-head videos, highlighting the effectiveness of reinforcement learning for portrait animation.
Figures & tables
Figure 1: Overview of FlowPortrait . We build upon a pretrained MLLM to enable audio-to-video generation for portrait animation. Using an improved Flow-GRPO objective, we post-train the AR-Flow generator with a composite reward that integrates MLLM-based evaluations with perceptual and temporal consistency terms, leading to consistent improvements in generation quality.
| Base Model | Method | Acc | Acc | Cov. (%) |
|---|---|---|---|---|
| Feature / Pixel Models | ||||
| Inception-V3 [ 25 ] | FID | 27.5 | – | – |
| SyncNet [ 14 ] | LSE-C | 53.4 | – | – |
| SyncNet | LSE-D | 51.6 | – | – |
| Pixel-based | SSIM | 61.2 | – | – |
| Comparison Models | ||||
Table 1: Evaluation against human preferences. For score-based methods, Acc assigns half credit to ties. Cov. is the proportion of non-tied predictions, and Acc is accuracy among non-tied samples.
| Model | Evaluation System | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| FVD ↓ | FID ↓ | LSE-D ↓ | LSE-C ↑ | SSIM ↑ | SAS-SA ↑ | MAS-SA ↑ | MAS-MA ↑ | Human ↑ | |
| Original | – | – | – | – | – | 3.32 | 3.76 | 3.93 | 4.77 |
| Sonic | 62.25 | 44.14 | 12.05 | 2.91 | 0.633 | 2.94 | 3.43 | 3.58 | 3.56 |
| Memo | 74.94 | 41.12 | 13.65 | 0.92 | 0.668 | 2.72 | 3.31 | 3.37 | 3.21 |
| Echomimics | 97.06 | 60.15 | 13.18 | 1.51 | 0.573 | 2.55 | 3.16 | 3.26 | 2.54 |
Table 2: Scores assigned to different generation models by automated metrics, MLLM-based evaluators, and human annotators. Arrows indicate whether higher ( ) or lower ( ) values are better. Vertical separators partition evaluation systems into (i) automated metrics, (ii) MLLM-based evaluators, and (iii) human judgments.
Figure 2: Error rates of different generator types. Each bar represents the error rate for a specific generator type, with annotations indicating the number of error cases.
Figure 3: Score distribution histograms for the three evaluation aspects: lip-sync quality, facial expressiveness, and motion smoothness. Each histogram illustrates the frequency of scores assigned by Human (blue) and MAS-MA evaluation system (orange) across the range of possible scores (1 to 5).
| In-domain Test Set | Out-domain Test Set | |||||||
| Generator | Lip-sync | Expressive | Motion | Avg | Lip-sync | Expressive | Motion | Avg |
| Original | 4.03 | 4.33 | 3.43 | 3.93 | 4.74 | 4.42 | 2.93 | 4.03 |
| Previous Generators | ||||||||
| Echo | 3.37 | 3.72 | 2.67 | 3.26 | 4.12 | 3.97 | 2.44 | 3.51 |
| Memo | 3.53 | 3.88 | 2.70 | 3.37 | 4.45 | 4.10 | 2.66 | 3.74 |
| Sonic | 3.73 | 4.07 | 2.93 | 3.58 | 4.45 | 4.15 | 2.63 | 3.74 |
Table 3: MLLM-based evaluation of different video generators on in-domain and out-domain test sets.
| Generator | Lip-sync | Expressive | Motion | Avg |
|---|---|---|---|---|
| Orig | 4.78 | 4.76 | 4.68 | 4.74 |
| Sonic | 3.59 | 3.47 | 3.21 | 3.42 |
| SFT | 3.87 | 3.75 | 3.55 | 3.72 |
| RL | 4.16 | 4.07 | 3.85 | 4.03 |
Table 4: Human annotation results comparing different generators, showing that RL post-training substantially improves generation quality and alignment with human preferences.
Figure 4: Cherrypicked failure cases from our SFT model, including blurred textures and hallucinated artifacts. RL post-training effectively mitigates these issues. The black masks are used to anonymize identities.
Figure 5: Comparison of reward curve during post-training for various noise levels during sampling stage.
| Setting | W | Noise | Lip-sync | Expressive | Motion | Avg |
|---|---|---|---|---|---|---|
| SFT Baseline | N/A | N/A | 3.74 | 4.07 | 3.16 | 3.66 |
| RL Baseline ( ) | 1 | 0.5 | 4.40 | 4.40 | 3.50 | 4.10 |
| Ablation 1: Noise Level | ||||||
| Noise = 0.8 | 1 | 0.8 | 4.29 ( -0.11 ) | 4.33 ( -0.07 ) | 3.48 ( -0.02 ) | 4.03 ( -0.07 ) |
| Noise = 1.0 | 1 | 1.0 | 4.12 ( -0.28 ) | 4.29 ( -0.11 ) | 3.41 ( -0.09 ) | 3.94 ( -0.16 ) |
| Ablation 2: Window Size | ||||||
Table 5: All ablations report score differences relative to the RL baseline. We found that small window size, small noise, and adding perceptual+consistency rewards with MLLM’s three aspect rewards produce the best results.
Figure 6: Comparing training-time metrics— and —reveals a clear gap between using only the MLLM-based reward and augmenting it with perceptual and consistency terms. Without these additional rewards, both metrics deteriorate sharply, aligning with the jittering and color drift observed in our qualitative results.
| Score | Guideline | Detailed Criteria (Precision, Timing, & Naturalness) |
|---|---|---|
| 5 | Excellent | Precision: Mouth shapes perfectly match all spoken phonemes. Timing: Lip movements are perfectly synchronized with the audio, with no noticeable delay or lead. Naturalness: Movements are fluid and human-like, with no warping or visual artifacts. |
| 4 | Good | Precision: Mouth shapes are correct for the vast majority of sounds, with only subtle inaccuracies. Timing: Synchronization is excellent, with only an almost imperceptible lag or lead. Naturalness: Movements look natural but may lack fine-grained nuances. |
| 3 | Fair | Precision: Mouth shapes are generally correct but imprecise or simplified. Timing: A slight but consistent delay or lead is noticeable. Naturalness: Movements appear somewhat robotic or overly smooth; minor blurring artifacts may be present. |
| 2 | Poor | Precision: Mouth shapes are frequently incorrect. Timing: Audio–visual synchronization is clearly off and distracting. Naturalness: Movements are jerky or exhibit unnatural “flapping,” with obvious warping. |
| 1 | Bad | Precision: Lip movements have little to no correlation with speech. Timing: Audio and lip movements are severely out of sync. Naturalness: Mouth motion is highly distorted, nonsensical, or completely static. |
Table 6: Human evaluation rubric for lip-sync quality (1–5).
| Score | Guideline | Detailed Criteria (Emotional Match & Realism) |
|---|---|---|
| 5 | Excellent | Emotional Match: Facial expressions perfectly reflect the emotional tone of the audio. Realism: Includes natural micro-expressions and appears fully human and genuine. |
| 4 | Good | Emotional Match: Expressions largely match the audio’s emotion, with minor inconsistencies. Realism: Looks realistic but may feel slightly acted or less nuanced. |
| 3 | Fair | Emotional Match: Emotion is generally appropriate but muted, delayed, or underspecified. Realism: The face appears mildly uncanny or artificial. |
| 2 | Poor | Emotional Match: Expression mismatches the audio emotion or remains stuck in a neutral state. Realism: Expressions look highly unnatural and artificial. |
| 1 | Bad | Emotional Match: Expression is completely incorrect, nonsensical, or emotionless. Realism: Face resembles a static mask or mannequin. |
Table 7: Human evaluation rubric for facial expressiveness (1–5).
| Score | Guideline | Detailed Criteria (Head Movement, Subtle Motion & Consistency) |
|---|---|---|
| 5 | Excellent | Head Movement: Smooth, fluid, and purposeful. Subtle Motion: Natural blinks, head tilts, and minor posture shifts are present. Consistency: Motion is coherent and human-like throughout. |
| 4 | Good | Head Movement: Smooth and stable, but somewhat simple or repetitive. Subtle Motion: Blinking appears natural; other subtle motions may be limited. Consistency: Motion remains plausible with no major artifacts. |
| 3 | Fair | Head Movement: Slightly stiff, jerky, or floaty. Subtle Motion: Blinking is unnatural in cadence or absent for periods. Consistency: Occasional robotic or locked head poses occur. |
| 2 | Poor | Head Movement: Robotic, jittery, or laggy. Subtle Motion: No natural blinks or fine motions are observed. Consistency: Motion is highly unnatural and distracting. |
| 1 | Bad | Head Movement: Head is completely static. Subtle Motion: No blinking or facial motion at all. Consistency: The lack of motion is constant and clearly artificial. |
Table 8: Human evaluation rubric for video smoothness and motion quality (1–5).
| Prompt Name | Lip-Sync Evaluation Prompt (Single-Video, Single-Aspect) |
|---|---|
| Role | You are a specialized analyst focusing on audio-visual synchronization in digital media. Your expertise lies in phonetics and the precise mapping of mouth movements to speech. |
| Context | You will analyze a single video. Your sole focus is to evaluate and score its lip-sync quality. You must ignore all other aspects like emotional expression or head movement. |
| Primary Objective | Conduct a rigorous analysis and scoring of the lip-sync quality for this video. Your analysis must justify the score given and return a JSON dictionary with the score. |
| Evaluation Criteria | • Precision: How accurately do the lip movements match vowel/consonant shapes (visemes)? • Timing: Any noticeable delay/lead between audio and corresponding lip movements? • Naturalness: Human/fluid vs. artificial/robotic/flappy; warping or artifacts around the mouth? |
| Scoring Rubric (1–5) | • 5 (Perfect): Precision matches all phonemes; timing perfectly synchronized; natural, fluid, no artifacts. • 4 (Excellent): Mostly correct visemes with subtle inaccuracies; near-imperceptible lag/lead; slightly scripted. • 3 (Good/Acceptable): Generally correct but simplified/puppet-like; slight consistent lag/lead; minor artifacts possible. • 2 (Poor): Frequently incorrect visemes; clearly off-sync and distracting; robotic/jerky, obvious warping/flapping. • 1 (Failure): No correlation to speech; severely out of sync; bizarre/distorted/nonsensical or static mouth. |
| Output Constraints | 1. Put all reasoning and analysis inside <think>...</think> . 2. After </think> , output only a valid JSON object with key "score" and an integer 1--5 . 3. Do not include natural language, commentary, markdown, or extra quotes outside the <think> block. 4. The final JSON must be valid and contain only the score. |
Table 9: Prompt used for lip-sync evaluation. The model is instructed to assess only audio-visual synchronization and to return a single 1–5 score in JSON format.
| Prompt Name | Facial Expressiveness Evaluation Prompt (Single-Video, Single-Aspect) |
|---|---|
| Role | You are a specialized analyst in human emotion and facial expression, with expertise in facial action coding systems (FACS) and psychological realism in digital actors. |
| Context | You will analyze a single video. Your sole focus is to evaluate and score its facial expressiveness and emotional authenticity. You must apply the scoring rubric below to assign a score from 1 to 5. |
| Primary Objective | Conduct a rigorous analysis of the video’s facial expressiveness, assign a score based on the provided rubric, and return a JSON dictionary with the score. |
| Evaluation Criteria | • Emotional Match: Does the facial expression accurately reflect the audio’s tone, sentiment, and emphasis? • Realism & Subtlety: Are there subtle micro-expressions (e.g., brow movements, eye crinkling, cheek twitches), or does the face appear uncanny or plastic? • Natural Dynamics: Do expressions transition smoothly with speech, or appear robotic, delayed, or disconnected? |
| Scoring Rubric (1–5) | • 5 (Excellent): Perfect emotional alignment with the audio; rich micro-expressions; fully genuine and human-like. • 4 (Good): Strong emotional match with minor inconsistencies; realistic but slightly acted or less nuanced. • 3 (Average): Generally appropriate emotion but muted, delayed, or underspecified; mildly uncanny or plastic. • 2 (Poor): Clear emotional mismatch or fixed neutral/awkward state; highly unnatural and artificial. • 1 (Very Poor): Completely incorrect or absent emotion; static, mask-like, or mannequin-like face. |
| Output Constraints | 1. Put all reasoning and analysis inside <think>...</think> . 2. After </think> , output only a valid JSON object with key "score" and an integer 1--5 . 3. Do not include natural language, commentary, markdown, or extra quotes outside the <think> block. 4. The final JSON must be valid and contain only the score. |
Table 10: Prompt used for evaluating facial expressiveness and emotional authenticity. The model assesses emotional alignment and realism of facial expressions and returns a single 1–5 score in JSON format.
| Prompt Name | Motion Smoothness and Background Quality Evaluation Prompt (Single-Video, Single-Aspect) |
|---|---|
| Role | You are a specialized analyst in motion quality and video composition. Your expertise lies in identifying motion artifacts, assessing movement naturalness, and spotting background anomalies in synthesized media. |
| Context | You will analyze a single video. Your sole focus is to evaluate and score its motion smoothness and background integrity. You must ignore all other aspects such as lip-sync or facial emotion. |
| Primary Objective | Conduct a rigorous analysis of the video’s motion quality and background stability, assign a score using the provided rubric, and return a JSON dictionary with the score. |
| Evaluation Criteria | • Head Movement: Are head motions smooth, fluid, and purposeful, or stiff, jittery, and robotic? • Subtle Motions: Are natural micro-movements (e.g., blinking, slight tilts, posture shifts) present and realistic? • Consistency: Does motion remain coherent and human-like throughout, without locking or abrupt artifacts? • Background Quality: Is the background stable, with clean edges around hair and shoulders, or does it exhibit warping, shimmering, or distortion? |
| Scoring Rubric (1–5) | • 5 (Excellent): Perfectly smooth and purposeful head motion; rich subtle movements; fully consistent and human-like; background entirely stable with no artifacts. • 4 (Good): Smooth and consistent motion but somewhat simple or repetitive; natural blinking; minor, non-distracting background artifacts. • 3 (Average): Slightly stiff, jerky, or floaty motion; unnatural blinking cadence or frequent stillness; noticeable but moderate background warping or blur. • 2 (Poor): Clearly robotic or jittery motion with unnatural jerks; no subtle movements; distracting instability; obvious background distortion or melting. • 1 (Failure): Completely static or locked head; frozen face with no movement; entirely artificial appearance; severe background and subject distortion. |
| Output Constraints | 1. Put all reasoning and analysis inside <think>...</think> . 2. After </think> , output only a valid JSON object with key "score" and an integer 1--5 . 3. Do not include natural language, commentary, markdown, or extra quotes outside the <think> block. 4. The final JSON must be valid and contain only the score. |
Table 11: Prompt used for evaluating motion smoothness and background integrity. The model assesses head motion dynamics, subtle movements, and background stability, and returns a single 1–5 score in JSON format.
Figure 7: Qualitative examples comparing prior methods with our SFT and RL post-trained models. Our approach generates videos with more natural movement, better lip-sync, and improved expressiveness. Black masks are used to anonymize identities.
Figure 8: Examples of generated video frames illustrating the effect of different post-training configurations. Good cases (Rows 1 and 3): Our final post-trained model (all three aspect rewards with perceptual and consistency rewards, , ) produces consistent identity, stable colors, and smooth temporal dynamics. Bad case 1 (Row 2): Training with a large stochastic window leads to strong temporal color drift and bad texture. Bad case 2 (Row 4): Training with only the lip-sync reward introduces jittering and unnatural motion, including exaggerated eye movements. Black masks are used to anonymize identities.