DynaForcing: Overcoming Dynamic Collapse in Self-Forcing Distillation for Streaming Avatar Generation

Yubo Huang; Sirui Zhao; Xinchen Yao; Zhengye Zhang; Jinyang Huang; Fengqi Cui; Shiwei Wu; Enhong Chen

cs.CVAug 18, 2026

DynaForcing: Overcoming Dynamic Collapse in Self-Forcing Distillation for Streaming Avatar Generation

Authors: Yubo Huang, Sirui Zhao, Xinchen Yao, Zhengye Zhang, Jinyang Huang, Fengqi Cui, Shiwei Wu, Enhong Chen

Organizations: University of Science and Technology of China · Nanjing University · Hefei University of Technology · Tsinghua University

Abstract

Audio-driven avatar generation requires realistic lip-sync, expressive motion, and real-time streaming. Recent work achieves the latter via self-forcing with Distribution Matching Distillation (DMD), but this paradigm suffers from a critical failure that has not been systematically characterized: dynamic collapse, where the student model converges to a near-static optimum with high perceptual quality but severely suppressed temporal dynamics. We trace this to two causes: the reverse KL objective in DMD, which biases toward low-motion modes, and unanchored self-conditioning, which creates a feedback loop that amplifies collapse. This is especially harmful for avatars, where even subtle motion loss breaks lip-sync and expression. To address this, we propose DynaForcing, a training framework with three complementary strategies applied at different levels. Specifically, Hybrid Forcing anchors rollouts to ground-truth dynamics at the data level to break the feedback loop. Dynamics-Aware Reward Regularization introduces explicit motion rewards via the RL interpretation of DMD to counteract the reverse KL bias at the loss level. Reference Perturbation perturbs reference images to decouple identity from static details, forcing the model to rely on audio for motion at the conditioning level. We further introduce computation graph pruning and gradient replay, reducing the GPU footprint of self-forcing by over an order of magnitude. Experiments show that DynaForcing recovers dynamics to teacher-comparable levels (Dyn-Deg: 0.31 -> 0.73, Sync-C: 7.03 -> 7.68) while improving visual quality, resolving the quality-dynamics trade-off throughout training without early stopping.

Figures & tables

Figure 1. Dynamic collapse in self-forcing distillation for streaming avatar generation. Given the same reference image and driving audio (left), the self-forcing baseline (top) produces visually appealing but temporally frozen outputs—nearly identical lip shapes across 10 seconds of speech (see mouth crops). DynaForcing (bottom) restores faithful audio-driven dynamics with natural expression variation and accurate lip articulation, at 45.2 FPS real-time streaming.

Figure 2. Dynamic collapse in self-forcing distillation. (a) Text-to-video (Self-Forcing ( Huang et al., 2025a ) ): VBench ( Huang et al., 2023 ) dynamic degree drops from 0.80 to 0.00 while visual quality improves. (b) Audio-driven avatar (LiveAvatar ( Huang et al., 2025b ) ): Sync-C and ExpVar degrade while IQA and ASE improve. (c) Comparing self-forcing with CausVid ( Yin et al., 2025 ) (GT-anchored): CausVid’s dynamic degree fluctuates around 0.43 without collapsing, confirming that unanchored self-conditioning, not DMD distillation itself, drives the collapse.

Figure 3. Overview of DynaForcing. Three strategies intervene at different levels of the self-forcing training pipeline: (a) Hybrid Forcing at the input level, (b) dynamics-aware reward regularization at the loss level, and (c) reference perturbation at the conditioning level.

Method	Standard Metrics						Dynamics
	ASE $\uparrow$	IQA $\uparrow$	Sync-C $\uparrow$	Sync-D $\downarrow$	Dino-S $\uparrow$	FPS $\uparrow$	ExpVar $\uparrow$	Dyn-Deg $\uparrow$
GenBench-ShortVideo
Non-real-time
WanS2V ( Gao et al., 2025 )	3.36	4.29	5.89	9.08	0.95	0.25	1.81	0.72
OmniAvatar ( Gan et al., 2025 )	3.53	4.49	6.77	8.22	0.95	0.16	1.95	0.75
Hallo3 ( Cui et al., 2025a )	3.12	3.97	4.74	10.19	0.94	0.26	1.48	0.57

Table 1. Quantitative comparison on GenBench-ShortVideo (

\sim

10 s) and GenBench-LongVideo (

>

5 min) ( Huang et al., 2025b ) . Bold : best. Underline : second best.

\uparrow

\downarrow

: higher/lower is better. FPS is resolution- and hardware-dependent (720

\times

400, single H800 node) and identical across benchmarks.

Figure 4. Qualitative comparison on GenBench-LongVideo. Frames sampled at 2 s, 200 s, and 400 s from the same driving audio. Please refer to the supplementary video for dynamics comparison.

Figure 5. Training dynamics of DynaForcing. Unlike vanilla self-forcing (Figure 2 b), dynamics metrics (Sync-C, ExpVar) stabilize after

\sim

1K steps without degradation, while quality metrics improve steadily.

Configuration	ASE $\uparrow$	Sync-C $\uparrow$	ExpVar $\uparrow$	Dyn-Deg $\uparrow$
Self-forcing	3.44	5.48	0.69	0.31
GT-anchored (CausVid-style)	3.38	6.85	1.36	0.52

Table 2. Diagnostic ablation: isolating rollout feedback. GT-anchored training conditions on ground-truth latents; self-forcing conditions on the student’s own outputs.

Configuration	ASE $\uparrow$	Sync-C $\uparrow$	ExpVar $\uparrow$
DynaForcing w/o Hybrid Forcing	3.57	6.78	1.18
DynaForcing w/o Reward Reg.	3.55	7.35	1.75
DynaForcing w/o Ref. Perturbation	3.52	7.58	1.92
DynaForcing (full)	3.55	7.68	2.02

Table 3. Component ablation. Each row removes one component from full DynaForcing.

$p_{\text{data}}$	ASE $\uparrow$	Sync-C $\uparrow$	ExpVar $\uparrow$	Dino-S $\uparrow$
0.1	3.56	7.19	1.42	0.96
0.3	3.55	7.68	2.02	0.96
0.5	3.53	7.64	2.03	0.95
1.0	3.42	7.62	2.10	0.93

Table 4. Effect of Hybrid Forcing probability

p_{\text{data}}

on full DynaForcing (varying

p_{\text{data}}

only).

	GPUs	GPU $\cdot$ h	ASE $\uparrow$	Sync-C $\uparrow$	ExpVar $\uparrow$
w/o eff. optim.	128 $\times$ H100	7,111	3.56	7.66	2.04
w. eff. optim.	8 $\times$ H100	667	3.55	7.68	2.02

Table 5. Training efficiency: full DynaForcing with and without our efficiency optimizations (§ 4.5 ), both trained for 4,000 steps.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Category 1: Background Replacement

Change the background to {a studio with plain gray backdrop / an outdoor park scene / a dimly lit office}; keep the person’s face, hairstyle, expression, and clothing unchanged.

Category 2: Lighting Variation

Adjust the lighting to {warm golden-hour sunlight from the left / cool fluorescent overhead lighting / dramatic side lighting with strong shadows}; keep the person’s appearance unchanged.

Category 3: Viewpoint Shift

Slightly rotate the person’s head to {face 15 degrees to the left / tilt slightly upward / turn to a three-quarter view}; keep the person’s identity, hairstyle, and clothing unchanged.

Category 4: Accessory Modification

Appendix

Table 6. Full prompts for reference image perturbation via Qwen-Image-Edit. Each category targets a specific visual attribute while explicitly constraining identity-related features to remain unchanged. {

\cdot

} denotes a slot randomly sampled from the provided options.

$\alpha$	ASE $\uparrow$	Sync-C $\uparrow$	ExpVar $\uparrow$
0.0	3.55	7.35	1.75
0.05	3.55	7.52	1.88
0.1	3.55	7.68	2.02
0.2	3.53	7.70	2.08
0.5	3.48	7.65	2.06

Appendix

Table 7. Effect of reward strength

\alpha

on full DynaForcing.

Distribution	ASE $\uparrow$	Sync-C $\uparrow$	ExpVar $\uparrow$
Uniform	3.54	7.51	1.86
Conc. $t_{0}$	3.55	7.42	1.78
Conc. $t_{1}$	3.55	7.68	2.02
Conc. $t_{2}$	3.52	7.55	1.91

Appendix

Table 8. Effect of starting-step probability distribution.

Reward	ASE $\uparrow$	Sync-C $\uparrow$	ExpVar $\uparrow$
None ( $\alpha=0$ )	3.55	7.35	1.75
Sync-C only	3.55	7.62	1.85
ExpVar only	3.54	7.48	1.96
Both (full)	3.55	7.68	2.02

Appendix

Table 9. Ablation on reward components in

R_{\text{dyn}}

Perturbation	ASE $\uparrow$	Sync-C $\uparrow$	ExpVar $\uparrow$
None	3.52	7.58	1.92
Traditional Aug.	3.48	7.70	2.03
Qwen-Image-Edit (Ours)	3.55	7.68	2.02

Appendix

Table 10. Perturbation strategy comparison on GenBench-ShortVideo.

Method	Reward metrics		Reward-free
	Sync-C $\uparrow$	ExpVar $\uparrow$	Dyn-Deg $\uparrow$	Lip-Vel $\uparrow$	Lip-Jit $\downarrow$
LiveAvatar	7.03	0.69	0.31	3.50	8.64
DynaForcing w/o Reward Reg.	7.35	1.75	0.55	3.41	4.23
DynaForcing (full)	7.68	2.02	0.73	3.78	4.26

Appendix

Table 11. Reward-free dynamics evaluation. Sync-C and ExpVar are training rewards; Dyn-Deg, Lip-Vel and Lip-Jit are not optimized.

vs.	Visual	Lip-Sync	Motion	Overall
LiveAvatar	58.4	71.2	80.6	72.8
SoulX-FlashTalk	55.2	64.8	74.2	66.4

Appendix

Table 12. User study: preference rate (%) for DynaForcing over the two strongest competing real-time methods.

Audio-Driven Facial Animation

Audio-driven facial animation maps speech to synchronized facial motion for realistic, expressive, and controllable talking faces and avatars.

See all 83 papers

Where and When to Force: Routed Forcing for Streaming Avatars
Sep 25, 2026Zihan Su, Siwen Lu, Junhao Zhuang +6Streaming Video Generation Diffusion Model Distillation
Tsinghua University · Joy Future Academy, JD · Southeast University
AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising
Mar 15, 2026Liyuan Cui, Wentao Hu, Wenyuan Zhang +3Audio-Video Generation Video Diffusion Models
Zhejiang University · Kling Team, Kuaishou Technology · Beijing University of Posts and Telecommunications +1
AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars
Jul 27, 2026Hengyuan Zhang, Jingna Sun, Meiguang Jin +1Video Diffusion Models Long-Horizon Video Generation
Alibaba Group, Taobao and Tmall Group

Sep 25, 2026J/K move · Enter open · S save

Where and When to Force: Routed Forcing for Streaming Avatars

Zihan Su, Siwen Lu, Junhao Zhuang, Zeyue Xue+5

Tsinghua University · Joy Future Academy, JD · Southeast University

Streaming Video Generation Diffusion Model Distillation Audio-Driven Facial Animation Mode Collapse

Open paper AskPDF arXiv

Audio-driven streaming avatar generation requires real-time synthesis of speech-synchronized videos with dynamic and diverse motion. Self Forcing uses Distribution Matching Distillation (DMD) to distill bidirectional video diffusion models into causal, few-step generators for real-time streaming. However, DMD minimizes a reverse KL divergence, which is inherently mode-seeking: it causes the student to discard high-dynamic modes and collapse onto static outputs, compressing both dynamics and diversity of generated videos. We find that this collapse is region-heterogeneous: person regions involving pose and gesture variations suffer the largest diversity loss, the audio-driven mouth region shows a small loss, and the background remains nearly stable. Based on this observation, we propose Routed Forcing, which routes the distillation objective by semantic region and noise stage to improve dynamics and diversity while preserving visual quality. Specifically, (1) Where to Force: Semantic-Region Routing applies Data-Forcing Distillation (DFD), which supervises the student with real videos, to the person region where diversity collapse is most severe, while retaining DMD for the mouth and background to preserve lip synchronization and scene stability. (2) When to Force: Noise-Stage Routing activates DFD at high noise stages, where real video serves as effective supervision to inject diverse and dynamic motion patterns. At low noise stages, DMD is used to refine details, avoiding blur and artifacts from spatial differences between real video and student-generated video. Experiments show that Routed Forcing improves dynamics by up to 45% and diversity by 7-25% over Self Forcing, while preserving video quality and lip synchronization.

Abstract

Audio-Driven Facial Animation

Explore similar work

Where and When to Force: Routed Forcing for Streaming Avatars

AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising

AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars