Autoregressive (AR) diffusion models offer a promising framework for sequential generation tasks such as video synthesis by combining diffusion modeling with causal inference. Although they support streaming generation, existing AR diffusion methods struggle to scale efficiently. In this paper, we identify two key challenges in hour-scale real-time human animation. First, most forcing strategies propagate sample-level representations with mismatched diffusion states, causing inconsistent learning signals and unstable convergence. Second, historical representations grow unbounded and lack structure, preventing effective reuse of cached states and severely limiting inference efficiency. To address these challenges, we propose Neighbor Forcing, a diffusion-step-consistent AR formulation that propagates temporally adjacent frames as latent neighbors under the same noise condition. This design provides a distribution-aligned and stable learning signal while preserving drifting throughout the AR chain. Building upon this, we introduce a structured ConvKV memory mechanism that compresses the keys and values in causal attention into a fixed-length representation, enabling constant-memory inference and truly infinite video generation without relying on short-term motion-frame memory. Extensive experiments demonstrate that our approach significantly improves training convergence, hour-scale generation quality, and inference efficiency compared to existing AR diffusion methods. Numerically, LiveAct enables hour-scale real-time human animation and supports 20 FPS real-time streaming inference on as few as two NVIDIA H100 or H200 GPUs. Quantitative results demonstrate that our method attains state-of-the-art performance in lip-sync accuracy, human animation quality, and emotional expressiveness, with the lowest inference cost.
Table 1: Comparison of AR diffusion-based methods from an AR formulation perspective. ARPP denotes the representation propagated along the AR chain, and KV Reuse indicates whether cached key–value states can be reused during inference. Here, subscript t denotes the diffusion step and superscript f denotes the f -th frame latent.
Figure 1: This phenomenon is observed when a causal attention mask is directly applied to a pretrained non-AR diffusion model, indicating that Neighbor Forcing effectively mitigates the mismatch between AR and non-AR diffusion.
Figure 2: Overall training pipeline of SoulX-LiveAct. Left : The architecture of DiT block. Right : The training pipeline consists of two stages: (i) training with step-aligned noisy references and a diffusion loss computed at the same step, and (ii) joint training of ConvKV memory and step distill.
Figure 3: Memory mechanism. Left : The KV states corresponding to the long-term memory are compressed via a lightweight 1D convolution operator; Right : The final inference pipeline unifies the Neighbor Forcing formulation and the ConvKV Memory mechanism.
Dataset
Model
Lip-sync Accuracy
Distribution Similarity
VBench
VBench-2.0
Sync-C ↑
Sync-D ↓
FID ↓
FVD ↓
Temporal Quality ↑
Image Quality ↑
Human Fidelity ↑
HDTF
OmniAvatar ( 5 )
5.13
10.19
27.90
268.47
86.1
61.6
96.8
InfiniteTalk ( 28 )
7.12
8.01
18.15
169.88
94.5
59.9
99.4
Live-Avatar ( 12 )
7.68
8.38
15.85
206.20
91.8
59.2
99.8
Ours
9.40
6.76
10.05
69.43
97.6
63.0
99.9
EMTD
OmniAvatar ( 5 )
6.24
8.63
33.63
589.5
91.9
63.2
96.7
Table 2: Quantitative comparison on lip-sync accuracy and video quality metrics.
Model
Thoughput(FPS)
Latency(s) ↓
GPUs ↓
TFLOPs per frame ↓
InfiniteTalk † † † Applied the LoRA-weighted 4-step distillation provided by LightX2V.
25 FPS
3.20 s
8
50.2
Live-Avatar
20 FPS
2.89 s
5
39.1
Neighbor-forcing
20 FPS
0.94 s
2
27.2
Table 3: Inference efficiency comparison between other real-time methods.
Figure 4: Qualitative comparison of lip-motion accuracy and emotion–action coordination . SoulX-LiveAct achieves more precise lip–phoneme alignment and maintains coherent facial expressions and body movements under emotion–action interactions, while baseline methods show misalignment or temporal jitter.
Figure 5: Long-video consistency comparison. Red boxes indicate identity drift (notably severe in OmniAvatar ), while yellow boxes highlight fine-grained detail inconsistencies (e.g., missing accessories in InfiniteTalk and Live-Avatar ). SoulX-LiveAct maintains stable identity and details throughout the sequence.
Figure 6: ConvKV Memory Mechanism consistency comparison. Red boxes highlight inconsistencies in clothing or hand details (e.g., inconsistencies in clothing patterns and nail color). The ConvKV Memory mechanism ensures stable identity and detail preservation throughout the sequence.
Figure 7: Effect of the Emotion and Action Editing Module . Given a reference image (Ref), our motion editing module enables controllable modification of facial expressions and gestures, including sad expression, heart gesture, covering face, and laughing.
Method
ODE Init. Training
DMD Distill Training
Self Forcing
Yes
1000 Steps
Neighbor Forcing
No
300 Steps
Table 4: Difference between Neighbor Forcing and Self Forcing.
Memory-block Size
Current-block Size
Cost/frame
Meat Real-time Requirement
6
6
52ms
No
8
8
60ms
No
6
8
49ms
Yes
Table 5: Ablation study on the memory-block size and current-block size hyperparameters.
Autoregressive video diffusion models support real-time synthesis but suffer from error accumulation and context loss over long horizons. We discover that attention heads in AR video diffusion transformers serve functionally distinct roles as local heads for detail refinement, anchor heads for structural stabilization, and memory heads for long-range context aggregation, yet existing methods treat them uniformly, leading to suboptimal KV cache allocation. We propose Head Forcing, a training-free framework that assigns each head type a tailored KV cache strategy: local and anchor heads retain only essential tokens, while memory heads employ a hierarchical memory system with dynamic episodic updates for long-range consistency. A head-wise RoPE re-encoding scheme further ensures positional encodings remain within the pretrained range. Without additional training, Head Forcing extends generation from 5 seconds to minute-level duration, supports multi-prompt interactive synthesis, and consistently outperforms existing baselines. Project Page: https://jiahaotian-sjtu.github.io/headforcing.github.io/.
Jiahao Tian, Yiwei Wang, Gang Yu +1
AGI Lab, Westlake University · University of California at Merced · StepFun
Autoregressive (AR) diffusion offers a promising framework for generating videos of theoretically infinite length. However, a major challenge is maintaining temporal continuity while preventing the progressive quality degradation caused by error accumulation. To ensure continuity, existing methods typically condition on highly denoised contexts; yet, this practice propagates prediction errors with high certainty, thereby exacerbating degradation. In this paper, we argue that a highly clean context is unnecessary. Drawing inspiration from bidirectional diffusion models, which denoise frames at a shared noise level while maintaining coherence, we propose that conditioning on context at the same noise level as the current block provides sufficient signal for temporal consistency while effectively mitigating error propagation. Building on this insight, we propose HiAR, a hierarchical denoising framework that reverses the conventional generation order: instead of completing each block sequentially, it performs causal generation across all blocks at every denoising step, so that each block is always conditioned on context at the same noise level. This hierarchy naturally admits pipelined parallel inference, yielding a 1.8 wall-clock speedup in our 4-step setting. We further observe that self-rollout distillation under this paradigm amplifies a low-motion shortcut inherent to the mode-seeking reverse-KL objective. To counteract this, we introduce a forward-KL regulariser in bidirectional-attention mode, which preserves motion diversity for causal inference without interfering with the distillation loss. On VBench (20s generation), HiAR achieves the best overall score and the lowest temporal drift among all compared methods.
Kai Zou, Dian Zheng, Hongbo Liu +3
University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security, USTC · The Chinese University of Hong Kong +2
Real-time interactive video generation requires low-latency, streaming, and controllable rollout. Existing autoregressive (AR) diffusion distillation methods have achieved strong results in the chunk-wise 4-step regime by distilling bidirectional base models into few-step AR students, but they remain limited by coarse response granularity and non-negligible sampling latency. In this paper, we study a more aggressive setting: frame-wise autoregression with only 1--2 sampling steps. In this regime, we identify the initialization of a few-step AR student as the key bottleneck: existing strategies are either target-misaligned, incapable of few-step generation, or too costly to scale. We propose Causal Forcing++, a principled and scalable pipeline that uses causal consistency distillation (causal CD) for few-step AR initialization. The core idea is that causal CD learns the same AR-conditional flow map as causal ODE distillation, but obtains supervision from a single online teacher ODE step between adjacent timesteps, avoiding the need to precompute and store full PF-ODE trajectories. This makes the initialization both more efficient and easier to optimize. The resulting pipeline, \ours, surpasses the SOTA 4-step chunk-wise Causal Forcing under the frame-wise 2-step setting by 0.1 in VBench Total, 0.3 in VBench Quality, and 0.335 in VisionReward, while reducing first-frame latency by 50% and Stage 2 training cost by \sim$$4\times. We further extend the pipeline to action-conditioned world model generation in the spirit of Genie3. Project Page: https://github.com/thu-ml/Causal-Forcing and https://github.com/shengshu-ai/minWM .
Min Zhao, Hongzhou Zhu, Kaiwen Zheng +6
Tsinghua University · ShengShu · Renmin University of China