HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising

Kai Zou; Dian Zheng; Hongbo Liu; Tiankai Hang; Bin Liu; Nenghai Yu

cs.CVMar 9, 2026

HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising

Authors: Kai Zou, Dian Zheng, Hongbo Liu, Tiankai Hang, Bin Liu, Nenghai Yu

Organizations: University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security, USTC · The Chinese University of Hong Kong · Tongji University ·

Abstract

Autoregressive (AR) diffusion offers a promising framework for generating videos of theoretically infinite length. However, a major challenge is maintaining temporal continuity while preventing the progressive quality degradation caused by error accumulation. To ensure continuity, existing methods typically condition on highly denoised contexts; yet, this practice propagates prediction errors with high certainty, thereby exacerbating degradation. In this paper, we argue that a highly clean context is unnecessary. Drawing inspiration from bidirectional diffusion models, which denoise frames at a shared noise level while maintaining coherence, we propose that conditioning on context at the same noise level as the current block provides sufficient signal for temporal consistency while effectively mitigating error propagation. Building on this insight, we propose HiAR, a hierarchical denoising framework that reverses the conventional generation order: instead of completing each block sequentially, it performs causal generation across all blocks at every denoising step, so that each block is always conditioned on context at the same noise level. This hierarchy naturally admits pipelined parallel inference, yielding a 1.8 wall-clock speedup in our 4-step setting. We further observe that self-rollout distillation under this paradigm amplifies a low-motion shortcut inherent to the mode-seeking reverse-KL objective. To counteract this, we introduce a forward-KL regulariser in bidirectional-attention mode, which preserves motion diversity for causal inference without interfering with the distillation loss. On VBench (20s generation), HiAR achieves the best overall score and the lowest temporal drift among all compared methods.

Figures & tables

Figure 1: Motivation. (a) Bidirectional diffusion (Wan2.1) proves that a shared noise level provides sufficient context for temporal coherence, though limited to a fixed horizon. (b) Standard AR (Self-Forcing) scales length but suffers quality drift, as conditioning on fully clean context amplifies error propagation. (c) Applying our hierarchical denoising (matched-noise context) only at inference ( w/o training ) mitigates drift but breaks continuity due to train–test mismatch; HiAR retrains under the hierarchical pipeline ( w/ training ), achieving scalable long-video generation with stable quality and seamless continuity.

Figure 2: Overview of HiAR. Left: Existing block-first AR (e.g., Self-Forcing) fully denoises each block before generating the next, conditioning every step on predicted clean context and thus amplifying inter-block error propagation. Right: Our hierarchical denoising performs causal generation across all blocks within each denoising step, conditioning on context at the matched noise level to suppress error accumulation. Bottom: Training combines causal self-rollout with a reverse-KL (DMD) loss for distillation, and a forward-KL regulariser computed in bidirectional-attention mode via teacher trajectory sampling to preserve motion diversity.

Bidirectional video diffusion models
Model	Thru.	Lat.	Total $\uparrow$	Quality $\uparrow$	Semantic $\uparrow$	Dynamic $\uparrow$	Drift $\downarrow$
LTX-Video HaCohen et al. (2025)	8.98	13.5	0.766	0.789	0.685	0.458	–
Wan2.1-1.3B Team (2025)	0.78	103	0.802	0.813	0.766	0.690	–
Autoregressive video diffusion models
NOVA Deng et al. (2025)	0.88	4.1	0.773	0.777	0.757	0.444	–
Pyramid Flow Jin et al. (2024b)	6.70	2.5	0.775	0.804	0.670	0.161	–

Table 1: Quantitative comparison on 20 s generation. Throughput is in frames/s; Latency is in seconds; VBench scores (Total/Quality/Semantic/Dynamic) are on a 0–1 scale; Drift is our proposed drift metric. “–” indicates the model is non-autoregressive and drift is not applicable. Best distilled AR results are bolded .

Figure 3: Qualitative comparison of distilled AR models at 20 s. We show temporally sampled frames from six diverse prompts covering natural scenery, objects, and human subjects. HiAR maintains consistent colour and detail throughout, while baselines exhibit progressive degradation.

Context noise	Quality $\uparrow$	Semantic $\uparrow$	Smooth. $\uparrow$	Drift $\downarrow$
$t_{c}=t_{j}$ (input level)	0.799	0.692	0.978	0.184
$t_{c}=t_{j+1}$ (output level; default)	0.846	0.723	0.988	0.257
$t_{c}=0$ (Self-Forcing)	0.829	0.708	0.991	0.355

Table 2: Ablation study on context noise level

t_{c}

. Quality, Semantic, and Smooth are VBench sub-scores; Drift is our proposed drift metric.

Configuration	Quality $\uparrow$	Semantic $\uparrow$	Dynamic $\uparrow$	Drift $\downarrow$
bi-attn + 1 step (default)	0.846	0.723	0.686	0.257
causal + 1 step	0.828	0.701	0.625	0.271
bi-attn + 2 steps	0.835	0.708	0.693	0.296
bi-attn + 4 steps	0.813	0.684	0.691	0.306
w/o $\mathcal{L}_{\text{FKL}}$	0.839	0.732	0.445	0.218
w/o re-training	0.767	0.559	0.512	0.309

Table 3: Ablation on forward-KL regulariser design. “bi-attn”, “causal” denotes the attention mode used for

\mathcal{L}_{\text{FKL}}

; “

K

step” is the number of denoising steps.

Figure 4: Correlation between bidirectional and causal dynamics during training (w/o $\mathcal{L}_{\text{FKL}}$ ). Each point represents one training checkpoint; colour encodes the training step. A strong positive correlation (Pearson $r=0.968$ ) confirms that the low-motion shortcut affects both attention modes simultaneously and that regularising the bidirectional mode effectively constrains causal-mode dynamics.

Figure 5: Comparison of single-step denoising under bidirectional vs. causal attention. Bidirectional attention produces frames of uniform quality and blur across all positions, while causal attention yields progressively sharper frames as preceding context reduces uncertainty for later positions.

Autoregressive Video Generation

Autoregressive video generation creates videos frame by frame or in chunks, conditioning each prediction on prior context for streaming and interactive use.

See all 211 papers

Sep 29, 2026

Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation

Autoregressive (AR) video diffusion enables low-latency, streamable video generation, but prediction errors often accumulate over long rollouts. Training the generator on its own rollouts exposes it to these imperfect histories. However, existing video-level distribution matching distillation (DMD) scores the whole rollout jointly. Because a chunk is evaluated together with its past and future, its correction can favor matching artifacts in the surrounding context merely to preserve temporal consistency. To provide a clearer visual-quality signal, we introduce Rollout-Marginal Distillation (RMD). RMD retains the generated history for AR prediction but scores each chunk independently against a chunk teacher, ensuring its quality correction is not compromised by an imperfect temporal context. To compensate for the lack of temporal context in independent chunk scoring, RMD subsequently applies video-level DMD to restore temporal coherence. Extensive experiments demonstrate that RMD maintains high visual quality far beyond its training horizon and outperforms video-level DMD baselines. Code and video results are available at https://cjeen.github.io/RMD

Chenjian Gao, Zhihao Hu, Jianqi Ma +3

MMLab, The Chinese University of Hong Kong · Tencent AIPD

Autoregressive Diffusion Long-Horizon Video Generation Diffusion Model Distillation Autoregressive Video Generation Temporal Consistency in Video Generation

Ask PDF

Sep 17, 2026

Recency Forcing: Bridging the Long-Horizon Gap in Autoregressive Video Generation

Autoregressive (AR) video generation degrades over long horizons due to an overlooked train-inference discrepancy we term KV eviction mismatch: models train on short clips where all context frames reside in the KV cache, but at inference, memory constraints force distant frames to be evicted from the KV cache - removing context the model was conditioned on. Rather than simulating eviction via context truncation - which discards temporal information the model still needs and degrades motion coherence - we keep the context but while progressively reducing the influence of distant frames, making their eventual eviction negligible. To guide this design, we introduce the positional response

R( Δ, \, t_{\text{denoise}})

, a perturbation-based sensitivity measure revealing that context influence decays steeply with temporal distance and varies systematically across denoising steps. Motivated by this analysis, we propose Recency Forcing, which applies a non-positive, timestep-dependent bias, termed Temporal Response Bias (TRB), on pre-softmax attention logits derived directly from

R

, closing the train-inference gap without modifying context length or training objectives. We further introduce Biased Attention Reparameterization (BAR), an exact reformulation that moves the bias outside the softmax, making TRB a standard FlashAttention call at zero overhead. Recency Forcing operates in both training-free mode and training-based mode. Experiments on VBench and VBench-Long demonstrate state-of-the-art long-horizon generation quality at no additional inference cost.

Tri Cao, Hung Nguyen, Phong Nguyen +1

Qualcomm AI Research

Long-Horizon Video Generation KV-Cache Management Autoregressive Video Generation Temporal Consistency in Video Generation

Ask PDF

Jun 17, 2026

UniTemp: Unlocking Video Generation in Any Temporal Order via Bidirectional Distillation

Autoregressive video diffusion models have emerged as a promising approach for long video generation, achieving strong performance in streaming settings. However, existing methods are restricted to forward temporal generation, whereas practical video creation often requires flexible generation order, e.g., conditioning on future context to extend backward, or on both past and future context for inbetween generation. We bridge this gap by training an autoregressive model that supports generation in arbitrary temporal directions. A key technical challenge arises from the Causal 3D VAE widely used in video diffusion models, which encodes latents strictly conditioned on past context. While suited for forward generation, this causal structure causes inter-block discontinuities when generation proceeds backward. To address this, we introduce blockwise anchor latents, a set of auxiliary latents that restore the missing past context at block boundaries during backward generation. Built on this design, we propose UniTemp, a bidirectional distillation framework that trains a single autoregressive student model for any-direction video generation. At inference time, UniTemp conditions on arbitrary past and/or future frames, improving controllability for both bidirectional and inbetween generation. Experiments show that UniTemp maintains competitive performance on short and long video generation compared to forward-only methods, while enabling diverse workflows such as bidirectional video extension, inbetween generation, looping video generation, scene transition, and visual story generation. Project website: https://lzhangbj.github.io/projects/unitemp/

Lin Zhang, Sicheng Mo, Zefan Cai +6

University of Wisconsin Madison · University of California Los Angeles · University of California Davis +1

Autoregressive Video Generation Controllable Video Generation

Ask PDF

Abstract

Autoregressive Video Generation

Explore similar work

Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation

Recency Forcing: Bridging the Long-Horizon Gap in Autoregressive Video Generation

UniTemp: Unlocking Video Generation in Any Temporal Order via Bidirectional Distillation