cs.CVDec 23, 2024

ViDE: Tuning-Free Video Coherence via Temporal Attention Reweighting and Prompt Blending

Authors: Xingyao Li, Hao Zhang, Fengzhuo Zhang, Jiachun Pan, Yunlong Hou, Vincent Y. F. Tan, Zhuoran Yang

Organizations: National University of Singapore · Yale University

Abstract

Despite the substantial progress in long video generation, multi-prompt synthesis often encounters inconsistencies resulting from the training-inference gap caused by length extension techniques and coarse prompt interpolation. To overcome these issues, we propose the Video Diffusion with hidden states Editing (ViDE) framework, which consists of two key components. The first is the Time-frequency based Temporal Attention Reweighting (TiTAR) algorithm, which leverages the relationship between inconsistencies and diagonal elements of temporal attention. By reweighting the attention scores via the Discrete Short-Time Fourier Transform (DSTFT), TiTAR effectively reduces frame inconsistencies, a capability further corroborated by a Fourier-based analysis. The second component, PromptBlend, reduces inconsistencies in multi-prompt settings through fine-grained prompt alignment and adaptive interpolation, enabling smooth semantic transitions. Extensive experiments demonstrate the effectiveness of ViDE, demonstrating consistent and significant improvements over multiple baselines.

Figures & tables

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SwitchCraft: Training-Free Multi-Event Video Generation with Attention Controls

    Feb 27, 2026Qianxun Xu, Chenxi Song, Yujun Cai +1Video GenerationControllable Video Generation

  2. SWIFT: Prompt-Adaptive Memory for Efficient Interactive Long Video Generation

    May 10, 2026Shanwen Tan, Hao Li, Jingtao Zhang +4Long Video GenerationMemory-Augmented Video Generation

  3. Diff-VF: Training-free High-quality Long Video Generation via Diffusion Model

    Aug 6, 2026Haoning Yang, Xinyuan Chen, Yaohui Wang +1Diffusion Model GuidanceVideo Diffusion Models