Video diffusion models have achieved remarkable progress in generating high-quality videos. However, these models struggle to represent the temporal succession of multiple events in real-world videos and lack explicit mechanisms to control when semantic concepts appear, how long they persist, and the order in which multiple events occur. Such control is especially important for movie-grade video synthesis, where coherent storytelling depends on precise timing, duration, and transitions between events. When using a single paragraph-style prompt to describe a sequence of complex events, models often exhibit semantic entanglement, where concepts intended for different moments in the video bleed into one another, resulting in poor text-video alignment. To address these limitations, we propose Prompt Relay, an inference-time, plug-and-play method to enable fine-grained temporal control in multi-event video generation, requiring no architectural modifications and no additional computational overhead. Prompt Relay introduces a penalty into the cross-attention mechanism, so that each temporal segment attends only to its assigned prompt, allowing the model to represent one semantic concept at a time and thereby improving temporal prompt alignment, reducing semantic interference, and enhancing visual quality.
Figures & tables
Figure 2: Temporal Cross-Attention Routing. Each textual prompt is associated with a specific temporal segment of the video. The attention penalty varies smoothly across time, allowing video tokens to attend strongly to their corresponding prompt within the assigned interval while suppressing attention to temporally irrelevant prompts. This enables multiple events (e.g., pouring cereal followed by pouring milk) to occur in the correct order without semantic interference.
Figure 3: Ablation Study of the Temporal Penalty Function. The curves show the attention fraction retained between a query token and the prompt tokens of a given segment, as a function of the query’s latent frame offset from that segment’s midpoint ms , after applying the penalty exp(−C(i,j)) . (Top) Effect of the window parameter w . w=L−2 preserves full attention within the segment and only suppresses attention near the segment boundaries. (Bottom) Effect of the decay threshold ϵ . Smaller values enforce stronger attenuation outside the ’free-attention’ window; however, we find that the choice among small values has negligible perceptual impact. We adopt ϵ=0.1 as our default.
Figure 4: Hard Masking vs Boundary-Attention Decay. Hard masking enforces an abrupt semantic switch in cross-attention at segment boundaries while self-attention remains continuous across the segments. This creates a discontinuity at the boundary, forcing the model to reconcile conflicting signals (Woman eats the pasta instead of the man). Boundary-attention decay avoids this conflict by smoothly co-activating both neighboring prompts near the boundary, giving the model a gradual handoff region in which the transition can be planned jointly before being committed to in the visual representation.
Figure 5: Qualitative Comparison. Given a multi-event prompt describing a deliberate scene transition, Prompt Relay preserves correct temporal structure, ensuring that each semantic instruction influences only its intended segment while maintaining global visual coherence.
Metric
Storyboard
Kling 2.6
Veo 3.1
Wan 2.2
Wan 2.2 + Prompt Relay
Temporal Prompt Alignment ( ↓ )
4.67
1.30
3.93
4.00
1.10
Transition Naturalness ( ↓ )
4.60
4.43
1.30
3.50
1.17
Visual Quality ( ↓ )
3.67
2.50
2.0
4.00
2.83
Table 1: Human preference scores for multi-event video generation. (lower values indicate better rankings)
Recent advances in text-to-video diffusion models have enabled high-fidelity and temporally coherent videos synthesis. However, current models are predominantly optimized for single-event generation. When handling multi-event prompts, without explicit temporal grounding, such models often produce blended or collapsed scenes that break the intended narrative. To address this limitation, we present SwitchCraft, a training-free framework for multi-event video generation. Our key insight is that uniform prompt injection across time ignores the correspondence between events and frames. To this end, we introduce Event-Aligned Query Steering (EAQS), which steers frame-level attention to align with relevant event prompts. Furthermore, we propose Auto-Balance Strength Solver (ABSS), which adaptively balances steering strength to preserve temporal consistency and visual fidelity. Extensive experiments demonstrate that SwitchCraft substantially improves prompt alignment, event clarity, and scene consistency compared with existing baselines, offering a simple yet effective solution for multi-event video generation.
Qianxun Xu, Chenxi Song, Yujun Cai +1
Westlake University · Duke Kunshan University · The University of Queensland
Generating high-quality videos from complex temporal descriptions that contain multiple sequential actions is a key unsolved problem. Existing methods are constrained by an inherent trade-off: using multiple short prompts fed sequentially into the model improves action fidelity but compromises temporal consistency, while a single complex prompt preserves consistency at the cost of prompt-following capability. We attribute this problem to two primary causes: 1) temporal misalignment between video content and the prompt, and 2) conflicting attention coupling between motion-related visual objects and their associated text conditions. To address these challenges, we propose a novel, training-free attention mechanism, Temporal-wise Separable Attention (TS-Attn), which dynamically rearranges attention distribution to ensure temporal awareness and global coherence in multi-event scenarios. TS-Attn can be seamlessly integrated into various pre-trained text-to-video models, boosting StoryEval-Bench scores by 33.5% and 16.4% on Wan2.1-T2V-14B and Wan2.2-T2V-A14B with only a 2% increase in inference time. It also supports plug-and-play usage across models for multi-event image-to-video generation. The source code and project page are available at https://github.com/Hong-yu-Zhang/TS-Attn.
Hongyu Zhang, Yufan Deng, Zilin Pan +6
Peking University, Shenzhen Graduate School · Zhejiang University · Nanjing University +3
Text-to-video models often generate the visual concepts requested in natural language prompts but fail to satisfy their intended ordering, timing, and persistence requirements. We propose representing these requirements as specifications in a formal Temporal Logic, and introduce Temporal Specification Guidance (TSG), a training-free inference-time framework that turns temporal specifications into executable objectives for guiding video generation during sampling. TSG scores traces of visual concept evidence using differentiable quantitative semantics and uses the score's gradients to construct a specification-guided denoising direction. We instantiate this framework with video-text attention traces, efficient differentiable scores for Metric Temporal Logic, and gradient-derived attention biases. This instantiation requires neither an external concept classifier nor backpropagation through the video transformer. Its attention-steered forward pass yields a denoising direction that augments classifier-free guidance (CFG). On a benchmark of 220 prompts spanning 11 categories of temporal constraints, including ordered object entry and state transitions, this instantiation improves specification satisfaction accuracy over CFG by 9.5-20 percentage points across five models, with minimal memory overhead. These results demonstrate the potential of temporal specifications and differentiable satisfaction objectives as a flexible interface for controlling video generation.
Avishree Khare, Hideki Okamoto, Bardh Hoxha +2
University of Pennsylvania · Toyota Motor North America (R&D)