Complementary Retrieval-Augmented Prompting for Consistent Long-Form Video Generation

Xianghan Wei; Xiaoda Yang; Zhi Wang; An Pan; Daoan Zhang; Huayi Zhang; Yan Zhang; Wei Xu

cs.CVSep 29, 2026

Complementary Retrieval-Augmented Prompting for Consistent Long-Form Video Generation

Authors: Xianghan Wei, Xiaoda Yang, Zhi Wang, An Pan, Daoan Zhang, Huayi Zhang, Yan Zhang, Wei Xu, +2 more

Abstract

While recent video foundation models excel at generating high-quality short videos, long-form video generation remains a critical challenge, where a major bottleneck lies in conditioning independently generated shots to preserve consistent characters, scenes, and objects throughout a story. Existing training-free approaches typically condition target shots using retrieved historical visuals. However, these references often suffer from severe informational mismatch, either introducing irrelevant contextual redundancy or failing to provide the full combination of required elements for the target shot. To resolve this, we present Complementary Retrieval-Augmented Prompting, an agentic framework that strategically aggregates a compact set of mutually supportive historical references to achieve complete and targeted conditioning for long-form video generation without retraining or modifying the underlying generator. Specifically, our framework explicitly models the visual elements required by each target shot by parsing the narrative script into a text-grounded visual element registry that tracks characters, objects, scenes, and their shot-level states. A VLM-annotated keyframe library further maps these elements to past visual observations. Guided by the required elements, our agent retrieves complementary references that maximize target-element coverage while minimizing historical noise. Finally, the retrieved references, structured element states, and grounding instructions are assembled into a unified prompt for the frozen video generator. This element-aware process provides comprehensive conditioning while remaining fully interpretable. Quantitative and qualitative evaluations on multi-shot story generation demonstrate that our method consistently outperforms recent-frame, memory-based, and entity-level retrieval baselines in cross-shot consistency and text-controllability.

Figures & tables

Figure 1: Overview of Complementary Retrieval-Augmented Prompting. An LLM updates the visual element registry and shot-level plan; greedy coverage retrieves complementary keyframes from the VLM-annotated library; and a second LLM produces per-reference guidance for structured prompting. After generation, representative frames are mined and annotated before the library is updated. The dashed arrow denotes retrieval from the updated library for the next shot.

Figure 2: Complementary reference retrieval example. Top- $K$ spends its three-frame budget on redundant views $f_{1}$ – $f_{3}$ and covers only three of five required elements. Iterative reweighting instead selects $f_{1}$ , $f_{4}$ , and $f_{5}$ , covering all required elements.

Figure 3: Effect of grounded prompting. Without grounding, obsolete scene elements leak into the target shot; ours preserves the character while suppressing conflicting context.

Training-based methods
Method	Overall	DINO	Char.	Obj.	Scene
CineTrans	0.2964	0.5659	0.2080	0.2235	0.3228
HoloCine	0.4471	0.6268	0.3125	0.5219	0.4171
LongLive2.0	0.6512	0.8690	0.5639	0.6464	0.6344
ShotStream	0.6703	0.8280	0.5789	0.5767	0.7766
Explicit workflows (all reproduced with Seedance2.0 backend)

Table 1: Cross-Shot Consistency on the same 35-episode EntityBench subset. Overall averages all 21 metrics. Best and second-best results are shown in bold and underlined, respectively.

Method	Overall	DINO	Char.	Obj.	Scene
Full	0.8670	0.8249	0.8930	0.8780	0.8510
w/o Cover	0.8482	0.7855	0.8658	0.8623	0.8479
w/o Prompting	0.8491	0.8134	0.8513	0.8741	0.8396
CLIP Retrieval	0.8456	0.8233	0.8407	0.8584	0.8488
No Reference	0.5456	0.7241	0.4058	0.5788	0.5628

Table 2: Cross-Shot Consistency of ablations and retrieval baselines on the same 35-episode subset.

Figure 4: Qualitative comparison of explicit workflows on EntityBench Hard_5 ; all methods use Seedance 2.0. Colored text and boxes identify the relevant characters and scene. After Fiona changes into Leo’s smiley-face hoodie in Scene 10, Shot 3, VideoMemory and ViMax exhibit outfit inconsistency in Scene 26, Shot 3; ViMax also exhibits background drift. StoryMem exhibits character confusion for Fiona and character drift for Silas. Our method preserves the marked character, outfit, and background states.

Figure 5: Qualitative ablation on Scene 3 of EntityBench Easy_2 . Without complementary coverage, Priya has an outfit inconsistency in Shot 5. Without structured prompting, Priya is incorrectly generated in Shot 5. CLIP retrieval causes clothing drift and loses her head covering. The complete pipeline maintains the intended character and Old Quarter scene states.

Explore similar work

Jun 15, 2026cs.CV

Closed-Loop Triplet Synergistic Generation for Long-Form Video

Multi-shot long-form video generation remains challenging due to identity drift and compounding inconsistencies across shots. While storyboard-driven pipelines improve controllability, they are often executed in a feed-forward manner, with limited mechanisms to incorporate generated visual evidence back into subsequent conditioning. We propose CoTriSyGen, an agentic framework that formulates multi-shot long video generation as a closed-loop visual-text-memory synergy process, where planned intent, persistent memory, and generated visuals are jointly leveraged for iterative correction and long-range coherence. A vision-language-model-based analyzer reasons over this triplet and produces updates to both prompts and memory along two pathways: (i) intra-shot refinement, which triggers targeted regeneration when semantic or compositional violations are detected and refines image-to-video prompt for coherent motions; and (ii) inter-shot refinement, which rewrites subsequent-shot prompts to propagate newly manifested entities or attributes and improve prompt quality (e.g., compositional grounding and cinematic fluency) based on generated evidence. The loop is grounded in an entity-centric memory modeled as a mutable visual state that evolves as the story progresses, which is continuously updated by both the generator and the analyzer by adding new and evolved entities to reflect appearance changes, accumulated multi-view evidence, and multi-entity compositions. Experiments on our curated StoryBench benchmark demonstrate substantial improvements in cross-shot consistency, prompt adherence, and cinematic continuity over representative methods.

Xinlei Yin, Xiulian Peng, Xiao Li +2

Microsoft Research Asia · University of Science and Technology of China

Interactive Video Generation Visual Memory Closed Loop Fluency

Ask PDF

Jun 12, 2026cs.CV

Memento: Reconstruct to Remember for Consistent Long Video Generation

Long-form video generation requires recurring subjects to remain consistent across various shots, viewpoints, motions, and scene transitions. Existing temporal decomposition methods improve scalability by generating videos shot by shot. However, they mainly focus on optimizing plausible next-shot continuations without verifying whether the historical memory preserves identity-critical subject evidence. Consequently, as generation proceeds, recurring subjects may be diluted, overwritten, or forgotten. In this paper, we propose Memento, a subject-reconstruction-guided framework that treats subject preservation as an explicit identity grounding problem, based on the premise that a memory bank faithfully preserving a subject should support reconstructing that subject from memory alone. Specifically, Memento jointly trains autoregressive next-shot generation with memory-based subject reconstruction, recovering target appearances using historical memory and global story captions. To disentangle long-range subject evidence from short-range cues, Memento introduces a dual-query memory mechanism, where one query retrieves identity-relevant memory and the other selects short-context keyframes for coherent continuation. Additionally, a subject-aware cinematic data pipeline provides precise reconstruction supervision via consistent, pronoun-free subject descriptions. Experiments demonstrate that Memento achieves state-of-the-art performance in long-term subject consistency, cross-shot coherence, and visual quality.

Xuan Wei, Longbin Ji, Guan Wang +5

Xiamen University · ERNIE Team, Baidu Inc.

Visual Memory Long-Term Memory Motion Coherence Memo Data-Curation

Ask PDF

Sep 27, 2026cs.CV

StoryEngine: A State-Grounded Agentic Framework for Video Storytelling

Despite recent progress in agentic multi-shot video generation, producing coherent and consistent long-form stories remains challenging. Existing agentic pipelines typically rely on textual shot plans or previously generated pixels, yet lack an explicit mechanism for propagating the consequences of story events and maintaining the video world state across shots. As a result, missing visual details may be reconstructed inaccurately, while visual drift may propagate across subsequent shots, undermining both narrative coherence and visual consistency. To address these challenges, we propose StoryEngine, a state-grounded agentic framework for video storytelling. StoryEngine establishes a separation between authoritative semantic plans and unreliable visual observations. Specifically, StoryEngine maintains a structured representation of entity placement and story-relevant states, and propagates event-induced changes to define the intended start and end states of each shot. To visually realize these states, StoryEngine constructs canonical references for recurring entities and environments, and compiles state and visual constraints into executable render plans. Meanwhile, to realize these states correctly, a bounded evaluation-guided repair loop further corrects local state inconsistencies. Together, these mechanisms preserve causal story progression and prevent local visual errors from propagating across shots. To comprehensively evaluate long-form storytelling, we construct a benchmark across diverse scenarios and visual styles, with metrics assessing storytelling quality, narrative coherence, and visual consistency. Experimental results demonstrate that StoryEngine consistently outperforms state-of-the-art methods across all evaluation dimensions, validating its effectiveness for coherent and consistent video storytelling.

Yingrui Wang, Zeqing Wang, Yeying Jin

Tencent · National University of Singapore

Video Storytelling Narratives Agentic Framework

Ask PDF