StoryTailor:A Zero-Shot Pipeline for Action-Rich Multi-Subject Visual Narratives

Jinghao Hu; Yuhe Zhang; GuoHua Geng; Kang Li; Han Zhang

cs.CVFeb 24, 2026

StoryTailor:A Zero-Shot Pipeline for Action-Rich Multi-Subject Visual Narratives

Authors: Jinghao Hu, Yuhe Zhang, GuoHua Geng, Kang Li, Han Zhang

Organizations: College of Computer Science, Northwest University

Abstract

Generating multi-frame, action-rich visual narratives without fine-tuning faces a threefold tension: action text faithfulness, subject identity fidelity, and cross-frame background continuity. We propose StoryTailor, a zero-shot pipeline that runs on a single RTX 4090 (24 GB) and produces temporally coherent, identity-preserving image sequences from a long narrative prompt, per-subject references, and grounding boxes. Three synergistic modules drive the system: Gaussian-Centered Attention (GCA) to dynamically focus on each subject core and ease grounding-box overlaps; Action-Boost Singular Value Reweighting (AB-SVR) to amplify action-related directions in the text embedding space; and Selective Forgetting Cache (SFC) that retains transferable background cues, forgets nonessential history, and selectively surfaces retained cues to build cross-scene semantic ties. Compared with baseline methods, experiments show that CLIP-T improves by up to 10-15%, with DreamSim lower than strong baselines, while CLIP-I stays in a visually acceptable, competitive range. With matched resolution and steps on a 24 GB GPU, inference is faster than FluxKontext. Qualitatively, StoryTailor delivers expressive interactions and evolving yet stable scenes.

Figures & tables

Figure 1: Zero-shot on a single 24-GB GPU, our pipeline produces action-rich multi-subject narratives with consistent identities and smooth scene transitions. Right: baselines show background drag, weak interaction, and close-range conflicts; our GCA stabilizes subjects, AB-SVR boosts actions, and SFC retains transferable background cues.

Figure 2: Overview of our pipeline: multi-subject conditioning and grounding, Gaussian-Centered Attention with Selective Forgetting Cache for layout recall, and AB-SVR on text embeddings to stabilize verbs and relations; frames are rendered sequentially into a coherent story.

Figure 3: GCA with SFC: the first 77 tokens run the main cross-attention with a subject mask; the remaining tokens form an IP branch where overlaps yield a dynamic Gaussian-decay mask applied to Q,K. Cross frame K,V caches recall layout while limiting background carry-over, and the IP output is scaled and fused with the main state.

Figure 4: Single frame image consistency on single- and multi-subject tasks. Top: single-subject action sequences. Bottom: multi-subject interaction and attribute binding—dog and cat rubbing each other, teddy bear with a backpack. Only Ours and Qwen-Edit attach the backpack.

Emphasis	$\tau$ range	$k$ range	Rec. $\tau$
Identity / core shape	55–65%	$2$ – $3$	0.60
Actions / interactions	75–85%	$3$ – $4$	0.80
Background continuity	88–92%	$5$ – $6$	0.90
Style tone, lighting	92–95%	$5$ – $8$	0.93
Fine details, textures	97–99%	$9$ – $14$	0.98

Table 1: Cumulative-energy bands (

\tau

), typical

k

, and recommended

\tau

by emphasis.

Figure 5: Prompt-following for actions and interactions in single- and multi-subject cases. With the same references and prompts, our method preserves identity and binds attributes to the correct subject while rendering actions and interactions—standing in fog, running in a forest, jumping on a beach; dancing and holding hands, play-fighting, hugging, nestling—with cleaner backgrounds than $\lambda$ -Eclipse, MS-Diffusion, FluxKontext, Qwen-Edit, Nano-Banana, and MS+1P1S.

Adapters & MLLM
Method	CLIP-I (↑)	M-DINO (↑)	CLIP-T (↑)
MS-Diffusion	0.692	0.108	0.340
SSR-Encoder	0.723	0.104	0.311
$\lambda$ -ECLIPSE	0.719	0.097	0.304
In-context
FluxKontext	0.732	0.107	0.372

Table 2: Multi-subject image consistency on MSBench (micro-averaged means). Best in bold; second-best underlined.

	Single-subject			Multi-subject			Efficiency
Method	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DreamSim $\downarrow$	CLIP-I $\uparrow$	CLIP-T $\uparrow$	DreamSim $\downarrow$	Time (s) $\downarrow$	VRAM (GB) $\downarrow$
IP-Adapter	0.872	0.283	0.214	–	–	–	4.82	19.39
IP-Adapter + 1Prompt1Story	0.928	0.215	0.472	–	–	–	7.25	21.64
$\lambda$ -Eclipse	0.826	0.299	0.268	–	–	–	8.31	14.21
MS-Diffusion	0.884	0.315	0.197	0.687	0.318	0.323	5.36	19.47
FluxKontext	0.913	0.381	0.187	0.732	0.373	0.212	32.31	22.99

Table 3: Short-frame visual narratives including single and multi. Higher is better for CLIP-I and CLIP-T; lower is better for DreamSim. Efficiency is per frame on a single RTX 4090 (24 GB), 1024

\times

1024, 30 steps; API rows omit efficiency.

Variant	CLIP-I $\uparrow$	CLIP-T $\uparrow$	Time $\downarrow$	Subject Confuse
Core modules
Ours (GCA + SFC + SVR)	0.671	0.406	7.026	N
w/o AB-SVR	0.636	0.364	6.715	N
w/o SFC	0.663	0.341	6.412	N
w/o GCA	0.684	0.319	5.894	Y
w/o GCA and SVR	0.675	0.304	5.431	Y

Table 4: Ablation summary in Multi-subject Narratives task. Higher is better for CLIP-I/CLIP-T; lower is better for Time; Subject confuse(Y/N).

Figure 6: Additional single-frame image consistency visual experiments.

Fine-tuning
	Single-subject			Multi-subject			Efficiency
Method	CLIP-I $\uparrow$	DINO - v2 $\uparrow$	CLIP-T $\uparrow$	CLIP-I $\uparrow$	M - DINO $\uparrow$	CLIP-T $\uparrow$	Time (s) $\downarrow$	VRAM (GB) $\downarrow$
Textual Inversion	0.772	0.574	0.264	–	–	–	3.51	13.73
LoRA	0.795	0.591	0.270	–	–	–	3.85	15.82
DreamBooth	0.802	0.589	0.309	–	–	–	4.25	13.12
Adapters & MLLM

Table 5: Single-frame Image Consistency on MSBench (micro-averaged means). Single-subject reports CLIP-I, DINO-v2, CLIP-T; Multi-subject reports CLIP-I, M-DINO, CLIP-T. Best values are bold and second-best values are underlined for each column . Inference Time and Peak GPU VRAM are overall efficiency metrics.

Figure 7: Additional animals and inanimate objects visual narrative task experiments.

Figure 8: Additional GCA anisotropic comparison experiments.

Method	CLIP-I $\uparrow$	CLIP-T $\uparrow$	Bkg Drag $\downarrow$	Confusion(Y/N)	Latency $\downarrow$	Peak VRAM $\downarrow$
Box Binary	0.815	0.285	13%	Y	6.23	20.24
XOR Split	0.837	0.279	22%	N	6.27	21.51
Gaussian Diffusion Split	0.819	0.313	28%	Y	31.32	22.01
Static Two-Stage	0.878	0.354	11%	N	7.13	20.58
Single-Stage	0.856	0.349	6%	Y	6.96	20.74
GCA	0.893	0.416	2%	N	7.57	20.44

Table 6: GCA anisotropy ablation on multi-subject action scenes, 1024

\times

1024, 30 steps, shared seed, single 24 GB GPU. Metrics: CLIP-I

\uparrow

, CLIP-T

\uparrow

, background-drag

\downarrow

, subject confusion(Y/N), latency (s)

\downarrow

, peak VRAM (GB)

\downarrow

. Best = bold , second = underline .

Figure 9: Some visual results of singluar value decay.

Figure 10: Singular-value decay polylines for different numbers of sampled prompts. Each subplot visualizes $n\!=\!5,10,15,20$ text embeddings, respectively. For every curve we plot the singular-value spectrum and highlight the three largest energy drops with markers: the first elbow (circles, $\tau\!\approx\!0.60$ ), the second elbow (squares, $\tau\!\approx\!0.85$ ), and the third elbow (triangles, $\tau\!\approx\!0.95$ ). The knees cluster into three bands, corresponding to subject-, action-, and background-related energy regions in the text-embedding space.

Figure 11: Singluar value energy $\tau$ setting experiments on single- and multi-subject tasks.

$\tau$ range	$k_{\min}$	$k_{\max}$	$k_{\text{avg}}$	Knees
0.50–0.60	2	3	2.1	18
0.61–0.70	2	4	2.8	32
0.71–0.80	3	6	4.1	41
0.81–0.90	3	8	5.0	53
0.91–0.95	5	18	10.2	37
0.96–0.99	8	39	19.5	19

Table 7: Statistical distribution of knee positions

k

across cumulative energy thresholds

\tau

in 200 samples.

Figure 12: DreamSim vs. sequence length on MSBench (lower is better).

Figure 13: CLIP-I vs. sequence length on MSBench (higher is better).

Figure 14: CLIP-T vs. sequence length on MSBench (higher is better).

Figure 15: Long-frame narrative effect experiment on single-subject tasks (a 20-frame example).

Figure 16: Long-frame narrative effect experiment on multi-subject tasks (a 20-frame example).

Figure 17: The web questionaire of StoryTailor about visual narrative tasks for users.

Figure 18: The Likert scores for single-subject(a) and multi-subject(b) tasks for user study

Figure 19: The pairwise preference for single-subject(a) and multi-subject(b) tasks for user study

Visual Storytelling

See all 23 papers

SubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytelling
Sep 28, 2026Xinyu Wang, Huafeng Shi, Zian Li +4Video Diffusion Models Memory-Augmented Video Generation
Shenzhen International Graduate School, Tsinghua University · Kling Team · Peking University +1
ReCap: Lightweight Referential Grounding for Coherent Story Visualization
Apr 20, 2026Aditya Arora, Akshita Gupta, Pau Rodriguez +1Conditional Image Generation Visual Storytelling
Technical University of Darmstadt & hessian.AI, Germany
CANVAS: Continuity-Aware Narratives via Visual Agentic Storyboarding
Apr 15, 2026Ishani Mondal, Yiwen Song, Mihir Parmar +4Visual Storytelling Narrative Generation
University of Maryland, College Park · Google

Sep 28, 2026J/K move · Enter open · S save

SubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytelling

Xinyu Wang, Huafeng Shi, Zian Li, Yan Zhou+3

Shenzhen International Graduate School, Tsinghua University · Kling Team · Peking University+1

Video Diffusion Models Memory-Augmented Video Generation Identity-Preserving Video Generation Multi-Shot Video Generation Temporal Consistency in Video Generation Visual Storytelling

Open paper AskPDF arXiv

We present SubjectAnchor, a Subject-Aware Memory-to-Video paradigm for multi-shot storytelling in which the current shot is generated by conditioning on explicit visual memories extracted from previous shots. The objective is to preserve subject identity and scene consistency across cuts while retaining the controllability of shot-wise prompting. Built on Wan2.2-I2V-A14B, SubjectAnchor contains three key components: subject-related memory construction, subject-aware temporal rotary position encoding, and memory-aware attention partition. For each target shot, the method constructs a compact memory bank by tracing each required subject to its historical appearance and retrieving the most relevant precomputed keyframes. These memory frames are encoded into the model input as explicit visual conditions, while different subjects are assigned to separated negative temporal slots to reduce identity interference. In addition, memory-aware attention partition regulates the interaction between memory tokens and generated content within a shared backbone. This formulation preserves the appearance anchoring of explicit visual memory while remaining compatible with script-driven shot-by-shot generation. Experiments show that SubjectAnchor improves cross-shot identity consistency over representative memory-based and holistic baselines while maintaining competitive visual quality.

Abstract

Visual Storytelling

Explore similar work

SubjectAnchor: Subject-Aware Memory-to-Video for Multi-Shot Storytelling

ReCap: Lightweight Referential Grounding for Coherent Story Visualization

CANVAS: Continuity-Aware Narratives via Visual Agentic Storyboarding