EvoTale: Continual Character Customization for Expanding Story Worlds
Authors: Jinlu Zhang, Qiyun Wang, Baoxiang Du, Jiayi Ji, Jing He, Rongsheng Zhang, Tangjie Lv, Xiaoshuai Sun, Rongrong Ji
Organizations: Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R., China. · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 510000, P.R., China. · Fuxi AI Lab, NetEase Inc., Hangzhou, 310000, P.R., China.
Character-centric story visualization aims to synthesize coherent image sequences that depict narrative events and interactions while preserving recurring character identities. In expanding story worlds, new user-specified characters must be continually incorporated despite varying customization difficulty and identity conflicts in multi-character scenes, without disrupting previously learned identities. In this paper, we propose EvoTale, a continual character customization framework for expanding stylized story worlds. We first introduce an All-in-One-World Character Integrator, which accumulates character-specific residual components within a unified LoRA branch using sparsely overlapping subspaces spanned by a shared orthonormal basis, while freezing previously learned components to limit cross-character coupling. We then develop a Character Quality Gate that uses rubric-guided MLLM feedback as a bounded controller to adapt the optimization budget based on the assessed customization quality. Finally, we propose Character-Aware Region-Focus Sampling, which combines bounding-box-guided regional denoising with identity-aware global denoising to preserve character identities within their designated regions while maintaining global narrative coherence. Experimental results show that EvoTale achieves a favorable balance across character fidelity, continual identity retention, multi-character generation quality, and story-text alignment compared with representative story visualization and customization methods.
Figures & tables
Figure 1: Visual comparisons and continual character integration in expanding stylized story worlds. (a) Compared with representative methods, EvoTale better preserves user-specified identities and scene coherence in single- and multi-character settings. (b) EvoTale continually expands the character set while retaining previously integrated identities, as illustrated by V2 after adding V7 in Pororo and U2 after adding U4 in Frozen .
Figure 2: Overview of EvoTale’s continual character customization pipeline. The All-in-One-World Character Integrator sequentially incorporates newly introduced characters into a unified framework while preserving previously acquired character knowledge. After each customization attempt, the Character Quality Gate (CQG) employs an MLLM as a quality judge to generate an evaluation prompt, assess the customized result, and determine whether to proceed to the next character or further refine the current one.
Figure 3: Overview of Character-Aware Region-Focus Sampling. RS-CD predicts character-specific regional noise, while IA-GD preserves global context through character-conditioned attention. A time-aware schedule initially favors RS-CD for identity preservation and gradually shifts toward IA-GD for coherent scene composition.
Type
Method
Pororo
Frozen
D-I ↑
C-I ↑
C-T ↑
D-I ↑
C-I ↑
C-T ↑
SD1.5-Based Comparison
Tuning-free
DisEnvisioner
47.03
74.76
34.05
49.63
83.53
32.80
Tuning-based
Mix-of-Show
49.89
79.38
34.17
45.05
80.66
35.08
Tuning-based
Orth-Adapt
52.01
79.99
34.06
49.61
82.56
35.10
CL
Textual Inversion
42.99
73.32
34.92
36.40
73.40
35.46
Table 1: Quantitative comparisons on single-character story settings. The upper block reports comparisons among SD1.5-based methods, while the lower block provides cross-backbone reference results. Best results are shown in bold and the second-best are underlined . ⋆ denotes DiT-based models and † denotes SDXL-based models; unmarked methods use SD1.5.
Figure 4: Qualitative comparison of single-character story visualization. The arrows indicate the character learning order for continual-learning methods. EvoTale preserves character identity while maintaining favorable alignment with the story descriptions across both story settings.
Figure 5: Identity preservation during continual integration. (a)–(b) Identity scores of the first learned character across learning stages on Pororo and Frozen . (c) Qualitative results across stages, with arrows indicating the learning order.
Dataset
Method
D-I ↑
C-I ↑
C-T ↑
Before
After
Δ
Before
After
Δ
Before
After
Δ
Pororo
Mix-of-Show
53.40
49.89
-3.51
81.13
79.38
-1.75
32.71
34.17
+1.46
Orth-Adapt
54.18
52.01
-2.17
80.76
79.99
-0.77
33.60
34.06
+0.46
EvoTale
54.85
54.46
-0.39
80.92
80.89
-0.03
34.73
34.42
-0.31
Frozen
Mix-of-Show
49.99
45.05
-4.94
83.26
80.66
-2.60
34.46
35.08
+0.62
Orth-Adapt
51.96
49.61
-2.35
84.51
82.56
-1.95
34.65
35.10
+0.45
Table 2: Before-and-after comparison of character customization performance. For Mix-of-Show and Orth-Adapt, Before denotes independently trained character modules, while After denotes the merged model. For EvoTale, Before denotes the average performance of each character immediately after its introduction, while After denotes its average performance in the final continually updated model. Δ denotes After minus Before .
Figure 6: Character identity before and after multi-character integration. EvoTale exhibits smaller identity changes than post-hoc composition methods after integrating multiple characters.
Type
Method
Pororo
Frozen
D-I ↑
C-I ↑
C-T ↑
D-I ↑
C-I ↑
C-T ↑
SD1.5-Based Comparison
Tuning-based
LoRA-S
37.24
65.49
33.60
37.47
72.13
33.90
LoRA-C
37.31
65.54
33.85
42.38
74.20
34.28
Mix-of-Show
40.27
69.13
34.72
43.57
74.34
36.11
Mix-of-Show ∗
36.23
68.04
34.56
39.24
75.49
36.08
Table 3: Quantitative comparison on multi-character story visualization. The upper block compares methods in SD1.5, while the lower block reports SDXL reference results. Best results are shown in bold and the second-best are underlined , while only the best result is highlighted in the SDXL block. † denotes SDXL-based models; unmarked methods use SD1.5.
Figure 7: Qualitative comparison of multi-character story visualization. EvoTale better preserves distinct character identities while maintaining natural interactions and strong alignment with the story descriptions.
Method
C-A ↑
S-A ↑
V-Q ↑
Single Character
DreamStory †
1.56
3.52
3.80
CharaConsist ⋆
1.80
3.44
3.56
DisEnvisioner
1.44
2.36
2.12
FLUX.1 Kontext ⋆
3.12
3.76
3.28
Mix-of-Show
2.64
2.88
2.92
Table 4: User study on single- and multi-character story visualization. C-A, S-A, and V-Q denote character alignment, story alignment, and visual quality, respectively. Best results are shown in bold and the second-best are underlined . ⋆ denotes DiT-based models and † denotes SDXL-based models; unmarked methods use SD1.5.
Setting
Pororo
Frozen
D-I ↑
C-I ↑
C-T ↑
D-I ↑
C-I ↑
C-T ↑
Effect of Main Components
Shared LoRA (Sequential, No Replay)
45.75
77.04
35.61
42.84
78.75
36.08
w/o CQG (Fixed Base Budget)
51.85
79.53
35.45
47.63
80.88
35.73
w/o CQG (Fixed Large Budget)
59.32
83.84
31.96
56.42
85.03
33.19
Full Model
54.46
80.89
34.42
51.50
82.84
35.23
Table 5: Ablation study of the main components and CI parameterization on single-character story visualization. For each dataset, best results are shown in bold and the second-best are underlined . LT denotes learnable token embeddings.
Setting
Pororo
Frozen
D-I ↑
C-I ↑
C-T ↑
D-I ↑
C-I ↑
C-T ↑
RS-CD + Global (Fixed Fusion)
42.87
68.18
36.43
43.32
76.19
36.05
RS-CD + Global (Time-Aware Fusion)
37.29
65.95
36.31
38.85
75.11
35.54
RS-CD + IA-GD (Cross-AR)
42.68
68.13
36.79
42.53
77.16
36.04
RS-CD + IA-GD (Cross-AR + Regional-SA)
46.35
69.73
36.25
45.65
76.97
36.57
Full CA-RF
44.95
69.54
36.59
45.32
77.98
36.44
Table 6: Ablation study of CA-RF on multi-character story visualization. Best results are shown in bold and the second-best are underlined . RS-CD and IA-GD denote Role-Specific Crop Denoising and Identity-Aware Global Denoising, and Global denotes standard global denoising. Cross-AR and Regional-SA denote character-specific cross-attention routing and region-restricted self-attention, respectively.
Figure 8: Qualitative ablation results for the key components of EvoTale. (a) CI and CQG in single-character story settings. (b) CA-RF sampling in multi-character story settings.
Story Visualization aims to generate a sequence of images that faithfully depicts a textual narrative that preserve character identity, spatial configuration, and stylistic coherence as the narratives unfold. Maintaining such cross-frame consistency has traditionally relied on explicit memory banks, architectural expansion, or auxiliary language models, resulting in substantial parameter growth and inference overhead. We introduce ReCap, a lightweight consistency framework that improves character stability and visual fidelity without modifying the base diffusion backbone. ReCap's CORE (COnditional frame REferencing) module treats anaphors, in our case pronouns, as visual anchors, activating only when characters are referred to by a pronoun and conditioning on the preceding frame to propagate visual identity. This selective design avoids unconditional cross-frame conditioning and introduces only 149K additional parameters, a fraction of the cost of memory-bank and LLM-augmented approaches. To further stabilize identity, we incorporate SemDrift (Guided Semantic Drift Correction) applied only during training. When text is vague or referential, the denoiser lacks a visual anchor for identity-defining attributes, causing character appearance to drift across frames, SemDrift corrects this by aligning denoiser representations with pretrained DINOv3 visual embeddings, enforcing semantic identity stability at zero inference cost. ReCap outperforms previous state-of-the-art, StoryGPT-V, on the two main benchmarks for story visualization by 2.63% Character-Accuracy on FlintstonesSV and by 5.65% on PororoSV, establishing a new state-of-the-art character consistency on both benchmarks. Furthermore, we extend story visualization to human-centric narratives derived from real films, demonstrating the capability of ReCap beyond stylized cartoon domains.
Aditya Arora, Akshita Gupta, Pau Rodriguez +1
Technical University of Darmstadt & hessian.AI, Germany
Story visualization requires generating sequential imagery that aligns semantically with evolving narratives while maintaining rigorous consistency in character identity and visual style. However, existing methodologies often struggle with subject inconsistency and identity drift, particularly when depicting complex interactions or extended narrative arcs. To address these challenges, we propose a cohesive two-stage framework designed for robust and consistent story generation. First, we introduce Group-Shared Attention (GSA), a mechanism that fosters intrinsic consistency by enabling lossless cross-sample information flow within attention layers. This allows the model to structurally encode identity correspondence across frames without relying on external encoders. Second, we leverage Direct Preference Optimization (DPO) to align generated outputs with human aesthetic and narrative standards. Unlike conventional methods that rely on conflicting auxiliary losses, our approach simultaneously enhances visual fidelity and identity preservation by learning from holistic preference data. Extensive evaluations on the ViStoryBench benchmark demonstrate that our method establishes a new state-of-the-art, significantly outperforming strong baselines with gains of +10.0 in Character Identity (CIDS) and +18.7 in Style Consistency (CSD), all while preserving high-fidelity generation.
Jianzhang Zhang, Yijing Tian, Jiwang Qu +1
Department of Management Science and Engineering, Hangzhou Normal University, Hangzhou, Zhejiang, P.R. China
Visual storytelling aims to generate image sequences that are both aligned with narrative prompts and consistent in character appearance across images. Recent training-free methods improve character consistency by reusing attention features, but rely on structured prompts where full character descriptions are repeated in every prompt. This assumption simplifies the task but deviates from natural storytelling, where characters are typically introduced once and later referred to using pronouns or type-based expressions. We propose FreeStory, a training-free framework that reformulates character consistency under free-form prompts as entity-grounded feature reuse. Our method associates reference mentions with their corresponding character descriptions and combines dynamic character masks, correspondence-aware feature matching, key-value injection, and query blending to preserve identity while retaining generation diversity. We also introduce FreeStoryBench, a benchmark for this setting that includes both single- and multi-character stories. Experiments show that FreeStory achieves state-of-the-art performance among training-free methods on structured benchmarks and stronger overall consistency over baselines under free-form prompts.
Sibo Dong, Ismail Shaheen, Sarah Adel Bargal
Department of Computer Science, Georgetown University