Manga2Animate: An Agentic Framework for Faithful Manga Animation with Script and Storyboard Planning

Sheng-Wei Zhang; Hsi-An Chen; Yan-Ting Liou; Chih-Hsuan Chen; Shih-Chen Tseng; Yu-Lun Liu

cs.CVOct 19, 2026

Manga2Animate: An Agentic Framework for Faithful Manga Animation with Script and Storyboard Planning

Authors: Sheng-Wei Zhang, Hsi-An Chen, Yan-Ting Liou, Chih-Hsuan Chen, Shih-Chen Tseng, Yu-Lun Liu

Organizations: Department of Computer Science, National Yang Ming Chiao Tung University

Abstract

Adapting existing manga into coherent animation that stays faithful to the source is a valuable yet challenging task. Existing methods either access the manga mainly through text, losing its characters and art style, or feed manga pages directly into video generation models, which misinterpret the manga's structure and produce incorrect storylines. In this paper, we propose Manga2Animate, a multi-agent framework that grounds every stage of generation in the source manga. We decompose the task into two intermediate representations, an animation script and a visual storyboard. Each representation is produced by a creator agent and reviewed against the source panels by a paired reviewer agent. We tailor representation-specific selection and dual-level feedback to progressively refine both before synthesizing the final animation. Extensive experiments and a user study demonstrate that Manga2Animate produces animations more faithful to the source manga and more consistent across shots than state-of-the-art approaches. Project page: https://annchenn.github.io/Manga2Animate/

Figures & tables

Figure 1: Manga2Animate turns existing manga pages into coherent animation that stays faithful to the original manga. Given a few manga pages (left) , our multi-agent framework first builds an animation script and a visual storyboard as intermediate representations, iteratively refining both through creator-reviewer collaboration, and then generates the final animation from them (right) . The resulting animation preserves the manga’s plot, character identity, and art style, while flowing coherently from clip to clip.

Figure 2: Limitations of existing methods to manga-to-animation generation. (a) Text-based storytelling methods (AniMaker, ViMax, StoryMem) access the manga mainly through text, losing its art style ( 1 ) or distorting characters ( 2 ). (b) Feeding manga pages into a video model (e.g., Seedance 2.0) tells the wrong story ( 3 ) and shows manga panels as video frames ( 4 ).

Figure 3: Overview of the Manga2Animate pipeline. (a) Pre-processing extracts ordered panels, a transcript, and character descriptions. (b) Creator and reviewer agents refine an animation script and a visual storyboard over $N$ rounds and select the best script and best storyboard, dual-level feedback also guides the next round. (c) Video generation runs in first-frame or reference-conditioned mode.

Figure 4: Compositional (clip-wise) selection by the Script Reviewer. The reviewer selects the best character and clip description at each panel position across the $N_{s}$ candidate scripts and reassembles them into one best script (left). For one panel (right), it flags inaccurate details (red) in two candidates and selects the one matching the panel (green).

Method	Model	Alignment		Video Quality				Aesthetic Quality		Cost
Method	Model	I2VS $\uparrow$	I2VB $\uparrow$	SC $\uparrow$	BC $\uparrow$	MS $\uparrow$	DD $\uparrow$	Shadow-v2 $\uparrow$	Waifu-v3 $\uparrow$	Calls (VLM)	Calls (Video Gen)	Time $\downarrow$
AniMaker	–	0.775	0.749	0.922	0.950	0.979	7.680	0.410	2.694	82.500	–	8h9m
ViMax	Veo 3.1 Fast	0.774	0.752	0.881	0.932	0.978	1.639	0.368	4.394	15.125	6.000	12m18s
ViMax	Seedance 2.0	0.753	0.747	0.927	0.948	0.977	2.541	0.385	4.764	15.125	6.000	16m33s
ViMax	MiniMax-H3	0.771	0.767	0.938	0.957	0.977	0.977	0.326	4.505	15.500	–	28m58s
StoryMem	–	0.728	0.731	0.885	0.922	0.956	5.203	0.275	3.602	10.300	–	2h16m8s

Table 1: Quantitative comparison of multi-shot video generation. Our main setting (Gemini + Seedance 2.0) performs better than AniMaker, ViMax, and StoryMem in I2VS, I2VB, BC, Shadow-v2, and Waifu-v3. For a fair comparison with ViMax, both methods use the same script, image model, and video model. Our FLUX.2 + MiniMax-H3 setting also performs well, showing that the improvement is not tied to one model combination. Direct generation has higher DD, but it may animate the page layout or produce the wrong actions (Fig. 2 (b)); a higher DD means more movement, not more accurate movement. Red , orange , and yellow mark the top three results;

\uparrow

\downarrow

show whether higher or lower is better. Call counts are not ranked; – means no calls.

Panel of Figure 5 — Figure 5: Qualitative comparison of storyboard generation on two manga examples (source pages on the left). Manga2Animate keeps the source art style, character identity, and plot across all six panels. AniMaker and AnyStory sometimes leave panels uncolored, StoryDiffusion uses a generic illustration style and repeats content, and Story-Iter becomes photorealistic and changes character appearance. In (b), the baselines also show the falling winged creature incorrectly or leave it out. Red boxes mark example failures.

Panel of Figure 6 — Figure 6: Qualitative comparison of video generation on two manga examples (source pages on the left). Manga2Animate keeps the manga’s content, characters, and art style. AniMaker adds unrelated people or backgrounds, ViMax mixes anime and photorealistic styles and adds text, and StoryMem produces blurry frames and mistakes the basketball scene for tennis. Baselines may also switch to black-and-white frames or change characters within a video; red boxes mark example failures.

Panel of Figure 7 — Figure 7: User study on storyboard and video generation. Thirty-seven participants evaluated eight storyboard and eight video cases shown with the input manga; method names were hidden and outputs randomized. For each criterion, participants selected the best result. Manga2Animate achieves the highest preference on every storyboard criterion (Character Identity, Manga Plot, Absence of AI Hallucination, and Overall Performance) and every video criterion (the same four plus Visual Comfort and Consistency).

Method	Alignment		Video Quality				Aesthetic Quality
Method	I2VS $\uparrow$	I2VB $\uparrow$	SC $\uparrow$	BC $\uparrow$	MS $\uparrow$	DD $\uparrow$	Shadow-v2 $\uparrow$	Waifu-v3 $\uparrow$
Without Script Reviewer	0.766	0.752	0.912	0.949	0.963	8.592	0.523	4.789
Without Storyboard Reviewer	0.783	0.756	0.923	0.948	0.971	5.616	0.479	4.540
Without Video Reviewer	0.768	0.749	0.902	0.941	0.957	8.330	0.517	4.996
Full Model	0.791	0.768	0.931	0.958	0.960	9.457	0.518	5.090

Table 2: Effect of removing each stage-specific reviewer. When a reviewer is removed, that stage generates only one candidate and receives no review or feedback; all other settings stay the same. Removing any reviewer lowers the overall alignment and consistency scores. Removing the Script Reviewer causes the largest drop in I2VS. Some variants score higher on MS or Shadow-v2, but score lower on alignment, consistency, or DD. Red and orange mark the best and second-best results; higher is better.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: Correspondence among intermediate representations. Each manga panel (left) corresponds to one clip in the animation script (middle) , which describes the environment, character positions, actions, and expressions of that panel in a structured JSON format. Each script clip is then rendered as one storyboard panel (right) . In first-frame mode, that panel conditions the corresponding video clip directly; in reference-conditioned mode, the complete storyboard jointly conditions the multi-shot video.

Method	V.C.		C.A.		A.Q.		Cost
Method	CSD $\uparrow$	CIDS $\uparrow$	PA $\uparrow$	OCCM $\uparrow$	Shadow-v2 $\uparrow$	Waifu-v3 $\uparrow$	Calls (VLM)	Calls (Img Gen)	Time $\downarrow$
StoryDiffusion	0.509	0.713	2.05	57.5	0.697	5.789	–	–	49s
Story-Iter	0.486	0.652	1.85	66.3	0.615	6.308	60.000	11.000	13m20s
AnyStory	0.605	0.792	2.66	69.3	0.606	3.781	10.000	1.000	2m7s
AniMaker	0.508	0.797	3.34	67.5	0.448	4.522	82.500	18.000	8h9m
Ours (FLUX.2)	0.622	0.797	3.14	73.5	0.552	3.423	58.000	–	1h59m

Appendix

Table A1: Quantitative comparison of storyboard generation. Red , orange , and yellow cells denote the best, second-best, and third-best results;

\uparrow

\downarrow

indicate that higher/lower is better. Call counts are not ranked; – denotes no calls of that type.

Method	Alignment		Video Quality				Aesthetic Quality
Method	I2VS $\uparrow$	I2VB $\uparrow$	SC $\uparrow$	BC $\uparrow$	MS $\uparrow$	DD $\uparrow$	Shadow-v2 $\uparrow$	Waifu-v3 $\uparrow$
Script-wise Selection	0.785	0.772	0.928	0.956	0.963	5.951	0.473	4.423
Clip-wise Selection (Full Model)	0.791	0.768	0.931	0.958	0.960	9.457	0.518	5.090

Appendix

Table A2: Ablation on script selection strategy. We compare Manga2Animate with clip-wise and script-wise selection. The full model selects the best clip description for each manga panel in a clip-wise manner and composes them into the final script, whereas script-wise selection directly picks one complete candidate script. All other models and settings are unchanged. Red cells denote the better result in each column.

Method	V.C.		C.A.		A.Q.
Method	CSD $\uparrow$	CIDS $\uparrow$	PA $\uparrow$	OCCM $\uparrow$	Shadow-v2 $\uparrow$	Waifu-v3 $\uparrow$
Without Script Feedback	0.597	0.786	3.33	70.8	0.334	3.813
Without Storyboard Feedback	0.588	0.770	3.39	72.6	0.249	3.257
Full Model	0.605	0.808	3.50	72.2	0.350	3.798

Appendix

Table A3: Ablation on dual-level feedback. The reviewers still select the best script and storyboard, but the feedback of one reviewer is withheld from the next-round Script Creator. All variants use the closed-source (Gemini) configuration. The Full Model row is the run reported in Table A1 . Cost is identical to the full pipeline and is therefore omitted. Red and orange mark the best and second-best results; higher is better.

Method	Alignment		Video Quality				Aesthetic Quality
Method	I2VS $\uparrow$	I2VB $\uparrow$	SC $\uparrow$	BC $\uparrow$	MS $\uparrow$	DD $\uparrow$	Shadow-v2 $\uparrow$	Waifu-v3 $\uparrow$
w/o Refinement	0.778	0.754	0.914	0.952	0.965	7.872	0.521	4.818
w/o Storyboard Gen.	0.766	0.745	0.896	0.947	0.961	8.289	0.515	5.050
Full Model	0.791	0.768	0.931	0.958	0.960	9.457	0.518	5.090

Appendix

Table A4: Ablation on the refinement loop and storyboard generation. w/o Refinement uses the first-pass script and storyboard directly, skipping the second-round feedback; w/o Storyboard Gen. keeps the same fully refined script but replaces the generated storyboard with a composite of the original manga panels. Removing either stage lowers I2VS, I2VB, SC, and BC relative to the full model, and removing storyboard generation causes the larger drop. Some variants score higher on MS or Shadow-v2, but lower on alignment, consistency, or DD. Red and orange mark the best and second-best results; higher is better.

Mode	Alignment		Video Quality				Aesthetic Quality
Mode	I2VS $\uparrow$	I2VB $\uparrow$	SC $\uparrow$	BC $\uparrow$	MS $\uparrow$	DD $\uparrow$	Shadow-v2 $\uparrow$	Waifu-v3 $\uparrow$
Reference-conditioned	0.791	0.768	0.931	0.958	0.960	9.457	0.518	5.090
First-frame	0.772	0.778	0.935	0.941	0.962	4.045	0.302	4.042

Appendix

Table A5: Comparison of Video Generation Modes. Both modes use Gemini for storyboard generation and Seedance 2.0 for video generation. Bold indicates the higher value for each metric. Reference-conditioned generation gives better subject alignment, background consistency, and aesthetic scores, while first-frame generation gives better background alignment and slightly better subject consistency and motion smoothness.

Table 16

Agentic Video Generation

Agentic video generation uses AI agents to plan, create, evaluate, and refine videos through coordinated tools and feedback.

See all 40 papers

MangaFlow: An End-to-End Agentic Framework for Controllable Story to Manga Generation
May 27, 2026Muyao Wang, Zeke Xie, Yanhao Chen +2Image Generation Controllable Image Generation
The University of Tokyo · Hong Kong University of Science and Technology (Guangzhou)
AnimeAgent: Is the Multi-Agent via Image-to-Video models a Good Disney Storytelling Artist?
Feb 24, 2026Hailong Yan, Shice Liu, Tao Wang +5Agentic Video Generation Temporal Consistency in Video Generation
UESTC · vivo Mobile Communication Co. Ltd
ManGo: Manga Active Narrative Grounding Optimization
Aug 30, 2026Hao Qiu, Junyan Wang, Zheyuan Liu +4Visual Question Answering RL for Language Model Reasoning
1Communication University of China · 2Australian Institute for Machine Learning, Adelaide University · University of New South Wales +2

May 27, 2026J/K move · Enter open · S save

MangaFlow: An End-to-End Agentic Framework for Controllable Story to Manga Generation

Muyao Wang, Zeke Xie, Yanhao Chen, Lixin Xiu+1

The University of Tokyo · Hong Kong University of Science and Technology (Guangzhou)

Image Generation Controllable Image Generation

Open paper AskPDF arXiv

End-to-end manga generation is a structured visual storytelling task that requires story decomposition, recurring character and scene grounding, page layout design, panel rendering, page composition, and lettering. However, existing generative models often perform direct page synthesis, entangling these factors in a single visual output and limiting precise control over layout geometry, visual references, and cross-panel consistency. To address these limitations, we propose MangaFlow, an agentic framework for controllable long-form manga generation that decomposes manga creation into planning, grounding, layout construction, reference-conditioned rendering, composition, and text placement. By treating layout and visual references as explicit intermediate variables, MangaFlow enables both simple text-to-manga generation and more precise user-controlled manga creation. This design exposes layout, visual assets, and lettering as editable intermediate controls for refining panel geometry, references, and text placement. To support long-form consistency, MangaFlow introduces a story section memory that links section descriptions with corresponding character, scene, and object references for reuse across panels. We further present a meta-benchmark for evaluating layout controllability, visual consistency, and generation quality. Experiments show that MangaFlow improves layout adherence and cross-panel consistency over direct generation baselines while supporting flexible human control.

Abstract

Agentic Video Generation

Explore similar work

MangaFlow: An End-to-End Agentic Framework for Controllable Story to Manga Generation

AnimeAgent: Is the Multi-Agent via Image-to-Video models a Good Disney Storytelling Artist?

ManGo: Manga Active Narrative Grounding Optimization