DataMagic: Authoring Data Videos through Declarative Multi-Agent Orchestration

Yupeng Xie; Zhenyang Wang; Liangwei Wang; Jiayi Zhu; Zhouan Shen; Yuyu Luo

cs.HCSep 27, 2026

DataMagic: Authoring Data Videos through Declarative Multi-Agent Orchestration

Authors: Yupeng Xie, Zhenyang Wang, Liangwei Wang, Jiayi Zhu, Zhouan Shen, Yuyu Luo

Organizations: The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China

Abstract

Data videos communicate data insights through dynamic charts, voice narration, and synchronized animations, and have become a widely adopted form of data storytelling. However, producing them requires expertise in data analysis, narrative design, and video editing. Static visualization tools lack narrative and animation capabilities; authoring tools rely on pre-prepared charts rather than raw data; and pixel-level models generate videos end-to-end but cannot guarantee data accuracy or provenance. End-to-end automatic generation faces two core challenges: how to uniformly represent charts, narration, and animations together with their temporal relationships, and how to efficiently search a vast design space for narrative-coherent compositions. We present DataMagic, which authors data videos from raw tabular data through declarative multi-agent orchestration. First, the declarative specification DVSpec unifies charts, narration, and animations with data-bound references and declarative synchronization, ensuring data provenance and automatic audio-visual alignment. Second, a "Generate-then-Orchestrate" multi-agent strategy generates candidate scenes in parallel and then optimizes narrative coherence through global orchestration. DVSpec provides a shared state for three complementary interaction modes, bridging full automation with fine-grained human control. Evaluations on 109 real-world samples show that even the most advanced LLM (e.g., GPT-5) achieves only 2.13/5 with execution success rates between 48.62% and 86.24%; DataMagic improves quality to 3.89 (+83%) with success rates above 95%, with the most significant gains in animation and narrative dimensions. A user study shows that, compared to a conversational LLM workflow, DataMagic improves creation efficiency (79.7% reduction in task time) and reduces perceived cognitive load. Project page: https://github.com/HKUSTDial/DataMagic.

Figures & tables

Figure 1 : DVSpec structure and rendering flow.

Figure 2 : A DVSpec specification example for one scene, where the numbered markers (1–4) link each declarative field (chart, data, narration, and emphasis animation) to its effect in the rendered scene.

Figure 3 : System architecture of DataMagic .

Figure 4 : DataMagic web interface, showing (A–D) the main UI components and (D1–D3) three key interaction flows.

		Evaluation Dimensions
Method	Exec Rate (%)	Intent	Insight	Narrative	Animation	Aesthetic	Avg. Score
Direct Generation Methods
DeepSeek-V3.2	48.62%	1.95	1.98	1.88	1.65	2.09	1.91
Gemini-2.5-Pro	66.06%	2.38	2.25	2.07	1.94	2.44	2.22
GPT-5	86.24%	2.36	2.22	2.05	1.84	2.17	2.13
Claude-Sonnet-4	84.40%	2.28	2.01	1.98	1.91	2.71	2.18

Table 1 : End-to-End Video Generation Performance Comparison.

Variant	Intent	Insight	Narrative	Animation	Aesthetic	Avg. Score
DataMagic (Full)	3.79	3.37	3.84	4.39	4.05	3.89
w/o Story Planner	3.42	3.16	3.26	3.79	3.58	3.44
w/o Orchestration	3.32	3.21	3.42	3.95	3.79	3.54

Table 2 : Ablation study of DataMagic variants.

Figure 5 : User-study results (

N=12

). DataMagic reduced task time from 39.2 to 8.0 min (79.7%) and lowered ratings on five NASA-TLX workload dimensions, with no significant Performance difference. Error bars: 95% CIs; Wilcoxon signed-rank tests:

{}^{*}p<.05

{}^{**}p<.01

{}^{***}p<.001

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6 : Comparative Analysis of Video Generation Quality Across Different Methods

Figure 7 : A representative high-quality case generated by DataMagic (Agriculture, overall score: 4.6). Four key frames are shown with their corresponding narration (italicized). Underlined spans in the narration are the index triggers in DVSpec that fire the co-occurring visual highlights, demonstrating precise audio-visual synchronization without manual timeline adjustment.

Figure 8 : High-quality Case 1 (Environmental Science): High-fidelity tracking of meteorological fluctuations and solar radiation swings.

Figure 9 : High-quality Case 2 (Gender-based Compensation): Visualization of complex gender-based compensation disparities across divisions.

Figure 10 : Low-quality Case 1 (Agriculture): Analysis of layout truncation and numerical discrepancies in DataMagic under extreme data distributions.

Figure 11 : Low-quality Case 2 (Energy): Analysis of systemic failures in baseline models across layout logic and cross-modal alignment.

Source	Datasets	Samples	Small (<100 rows)	Med. (100-1K)	Large (>1K rows)
T2R-bench	36	85	4	14	18
DAComp-DA	24	24	2	7	15
Total	60	109	6	21	33

Appendix

Table 3 : Dataset sources and scale distribution.

Dimension	Min	Median	Max	Mean
Rows	23	1,436	150,000	17,065.58
Columns	3	16	555	32.42
Numeric columns	1	8	552	24.25
Categorical columns	0	7	57	8.17

Appendix

Table 4 : Dataset dimension statistics.

Query Type	Count	Percentage	Typical Patterns
Trend analysis	12	11.0%	Time-series changes; Periodic patterns
Comparative analysis	15	13.8%	Cross-category comparison; Ranking analysis
Distribution analysis	18	16.5%	Value distribution; Proportion
Correlation analysis	33	30.2%	Variable relationships; Correlation
Comprehensive analysis	31	28.4%	Multi-dimensional analysis
Total	109	100.0%	–

Appendix

Table 5 : Query type distribution of the sample dataset.

Domains	Sub-domains
Technology and Engineering	Electronics and Automation Manufacturing; Academic Research; Energy Production and Power Systems; Automotive Industry
Environmental Management	Environmental Protection; Agriculture and Forestry; Resource Management
Transportation Logistics	Communication and Digital Infrastructure; Transportation Networks and Logistics Management
Social Policy Administration	Education Policy and Public Education; Government Administration and Public Sector Services; Labor and Employment Administration; Healthcare Systems and Public Health; Demographics and Social Development
Commercial Services and Markets	Retail Trade and E-commerce Platforms; Tourism and Hospitality Services; Digital Entertainment and Gaming; Food and Beverage Services; Real Estate and Housing Market; Business Management and Supply Chain
Financial Economics	Economic Development and International Trade; Banking and Financial Services

Appendix

Table 6 : Domains and sub-domains.

Figure 12 : The online evaluation platform used for human expert validation. The interface integrates the original user query, synchronized video playback, a multi-dimensional scoring guide, and a feedback module for documenting qualitative observations.

Dimension	Pearson $r$	MAE	MSE
Overall	0.91	0.38	0.23
Intent	0.78	0.57	0.49
Insight	0.76	0.51	0.47
Narrative	0.83	0.60	0.59
Animation	0.90	0.47	0.48
Aesthetic	0.77	0.73	0.84

Appendix

Table 7 : Automated Evaluator–Human Expert Consistency Metrics.

Explore similar work

Jun 18, 2026cs.HC

DataMagic: Transforming Tabular Data into Data Insight Video

Data videos integrate dynamic charts, voice narration, and synchronized animations to communicate data insights as temporal narratives, making them an effective medium for improving data consumption efficiency in the data management lifecycle. However, producing high-quality data videos requires expertise spanning data analysis, narrative design, and video production. Existing approaches fall short: static visualization tools (e.g., BI dashboards) lack narrative logic and animation; authoring tools require users to pre-prepare visualizations rather than working from raw data; pixel-level video generation models cannot guarantee data fidelity or provenance. We demonstrate DataMagic, an end-to-end interactive system that transforms raw tabular data and natural language queries into narrative data-insight videos. To ensure data fidelity, DataMagic introduces the declarative specification DVSpec, which binds visual and animation elements to underlying data fields through data-driven semantic references. To address the combinatorial explosion of the design space, DataMagic adopts a Generate-then-Orchestrate multi-agent architecture that generates candidate scenes in parallel and then optimizes narrative coherence through global orchestration. Leveraging DVSpec's decoupling of logic and rendering, the system further supports three interaction modes and structured provenance-based data Q&A, transforming one-way videos into explorable interactive data interfaces. Evaluation on 109 real-world samples validates the effectiveness of the DataMagic. Homepage: https://datamagic-home.github.io/

Yupeng Xie, Chen Ma, Zhenyang Wang +6

HKUST (GZ) · China Unicom

Visual Analytics Large Databases Tabular Data Insights Narratives

Ask PDF

Apr 28, 2026cs.AI

DATAREEL: Automated Data-Driven Video Story Generation with Animations

Data videos are a powerful medium for visual data based storytelling, combining animated, chart-centric visualizations with synchronized narration. Widely used in journalism, education, and public communication, they help audiences understand complex data through clear and engaging visual explanations. Despite their growing impact, generating data-driven video stories remains challenging, as it requires careful coordination of visual encoding, temporal progression, and narration and substantial expertise in visualization design, animation, and video-editing tools. Recent advances in large language models offer new opportunities to automate this process; however, there is currently no benchmark for rigorously evaluating models on animated visualization-based video storytelling. To address this gap, we introduce DataReel, a benchmark for automated data-driven video story generation comprising 328 real-world stories. Each story pairs structured data, a chart visualization, and a narration transcript, enabling systematic evaluation of models' abilities to generate animated data video stories. We further propose a multi-agent framework that decomposes the task into planning, generation, and verification stages, mirroring key aspects of the human storytelling process. Experiments show that this multi-agent approach outperforms direct prompting baselines under both automatic and human evaluations, while revealing persistent challenges in coordinating animation, narration, and visual emphasis. We release DataReel at https://github.com/vis-nlp/DataReel.

Ridwan Mahbub, Syem Aziz, Mahir Ahmed +4

York University · Bangladesh University of Business and Technology · RBC, Canada +2

Video Storytelling Long Video Generation Long Videos Narratives Lotteries

Ask PDF

Aug 30, 2026cs.CL

DVBench: Benchmarking MLLMs for Understanding Dynamic Charts and Narratives in Data Videos

While MLLMs have made significant strides in chart comprehension and video understanding, current evaluations largely isolate these capabilities, leaving a critical gap in understanding temporally evolving structured visual information. To address this gap, we introduce DVBench, a benchmark for evaluating MLLMs on data videos, a storytelling medium that integrates dynamic charts with structured narratives. We decompose data video understanding into five dimensions. DVBench comprises 300 real-world data videos and 1,000 human-verified QA pairs curated through a rigorous semi-automated pipeline. Extensive evaluations of nine MLLMs show that Gemini-3.1-Pro achieves the best overall performance, while Kimi-k2.5 is the strongest open-source model. We further identify two notable phenomena: open-source model performance does not scale strictly with parameter size, and narrative proficiency does not guarantee visual capability. Fine-grained analyses and ablation studies further reveal dimension-specific weaknesses and the effects of frame configurations and subtitle inputs, informing future MLLM development. DVBench is publicly available at https://bomiaowang.github.io/DVBench/.

Bomiao Wang, Zekai Shao, Jiexiang Lan +3

Fudan University · The Hong Kong University of Science and Technology (Guangzhou)

Long Videos Chart Multimodal Large Language Models

Ask PDF