Organizations: State Key Lab of CAD&CG, Zhejiang University · HKUST(GZ) · National University of Singapore · University of Virginia · Cornell University
Interactive documents help readers engage with complex ideas through dynamic visualization, interactive animations, and exploratory interfaces. However, creating such documents remains costly, as it requires both domain expertise and web development skills. Recent Large Language Model (LLM)-based agents can automate content creation, but directly applying them to interactive document generation often produces outputs that are difficult to control. To address this, we present ViviDoc, to the best of our knowledge the first work to systematically address interactive document generation. ViviDoc introduces a multi-agent pipeline (Planner, Styler, Executor, Evaluator). To make the generation process controllable, we provide three levels of human control: (1) the Document Specification (DocSpec) with SRTC Interaction Specifications (State, Render, Transition, Constraint) for structured planning, (2) a content-aware Style Palette for customizing writing and interaction styles, and (3) chat-based editing for iterative refinement. We also construct ViviBench, a benchmark of 101 topics derived from real-world interactive documents across 11 domains, along with a taxonomy of 8 interaction types and a 4-dimensional automated evaluation framework validated against human ratings (Pearson r > 0.84). Experiments show that ViviDoc achieves the highest content richness and interaction quality in both automated and human evaluation. A 12-person user study confirms that the system is easy to use, provides effective control over the generation process, and produces documents that satisfy users.
Figures & tables
Figure 1. Eight interactive visualization examples generated by ViviDoc , covering all 8 interaction types in our taxonomy, with styles automatically adapted to each topic’s content. (a) Parameter Exploration : sliders adjust flow rate and elevation parameters in real time. (b) Direct Manipulation : drag the object or focal points to update the lens equation live. (c) Inspection : hover to reveal a Voronoi cell and nearest-neighbor envelope. (d) Freeform Construction : click to place neurons and trigger animated signal propagation. (e) Scroll-driven Narrative : scroll to mix two particle gases and watch entropy rise. (f) Spatial Navigation : drag to rotate a 3D Möbius strip freely in space. (g) State Switching : switch quantum orbital states to redraw the electron probability cloud. (h) Temporal Control : play/pause and tune harmonics to build a Fourier series.
Figure 2. The ViviDoc pipeline. Given a topic, the Planner generates a DocSpec consisting of knowledge units with text descriptions and SRTC Interaction Specifications. The Styler analyzes the DocSpec and generates a Style Palette for users to customize writing and interaction styles. The Executor generates the document code guided by the DocSpec and style instructions. The Evaluator checks the output for correctness. Users can intervene at three points: editing the DocSpec, customizing the Style Palette, and refining the document through chat.
Natural language description
“The reader can adjust the radius of a circle using a slider, and as the radius changes, the visualization updates the circle while recalculating and displaying the circumference, diameter, and ratio, allowing the reader to see that the ratio stays roughly the same.”
SRTC Interaction Specification
S
r : slider [0.5,5] , default 1; C : derived 2πr ; D : derived 2r ; ratio : derived C/D
R
A circle whose size reflects r ; labels showing C , D , and ratio
T
Dragging the slider changes r ; C , D , ratio update automatically
C
ratio ≈3.14159 regardless of r
Table 1. Comparison of a natural language interaction description and its SRTC Interaction Specification for a visualization about π .
Figure 3. The ViviDoc user interface. Top: the main view with (A) sidebar for history and new document creation, (B) center panel with topic input and four-stage navigation bar, and (C) AI chat assistant. Bottom (left to right): the Spec stage showing editable knowledge units, the Style stage with writing and interaction style options, and the Doc stage displaying the generated interactive document.
Figure 4. Automated evaluation results for ViviDoc vs. three multi-agent baselines across three backbone LLMs. Content Richness (CR) and Interaction Quality (IQ) are normalized to [0,1] . Interaction Functionality is on a 0–1 scale; Efficiency is measured in characters per second (chars/s).
Backbone
Method
CR ↑
IQ ↑
IF ↑
Eff ↑
Gemini 3 Flash
Naive Agent
4.00
3.24
0.85
539
ViviDoc
5.00
4.58
0.92
505
Mistral Small
Naive Agent
4.44
2.36
0.61
695
ViviDoc
5.00
3.24
0.65
562
Qwen3.5-35B
Naive Agent
4.11
2.76
0.70
355
ViviDoc
4.56
3.57
0.73
178
Table 2. DocSpec effectiveness: ViviDoc vs. Naive Agent across three backbone LLMs. CR: Content Richness (1–5), IQ: Interaction Quality (0–5), IF: Interaction Functionality (0–1), Eff: Efficiency (chars/s). Yellow : best per backbone.
AutoGen
MetaGPT
CAMEL
ViviDoc
CR ↑
2.79
2.51
1.60
4.43
ID ↑
3.55
1.85
1.76
4.33
Table 3. Mean human evaluation scores by method (Gemini 3 Flash backbone), averaged across 12 raters and 9 topics. Yellow : best per metric.
Item
Mean ± SD
Usability
Q1
Easy to learn and use
5.00±0.00
Q2
Interface intuitive and satisfying
5.00±0.00
Controllability
Q3
DocSpec gives meaningful control
4.50±0.76
Q4
Style Palette gives meaningful control
4.17±0.49
Table 4. User study ratings (5-point Likert scale, n=12 ).
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Subject Area
# Topics
Algorithms
25
Mathematics
24
Tools & Resources
13
Physics
10
Science
9
Other
6
Appendix
Table 5. Distribution of topics in ViviBench by subject area.
• A continuously growing 3D phase-space trajectory projected onto a 2D canvas • The trail fades over time to emphasize recent motion • A slow auto-rotation of the view around the vertical axis • Two sliders for σ and ρ displayed below the canvas
T
• Adjusting the σ slider resets the trajectory and restarts integration from the same initial point • Adjusting the ρ slider resets the trajectory, causing the attractor shape to deform or collapse into a stable orbit
C
For classical values ( σ=10 , ρ=28 , β=8/3 ), the trajectory never repeats and draws a distinctive butterfly-shape—demonstrating sensitive dependence on initial conditions.
Appendix
Table 10
S
object_x : drag-x [20,lens_x−10] ; object_y : drag-y [0,canvas_height] ; f : drag-x [40,300] ; u : derived lens_x−object_x ; v : derived (u⋅f)/(u−f) ; M : derived v/u ; image_type : derived if v>0 : Real Inverted, if v<0 : Virtual Upright, if v=∞ : Undefined
R
• A central convex lens whose thickness scales with focal length • An orange draggable object arrow on the left of the lens • Two green draggable focal points F and F′ on the optical axis • Three principal rays drawn from the object tip through the lens • A colored image arrow on the right (green for real, blue for virtual) • An instrument dashboard showing live values of f,u,v , and M • A status tag indicating image type and orientation
T
• Dragging the orange object arrow horizontally changes u and updates all derived optical values • Dragging the object arrow vertically changes the object height and redraws the ray diagram • Dragging a focal point ( F or F′ ) changes f , reshaping both the lens and all derived values simultaneously
C
The thin lens equation 1/u+1/v=1/f is always satisfied. When u<f , the image distance v becomes negative (virtual image). When u=f , the image forms at infinity.
Appendix
Table 11
S
seeds : 15 points initialized at random positions, moving with slow random velocity; mouse_pos : hover; nearest : derived argminkd(mouse_pos,seeds[k]) ; min_dist : derived d(mouse_pos,seeds[nearest]) ; cell_region : derived all pixels closest to seeds[nearest] within 150px from mouse_pos
R
• An animated canvas of 15 slowly drifting seed points on a dark background • On hover: the hovered Voronoi cell illuminated with a purple radial gradient • On hover: a dashed gold line connecting the cursor to its nearest seed • On hover: a transparent circle of radius min_dist visualizing the nearest-neighbor envelope • The nearest seed highlighted in gold; all others remain purple
T
• Moving the mouse over the canvas updates mouse_pos continuously • Each frame recalculates the nearest seed and updates the illuminated cell, connecting line, and envelope circle in real-time
C
The illuminated region always corresponds exactly to the Voronoi cell of the nearest seed. Every point within the highlighted region is provably closer to that seed than to any other—demonstrating the core definition of Voronoi partitioning.
Appendix
Table 12
S
hidden_nodes : click-to-place list of {x,y} , default [] ; weights : derived random initialization; activations : derived forward pass (sigmoid); output_val : derived average of output neuron activations
R
• Three fixed red input nodes ( x1,x2,x3 ) on the left • Two fixed blue output nodes ( y1,y2 ) on the right • User-placed yellow hidden nodes in the central region • Directed arrows connecting all layers; animate green (input → hidden) then orange (hidden → output) during forward pass • Activation values displayed on each node • An output activation progress bar and percentage readout • A ’Send Signal’ button and a ’Clear’ button
T
• Clicking in the central canvas zone places a new hidden neuron and immediately triggers an animated forward pass • Pressing ’Send Signal’ replays the forward pass with newly randomized weights • Pressing ’Clear’ removes all hidden nodes and resets the output bar to 50%
C
Adding more hidden neurons generally introduces more non-linearity. The output activation always falls in (0,1) due to the sigmoid function, regardless of network topology.
Appendix
Table 13
S
scroll_progress : scroll-wheel [0,1] , default 0; partition_y : derived scroll_progress×canvas_height ; particles : 150 red + 150 blue bouncing particles; entropy_S : derived fraction of particles that have crossed to the other side times 100
R
• A dark split-canvas with 300 bouncing particles: red on left, blue on right • A central vertical wall separating the two halves, present only from partition_y downward • A live entropy counter ( S=… ) displayed in red in the top-left corner • A scroll-progress fill bar on the left edge
T
• Scrolling downward increases scroll_progress , raising partition_y and shortening the wall from the top, allowing particles to mix • Scrolling upward decreases scroll_progress , lowering partition_y and re-blocking particle passage
C
As the wall is removed ( scroll_progress→1 ), particles irreversibly mix—entropy S increases monotonically. The constraint ΔS≥0 maps directly to the unidirectional scroll interaction, visualizing time’s arrow.
• A 3D polygon mesh of the Möbius strip rendered using the Painter’s Algorithm • Faces colored with a teal-to-sky-blue gradient mapped to the u parameter • Depth shading simulating a diffuse light source • Numeric overlays showing the current rotX and rotY viewing angles
T
• Clicking and dragging horizontally on the canvas updates rotY , rotating the strip around the vertical axis • Clicking and dragging vertically updates rotX , tilting the strip forward or backward • Releasing the mouse locks the current orientation
C
No matter how the strip is rotated, a continuous path along its surface always returns to the starting point in a mirrored orientation, demonstrating the strip’s single-sidedness.
Appendix
Table 15
S
orbital_state : segmented-button [ 1s , 2p , 3d ], default 1s ; wavefunction : derived ψ(r,θ)=radial_part(n,l)×angular_part(l,m) ; density_cloud : derived Monte Carlo sampling, accept point (x,y) with probability ∝∣ψ(x,y)∣2
R
• A 2D canvas with coordinate axes and a nucleus dot at the origin • Points progressively sampled and plotted, accumulating into a probability density cloud • A wavefunction equation display that updates to reflect the active orbital state • A segmented button control with options 1s , 2p , and 3d
T
• Clicking a segment button sets orbital_state to the selected orbital • Switching state clears all existing sample points and triggers a new Monte Carlo sampling run
C
Each orbital state produces a distinct, characteristic spatial density pattern: 1s is spherically symmetric, 2p is a dumbbell shape with a nodal plane, and 3d forms a four-lobed clover pattern—matching theoretical predictions.
Appendix
Table 16
S
time : playback [0,∞] , default 0; is_playing : toggle, default true; n_harmonics : slider [1,15] (step 2), default 5; wave : derived last 500 y -values of the epicycle tip
R
• A chain of rotating circles (epicycles), each at a frequency proportional to its harmonic index • A dot tracing the tip of the outermost epicycle • A reconstructed square wave drawn on the right by recording the tip’s y -position over time • A dashed line connecting the epicycle tip to the leading edge of the wave • A Play/Pause button and a harmonic count slider
T
• Pressing Play/Pause starts or freezes the rotation of all epicycles • Dragging the harmonic slider adds or removes outer epicycles in odd increments, immediately reshaping the output wave
C
As n_harmonics increases toward infinity, the reconstructed wave converges to a perfect square wave, demonstrating Fourier’s theorem that any periodic signal decomposes into sinusoids.
Interactive articles help readers engage with complex ideas through exploration, yet creating them remains costly, requiring both domain expertise and web development skills. Recent LLM-based agents can automate content creation, but naively applying them yields uncontrollable and unverifiable outputs. We present ViviDoc, a human-agent collaborative system that generates interactive educational documents from a single topic input. ViviDoc introduces a multi-agent pipeline (Planner, Executor, Evaluator) and the Document Specification (DocSpec), a human-readable intermediate representation that decomposes each interactive visualization into State, Render, Transition, and Constraint components. The DocSpec enables educators to review and refine generation plans before code is produced, bridging the gap between pedagogical intent and executable output. Expert evaluation and a user study show that ViviDoc substantially outperforms naive agentic generation and provides an intuitive editing experience. Our project homepage is available at https://vividoc-homepage.vercel.app/.
Yinghao Tang, Yupeng Xie, Yingchaojie Feng +2
State Key Lab of CAD&CG, Zhejiang University · HKUST(GZ) · National University of Singapore +1
Recent advances in visual language models have enabled autonomous agents for complex reasoning, tool use, and document understanding. However, existing document agents mainly transform papers into static artifacts such as summaries, webpages, or slides, which are insufficient for technical papers involving dynamic mechanisms and state transitions. In this work, we propose a Paper-to-Interactive-System Agent that converts research papers into executable interactive web systems. Given a PDF paper, the agent performs end-to-end processing without human intervention, including paper understanding, system modeling, and interactive webpage synthesis, enabling users to manipulate inputs and observe dynamic behaviors. To evaluate this task, we introduce a benchmark of 19 research papers paired with expert-built interactive systems as ground truth. We further propose PaperVoyager, a structured generation framework that explicitly models mechanisms and interaction logic during synthesis. Experiments show that PaperVoyager significantly improves the quality of generated interactive systems, offering a new paradigm for interactive scientific paper understanding.
Dasen Dai, Biao Wu, Meng Fang +2
1Vast Intelligence Lab · 2UTS · University of Liverpool
Recent advances in visual language models have enabled autonomous agents for complex reasoning, tool use, and document understanding. However, existing document agents mainly transform papers into static artifacts such as summaries, webpages, or slides, which are insufficient for technical papers involving dynamic mechanisms and state transitions. In this work, we propose a Paper-to-Interactive-System Agent that converts research papers into executable interactive web systems. Given a PDF paper, the agent performs end-to-end processing without human intervention, including paper understanding, system modeling, and interactive webpage synthesis, enabling users to manipulate inputs and observe dynamic behaviors. To evaluate this task, we introduce a benchmark of 19 research papers paired with expert-built interactive systems as ground truth. We further propose PaperVoyager, a structured generation framework that explicitly models mechanisms and interaction logic during synthesis. Experiments show that PaperVoyager significantly improves the quality of generated interactive systems, offering a new paradigm for interactive scientific paper understanding.
Dasen Dai, Biao Wu, Meng Fang +1
Vast Intelligence Lab · UTS · University of Liverpool