ViviDoc: Generating Interactive Documents through Human-Agent Collaboration

Yinghao Tang; Yupeng Xie; Yingchaojie Feng; Tingfeng Lan; Jiale Lao; Yue Cheng; Wei Chen

cs.HCMar 30, 2026

ViviDoc: Generating Interactive Documents through Human-Agent Collaboration

Authors: Yinghao Tang, Yupeng Xie, Yingchaojie Feng, Tingfeng Lan, Jiale Lao, Yue Cheng, Wei Chen

Organizations: State Key Lab of CAD&CG, Zhejiang University · HKUST(GZ) · National University of Singapore · University of Virginia · Cornell University

Abstract

Interactive documents help readers engage with complex ideas through dynamic visualization, interactive animations, and exploratory interfaces. However, creating such documents remains costly, as it requires both domain expertise and web development skills. Recent Large Language Model (LLM)-based agents can automate content creation, but directly applying them to interactive document generation often produces outputs that are difficult to control. To address this, we present ViviDoc, to the best of our knowledge the first work to systematically address interactive document generation. ViviDoc introduces a multi-agent pipeline (Planner, Styler, Executor, Evaluator). To make the generation process controllable, we provide three levels of human control: (1) the Document Specification (DocSpec) with SRTC Interaction Specifications (State, Render, Transition, Constraint) for structured planning, (2) a content-aware Style Palette for customizing writing and interaction styles, and (3) chat-based editing for iterative refinement. We also construct ViviBench, a benchmark of 101 topics derived from real-world interactive documents across 11 domains, along with a taxonomy of 8 interaction types and a 4-dimensional automated evaluation framework validated against human ratings (Pearson r > 0.84). Experiments show that ViviDoc achieves the highest content richness and interaction quality in both automated and human evaluation. A 12-person user study confirms that the system is easy to use, provides effective control over the generation process, and produces documents that satisfy users.

Figures & tables

Figure 1. Eight interactive visualization examples generated by ViviDoc , covering all 8 interaction types in our taxonomy, with styles automatically adapted to each topic’s content. (a) Parameter Exploration : sliders adjust flow rate and elevation parameters in real time. (b) Direct Manipulation : drag the object or focal points to update the lens equation live. (c) Inspection : hover to reveal a Voronoi cell and nearest-neighbor envelope. (d) Freeform Construction : click to place neurons and trigger animated signal propagation. (e) Scroll-driven Narrative : scroll to mix two particle gases and watch entropy rise. (f) Spatial Navigation : drag to rotate a 3D Möbius strip freely in space. (g) State Switching : switch quantum orbital states to redraw the electron probability cloud. (h) Temporal Control : play/pause and tune harmonics to build a Fourier series.

Figure 2. The ViviDoc pipeline. Given a topic, the Planner generates a DocSpec consisting of knowledge units with text descriptions and SRTC Interaction Specifications. The Styler analyzes the DocSpec and generates a Style Palette for users to customize writing and interaction styles. The Executor generates the document code guided by the DocSpec and style instructions. The Evaluator checks the output for correctness. Users can intervene at three points: editing the DocSpec, customizing the Style Palette, and refining the document through chat.

Natural language description
“The reader can adjust the radius of a circle using a slider, and as the radius changes, the visualization updates the circle while recalculating and displaying the circumference, diameter, and ratio, allowing the reader to see that the ratio stays roughly the same.”
SRTC Interaction Specification
S	r : slider $[0.5,5]$ , default 1; C : derived $2\pi r$ ; D : derived $2r$ ; ratio : derived $C/D$
R	A circle whose size reflects $r$ ; labels showing $C$ , $D$ , and ratio
T	Dragging the slider changes $r$ ; $C$ , $D$ , ratio update automatically
C	ratio $\approx 3.14159$ regardless of $r$

Table 1. Comparison of a natural language interaction description and its SRTC Interaction Specification for a visualization about

\pi

Figure 3. The ViviDoc user interface. Top: the main view with (A) sidebar for history and new document creation, (B) center panel with topic input and four-stage navigation bar, and (C) AI chat assistant. Bottom (left to right): the Spec stage showing editable knowledge units, the Style stage with writing and interaction style options, and the Doc stage displaying the generated interactive document.

Figure 4. Automated evaluation results for ViviDoc vs. three multi-agent baselines across three backbone LLMs. Content Richness (CR) and Interaction Quality (IQ) are normalized to $[0,1]$ . Interaction Functionality is on a 0–1 scale; Efficiency is measured in characters per second (chars/s).

Backbone	Method	CR $\uparrow$	IQ $\uparrow$	IF $\uparrow$	Eff $\uparrow$
Gemini 3 Flash	Naive Agent	4.00	3.24	0.85	539
Gemini 3 Flash	ViviDoc	5.00	4.58	0.92	505
Mistral Small	Naive Agent	4.44	2.36	0.61	695
Mistral Small	ViviDoc	5.00	3.24	0.65	562
Qwen3.5-35B	Naive Agent	4.11	2.76	0.70	355
Qwen3.5-35B	ViviDoc	4.56	3.57	0.73	178

Table 2. DocSpec effectiveness: ViviDoc vs. Naive Agent across three backbone LLMs. CR: Content Richness (1–5), IQ: Interaction Quality (0–5), IF: Interaction Functionality (0–1), Eff: Efficiency (chars/s). Yellow : best per backbone.

	AutoGen	MetaGPT	CAMEL	ViviDoc
CR $\uparrow$	2.79	2.51	1.60	4.43
ID $\uparrow$	3.55	1.85	1.76	4.33

Table 3. Mean human evaluation scores by method (Gemini 3 Flash backbone), averaged across 12 raters and 9 topics. Yellow : best per metric.

	Item	Mean $\pm$ SD
Usability
Q1	Easy to learn and use	$5.00\pm 0.00$
Q2	Interface intuitive and satisfying	$5.00\pm 0.00$
Controllability
Q3	DocSpec gives meaningful control	$4.50\pm 0.76$
Q4	Style Palette gives meaningful control	$4.17\pm 0.49$

Table 4. User study ratings (5-point Likert scale,

n=12

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Subject Area	# Topics
Algorithms	25
Mathematics	24
Tools & Resources	13
Physics	10
Science	9
Other	6

Appendix

Table 5. Distribution of topics in ViviBench by subject area.

S	sigma : slider $[1,30]$ , default 10; rho : slider $[10,60]$ , default 28; beta : constant 2.667; trajectory : derived numerical integration of $dx/dt=\sigma(y-x),dy/dt=x(\rho-z)-y,dz/dt=xy-\beta z$
R	• A continuously growing 3D phase-space trajectory projected onto a 2D canvas • The trail fades over time to emphasize recent motion • A slow auto-rotation of the view around the vertical axis • Two sliders for $\sigma$ and $\rho$ displayed below the canvas
T	• Adjusting the $\sigma$ slider resets the trajectory and restarts integration from the same initial point • Adjusting the $\rho$ slider resets the trajectory, causing the attractor shape to deform or collapse into a stable orbit
C	For classical values ( $\sigma=10$ , $\rho=28$ , $\beta=8/3$ ), the trajectory never repeats and draws a distinctive butterfly-shape—demonstrating sensitive dependence on initial conditions.

Appendix

Table 10

S	object_x : drag-x $[20,\text{lens\_x}-10]$ ; object_y : drag-y $[0,\text{canvas\_height}]$ ; f : drag-x $[40,300]$ ; u : derived $\text{lens\_x}-\text{object\_x}$ ; v : derived $(u\cdot f)/(u-f)$ ; M : derived $v/u$ ; image_type : derived if $v>0$ : Real Inverted, if $v<0$ : Virtual Upright, if $v=\infty$ : Undefined
R	• A central convex lens whose thickness scales with focal length • An orange draggable object arrow on the left of the lens • Two green draggable focal points $F$ and $F^{\prime}$ on the optical axis • Three principal rays drawn from the object tip through the lens • A colored image arrow on the right (green for real, blue for virtual) • An instrument dashboard showing live values of $f,u,v$ , and $M$ • A status tag indicating image type and orientation
T	• Dragging the orange object arrow horizontally changes $u$ and updates all derived optical values • Dragging the object arrow vertically changes the object height and redraws the ray diagram • Dragging a focal point ( $F$ or $F^{\prime}$ ) changes $f$ , reshaping both the lens and all derived values simultaneously
C	The thin lens equation $1/u+1/v=1/f$ is always satisfied. When $u<f$ , the image distance $v$ becomes negative (virtual image). When $u=f$ , the image forms at infinity.

Appendix

Table 11

S	seeds : 15 points initialized at random positions, moving with slow random velocity; mouse_pos : hover; nearest : derived $\arg\min_{k}d(\text{mouse\_pos},\text{seeds}[k])$ ; min_dist : derived $d(\text{mouse\_pos},\text{seeds}[\text{nearest}])$ ; cell_region : derived all pixels closest to seeds[nearest] within 150px from mouse_pos
R	• An animated canvas of 15 slowly drifting seed points on a dark background • On hover: the hovered Voronoi cell illuminated with a purple radial gradient • On hover: a dashed gold line connecting the cursor to its nearest seed • On hover: a transparent circle of radius min_dist visualizing the nearest-neighbor envelope • The nearest seed highlighted in gold; all others remain purple
T	• Moving the mouse over the canvas updates mouse_pos continuously • Each frame recalculates the nearest seed and updates the illuminated cell, connecting line, and envelope circle in real-time
C	The illuminated region always corresponds exactly to the Voronoi cell of the nearest seed. Every point within the highlighted region is provably closer to that seed than to any other—demonstrating the core definition of Voronoi partitioning.

Appendix

Table 12

S	hidden_nodes : click-to-place list of $\{x,y\}$ , default $[]$ ; weights : derived random initialization; activations : derived forward pass (sigmoid); output_val : derived average of output neuron activations
R	• Three fixed red input nodes ( $x_{1},x_{2},x_{3}$ ) on the left • Two fixed blue output nodes ( $y_{1},y_{2}$ ) on the right • User-placed yellow hidden nodes in the central region • Directed arrows connecting all layers; animate green (input $\to$ hidden) then orange (hidden $\to$ output) during forward pass • Activation values displayed on each node • An output activation progress bar and percentage readout • A ’Send Signal’ button and a ’Clear’ button
T	• Clicking in the central canvas zone places a new hidden neuron and immediately triggers an animated forward pass • Pressing ’Send Signal’ replays the forward pass with newly randomized weights • Pressing ’Clear’ removes all hidden nodes and resets the output bar to 50%
C	Adding more hidden neurons generally introduces more non-linearity. The output activation always falls in $(0,1)$ due to the sigmoid function, regardless of network topology.

Appendix

Table 13

S	scroll_progress : scroll-wheel $[0,1]$ , default 0; partition_y : derived $\text{scroll\_progress}\times\text{canvas\_height}$ ; particles : 150 red + 150 blue bouncing particles; entropy_S : derived fraction of particles that have crossed to the other side times 100
R	• A dark split-canvas with 300 bouncing particles: red on left, blue on right • A central vertical wall separating the two halves, present only from partition_y downward • A live entropy counter ( $S=\dots$ ) displayed in red in the top-left corner • A scroll-progress fill bar on the left edge
T	• Scrolling downward increases scroll_progress , raising partition_y and shortening the wall from the top, allowing particles to mix • Scrolling upward decreases scroll_progress , lowering partition_y and re-blocking particle passage
C	As the wall is removed ( $\text{scroll\_progress}\to 1$ ), particles irreversibly mix—entropy $S$ increases monotonically. The constraint $\Delta S\geq 0$ maps directly to the unidirectional scroll interaction, visualizing time’s arrow.

Appendix

Table 14

S	rotX : drag-y $[-3.14,3.14]$ , default 0.5; rotY : drag-x $[-3.14,3.14]$ , default -0.5; surface : derived parametric mesh $x(u,v)=(R+v\cos(u/2))\cos(u),y(u,v)=v\sin(u/2),z(u,v)=(R+v\cos(u/2))\sin(u)$
R	• A 3D polygon mesh of the Möbius strip rendered using the Painter’s Algorithm • Faces colored with a teal-to-sky-blue gradient mapped to the $u$ parameter • Depth shading simulating a diffuse light source • Numeric overlays showing the current rotX and rotY viewing angles
T	• Clicking and dragging horizontally on the canvas updates rotY , rotating the strip around the vertical axis • Clicking and dragging vertically updates rotX , tilting the strip forward or backward • Releasing the mouse locks the current orientation
C	No matter how the strip is rotated, a continuous path along its surface always returns to the starting point in a mirrored orientation, demonstrating the strip’s single-sidedness.

Appendix

Table 15

S	orbital_state : segmented-button [ 1s , 2p , 3d ], default 1s ; wavefunction : derived $\psi(r,\theta)=\text{radial\_part}(n,l)\times\text{angular\_part}(l,m)$ ; density_cloud : derived Monte Carlo sampling, accept point $(x,y)$ with probability $\propto\|\psi(x,y)\|^{2}$
R	• A 2D canvas with coordinate axes and a nucleus dot at the origin • Points progressively sampled and plotted, accumulating into a probability density cloud • A wavefunction equation display that updates to reflect the active orbital state • A segmented button control with options 1s , 2p , and 3d
T	• Clicking a segment button sets orbital_state to the selected orbital • Switching state clears all existing sample points and triggers a new Monte Carlo sampling run
C	Each orbital state produces a distinct, characteristic spatial density pattern: 1s is spherically symmetric, 2p is a dumbbell shape with a nodal plane, and 3d forms a four-lobed clover pattern—matching theoretical predictions.

Appendix

Table 16

S	time : playback $[0,\infty]$ , default 0; is_playing : toggle, default true; n_harmonics : slider $[1,15]$ (step 2), default 5; wave : derived last 500 $y$ -values of the epicycle tip
R	• A chain of rotating circles (epicycles), each at a frequency proportional to its harmonic index • A dot tracing the tip of the outermost epicycle • A reconstructed square wave drawn on the right by recording the tip’s $y$ -position over time • A dashed line connecting the epicycle tip to the leading edge of the wave • A Play/Pause button and a harmonic count slider
T	• Pressing Play/Pause starts or freezes the rotation of all epicycles • Dragging the harmonic slider adds or removes outer epicycles in odd increments, immediately reshaping the output wave
C	As n_harmonics increases toward infinity, the reconstructed wave converges to a perfect square wave, demonstrating Fourier’s theorem that any periodic signal decomposes into sinusoids.

Appendix

Table 17

Explore similar work

Demonstrating ViviDoc: Generating Interactive Documents through Human-Agent Collaboration
Mar 2, 2026Yinghao Tang, Yupeng Xie, Yingchaojie Feng +2Human-AI Collaboration Generative AI in Education
State Key Lab of CAD&CG, Zhejiang University · HKUST(GZ) · National University of Singapore +1
I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applications
May 30, 2026Dasen Dai, Biao Wu, Meng Fang +2AI for Science Web Application Generation
1Vast Intelligence Lab · 2UTS · University of Liverpool
PaperVoyager : Building Interactive Web with Visual Language Models
Mar 24, 2026Dasen Dai, Biao Wu, Meng Fang +1Web Application Generation Document Understanding
Vast Intelligence Lab · UTS · University of Liverpool

Mar 2, 2026J/K move · Enter open · S save

Demonstrating ViviDoc: Generating Interactive Documents through Human-Agent Collaboration

Yinghao Tang, Yupeng Xie, Yingchaojie Feng, Tingfeng Lan+1

State Key Lab of CAD&CG, Zhejiang University · HKUST(GZ) · National University of Singapore+1

Human-AI Collaboration Generative AI in Education Multi-Agent LLM Systems

Open paper AskPDF arXiv

Interactive articles help readers engage with complex ideas through exploration, yet creating them remains costly, requiring both domain expertise and web development skills. Recent LLM-based agents can automate content creation, but naively applying them yields uncontrollable and unverifiable outputs. We present ViviDoc, a human-agent collaborative system that generates interactive educational documents from a single topic input. ViviDoc introduces a multi-agent pipeline (Planner, Executor, Evaluator) and the Document Specification (DocSpec), a human-readable intermediate representation that decomposes each interactive visualization into State, Render, Transition, and Constraint components. The DocSpec enables educators to review and refine generation plans before code is produced, bridging the gap between pedagogical intent and executable output. Expert evaluation and a user study show that ViviDoc substantially outperforms naive agentic generation and provides an intuitive editing experience. Our project homepage is available at https://vividoc-homepage.vercel.app/.

Abstract

Explore similar work

Demonstrating ViviDoc: Generating Interactive Documents through Human-Agent Collaboration

I-WebGenBench : Evaluating Interactivity in LLM-Generated Scientific Web Applications

PaperVoyager : Building Interactive Web with Visual Language Models