Everyone can write their stories in freeform text format -- it's something we all learn in school. Yet storytelling via video requires one to learn specialized and complicated tools. In this paper, we introduce Doki, a text-native interface for generative video authoring, aligning video creation with the natural process of text writing. In Doki, writing text is the primary interaction: within a single document, users define assets, structure scenes, create shots, refine edits, and add audio. We articulate the design principles of this text-first approach and demonstrate Doki's capabilities through a series of examples. To evaluate its real-world use, we conducted a week-long deployment study with participants of varying expertise in video authoring. This work contributes a fundamental shift in generative video interfaces, demonstrating a powerful and accessible new way to craft visual stories.
Figures & tables
Figure 1. Doki is a text-native video authoring interface that creates generative videos within a single document. (a) Users define reusable assets and styles with mentions and hashtags, (b) view and adjust inline previews, (c) access commands through a slash menu, (d) write paragraphs that compile into video shots, and collaborate with both (e) a conversational AI agent and (f) inline agent.
Figure 2. A comparison of interface paradigms. (a) “Bento Box” style interfaces distribute authoring across multiple, separate representations. (b) Doki’s approach uses a text-native canonical representation where the document serves as the primary interface.
Figure 3. Two basic example workflows in Doki. Alice: (1) define assets and shots with slash commands → (2) write story and generate previews → (3) create video shots; Bob: (a) prompt the sidebar agent for a draft → (b) review the AI-generated draft → (c) refine with inline AI agent.
Figure 4. Creating a shot in Doki. A new shot is inserted inline with a slash command, and the description that follows serves as its prompt. The system first generates a preview image, which can then be turned into a video clip. Users can click to expand them for playback and additional controls.
Figure 5. Writing consecutive shots within a single paragraph. Later shots inherit context from earlier ones, enabling continuity across a sequence. We achieve strong consistency between shots without repetitive context description in the Doki document.
Figure 6. Creating definitions in Doki. Users open the command menu with a / , select a type, and provide a name and description. Optionally they can add a visual definition for even better consistency.
Figure 7. Adding variants to a definition and shot. The user selects \inter Add Variants from the context menu by right-clicking on the image. Three variations will be added inline. The user can then select one variant by moving it to the first position.
Figure 8. Authors can apply definitions globally ( \inter #all \inter = ) or scope them to specific sections via headings. In the example, a scoped definition ( \inter #Cartoon ) transforms subsequent shots under the heading level into a consistent cartoon style.
Figure 9. Three types of inline agent actions. (a) “Enhance” increases the descriptiveness of the selected text. (b) “Create Definition” converts the selection into a reusable definition. (c) “Custom Request” applies an in-context edit based on user instruction.
Figure 10. Doki’s shot generation pipeline. First, the user’s raw prompt and the document context is passed to reference resolution module to create a structured prompt and gather relevant visual reference images from the definitions. Then, the prompt rewriter rewrites and polishes this structured prompt for image and video generation. Doki first produces a static preview image and then uses that image as a the first frame to generate the final video clip.
Figure 11. Example frames from videos created by participants in our diary study. The videos demonstrate the breadth of video types Doki supports. Each row corresponds to a single video, shown left to right through four representative keyframes.
Id
Role
Video Creation Freq.
GenAI Use (Text)
GenAI Use (Image)
GenAI Use (Video)
SUS
1
UX Designer
A few times a year
A few times a year
Monthly
A few times a year
87.5
2
Filmmaker
Daily
Daily
Daily
Daily
77.5
3
Product Designer
Weekly
Rarely or never
Rarely or never
Rarely or never
75.0
4
Software Engineer
A few times a year
Daily
Weekly
Monthly
90.0
5
Animator
Daily
Rarely or never
Rarely or never
Rarely or never
80.0
6
Program Manager
A few times a year
Daily
Monthly
A few times a year
72.5
Table 1. System Usability Scale (SUS) scores and subscales (Usability and Learnability) across participant roles, video creation frequency, and generative AI tool use.
Id
Age
Gender
Role
Video Types
Video Editor Use
1
28
M
UX Designer
Professional films
DaVinci Resolve, GenAI tools
2
32
M
Filmmaker
Social media, Professional films
Adobe Premiere Pro, Final Cut Pro
3
24
F
Product Designer
Social media, Personal story or vlogs
Adobe Premiere Pro, iMovie, CapCut
4
57
M
Software Engineer
Personal Story or vlogs, Music video
Adobe Premiere Pro
5
28
F
Animator
School projects, Work-related content, Professional films
Adobe Premiere Pro
6
37
F
Program Manager
Social media, Personal story or vlogs
CapCut
Table 2. Participants information including demographics, roles, video practices, and generative AI tool usage.
Figure 12. Additional features in Doki: (1) Preview video player shows the generated video with shot segments. (2) Settings panel lets users select AI models, toggle cost/debug options, access tutorials, and manage projects. (3) Cost monitor tracks generation usage and expenses across images and videos in real time. (4) Download menu provides output in video, clip, or JSON formats.
Figure 13. Doki interface usage analytics. Left: Total cost by session duration, with point color indicating regenerations. Cost data are available for 35 of 50 sessions, as some sessions were logged under ongoing projects rather than new ones as instructed. Right: Distribution of per-minute activity rates for images, videos, and regenerations. Images show the highest median rate, followed by videos, with regenerations least frequent.
We present Visko Orbis 1.0, a Live Model for real-time, interactive long video generation. Users can change the prompt at any moment during generation, and the update becomes visible in real time. Visko Orbis 1.0 supports long-form text-to-video, image-to-video, and video continuation, with multilingual prompts and prompt switching while generation is in progress. A bounded multi-scale memory preserves subjects, scenes, and style across chunks, sustaining hour-scale rollouts without evident quality or color drift. The generator is factorized causally in time, matching the causal structure of physical dynamics, and is aligned with a latent world-model reward for predictive consistency. Built on a distilled chunk-wise streaming generator and a streaming video upscaler, Visko Orbis 1.0 delivers 4K video generation at 24 FPS in real time, using an optimized GPU serving engine. In quantitative evaluations, Visko Orbis 1.0 achieves the best DOVER aesthetic and technical scores and the best VideoAlign visual and motion quality, and leads three physical-plausibility protocols (VideoPhy-2, Physics-IQ, and VBench-2.0 Physics); in long-form Arena comparisons, it obtains the highest overall-preference and temporal-stability ratings among all the state-of-the-art real-time interactive video generation systems.
We introduce Vidu S1, a real-time interactive video generation model supporting voice control of digital characters. Users can control video generation content at any moment through voice instructions. Vidu S1 supports infinite-length real-time video generation without blurring, drift, or visual distortion. Built with TurboDiffusion and TurboServe, Vidu S1 outputs 540p real-time videos at up to 42 FPS on regular consumer GPUs. Users can upload custom images of real people, anime, and pets, and choose different voice tones for personalized experiences. Experiments show that Vidu S1 achieves the best performance across all test metrics while fully meeting real-time inference requirements. A playable online demo is available at https://vidu.com/vidu-stream.
Recent advances in video generation let users create high-quality videos from natural language descriptions, yet obtaining results that match user intent remains time-consuming: individual generations are computationally expensive, and underspecified prompts force repeated cycles of generation, inspection, and revision. Existing video prompt optimization methods improve generation through automatic rewriting, but cannot supply the ideas and preferences that were not in the initial description. We propose ProactGen, a video generation framework that learns proactive clarification, eliciting user intent through questions grounded in generated video drafts. Specifically, we train a vision-language model as a questioning policy with reinforcement learning: observing the initial prompt and a video draft, it learns to ask the questions most valuable to downstream generation, rewarded by the alignment between the final generated video and a target reference. We further find that effective questioning does not require drafts of final-output quality, and exploit this property by generating drafts with fewer denoising steps, keeping the added cost of clarification low. Experiments show that ProactGen improves target alignment over direct generation by at least 1.8x the gain of the strongest baselines. Moreover, a single round of ProactGen already exceeds the alignment that the strongest baseline reaches after four refinement rounds, in 29% of its time.
Zhixiao Wang, Yang Chen, Yifan Xu +5
Nanjing University · University of Science and Technology of China · The Hong Kong University of Science and Technology (Guangzhou) +1