From Video Clips to Creation Trajectory: Sora100K for AI-Native Video Creation

Sicong Yang; Ruihuan Yang; Jian Lu; Jianfei Yuan; Xiaodong Cun; Xiuli Bi

cs.CVOct 8, 2026

From Video Clips to Creation Trajectory: Sora100K for AI-Native Video Creation

Authors: Sicong Yang, Ruihuan Yang, Jian Lu, Jianfei Yuan, Xiaodong Cun, Xiuli Bi

Organizations: Chongqing University of Posts and Telecommunications, Chongqing, China · GVC Lab, Great Bay University, Dongguan, China

Abstract

AI-Native video creation is shifting from isolated video clips toward iterative video creation workflows. However, existing datasets remain largely video clips, representing video generation and editing as separate tasks rather than connected stages of a video creation workflow. In this paper, we introduce Sora100K, a dataset that represents the AI-Native video creation workflow as a structured video creation trajectory. Specifically, we first identify video creation trajectories and decompose them into three subsets according to their structural roles: text-to-video generation records as roots, single-turn video editing records as editing edges, and multi-turn video editing records as complete trajectories. Then, we use a VLM to assign semantic annotations for generation roots and editing-operation annotations for editing edges. A strict construction pipeline further reconstructs source-to-edit lineage, editing order, and intermediate video states while ensuring data quality. Finally, we perform lightweight adaptation on LTX-2 models to assess the supervision value of Sora100K. The results show improvements in visual quality, multi-shot generation, and cross-shot consistency, while successive-turn evaluation reveals that following multi-turn editing instructions remains challenging. Sora100K establishes a new data foundation for AI-Native video creation beyond isolated video clips and toward structured video creation trajectory. The dataset and supplementary materials are publicly available at https://huggingface.co/datasets/ysicong/Sora100K.

Figures & tables

	Structure			Modality
Dataset	Root	Edge	Trajectory	Multi-shot	Audio

ActivityNet (CVPR’15)	$\checkmark$	$\times$	$\times$	$\times$	$\checkmark$
MSR-VTT (CVPR’16)	$\checkmark$	$\times$	$\times$	$\times$	$\checkmark$
WebVid-10M (ICCV’21)	$\checkmark$	$\times$	$\times$	$\times$	$\times$
HD-VILA-100M (CVPR’22)	$\checkmark$	$\times$	$\times$	$\checkmark$	$\times$

Table 1: Structural and modality comparison of representative video datasets. Root and Edge denote text-to-video generation records and single-turn editing records, respectively, while Trajectory indicates whether they are connected through editing order and intermediate video states. Multi-shot and Audio specify the presence of multi-scene transitions and aligned audio tracks.

Figure 2: Comparison between representative video datasets and Sora100K. Prior datasets typically store video generation and editing as separate clip-level samples. Sora100K connects text-to-video generation roots, single-turn editing edges, and multi-turn editing trajectories, while preserving multi-shot structure and audio-visual synchronization.

Figure 3: Sample video creation trajectories from Sora100K. In each row, the initial text-to-video generation record corresponds to the root. Each pair of adjacent video states and its editing instruction forms a single-turn editing edge, while the complete ordered sequence forms a video creation trajectory.

Panel of Figure 4 — Figure 4: Statistical analysis of Sora100K, including overall task proportions, video-level statistics, editing category distribution, generation category distribution, and prompt length distribution.

Figure 5: Qualitative comparison on trajectory-level video editing. The Sora100K-adapted pipeline produces higher aesthetic quality while maintaining comparable subject consistency across successive editing turns.

Turn	Model	$S_{\mathrm{eff}}\uparrow$	$S_{\mathrm{Sub}}\uparrow$	$S_{\mathrm{Aes}}\uparrow$	Avg. $\uparrow$
Edit 1	LTX-2	52.0	97.5	41.5	63.7
Edit 1	Ours	54.0	97.0	45.4	65.5
Edit 2	LTX-2	34.0	97.0	41.5	57.5
Edit 2	Ours	32.0	97.7	48.4	59.4
Average	LTX-2	43.0	97.3	41.5	60.6
Average	Ours	43.0	97.4	46.9	62.4

Table 2: Multi-turn video editing results over two successive editing turns.

S_{\mathrm{eff}}

is a reference-based VLM score using the ground-truth edited video, while

S_{\mathrm{Sub}}

and

S_{\mathrm{Aes}}

are computed with VBench.

Figure 6: Qualitative comparison between LTX-2 ( HaCohen et al. 2026 ) and our model on video generation. Our model yields more consistent subjects, and fewer structural artifacts across shots.

Model	Subject Consistency $\uparrow$	Background Consistency $\uparrow$	Aesthetic Quality $\uparrow$	Imaging Quality $\uparrow$	Motion Smoothness $\uparrow$	ArcFace $\uparrow$	Avg. $\uparrow$
LTX-2(Base)	86.07	92.07	59.11	47.84	99.60	48.30	72.17
Panda-70M	86.73	91.23	49.00	46.66	99.63	49.23	70.41
Sora100K	86.90	92.24	62.98	57.22	99.56	59.70	76.43

Table 3: Quantitative comparison of the base LTX-2 model and its variants fine-tuned on 3,000 samples from Panda-70M and Sora100K, respectively. All fine-tuned variants use the same training configuration. Bold indicates the best result.

Figure 7: Qualitative comparison between LTX-2 ( HaCohen et al. 2026 ) and our model on video editing. Our model achieves better aesthetic quality, editing effectiveness and stronger instruction following.

Model	$S_{\mathrm{inst}}\uparrow$	$S_{\mathrm{eff}}\uparrow$	$S_{\mathrm{Sub}}\uparrow$	$S_{\mathrm{Aes}}\uparrow$	Avg. $\uparrow$
LTX-2	43.4	38.7	92.0	49.8	56.0
Ours	47.9	44.2	91.7	53.8	59.4

Table 4: Single-turn video editing results. The average is the arithmetic mean of the four reported metrics.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Representative text-to-video samples from Panda-70M, Koala-36M, and Sora100K, illustrating their differences in shot organization and visual content.

Figure 9: Representative video editing samples from OpenVE-3M and Sora100K. The source and edited videos illustrate different types of content and style modifications.

Figure 10: Representative samples from the Text-to-Video Generation subset across eight semantic categories: People, Animals, Architecture, Food & Drink, Plants, Scenery, Science & Technology, and Transportation.

Figure 11: Representative samples from the Text-to-Video Generation subset across eight semantic categories: People, Animals, Architecture, Food & Drink, Plants, Scenery, Science & Technology, and Transportation.

Figure 12: Representative samples from the Single-Turn Video Editing subset across four editing categories. Each example shows the source video on the left and the edited result on the right, covering object-level, style-level, scene-level, and narrative and temporal editing operations.

Figure 13: Representative multi-turn video editing trajectory.

Field	Type	Description	Example
Text-to-Video Generation
sample_id	string	Unique sample identifier	s_68dc...5177
prompt	string	Original generation prompt	Three guys getting lunch …
description_qwen	string	Qwen2.5-VL-generated description	Three young men walk along a busy city street on a sunny day.
duration/frame/scene	tuple	Duration, frame count, and shot count	10/300/1
category	string	Generation category	Real_People

Appendix

Table 5: Field definitions and representative examples for the three Sora100K subsets.

Configuration	Text-to-Video Generation	Single-Turn Video Editing
Base model	LTX-2	LTX-2-T2AV-IC
Training subset	Text-to-Video Generation	Single-Turn Video Editing
Training samples	3,000	3,000
Input	Video and audio	Source video, target video, and audio
Resolution	$512\times 768$	$512\times 768$
Number of frames	121	81

Appendix

Table 6: Training configurations for LoRA adaptation on the Text-to-Video Generation and Single-Turn Video Editing subsets.

Figure 14: Qualitative comparison between the Sora100K-adapted model and LTX-2 on text-to-video generation. Each example presents sampled frames from the two models under the same prompt, illustrating their differences in visual quality, subject preservation, and consistency across shots. Red boxes highlight representative subject-consistency differences.

Figure 15: Qualitative comparison between the Sora100K-adapted model and LTX-2 on single-turn video editing. The examples cover object removal and merging, lighting-effect addition, and watermark insertion.

Figure 16: Qualitative comparison of multi-turn video editing between LTX-2 and our model. The models sequentially modify the source video according to four editing instructions. Our model better preserves character appearance and scene structure while accumulating changes across turns.

Figure 17: Qualitative comparison of multi-turn video editing between LTX-2 and our model. Starting from the same source video, the models sequentially apply a high-motion anime style and add a halo to the character. Our model better preserves the visual content and previous editing effects across successive turns.

Explore similar work

Sep 22, 2026cs.AI

VideoX-Qwen: Data-Centric Instruction-Based Video Editing

Progress in general-purpose video editing depends on constructing large-scale paired supervision and effectively adapting video-generation backbones to instruction-driven editing. Unlike video generation, video editing must execute a requested transformation while preserving unrelated subjects, scene structure, motion, and temporal continuity. We present VideoX-Qwen, an integrated data-construction and model-training framework for general instruction-based video editing. Our scalable production pipeline organizes specialized generation and understanding models into complementary routes for addition, removal, replacement, and attribute editing, followed by quality screening and instruction enrichment. It produces more than 1.2 million directional video-editing records, including over 400,000 records in each major task group, with an automatic acceptance rate of 89%. The resulting corpus provides broad and structured coverage of common editing operations through a unified source-instruction-target interface. We further develop a unified Qwen-Wan editor that combines multimodal semantic conditioning with dense source-video latent guidance. A progressive image-video training strategy aligns the multimodal instruction interface, adapts the video generator to source-conditioned editing, and refines output quality with selected high-resolution data. In a 100-example comparison with UniVideo and Kling O1, VideoX-Qwen achieves the best mean result on nine of eleven reported metrics, including instruction following, editing quality, content preservation, structural and perceptual similarity, and video-distribution quality. Together, the large-scale data-production system and unified training framework provide a practical foundation for more capable instruction-driven video editing.

JJiahang Li, Dingbao Shao, Xinyu Chen +8

School of Intelligence Science and Technology, Nanjing University · Jiutian Research

Video Editing Instruction-Based Video Editing

Ask PDF

Jun 2, 2026cs.CV

JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation

While instruction-based video editing has seen significant progress, joint audio-visual editing remains constrained by the absence of dedicated datasets and benchmarks. To bridge this gap, we present JAVEdit-100k, the first large-scale, high-quality dataset tailored for instruction-guided joint audio-visual editing. Focusing on human-centric videos, JAVEdit-100k comprises approximately 100K editing triplets spanning five distinct categories, including subject editing and speech editing. This dataset is rigorously constructed via four meticulously designed generation pipelines, seamlessly paired with an agent-in-the-loop quality control mechanism. Furthermore, to address the lack of standardized evaluation within the field, we introduce JAVEditBench, a comprehensive benchmark featuring curated source videos and human-aligned instructions across all editing categories. Finally, we propose JAVEdit, a pioneering baseline model for instruction-guided joint audio-visual editing. Experiments show that \model\ outperforms all baselines on five of six evaluation metrics.

Yinan Chen, Chuming Lin, Zhennan Chen +12

1Zhejiang University · 2Tencent Youtu Lab · 3Nanjing University +3

Video Editing Benchmarks Training Data Curation Video Editing Instruction-Based Video Editing

Ask PDF

Jun 17, 2026cs.CV

Bridging Creative Intent and Visual Quality: Creator-Driven Recurrent Video Generation with Agentic Feedback Loops

Generative AI has made content creation increasingly accessible, but many AI-generated videos lack narrative coherence and creative direction, issues that become more substantial at longer durations. Unlike coding, where AI generation benefits from reliable feedback and techniques such as recurrent self-improvement, video generation requires subjective feedback about plot, scenes, and narrative, which naturally motivates approaches that incorporate human creative direction. We introduce CHIEF, a human-AI co-creation video generation framework that places the creator at the center of human-in-the-loop iterative video refinement, and supports them by providing automatic subjective feedback. The creator incorporates their creative direction by driving each iteration, while their revisions are incorporated by a specialized refiner agent. The feedback loop is generated by persona-conditioned multimodal LLMs that watch generated videos and produce subjective critique from the audience perspectives, providing feedback that self-evaluation alone cannot capture. To test the effectiveness of our proposed framework, we work with high school and college students with no prior filmmaking experience to create videos, from short 1-minute videos to a complete short 10-minute film with a complicated plot.

Denis Savytski, Aiden Lei, Heding Liu +4

Department of Computer Science, University of California, Davis, USA · The Harker School · Basis Independent Silicon Valley +1

Long Video Generation Human-in-the-Loop AI Video Generation Human-AI Co-Creation

Ask PDF