SkyReels V4 is a unified multi modal video foundation model for joint video audio generation, inpainting, and editing. The model adopts a dual stream Multimodal Diffusion Transformer (MMDiT) architecture, where one branch synthesizes video and the other generates temporally aligned audio, while sharing a powerful text encoder based on the Multimodal Large Language Models (MLLM). SkyReels V4 accepts rich multi modal instructions, including text, images, video clips, masks, and audio references. By combining the MLLMs multi modal instruction following capability with in context learning in the video branch MMDiT, the model can inject fine grained visual guidance under complex conditioning, while the audio branch MMDiT simultaneously leverages audio references to guide sound generation. On the video side, we adopt a channel concatenation formulation that unifies a wide range of inpainting style tasks, such as image to video, video extension, and video editing under a single interface, and naturally extends to vision referenced inpainting and editing via multi modal prompts. SkyReels V4 supports up to 1080p resolution, 32 FPS, and 15 second duration, enabling high fidelity, multi shot, cinema level video generation with synchronized audio. To make such high resolution, long-duration generation computationally feasible, we introduce an efficiency strategy: Joint generation of low resolution full sequences and high-resolution keyframes, followed by dedicated super-resolution and frame interpolation models. To our knowledge, SkyReels V4 is the first video foundation model that simultaneously supports multi-modal input, joint video audio generation, and a unified treatment of generation, inpainting, and editing, while maintaining strong efficiency and quality at cinematic resolutions and durations.
Figures & tables
Figure 1: Overview of the proposed method.
Task
Stage
Resolution
Data Volume
Epochs
Video Pretrain
T2I
Stage 1
256px
3B images
3
T2I + T2V
Stage 2
256px, 16fps, 2-10s
1B images / 400M videos
3
T2I + T2V + Inpaint
Stage 3
256px, 16fps, 2-15s
1B images / 400M videos
2
(Image Inpaint, I2V, V2V, Edit)
(Inpaint: 5% each)
Mixed Tasks
Stage 4
256/480px, 16fps, 2-15s
100M images / 100M videos
2
Table 1: Complete training schedule across all stages. The progressive strategy gradually increases resolution, temporal length, and task complexity.
Figure 2: The pipeline of the video super-resolution and frame interpolation method. F denotes the output latent of our base model. KF demotes the key frames latent of our base model.
Figure 3: Artificial Analysis Video Arena Leaderboards across four generation tracks (as of 2026-03-18). SkyReels V4 ranks #1 in Text-to-Video with Audio, #2 in Text-to-Video without Audio, #4 in Image-to-Video with Audio, and #7 in Image-to-Video without Audio, demonstrating strong and competitive audiovisual generation quality as evaluated by public user preferences.
Dimension
Sub-dimension
Evaluation Criteria
Instruction Following
Video Instruction Following
Subject description
Accurate representation of subjects, attributes, and appearances
Subject interaction
Correct execution of actions, interactions, and motion dynamics
Camera movement
Proper execution of camera operations (pan, tilt, zoom, dolly)
Style and aesthetics
Adherence to visual styles, color palettes, and artistic directions
Multi-shot consistency
Correct shot transitions, cross-shot coherence, and reference accuracy
Table 2: Comprehensive Evaluation Dimensions for Audio-Visual Generation