Unified multimodal models integrating visual understanding and generation face a fundamental challenge: visual generation incurs substantially higher computational costs than understanding, particularly for video. This imbalance motivates us to invert the conventional paradigm: rather than extending understanding-centric MLLMs to support generation, we propose Uni-ViGU, a framework that unifies video generation and understanding by extending a video generator as the foundation. We introduce a unified flow method that performs continuous flow matching for video and discrete flow matching for text within a single process, enabling coherent multimodal generation. We further propose a modality-driven MoE-based framework that augments Transformer blocks with lightweight layers for text generation while preserving generative priors. To repurpose generation knowledge for understanding, we design a bidirectional training mechanism with two stages: Knowledge Recall reconstructs input prompts to leverage learned text-video correspondences, while Capability Refinement fine-tunes on detailed captions to establish discriminative shared representations. Experiments demonstrate that Uni-ViGU achieves competitive performance on both video generation and understanding, validating generation-centric architectures as a scalable path toward unified multimodal intelligence. Project Page and Code: https://fr0zencrane.github.io/uni-vigu-page/.
Figures & tables
Figure 1: The DiT architecture of WAN and Uni-ViGU
Figure 2: Overview of Uni-ViGU framework. We formulate unified multimodal generation via a uni-flow process, where video generation is modeled with continuous flow matching in the latent space, while text generation is modeled with discrete flow matching over token embeddings. Both modalities are jointly learned within a single Transformer backbone: self- and cross-attention layers are shared to enable cross-modal interaction and alignment, whereas modality-specific FFN branches are employed to capture domain-specific knowledge for video and text, respectively. This design allows the model to unify generation and understanding while preserving modality-specialized capacity.
Figure 3: Qualitative Results on Video-Text Joint Generation