cs.CVOct 27, 2026

FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation

Authors: Yijing Lin, Zhendong Mao, Mengqi Huang, Yuxin Wang, Yongdong Zhang

Organizations: University of Science and Technology of China

Abstract

Unified conditional image generation requires a single model to preserve different aspects of a visual input while producing a target image that follows textual guidance. Pretrained video models provide a promising foundation by encoding visual transformations across time, yet existing approaches typically assign the condition and target to fixed temporal positions. We observe that condition-target relationships vary substantially across generation tasks, while video models naturally encode visual transformations at different temporal scales. This motivates us to model conditional generation through input-adaptive virtual temporal relations. We propose FrameForge, which captures two complementary aspects of such relations. Mixture of Temporal Experts (MoTE) models how far the target lies from the condition by dynamically selecting temporal RoPE experts with different virtual intervals. Chain of Temporal Reasoning (CoTR) models how the condition is transformed into the target through a latent intermediate state, with a reasoning-bottleneck attention mask encouraging information propagation through this pathway. Experiments on DreamBench, COCO2017, and GEdit-Bench show that FrameForge achieves superior performance in subject-driven generation, canny-to-image generation, and image editing among the compared methods.

Figures & tables

Explore similar work

CardsList
  1. PreCoG: Preview-Edit for Efficient Video Correction in Generation-Time

    Oct 9, 2026Yilai Liu, Wenqi Pei, Zhifeng Lu +3Efficient Video GenerationT2V Generation

  2. Image-to-Video Diffusion: From Foundations to Open Frontiers

    May 17, 2026Xianlong Wang, Wenbo Pan, Shijia Zhou +6Image-to-Video GenerationVideo Diffusion Models

  3. Edit2Interp: Adapting Image Foundation Models from Spatial Editing to Video Frame Interpolation with Few-Shot Learning

    Mar 16, 2026Nasrin Rahimi, Mısra Yavuz, Burak Can Biner +6Video Frame InterpolationVision Foundation Model Adaptation