Edit2Interp: Adapting Image Foundation Models from Spatial Editing to Video Frame Interpolation with Few-Shot Learning
Organizations: Codeway AI Research · Dept. of Electrical & Electronics Engineering, Koç University, İstanbul, Türkiye
Abstract
Pre-trained image editing models exhibit strong spatial reasoning and object-aware transformation capabilities acquired from billions of image-text pairs, yet they possess no explicit temporal modeling. This paper demonstrates that these spatial priors can be repurposed to unlock temporal synthesis capabilities through minimal adaptation - without introducing any video-specific architecture or motion estimation modules. We show that a large image editing model (Qwen-Image-Edit), originally designed solely for static instruction-based edits, can be adapted for Video Frame Interpolation (VFI) using only 64-256 training samples via Low-Rank Adaptation (LoRA). Our core contribution is revealing that the model's inherent understanding of "how objects transform" in static scenes contains latent temporal reasoning that can be activated through few-shot fine-tuning. While the baseline model completely fails at producing coherent intermediate frames, our parameter-efficient adaptation successfully unlocks its interpolation capability. Rather than competing with task-specific VFI methods trained from scratch on massive datasets, our work establishes that foundation image editing models possess untapped potential for temporal tasks, offering a data-efficient pathway for video synthesis in resource-constrained scenarios. This bridges the gap between image manipulation and video understanding, suggesting that spatial and temporal reasoning may be more intertwined in foundation models than previously recognized
Figures & tables
| Dataset | Method | PSNR | LPIPS | FID | FloLPIPS | PS |
|---|---|---|---|---|---|---|
| VIMEO90k | ||||||
| Baseline (image edit) | 19.80 | 0.207 | 11.26 | 0.258 | 51.53 | |
| LoRA-r64-N64 | 23.63 | 0.074 | 4.38 | 0.104 | 88.29 | |
| LoRA-r64-N128 | 26.17 | 0.053 | 3.50 | 0.766 | 100.35 | |
| LoRA-r128-N64 | 24.01 | 0.066 | 3.94 | 0.095 | 90.59 | |
| LoRA-r128-N128 | 26.66 | 0.051 | 3.47 | 0.072 | 101.04 | |
Video Frame Interpolation
Video frame interpolation estimates intermediate frames between observed video frames to create smoother playback and reconstruct temporal detail.
See all 26 papers