Frequency-Synchronized Boundary Coupling for Training-Free Multi-Prompt Long Video Generation
Organizations: Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University · Naval Medical Center of PLA, Second Military Medical University · Shanghai Institute of Virology, Shanghai Jiao Tong University · College of Electronic and Information Engineering, Tongji University
Abstract
Text-to-video generation has made remarkable progress with diffusion models and transformer-based video architectures, yet generating coherent long videos from evolving textual descriptions remains challenging. Training long-video models from scratch requires substantial computation and large-scale multi-event data, while training-free multi-prompt generation must activate new prompt semantics without breaking temporal continuity. We identify a frequency-agnostic boundary coupling problem in existing transition strategies: weak coupling causes abrupt changes, flicker, and background jumps, whereas overly strong or coarse coupling may over-preserve source semantics and produce ghosting or duplicated subjects around prompt boundaries. To address this problem, we propose FreqSync, a training-free framework that formulates prompt transitions as frequency-selective boundary coupling. FreqSync combines Frequency-Synchronized Source-Conditioned Attention (FS-SCA) with Residual-Clipped Spectral Stitching (RCSS), enabling adjacent segments to share stable low- and mid-frequency structure and motion cues while keeping high-frequency prompt-specific details target-dominant under explicit residual constraints. Extensive experiments show that FreqSync achieves state-of-the-art transition smoothness and prompt alignment while maintaining strong perceptual video quality, demonstrating its effectiveness for coherent multi-prompt long video generation.
Figures & tables
| Method | CSCV (%) | Motion Smooth. (%) | Text-Image Similarity | Visual Quality | T2V Align. | Physical Consist. |
| MEVG [ 33 ] | 67.22 | 95.66 | 29.56 | 1.970 | 2.773 | 2.280 |
| Video-Infinity [ 42 ] | 83.62 | 96.45 | 29.72 | 3.247 | 3.422 | 3.210 |
| DiTCtrl [ 7 ] | 86.11 | 98.44 | 30.33 | 3.147 | 3.758 | 3.228 |
| FreqSync (Ours) | 86.43 | 98.25 | 30.53 | 3.214 | 3.849 | 3.240 |
| Variant | FS-SCA | RCSS | CSCV (%) | Text-Image Sim. |
| Isolated | 76.96 | 30.89 | ||
| w/o FS-SCA (RCSS only) | 85.73 | 30.62 | ||
| w/o RCSS (FS-SCA only) | 82.08 | 30.56 | ||
| Full FreqSync | 89.14 | 30.71 |
| Method | Target semantic correctness | Source leakage and artifacts | Transition smoothness | Visual quality | Overall preference |
| MEVG [ 33 ] | 1.67 | 1.53 | 1.70 | 1.83 | 1.63 |
| Video-Infinity [ 42 ] | 2.70 | 2.57 | 2.83 | 2.97 | 2.70 |
| DiTCtrl [ 7 ] | 3.33 | 3.07 | 2.80 | 2.90 | 3.17 |
| FreqSync (Ours) | 4.30 | 4.23 | 4.27 | 4.07 | 4.30 |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Module | Hyperparameter | Value | Description |
| Basic inference | latent frames per segment | 13 | Number of latent frames generated for each segment. |
| Basic inference | sampling FPS | 16 | Frame rate used when saving the generated video. |
| Basic inference | overlap size | 9 | Number of shared latent frames between adjacent segments. |
| Basic inference | transition blocks | 2 | Number of transition segments inserted between neighboring prompts. |
| Basic inference | seed | 42 | Default seed used by the example configuration. |
| FS-SCA | active attention layers | Transformer layers where source-conditioned attention is applied. |
| Method | Seed | CSCV (%) | Motion Smooth. (%) | Text-Image Similarity | Visual Quality | T2V Align. | Physical Consist. |
| MEVG [ 33 ] | 42 | 68.09 | 95.54 | 29.86 | 1.957 | 2.778 | 2.261 |
| MEVG [ 33 ] | 123 | 67.23 | 96.04 | 29.29 | 2.008 | 2.868 | 2.306 |
| MEVG [ 33 ] | 999 | 66.34 | 95.39 | 29.52 | 1.946 | 2.674 | 2.272 |
| MEVG [ 33 ] | Average | 67.22 | 95.66 | 29.56 | 1.970 | 2.773 | 2.280 |
| Video-Infinity [ 42 ] | 42 | 83.72 | 96.53 | 29.92 | 3.255 | 3.511 | 3.200 |
| Video-Infinity [ 42 ] | 123 | 83.63 | 96.38 | 29.75 | 3.210 | 3.347 | 3.162 |
| Metric | FreqSync | DiTCtrl | Difference | 95% CI | Holm |
| CSCV (%) | 87.5133 | 85.7334 | +1.7799 | [0.8623, 2.7845] | 0.00124 |
| Text–Image Similarity | 30.5638 | 30.3236 | +0.2402 | [0.0244, 0.4653] | 0.04325 |
| Visual Quality | 3.2980 | 3.0855 | +0.2124 | [0.1536, 0.2725] | |
| T2V Alignment | 3.9049 | 3.7341 | +0.1708 | [0.1003, 0.2429] | 0.00012 |
| Physical Consistency | 3.3885 | 3.1571 | +0.2314 | [0.1396, 0.3251] |
| Backbone | Method | CSCV (%) | Motion (%) | Text–Image | VQ | T2V | Physical |
| VideoCrafter2 | DiTCtrl | 81.04 | 97.15 | 34.01 | 3.095 | 3.665 | 3.057 |
| VideoCrafter2 | FreqSync | 86.50 | 99.35 | 34.23 | 2.836 | 3.576 | 2.842 |
| Wan2.1-1.3B | DiTCtrl | 84.05 | 98.25 | 31.91 | 3.267 | 3.900 | 3.404 |
| Wan2.1-1.3B | FreqSync | 87.00 | 98.04 | 33.50 | 3.872 | 4.095 | 3.809 |
| Variant | Source routing | CSCV (%) | Text–Image Similarity | T2V Alignment |
| Full FreqSync | ||||
| Low+Mid-only | ||||
| High-only | ||||
| Reverse Routing | ||||
| Uniform Routing | ||||
| w/o FS-SCA | — |
| Family | Setting | CSCV (%) | Text–Image | VQ | T2V | Physical | Motion (%) |
| Default | low=2, low+mid=3, 8 layers | 88.993 | 34.083 | 3.395 | 3.984 | 3.419 | 98.184 |
| Low cutoff | DC+ non-DC | 89.326 | 34.161 | 3.414 | 3.987 | 3.377 | 98.191 |
| Low cutoff | DC+ non-DC | 88.890 | 34.092 | 3.410 | 3.960 | 3.453 | 98.183 |
| Low+mid cutoff | DC+ , mid collapsed | 88.625 | 34.112 | 3.300 | 3.862 | 3.295 | 98.190 |
| Low+mid cutoff | DC+ | 88.845 | 34.078 | 3.311 | 3.925 | 3.352 | 98.182 |
| Routing | 88.315 | 34.123 | 3.298 | 4.014 | 3.302 | 98.191 |
| Low partition | 10 layers | 8 layers | 6 layers |
| DC+ | 87.514 / 4.000 | 88.476 / 3.799 | 88.739 / 3.846 |
| DC+ | 87.000 / 3.948 | 88.289 / 3.918 | 89.494 / 3.866 |
| DC+ | 87.491 / 4.073 | 88.070 / 3.694 | 88.697 / 3.859 |
| Prompts | Sampling time (s) | Memory (GiB) | Counted TFLOPs | ||||||
| DiTCtrl | FreqSync | DiTCtrl | FreqSync | DiTCtrl | FreqSync | ||||
| 2 | +9.07% | 16.51 | 16.74 | +1.39% | 71,998.39 | 75,379.93 | +4.697% | ||
| 3 | +8.39% | 16.53 | 16.76 | +1.39% | 143,996.77 | 150,759.87 | +4.697% | ||
| 4 | +8.43% | 16.56 | 16.79 | +1.37% | 215,995.16 | 226,139.80 | +4.697% | ||
| 5 | +8.07% | 16.59 | 16.81 | +1.37% | 287,993.55 | 301,519.73 | +4.697% | ||
| Asset | Usage in this work | License / Terms |
| CogVideoX-2B [ 50 ] | Pre-trained video generation backbone | Apache-2.0 License |
| DiTCtrl / MPVBench [ 7 ] | Baseline method and multi-prompt evaluation setting | Apache-2.0 License for released inference code |
| VBench [ 22 ] | Motion Smoothness evaluation | Apache-2.0 License |
| CLIP [ 37 ] | Text-Image Similarity and CSCV-related feature similarity | MIT License |
| VideoScore2 [ 15 ] | Visual Quality, T2V Alignment, and Physical Consistency evaluation | MIT License |
| MEVG [ 33 ] | Baseline comparison | Original paper and official project terms |
Video Generation
Video generation studies models that create and edit moving images from text, images, audio, and structured controls.
See all 397 papers