Visual Autoregressive modeling (VAR) has emerged as a highly efficient alternative to diffusion-based frameworks, achieving comparable synthesis quality. However, as this paradigm extends to Spacetime Autoregressive modeling (STAR) for video generation, scaling resolution and frame counts leads to a "token explosion" that creates a massive computational bottleneck in the final refinement stages. To address this, we propose FastSTAR, a training-free acceleration framework designed for high-quality video generation. Our core method, Spatiotemporal Token Pruning, identifies essential tokens by integrating two specialized terms: (1) Spatial similarity, which evaluates structural convergence across hierarchical scales to skip computations in regions where further refinement becomes redundant, and (2) Temporal similarity, which identifies active motion trajectories by assessing feature-level variations relative to the preceding clip. Combined with a Partial Update mechanism, FastSTAR ensures that only non-converged regions are refined, maintaining fluid motion while bypassing redundant computations. Experimental results on InfinityStar demonstrate that FastSTAR achieves up to a 2.01x speedup with a PSNR of 28.29 and less than 1% performance degradation, proving a superior efficiency-quality trade-off for STAR-based video synthesis.
Figures & tables
Figure 1: FastSTAR achieves a 2.01 × end-to-end speedup on a single H100 GPU for T2V and I2V tasks, maintaining high fidelity with PSNR of 28.29 and 25.65, respectively.
Figure 2: (a) Latency breakdown for generating a 720p video (5s, 81 frames), comparing InfinityStar with FastSTAR. (b) Latency vs. PSNR trade-off curves for T2V synthesis. FastSTAR establishes the Pareto frontier, consistently surpassing existing baselines.
Figure 3: (a) Overview of the FastSTAR framework. (b) Overall Mechanism of FastSTAR; Spatiotemporal Token Pruning identifies converged regions to completely skip transformer blocks, while Partial Update preserves structural integrity.
Figure 4: (a) Scale-wise Fourier-based spectral analysis demonstrates early low-frequency convergence alongside sustained high-frequency updates. (b) High-frequency energy maps show intense localization. Comparing high-intensity locations with large incremental updates reveals significant opportunities for token pruning in converged regions.
Figure 5: Comparison of token pruning and merging. (a) Token merging at 40% obliterates high-frequency textures, while pruning preserves fine details even at high compression. (b) Merging exhibits higher average MSE and expanding variance across resolution scales, whereas pruning maintains tighter error bounds.
Figure 6: Overview of Spatiotemporal Token Pruning (STTP) and Partial Update (PU). (a) Spatial similarity and temporal similarity are fused into a unified spatiotemporal score to generate a per-clip pruning mask. (b) Only high-priority tokens are passed through the transformer and quantize at each scale, while unselected regions are filled with zeros via Partial Update to preserve the integrity of the cumulative feature map.
Figure 7: Visualization of similarity maps across scales and frames. Warmer colors ( red ) indicate low similarity , corresponding to dynamic or high-frequency regions requiring update, while cooler colors ( blue ) indicate high similarity in converged, static regions.
Table 1: Quantitative and efficiency benchmarking for 720p, 5s (81 frames) Text-to-Video generation. We compare FastSTAR with baselines across quality metrics and inference latency; best results are shown in bold and second-best results are underlined; † denotes results using prompt rewriting.
Figure 8: Qualitative results. (a) Pruning masks across scales and resulting outputs. (b) Frame-level results at the last scale with spatiotemporal similarity heatmaps and pruning masks. Black denotes pruned regions and white denotes computed regions.
Table 2: Comparison with state-of-the-art token reduction methods on 720p, 5s (81 frames) Image-to-Video generation; best results are shown in bold and second-best results are underlined.
Figure 10: Ablation studies of FastSTAR. (a) Effect of pruning ratios on runtime and PSNR. (b) Sensitivity to norm degree p in score fusion. (c) Comparison of pruning methods. Best results are shown in bold and second-best results are underlined.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1 : Visualization of attention score maps and their corresponding important tokens.
Figure A.2 : Latency breakdown and overhead analysis across different resolution scales.
Figure A.3 : 3D landscapes of the spatiotemporal score fusion function, illustrating the fusion geometries for (a) p=1 (Addition), (b) p=2 (Euclidean), and (c) p=∞ (Maximum).
Figure A.4 : Log-scale spatiotemporal score distribution analysis of fusion functions across resolution scales.
Table A.2 : Quantitative and efficiency benchmarking for 720p, 5s (81 frames) Text-to-Video generation. Best results are highlighted in bold and second-best results are underlined; † denotes results using prompt rewriting.
Figure A.5 : Additional Side-by-Side comparison of FastSTAR with ToMe, FastVAR.
Figure A.6 : Text-to-Video generation results at 720p resolution.
Figure A.7 : Image-to-Video generation results at 720p resolution.
Figure A.8 : Qualitative examples across multiple 480p generation tasks.
School of Artificial Intelligence, Beijing Normal University, Beijing, China · Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai, China · Shenzhen University of Advanced Technology, Shenzhen, China