cs.CVApr 7, 2026
SaveCross-Resolution Diffusion Models via Network Pruning
Organizations: University of Electronic Science and Technology of China · Westlake University
Abstract
Diffusion models have demonstrated impressive image synthesis performance, yet many UNet-based models are trained at certain fixed resolutions. Their quality tends to degrade when generating images at out-of-training resolutions. We trace this issue to resolution-dependent parameter behaviors, where weights that function well at the default resolution can become adverse when spatial scales shift, weakening semantic alignment and causing structural instability in the UNet architecture. Based on this analysis, this paper introduces CR-Diff, a novel method that improves the cross-resolution visual consistency by pruning some parameters of the diffusion model. Specifically, CR-Diff has two stages. It first performs block-wise pruning to selectively eliminate adverse weights. Then, a pruned output amplification is conducted to further purify the pruned predictions. Empirically, extensive experiments suggest that CR-Diff can improve perceptual fidelity and semantic coherence across various diffusion backbones and unseen resolutions, while largely preserving the performance at default resolutions. Additionally, CR-Diff supports prompt-specific refinement, enabling quality enhancement on demand.
Figures & tables
Figure 2: Effects of magnitude-based unstructured pruning on SDXL at unseen resolution . (a) Quantitative metrics improvement within moderate sparsity. (b) Qualitative illustration of improved semantic alignment as sparsity increases.
Figure 3: Overview of CR-Diff. Most UNet–based diffusion models exhibit resolution-dependent degradation when generating at unseen scales. CR-Diff addresses this issue through a two-stage pruning and optimizing process, consisting of a block-wise (B-W) pruning ratio strategy and a pruned output amplification (POA) mechanism. As shown in Figure 5 , block-wise pruning adopts a magnitude-based criterion with adaptive ratios across blocks to extract parameter directions that remain stable across resolutions. Pruned output amplification refines the model’s forward predictions by amplifying the pruned output with an amplification coefficient , which suppresses residual dense model’s influences that otherwise introduce artifacts. This leads to cleaner denoising trajectories and higher-quality final images with more stable structure and details.
Figure 4: Simulated annealing (SA) search process for determining the optimal block-wise pruning ratios.
Figure 5: Block-wise (B-W) pruning applies differentiated pruning ratios across blocks to preserve essential structure and provide a pruned backbone for subsequent optimization.
| Model | Resolution | FID | CLIP | ImageReward | PickScore | Aesthetic Score | |||||
| Dense | CR-Diff | Dense | CR-Diff | Dense | CR-Diff | Dense | CR-Diff | Dense | CR-Diff | ||
| SDXL | 83.827 | 37.918 | 0.295 | 0.321 | -0.498 | 0.735 | 20.296 | 22.140 | 4.335 | 5.525 | |
| 146.984 | 36.688 | 0.252 | 0.311 | -1.734 | 0.092 | 18.608 | 21.074 | 3.494 | 4.672 | ||
| 211.369 | 46.040 | 0.225 | 0.307 | -2.148 | -0.099 | 18.060 | 20.956 | 3.806 | 4.644 | ||
| SD1.5 | 39.047 | 39.291 | 0.309 | 0.310 | 0.061 | 0.151 | 21.146 | 21.188 | 4.736 | 4.779 | |
| 39.797 | 37.634 | 0.307 | 0.307 | -0.068 | -0.026 | 20.906 | 20.944 | 4.710 | 4.819 | ||
Table 1: Performance comparison on unseen resolutions. Across the evaluated models and resolutions, CR-Diff improves most metrics relative to the dense model, with bold values indicating superior performance of our CR-Diff against the dense model.
Figure 6: Visual comparison across three generation settings. Dense denotes the original unpruned model. Global CR-Diff applies a pruning ratio optimized for a specific model and resolution, shared across all prompts. Specific CR-Diff further refines this ratio for the given prompt, enabling prompt-specific optimization. Each group corresponds to a specific prompt, and the ImageReward (IR) scores are shown below each image. Global CR-Diff improves generative fidelity in a prompt-agnostic manner, while Specific CR-Diff further enhances semantic alignment and visual coherence for the specific prompt.
| Model | Resolution | FID | CLIP | ImageReward | PickScore | Aesthetic Score | |||||
| Dense | CR-Diff | Dense | CR-Diff | Dense | CR-Diff | Dense | CR-Diff | Dense | CR-Diff | ||
| SDXL | 33.186 | 33.562 | 0.322 | 0.322 | 0.788 | 0.946 | 22.512 | 22.639 | 6.123 | 6.106 | |
| SD1.5 | 38.368 | 37.773 | 0.315 | 0.314 | 0.239 | 0.203 | 21.539 | 21.377 | 5.205 | 5.233 | |
| SD2.1 | 45.583 | 36.792 | 0.308 | 0.309 | -0.100 | -0.052 | 20.943 | 20.960 | 4.728 | 5.082 | |
Table 2: Performance comparison on default resolutions. Across the evaluated models, CR-Diff consistently improves or maintains performance, with bold values indicating gains over the dense model.
| Model | Resolution | FID | CLIP | ImageReward | PickScore | Aesthetic Score | |||||
| Dense | CR-Diff | Dense | CR-Diff | Dense | CR-Diff | Dense | CR-Diff | Dense | CR-Diff | ||
| SD3Medium | 40.453 | 38.901 | 0.317 | 0.317 | 0.972 | 1.038 | 22.187 | 22.121 | 4.886 | 4.884 | |
| SD3Medium | 37.841 | 37.026 | 0.320 | 0.320 | 1.081 | 1.128 | 22.609 | 22.543 | 5.513 | 5.455 | |
| FLUX.dev | 35.799 | 35.708 | 0.311 | 0.312 | 0.945 | 0.935 | 22.793 | 22.775 | 6.295 | 6.263 | |
Table 3: Performance comparison on DiTs. Bold values indicate our CR-Diff is better than the dense model. The results show that CR-Diff preserves or even improves the generation quality, demonstrating the generalizability of our method beyond UNets.
Figure 7: Ablation results of block-wise pruning. (a) Performance comparison under uniform and block-wise pruning strategies across different models and resolutions. For block-wise pruning, the ratios are listed in the order down-sampling / middle / up-sampling . ImageReward (IR) of the better-performing strategy is highlighted in bold , showing that differentiated ratios improve image quality. (b) Conceptual illustration of ImageReward trends during the updating process for the optimal pruning configuration, showing generally higher and more stable values under block-wise pruning compared to uniform pruning.
Figure 8: Radar comparison across pruning strategies on SDXL at . Five metrics are normalized with direction alignment so that larger radial values denote better performance. CR-Diff achieves the strongest overall results, highlighting its superior perceptual and semantic quality.
| Resolution | ImageReward Improvement | ||
| SDXL | SD1.5 | SD2.1 | |
| +0.205 | +0.144 | +0.236 | |
| +0.364 | +0.155 | +0.266 | |
| +0.417 | +0.164 | +0.261 | |
Table 4: Ablation results of the pruned output amplification component. Values in the table denote the performance difference between models with and without POA, where consistent positive values demonstrate the effectiveness of POA.
| Metric | K = 1.5 | K = 2.0 | K = 2.5 |
| Best IR % | 54.31 | 29.89 | 15.80 |
| FID | 37.918 | 43.08 | 61.71 |
| CLIP | 0.321 | 0.305 | 0.290 |
| ImageReward | 0.735 | -0.003 | -0.557 |
| PickScore | 22.140 | 21.14 | 20.34 |
| Aesthetic Score | 5.525 | 5.040 | 4.630 |
Table 5: Ablation study on the amplification coefficient in POA, on SDXL ( ). A moderate amplification ( ) yields the most stable performance gains across evaluation metrics. Best per-metric values are shown in bold .
| Model | Resolution | Ratios (Down/Middle/Up) |
| SDXL | 0.295 / 0.194 / 0.236 | |
| 0.397 / 0.434 / 0.387 | ||
| 0.482 / 0.396 / 0.469 | ||
| 0.434 / 0.428 / 0.355 | ||
| 0.300 / 0.343 / 0.300 | ||
| SD1.5 | 0.433 / 0.345 / 0.300 |
Table 6: Block-wise pruning ratio configurations across different models and resolutions, showing the distinct ratio allocations for the downsampling, middle, and upsampling blocks (* indicates the model’s default resolution).
| Model | Resolution | FID | CLIP | ImageReward | PickScore | Aesthetic Score | |||||
| pruned | CR-Diff | pruned | CR-Diff | pruned | CR-Diff | pruned | CR-Diff | pruned | CR-Diff | ||
| SDXL | * | 33.397 | 33.562 | 0.322 | 0.322 | 0.834 | 0.946 | 22.594 | 22.639 | 6.058 | 6.106 |
| 40.068 | 37.918 | 0.320 | 0.321 | 0.530 | 0.735 | 22.100 | 22.140 | 5.508 | 5.525 | ||
| 43.348 | 36.688 | 0.308 | 0.311 | -0.272 | 0.092 | 20.948 | 21.074 | 4.752 | 4.672 | ||
| 56.182 | 46.040 | 0.301 | 0.307 | -0.516 | -0.099 | 20.636 | 20.956 | 4.472 | 4.644 | ||
| 39.362 | 40.380 | 0.312 | 0.312 | 0.108 | 0.208 | 21.394 | 21.399 | 5.806 | 5.855 | ||
Table 7: Performance comparison of pruned models (the model pruned block-wisely without pruned output amplification) and CR-Diff ( with pruned output amplification) across default and unseen resolutions. Resolutions are reported as height width, where resolutions marked with * denote the native (default) setting of the model. Bold values indicate that CR-Diff outperforms the pruned baseline. These results verify that POA provides a stable refinement effect that generalizes across models and resolutions.
| Model | Resolution | FID | CLIP | ImageReward | PickScore | Aesthetic Score | |||||
| dense | ours | dense | ours | dense | ours | dense | ours | dense | ours | ||
| SDXL | 46.563 | 40.380 | 0.315 | 0.312 | 0.300 | 0.208 | 21.675 | 21.399 | 5.952 | 5.855 | |
Table 8: Evaluation of SDXL at the unseen higher resolution . The results show that the model remains stable at this scale and CR-Diff maintains comparable performance without altering semantic structure or perceptual characteristics.
Figure 9: Additional cross-resolution comparisons between SDXL [ 33 ] and its CR-Diff counterpart. CR-Diff consistently improves cross-resolution image quality compared to the dense model.
Figure 10: Additional cross-resolution comparisons between SDXL [ 33 ] and its CR-Diff counterpart. CR-Diff consistently improves cross-resolution image quality compared to the dense model.
Figure 11: Additional cross-resolution comparison on a subset of 5K prompts from the MS-COCO 2014 validation set [ 27 ] . CR-Diff shows consistent gains in both ImageReward and visual fidelity compared to the original SDXL. Dense denotes the original unpruned model. Each group corresponds to a specific prompt, and the ImageReward (IR) scores are shown below each image.
Figure 12: Additional cross-resolution comparison on a subset of 5K prompts from the MS-COCO 2014 validation set [ 27 ] . CR-Diff shows consistent gains in both ImageReward and visual fidelity compared to the original SD1.5. Dense denotes the original unpruned model. Each group corresponds to a specific prompt, and the ImageReward (IR) scores are shown below each image.
Figure 13: Additional cross-resolution comparison on a subset of 5K prompts from the MS-COCO 2014 validation set [ 27 ] . CR-Diff shows consistent gains in both ImageReward and visual fidelity compared to the original SD2.1. Dense denotes the original unpruned model. Each group corresponds to a specific prompt, and the ImageReward (IR) scores are shown below each image.
Figure 14: Visual comparison across two generation settings. Dense denotes the original unpruned model. Specific CR-Diff adjusts the pruning ratio based on each input prompt, enabling prompt-tailored optimization. Each group corresponds to a specific prompt, and the ImageReward (IR) scores are shown below each image. Both quantitative and qualitative results show that Specific CR-Diff improves semantic alignment and visual coherence for the given prompt.