TileCache: Serving Multi-Round Diffusion Image Editing with Compressed Activation Caches
Organizations: Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, Canada · Department of Computer Science, George Mason University, Fairfax, VA, USA
Abstract
Multi-round diffusion image editing can reduce computation by reusing intermediate activations, but retaining these activations across editing rounds introduces substantial memory and bandwidth overheads. Compression can reduce these overheads, but efficient cache construction is challenging: compressing only after editing requires buffering over 100 GB of activations per request. Moreover, the compressed cache must support selective reuse, as future masked edits require only a subset of the cached activations. It also exposes a scheduling challenge, as cached edits are faster than initial generations, causing naive policies to severely delay either initial generations or editing requests. In this paper, we aim to address both challenges through compression-informed serving. We develop TileCache, a serving system for multi-round diffusion editing. It compresses reusable activations and incorporates a scheduler that balances both edits and initial generation requests. Compared with the state-of-the-art image editing baseline, TileCache sustains over 2x the request arrival rate at 90% SLO attainment. It completes edits 2.4-4.5x and initial generations 1.3-2.2x faster than the baseline, while preserving image quality.
Figures & tables
| CLIP | Deviation from the Diffusers edit (no reuse) | Error added by compression | |||||||||||
| PSNR | SSIM | FID | TileCache vs. FlashPS edit | ||||||||||
| Model | Dataset | Ratio | Diffusers | FlashPS | TileCache | FlashPS | TileCache | FlashPS | TileCache | FlashPS | TileCache | PSNR | SSIM |
| FLUX.2 distilled | MagicBrush | 4.6 | 26.1 | 26.2 | 26.2 | 27.1 | 26.5 | 0.937 | 0.937 | 31.5 | 31.5 | 43.4 | 0.994 |
| PIE-Bench | 4.6 | 26.2 | 26.0 | 26.0 | 21.6 | 21.6 | 0.868 | 0.868 | 60.6 | 60.8 | 38.2 | 0.989 | |
| ImgEdit (5 rounds) | 4.6 | 27.6 | 27.8 | 27.8 | 23.9 | 23.9 | 0.911 | 0.911 | 21.4 | 21.5 | 42.2 | 0.993 | |
| FLUX.2 base | MagicBrush | 25.1 | 26.0 | 26.1 | 26.1 | 33.4 | 33.1 | 0.974 | 0.974 | 9.8 | 9.6 | 46.4 | 0.996 |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| 0.25 requests/s | 0.275 requests/s | ||||
| Delay budget | Goodput (%) | SLO attainment (%) | Fairness | SLO attainment (%) | Fairness |
| 1.5 | -0.1 | 94.6 | 0.91 | 84.4 | 0.69 |
| 2 (default) | 0 | 94.7 | 0.91 | 84.6 | 0.68 |
| 3 | +0.4 | 95.3 | 0.86 | 84.8 | 0.68 |
| 5 | +0.2 | 94.8 | 0.81 | 85.0 | 0.62 |
| (none) | +1.9 | 95.1 | 0.80 | 87.4 | 0.56 |
| Host cache (GB) | 50 | 100 | 200 | 400 | 800 |
| Hit rate (%) | 32 | 58 | 82 | 93 | 96 |
Diffusion Model Serving
Diffusion model serving optimizes the scheduling, execution, and resource use of diffusion models to meet latency and throughput goals.
See all 21 papers