Recent zero-shot image compression methods exploit the powerful priors of pretrained diffusion models to produce perceptually pleasing reconstructions without compression-specific training. However, image compression serves not only human viewing but also machine analysis, while existing zero-shot codecs have limited flexibility in meeting the diverse reconstruction requirements of these applications. In this paper, we propose Flex-DDCM, a framework for zero-shot task-oriented image compression via diffusion trajectory guidance. Building on Turbo-DDCM, we adapt the coded diffusion trajectory through task-guided noise selection, embedding the effects of task guidance into the compressed representation. Guidance terms can be applied individually or combined, with timestep-dependent strengths controlling their influence throughout denoising. This design enables task adaptation without introducing additional decoding complexity. Experiments demonstrate flexible trade-offs among pixel fidelity, perceptual quality, and downstream task performance, with competitive performance against trained task-oriented codecs and faster decoding than Diff-ICMH.
Figures & tables
Fig. 1: Overview of Flex-DDCM for task-oriented image compression. Task-guided noise selection during encoding enables adaptation to different task requirements, such as pixel fidelity and object detection performance. This adaptation introduces no additional decoding complexity.
Fig. 2: Task-guided noise selection in Flex-DDCM. Normalized task gradients are weighted and combined with the latent residual to select codebook atoms, whose indices and signs convey task guidance.
Fig. 3: Performance comparison on Kodak, COCO 2017, and Flickr30K. Colored areas represent the adjustable range of Flex-DDCM.
DiffC
PerCo (SD) †
DiffEIC †
Diff-ICMH †
Turbo-DDCM
Flex-DDCM-D
Flex-DDCM-M
Flex-DDCM-D+M
Enc. (s)
3.560
0.365
0.129
0.160
1.068
1.939
1.631
2.537
Dec. (s)
3.210
1.170
3.291
6.466
0.986
0.994
1.032
1.033
TABLE I: Average runtime on 512×512 Kodak images using an NVIDIA RTX 4090. Enc. and Dec. denote encoding and decoding time. † Trained or fine-tuned for compression.
Fig. 4: Effect of guidance strength and temporal scheduling on Kodak distortion and COCO 2017 object detection and instance segmentation.
Variant
Detection (%)
Segmentation (%)
w/o tags
+8.06
+7.32
w/ CFG
+22.37
+23.04
Flex-DDCM-M (Ours)
0.00
0.00
TABLE II: Ablation of tag conditioning and classifier-free guidance (CFG) on COCO 2017. BD-rate is relative to Flex-DDCM-M (lower is better).
Diffusion-based image compression has recently shown outstanding perceptual fidelity, yet its practicality is hindered by prohibitive sampling overhead and high memory usage. Most existing diffusion codecs employ U-Net architectures, where hierarchical downsampling forces diffusion to operate in shallow latent spaces (typically with only 8x spatial downscaling), resulting in excessive computation. In contrast, conventional VAE-based codecs work in much deeper latent domains (16x - 64x downscaled), motivating a key question: Can diffusion operate effectively in such compact latent spaces without compromising reconstruction quality? To address this, we introduce DiT-IC, an Aligned Diffusion Transformer for Image Compression, which replaces the U-Net with a Diffusion Transformer capable of performing diffusion in latent space entirely at 32x downscaled resolution. DiT-IC adapts a pretrained text-to-image multi-step DiT into a single-step reconstruction model through three key alignment mechanisms: (1) a variance-guided reconstruction flow that adapts denoising strength to latent uncertainty for efficient reconstruction; (2) a self-distillation alignment that enforces consistency with encoder-defined latent geometry to enable one-step diffusion; and (3) a latent-conditioned guidance that replaces text prompts with semantically aligned latent conditions, enabling text-free inference. With these designs, DiT-IC achieves state-of-the-art perceptual quality while offering up to 30x faster decoding and drastically lower memory usage than existing diffusion-based codecs. Remarkably, it can reconstruct 2048x2048 images on a 16 GB laptop GPU.
Junqi Shi, Ming Lu, Xingchen Li +3
School of Electronic Science and Engineering, Nanjing University, Nanjing 210023, China
We propose FluxC, a zero-shot codec that uses Flux as a compression prior. Although DiffC has shown that pretrained diffusion models can serve as zero-shot codecs, we ask how the choice of coding path and reconstruction method affects rate--distortion--perception behavior. We analyze FluxC along three axes: coding cost and bit allocation, reconstruction method, and low rate distortion. At matched signal-to-noise ratio (SNR), we show that the path choice does not affect the ideal information rate, but changes how bits are allocated across time steps. For reconstruction, we compare ancestral sampling, one-step ODE reconstruction, and many-step ODE reconstruction. Our analysis shows that ODE-based reconstruction can achieve lower distortion than ancestral sampling, especially in the low rate regime. Experiments on Gaussian sources and natural images support this prediction. Moreover, FluxC performs competitively with, and in some cases outperforms, recent neural image compression methods, suggesting that Flux can serve as a principled zero-shot codec.
With the great success of diffusion models in image generation, diffusion-based image compression is attracting increasing interests. However, due to the random noise introduced in the diffusion learning, they usually produce reconstructions with deviation from the original images, leading to suboptimal compression results. To address this problem, in this paper, we propose a Noise Constrained Diffusion (NC-Diffusion) framework for high fidelity image compression. Unlike existing diffusion-based compression methods that add random Gaussian noise and direct the noise into the image space, the proposed NC-Diffusion formulates the quantization noise originally added in the learned image compression as the noise in the forward process of diffusion. Then a noise constrained diffusion process is constructed from the ground-truth image to the initial compression result generated with quantization noise. The NC-Diffusion overcomes the problem of noise mismatch between compression and diffusion, significantly improving the inference efficiency. In addition, an adaptive frequency-domain filtering module is developed to enhance the skip connections in the U-Net based diffusion architecture, in order to enhance high-frequency details. Moreover, a zero-shot sample-guided enhancement method is designed to further improve the fidelity of the image. Experiments on multiple benchmark datasets demonstrate that our method can achieve the best performance compared with existing methods.
Zhenyu Du, Yanbo Gao, Shuai Li +3
Shandong University, Jinan, China · University of Electronic Science and Technology of China, Sichuan, China