CADC: Content Adaptive Diffusion-Based Generative Image Compression

Xihua Sheng; Lingyu Zhu; Tianyu Zhang; Dong Liu; Shiqi Wang; Jing Wang

cs.CVFeb 25, 2026

CADC: Content Adaptive Diffusion-Based Generative Image Compression

Authors: Xihua Sheng, Lingyu Zhu, Tianyu Zhang, Dong Liu, Shiqi Wang, Jing Wang

Organizations: City University of Hong Kong · University of Science and Technology of China · Central Media Technology Institute, Huawei

Abstract

Diffusion-based generative image compression has demonstrated remarkable potential for achieving realistic reconstruction at ultra-low bitrates. The key to unlocking this potential lies in making the entire compression process content-adaptive, ensuring that the encoder's representation and the decoder's generative prior are dynamically aligned with the semantic and structural characteristics of the input image. However, existing methods suffer from three critical limitations that prevent effective content adaptation. First, isotropic quantization applies a uniform quantization step, failing to adapt to the spatially varying complexity of image content and creating a misalignment with the diffusion model's noise-dependent prior. Second, the information concentration bottleneck -- arising from the dimensional mismatch between the high-dimensional noisy latent and the diffusion decoder's fixed input -- prevents the model from adaptively preserving essential semantic information in the primary channels. Third, existing textual conditioning strategies either need significant textual bitrate overhead or rely on generic, content-agnostic textual prompts, thereby failing to provide adaptive semantic guidance efficiently. To overcome these limitations, we propose a content-adaptive diffusion-based image codec with three technical innovations: 1) an Uncertainty-Guided Adaptive Quantization method that learns spatial uncertainty maps to adaptively align quantization distortion with content characteristics; 2) an Auxiliary Decoder-Guided Information Concentration method that uses a lightweight auxiliary decoder to enforce content-aware information preservation in the primary latent channels; and 3) a Bitrate-Free Adaptive Textual Conditioning method that derives content-aware textual descriptions from the auxiliary reconstructed image, enabling semantic guidance without bitrate cost.

Figures & tables

Figure 2: On the encoder side, an analysis transform $g_{a}$ encodes the input image $\mathbf{x}$ into a compact latent representation $\mathbf{y}$ . An uncertainty map $\mathbf{m}$ is estimated by $f_{u}$ to guide the quantization of $\mathbf{y}$ . The quantized latent $\mathbf{\hat{y}}$ is encoded into a bitstream via an arithmetic encoder (AE) and transmitted. On the decoder side, a synthesis transform $g_{s}$ upsamples $\mathbf{\hat{y}}$ to produce a noisy latent $\bm{l}_{T}$ at the spatial resolution required by the pre-trained Stable Diffusion VAE decoder $\mathcal{D}_{SD}$ [ 52 ] . In learned codecs, $\bm{l}_{T}$ typically has a high channel count (e.g., 320), while $\mathcal{D}_{SD}$ is fixed to accept only 4-channel input. To resolve this, the entire $\bm{l}_{T}$ is commonly input to the Unet $\epsilon_{SD}$ (a new input channel number is set to the first convolutional layer of the Unet) to utilize all available context for estimating more accurate 4-channel noise (the output channel number of the Unet is still 4) [ 66 ] . The denoising process is applied exclusively to the first four noisy channels $\bm{l}_{T}^{(1:4)}$ , yielding the standard 4-channel clean latent $\bm{l}_{0}$ for $\mathcal{D}_{SD}$ . To concentrate essential semantic information into $\bm{l}_{T}^{(1:4)}$ , a lightweight auxiliary decoder $g_{aux}$ takes $\bm{l}_{T}^{(1:4)}$ as inputs to reconstruct an auxiliary image $\mathbf{\hat{x}}_{aux}$ . To produce a content-adaptive textual description $c_{aux}$ , $\mathbf{\hat{x}}_{aux}$ is captioned by $f_{c}$ (a frozen BLIP [ 36 ] ). $c_{aux}$ is then combined with a fixed description $c_{fix}$ to condition a one-step diffusion denoising process [ 53 ] .

Figure 3: Quantitative comparisons of different generative image codecs on Kodak, DIV2K Val, and CLIC 2020 Test.

Figure 4: Qualitative comparison of different generative image codecs on the Kodak dataset under ultra-low bitrate conditions.

Models	UGAQ	ADGIC	BFATC	LPIPS	DISTS
$M_{0}$	✗	✗	✗	0.00	0.00
$M_{1}$	✓	✗	✗	–3.7	–2.7
$M_{2}$	✓	✓	✗	–5.3	–3.5
$M_{3}$	✓	✓	✓	–6.8	–5.5

Table 1: Ablation studies of our proposed methods on Kodak. Negative BD-rate (%) values indicate better compression performance. The distortion is measured by LPIPS and DISTS.

Figure 5: Analysis of Uncertainty-Guided Adaptive Quantization (UGAQ) and the isotropic quantization on the DIV2K dataset.

Figure 6: Energy comparison of the first four channels of the noisy latents of the codecs with and without Auxiliary Decoder-Guided Information Concentration (ADGIC) on the Kodak dataset.

Figure 7: Illustration of the textual descriptions extracted from auxiliary reconstructed images under different bitrate conditions.

Figure 8: Network structures of the main modules, including the main analysis transform

g_{a}

, main synthesis transform

g_{s}

, hyper analysis transform

h_{a}

, hyper synthesis transform

h_{s}

, auxiliary decoder

g_{aux}

, uncertainty estimation network

f_{u}

, and the context model. For efficient network construction, we primarily rely on modified versions of InceptionNeXt [ 62 ] , GatedCNN [ 61 ] , and their combination (StarBlock).

Figure 9: Illustration of our entropy modeling process.

Figure 10: All rate-distortion curves of different generative image codecs on Kodak, the validation set of DIV2K, and the test set of CLIC 2020 Professional in terms of LPIPS, DISTS, FID, KID, MS-SSIM, and PSNR metrics.

Figure 11: Visual examples and comparisons on 2K-resolution images from the test set of CLIC 2020 Professional.

Figure 12: Visual examples and comparisons on 2K-resolution images from the test set of CLIC 2020 Professional.

Figure 13: Visual examples and comparisons on 2K-resolution images from the validation set of DIV2K.

Figure 14: Visual examples and comparisons on 2K-resolution images from the validation set of DIV2K.

Method	Encoding Time (s)	Decoding Time (s)
Ours	0.034	0.355
StableCodec	0.030	0.263
DLF	0.181	0.195

Table 2: Runtime comparison of in seconds averaged on the Kodak dataset.

Method	Ours	StableCodec	DLF
Bitrate (bpp)	0.0076	0.0072	0.0079
Top-1 Percentage (%)	58.5	29.0	12.5

Table 3: Top-1 user preference on the Kodak dataset.

Generative Image Compression

Generative image compression uses compact codes and learned generative priors to reconstruct plausible images at very low bitrates.

See all 35 papers

Dec 6, 2025

SDGIC: A Semantic Disambiguation-Guided Generative Image Compression Method for Ultra-Low Bitrates

Generative image compression has recently shown impressive perceptual quality, but often suffers from semantic inconsistency at ultra-low bitrates (bpp < 0.05), limiting its reliable deployment in bandwidth-constrained scenarios such as 6G semantic communications. This inconsistency stems from incomplete guidance information, which introduces semantic ambiguity into the generation process and may lead to natural-looking but source-inconsistent content. In this work, we propose a Semantic-Disambiguation-Guided Generative Image Compression (SDGIC) framework to constrain diffusion-based reconstruction at ultra-low bitrates. Specifically, SDGIC compresses the source image into three compact and complementary guidance streams: a concise text caption for global semantics, a highly compressed image (HCI) for dense visual evidence, and Reconstruction-Aware Semantic Residual Tokens (RSRTs) for reconstruction-relevant residual semantics that remain ambiguous under the text caption and HCI conditions. The RSRTs are directly optimized toward the downstream denoising objective, enabling them to provide source-specific semantic constraints for disambiguating diffusion-based reconstruction. To inject these three guidance streams into the generation process effectively, we design a Dual-Path Conditioned Diffusion Decoder (DPCD), which uses cross-attention for semantic conditions and ControlNet residuals for dense visual guidance. Extensive experiments demonstrate that SDGIC improves semantic consistency at ultra-low bitrates while maintaining favorable perceptual quality, with a 23.4% reduction in AFINE on the CLIC2020 dataset.

Kaile Wang, Lijun He, Haisheng Fu +2

School of Information and Communications Engineering, Xi’an Jiaotong University, Xi’an 710049, China · Department of Electrical and Computer Engineering, Faculty of Applied Science, The University of British Columbia, Vancouver, BC, Canada

Diffusion Model Guidance Learned Image Compression Conditional Diffusion Models Generative Image Compression

Ask PDF

May 16, 2026

Adaptive Fused Prior Transfer for Controllable Generative Image Compression

At very low bitrates, image compression discards fine textures and local structures, while distortion-oriented reconstruction often produces over-smoothed images. Generative codecs synthesize missing details, but existing codebook-based controllable designs generally rely on single-codebook reconstruction priors. We propose Adaptive Fused Prior Transfer for Controllable Generative Image Compression (AFP-GIC), which transfers an image-adaptive fused prior from a frozen pretrained AdaCode model. Encoder-side prior features guide latent formation, while the decoder predicts a compatible fused prior from the compressed representation and control variables, without transmitting the prior itself. A motivating analysis shows that better decoder-side prior alignment tightens a reconstruction-error upper bound and that the fused-prior family includes single-codebook choices as special cases. A single pretrained model supports five evaluated bitrate operating points. Under the unified benchmark, AFP-GIC achieves 18.1% lower decoder latency and uses 31.10 million (20.5%) fewer inference parameters than DC-VIC. Experiments on Kodak, CLIC2020, and DIV2K show competitive PSNR and SSIM, with the clearest naturalness gains in NIQE scores and very-low-bitrate visual comparisons. Code: https://github.com/yifeipet/AFP_GIC.

Yifei Pei, Ying Liu, Nam Ling

Department of Computer Science and Engineering, Santa Clara University, Santa Clara, CA 95053 USA

Learned Image Compression Generative Image Compression

Ask PDF

Jun 19, 2026

Gen2-IC: Bridging Generative Models and Image Codecs through Latent Transport

Diffusion-based image compression has achieved strong perceptual quality at ultra-low bitrates. However, existing codecs are often tied to specific backbones and specialized components, making diverse, rapidly evolving generative models difficult to reuse. This raises a natural question: Can modern generative foundation models be connected to image compression through a simple and extensible interface? Two insights guide our design: stronger generative priors make a simpler codec interface viable, and generation and compression can be intrinsically linked through latent transport. We therefore propose Gen2-IC with two stages: (1) Latent Compression maps clean image latents to entropy-constrained latents; and (2) Latent Transport refines them with one near-terminal update based on the pretrained model. Gen2-IC requires neither auxiliary conditioning signals nor task-specific backbone modifications. With lightweight adaptation and no distillation, it supports fast encoding and one-step decoding across multiple bitrates. We validate Gen2-IC on SD-2.1, SANA-1.5, FLUX.1-dev, and Qwen-Image-2512, spanning U-Net and Transformer architectures as well as diffusion and flow-matching formulations. With stronger priors, Gen2-IC delivers gains below 0.05 bpp: the Qwen variant leads diffusion-based generative codecs in reconstruction fidelity (PSNR), perceptual similarity (LPIPS and DISTS), and recognizer-based semantic fidelity (OCR CER/WER and face-ROI similarity) across four benchmarks.

Yinhuan Huang, Hao Cao, Pu chen +3

Tsinghua University

Learned Image Compression Generative Image Compression Diffusion Model Compression

Ask PDF

Abstract

Generative Image Compression

Explore similar work

SDGIC: A Semantic Disambiguation-Guided Generative Image Compression Method for Ultra-Low Bitrates

Adaptive Fused Prior Transfer for Controllable Generative Image Compression

Gen2-IC: Bridging Generative Models and Image Codecs through Latent Transport