Efficient Generative Image Compression with Quantized Latents Refinement

Hao Cao; Yinhuan Huang; Wenqi Guo; Zhijin Qin; Jungong Han

cs.CVOct 21, 2026

Efficient Generative Image Compression with Quantized Latents Refinement

Authors: Hao Cao, Yinhuan Huang, Wenqi Guo, Zhijin Qin, Jungong Han

Organizations: Department of Electronic Engineering, Tsinghua University · Department of Automation, Tsinghua University · State Key Laboratory of Space Network and Communications · Beijing National Research Center for Information Science and Technology

Abstract

Generative image compression (GIC) delivers strong perceptual quality at low bitrates. However, GIC methods with strong perceptual quality often rely on pretrained diffusion models with high latency, while recent lightweight GIC methods reduce latency but deliver lower perceptual quality. We observe that, for lightweight GIC methods at low bitrates, coarse deterministic quantization can collapse nearby encoder latents corresponding to different fine details to the same point, making these details difficult to recover. Motivated by this, we propose GR-GIC, an efficient GIC that delivers competitive reconstruction performance with fast inference. For better performance, we introduce Guided Refinement (GR), which combines latent refinement guided by entropy context with adaptive perturbations for training regularization, jointly improving reconstruction at low bitrates. For fast inference, we introduce reorganized projected residual vector quantization (P-RVQ), which equivalently reorganizes residual updates into low-dimensional lookups to expose greater parallelism and reduce sequential computation. Experiments on Kodak show that for performance, compared with OneDC, GR-GIC achieves BD-rate savings of 23.19%, 37.71%, 22.30%, and 22.14% for LPIPS, DISTS, PSNR, and FID, respectively. For fast inference, the highly optimized implementation of GR-GIC encodes and decodes in 4.60 and 5.98 ms on an NVIDIA A100. Specifically, the reorganized P-RVQ implementation achieves a $3.14\times$ average speedup over the original implementation under the same FP32 setting. On an NVIDIA Jetson Orin NX, GR-GIC achieves latency comparable to lightweight GIC methods.

Figures & tables

Figure 1: Motivation and overview of GR-GIC . (a) Images with similar coarse content but different fine details are mapped to nearby points in the continuous latent space. (b) At low bitrates, coarse deterministic quantization can collapse these distinct latents to the same quantized point, leaving their fine detail differences unspecified. (c) GR uses entropy context to define a Gaussian refinement distribution around the recovered latent and introduces adaptive perturbations during training. (d) GR-GIC combines competitive reconstruction performance with fast inference.

Figure 2: Overview of GR-GIC . The encoder $g_{a}$ produces the latent $\bm{y}$ , while the hyper encoder $h_{a}$ and hyper decoder $h_{s}$ provide the context feature $\bm{\phi}$ . The context model predicts $(\bm{\mu},\bm{\sigma})$ for the groupwise affine projections (APs) and P-RVQ. The GR head refines the recovered latent $\hat{\bm{y}}$ into $\tilde{\bm{y}}$ before the decoder $g_{s}$ reconstructs $\hat{\bm{x}}$ .

Figure 3: Reorganized P-RVQ. (a) Original P-RVQ alternates high dimensional residual updates, projections, and greedy search. (b) The equivalent form stacks source projections and uses precomputed low dimensional lookups. Red arrows mark the critical dependency path.

Figure 4: Rate and reconstruction quality across four datasets. Lower LPIPS, DISTS, and FID and higher PSNR indicate better quality. FID uses base-2 logarithmic vertical axes.

Family	Method	Complexity				BD-rate (%)
Family	Method	Enc.	Dec.	FLOPs	Params	LPIPS	DISTS	PSNR	FID
Pretrained diffusion	ResULIC	96.24	428.32	8549.01	1390.00	206.40	260.74	10.77	304.92
	StableCodec	100.18	200.03	6820.28	1065.81	8.03	6.77	-17.10	-4.45
	RDEIC	157.25	426.68	7767.46	1380.27	69.79	166.61	-12.14	80.26
	OneDC	76.98	204.47	8557.18	1406.42	0.00	0.00	0.00	0.00
	OSCAR	53.04	167.56	6485.61	1009.30	127.15	131.50	154.09	159.92

Table 1: Results on the Kodak dataset, with latency measured on one NVIDIA A100 GPU. Encoding and decoding latencies are in milliseconds (ms), FLOPs in billions (G), parameters in millions (M), and BD-rate in percent. OneDC is the BD-rate anchor, and lower values are better for every reported measure. Bold and underlined values indicate the best and second best results, respectively.

Figure 5: Qualitative comparisons of GR-GIC with high-capacity diffusion and lightweight codecs at similar rates. Boxes mark local details. Labels report LPIPS and BPP, where lower is better.

Model	GR configuration				BPP $=0.036$		BPP $=0.071$		BPP $=0.107$		BPP $=0.143$
Model	Offset	Ctx.	Stoch.	Adapt.	LPIPS	FID	LPIPS	FID	LPIPS	FID	LPIPS	FID
$\mathrm{M}_{0}$	✗	✗	✗	✗	0.277	66.2	0.214	51.2	0.184	43.6	0.165	38.1
$\mathrm{M}_{1}$	✓	✗	✗	✗	0.271	63.6	0.212	46.8	0.184	39.2	0.164	35.4
$\mathrm{M}_{2}$	✓	✓	✗	✗	0.264	62.4	0.208	41.2	0.182	36.5	0.162	31.9
$\mathrm{M}_{3}$	✓	✓	✓	✗	0.264	59.5	0.207	37.9	0.182	33.8	0.162	29.5
$\mathrm{M}_{4}$	✓	✓	✓	✓	0.261	54.6	0.206	36.0	0.180	32.3	0.160	27.6

Table 2: Guided Refinement configurations on Kodak at four nominal rates. The GR columns indicate a learned offset, decoder context, stochastic scale, and adaptive scale.

Method	$512\!\times\!768$		1080p		2K		4K
Method	Enc.	Dec.	Enc.	Dec.	Enc.	Dec.	Enc.	Dec.
EF-LIC	71.253	143.040	347.490	633.721	623.651	1137.985	1441.824	2628.537
ProGIC	83.204	164.517	413.956	781.443	748.899	1412.819	1670.957	3182.561
AEIC	96.784	1290.483	–	$>10$ s	–	$>10$ s	OOM
CoD-Lite	114.456	239.371	564.422	1348.999	974.139	2489.412	–	$>10$ s
OneDC	773.388	1981.831	OOM		OOM		OOM

Table 3: Encoding (Enc.) and decoding (Dec.) latency (ms) on an NVIDIA Jetson Orin NX. Bold and underline mark the two best reported values in each column.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Resolution	$M$	Original FP32	Reorganized FP32	Reorganized FP16	CUDA FP16
$512\!\times\!768$	1	$3.84/1.00\times$	$1.54/2.50\times$	$1.54/2.49\times$	$0.93/4.12\times$
	2	$5.38/1.00\times$	$1.74/3.09\times$	$1.74/3.09\times$	$0.64/8.43\times$
	3	$10.80/1.00\times$	$3.29/3.29\times$	$3.33/3.24\times$	$0.87/12.38\times$
	4	$14.09/1.00\times$	$4.03/3.49\times$	$4.15/3.40\times$	$0.87/16.12\times$
1080p	1	$2.82/1.00\times$	$1.14/2.47\times$	$1.14/2.48\times$	$0.67/4.18\times$
	2	$5.42/1.00\times$	$1.77/3.07\times$	$1.83/2.97\times$	$0.66/8.26\times$

Appendix

Table 4: P-RVQ latency on an NVIDIA A100. Each entry reports the median CUDA event latency in ms followed by speedup over the corresponding original FP32 result. All implementations use weights converted from the same checkpoints.

Module	Implemented topology	Output or role
Analysis transform	$8\times$ pixel unshuffle, $192\!\to\!368$ entry DCB, six 368-channel body DCBs, $3\!\times\!3$ Conv with stride 2 to 256 channels	Main latent at $1/16$ resolution
Hyper analysis	$1\!\times\!1$ Conv $256\!\to\!512$ , two 512-channel DCBs, two residual downsampling stages with stride 2, final 128-channel projection	Hyper latent at $1/64$ resolution
Hyper synthesis	SP $128\!\to\!512$ , one DCB, SP $512\!\to\!512$ , two DCBs, $1\!\times\!1$ Conv to 512 channels	256-channel normalization scale $\bm{q}_{\mathrm{dec}}$ and 256-channel context feature $\bm{\phi}$
Context path	Quadtree support projection, projection of previously decoded groups, one group adaptor DCB, three shared 512-channel DCBs, final $1\!\times\!1$ Conv	256-channel entropy context mean $\bm{\mu}$ and scale $\bm{\sigma}$
GR head	$1\!\times\!1$ Conv $768\!\to\!768$ , three 768-channel DCBs, $1\!\times\!1$ Conv $768\!\to\!512$	256-channel offset $\bm{\delta}$ and adaptive perturbation scale $\bm{\beta}$
Synthesis transform	SP $256\!\to\!368$ , twelve 368-channel body DCBs, one final refinement DCB, $1\!\times\!1$ Conv $368\!\to\!192$ , $8\times$ pixel shuffle	RGB reconstruction

Appendix

Table 5: Detailed GR-GIC architecture. Conv, DCB, and SP denote convolution, DepthConvBlock, and subpixel upsampling, respectively. The analysis and synthesis depths count the repeated body DCBs, while the entry and final refinement blocks are listed separately.

Stream	$y_{1}$	$y_{2}$	$y_{3}$	$y_{4}$	$z$
Input dim.	256	256	256	256	128
Code dim.	16	16	16	16	8
Entries	1024	512	256	128	1024
Bits per stage	10	9	8	7	10

Appendix

Table 6: P-RVQ configuration. The four main latent quantizers operate on quadtree groups. The fifth quantizes the hyper latent. “Bits per stage” is the fixed index width.

Resolution	BPP	Params	FLOPs	Enc.	Dec.	Mem.
$512\!\times\!768$	0.0356	79.389	456.491	4.475	5.949	0.183
	0.0713	79.389	456.531	4.523	5.947	0.185
	0.1069	79.389	456.571	4.717	6.028	0.187
	0.1426	79.389	456.610	4.674	6.009	0.189
1080p	0.0356	79.389	2425.111	13.817	21.428	0.250
	0.0713	79.389	2425.321	14.111	21.483	0.250

Appendix

Table 7: Per operating point GR-GIC complexity, CUDA latency, and peak memory on one A100. FLOPs are reported in G for one encoder and decoder pass under the profiler-supported operator scope. Latency is in ms and peak GPU memory is in GiB.

Resolution	Backend	Enc.	Dec.	Online E2E	Mem.
$512\!\times\!768$	PyTorch FP32	30.453	44.708	66.519	0.438
	PyTorch FP16	15.439	20.298	37.784	0.222
	CUDA FP16	4.597	5.983	10.353	0.186
1080p	PyTorch FP32	109.967	176.403	282.883	0.965
	PyTorch FP16	32.895	57.600	86.817	0.465
	CUDA FP16	14.097	21.442	35.567	0.252

Appendix

Table 8: GR-GIC backend comparison on one A100. Latency is in ms and peak GPU memory is in GiB. Online E2E denotes one consecutive pair of in-memory encode and decode calls.

Figure 6: Rate and reconstruction quality of GR-GIC and lightweight codecs on four datasets. GR-GIC uses four operating points, while EF-LIC uses five. Lower LPIPS, DISTS, and FID and higher PSNR indicate better reconstruction. FID uses a base-2 logarithmic vertical axis.

Metric	Family	Method	Kodak	Tecnick	DIV2K	CLIC 2020
LPIPS	Diffusion	OneDC	0.00	0.00	0.00	0.00
		StableCodec	8.03	32.28	27.06	36.13
		AEIC	41.65	34.58	25.27	32.18
		DiT-IC	35.99	61.79	49.18	52.00
		OSCAR	127.15	225.78	201.57	–
	VAE-based	GLC	21.53	47.87	33.19	39.53

Appendix

Table 9: Pairwise OneDC-anchored BD-rate (%). Bold and underline mark the two lowest displayed values per dataset.

Method	1080p			2K			4K
Method	Enc.	Dec.	Mem.	Enc.	Dec.	Mem.	Enc.	Dec.	Mem.
ResULIC	481.75	3702.45	20.38	–	$>10$ s	–	–	$>10$ s	–
StableCodec	641.76	1657.43	21.80	1085.39	3161.54	30.65	–	$>10$ s	–
RDEIC	683.20	2296.58	13.55	1436.00	5963.77	20.17	–	$>10$ s	–
OneDC	303.33	742.06	12.81	585.72	1554.82	18.86	1542.18	4846.63	35.38
OSCAR	513.20	1123.38	24.75	2382.55	2719.79	79.44	–	–	$>80$

Appendix

Table 10: High-resolution encoding and decoding latency (ms) and peak GPU memory (GiB) on one A100. Bold and underline mark the two best results.

Figure 7: Additional Kodak visuals with LPIPS and BPP labels, where lower is better.

Panel of Figure 8 — Figure 8: Additional Tecnick visuals with LPIPS and BPP labels, where lower is better.

Panel of Figure 11 — Figure 11: Additional Tecnick visuals with LPIPS and BPP labels, where lower is better.

Generative Image Compression

Generative image compression uses compact codes and learned generative priors to reconstruct plausible images at very low bitrates.

See all 35 papers

Mar 3, 2026

ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization

Recent advances in generative image compression (GIC) have delivered remarkable improvements in perceptual quality. However, many GICs rely on large-scale and rigid models, which severely constrain their utility for flexible transmission and practical deployment in low-bitrate scenarios. To address these issues, we propose Progressive Generative Image Compression (ProGIC), a compact codec built on residual vector quantization (RVQ). In RVQ, a sequence of vector quantizers encodes the residuals stage by stage, each with its own codebook. The resulting codewords sum to a coarse-to-fine reconstruction and a progressive bitstream, enabling previews from partial data. We pair this with a lightweight backbone based on depthwise-separable convolutions and small attention blocks, enabling practical deployment on both GPUs and CPU-only devices. Experimental results show that ProGIC attains comparable compression performance compared with previous methods. It achieves bitrate savings of up to 57.57% on DISTS and 58.83% on LPIPS compared to MS-ILLM on the Kodak dataset. Beyond perceptual quality, ProGIC enables progressive transmission for flexibility, and also delivers over 10 times faster encoding and decoding compared with MS-ILLM on GPUs for efficiency.

Hao Cao, Chengbin Liang, Wenqi Guo +2

Tsinghua University · State Key Laboratory of Space Network and Communications

Generative Image Compression Learned Image Compression Residual VQ

Ask PDF

Dec 6, 2025

SDGIC: A Semantic Disambiguation-Guided Generative Image Compression Method for Ultra-Low Bitrates

Generative image compression has recently shown impressive perceptual quality, but often suffers from semantic inconsistency at ultra-low bitrates (bpp < 0.05), limiting its reliable deployment in bandwidth-constrained scenarios such as 6G semantic communications. This inconsistency stems from incomplete guidance information, which introduces semantic ambiguity into the generation process and may lead to natural-looking but source-inconsistent content. In this work, we propose a Semantic-Disambiguation-Guided Generative Image Compression (SDGIC) framework to constrain diffusion-based reconstruction at ultra-low bitrates. Specifically, SDGIC compresses the source image into three compact and complementary guidance streams: a concise text caption for global semantics, a highly compressed image (HCI) for dense visual evidence, and Reconstruction-Aware Semantic Residual Tokens (RSRTs) for reconstruction-relevant residual semantics that remain ambiguous under the text caption and HCI conditions. The RSRTs are directly optimized toward the downstream denoising objective, enabling them to provide source-specific semantic constraints for disambiguating diffusion-based reconstruction. To inject these three guidance streams into the generation process effectively, we design a Dual-Path Conditioned Diffusion Decoder (DPCD), which uses cross-attention for semantic conditions and ControlNet residuals for dense visual guidance. Extensive experiments demonstrate that SDGIC improves semantic consistency at ultra-low bitrates while maintaining favorable perceptual quality, with a 23.4% reduction in AFINE on the CLIC2020 dataset.

Kaile Wang, Lijun He, Haisheng Fu +2

School of Information and Communications Engineering, Xi’an Jiaotong University, Xi’an 710049, China · Department of Electrical and Computer Engineering, Faculty of Applied Science, The University of British Columbia, Vancouver, BC, Canada

Diffusion Model Guidance Learned Image Compression Conditional Diffusion Models Generative Image Compression

Ask PDF

Jun 19, 2026

Gen2-IC: Bridging Generative Models and Image Codecs through Latent Transport

Diffusion-based image compression has achieved strong perceptual quality at ultra-low bitrates. However, existing codecs are often tied to specific backbones and specialized components, making diverse, rapidly evolving generative models difficult to reuse. This raises a natural question: Can modern generative foundation models be connected to image compression through a simple and extensible interface? Two insights guide our design: stronger generative priors make a simpler codec interface viable, and generation and compression can be intrinsically linked through latent transport. We therefore propose Gen2-IC with two stages: (1) Latent Compression maps clean image latents to entropy-constrained latents; and (2) Latent Transport refines them with one near-terminal update based on the pretrained model. Gen2-IC requires neither auxiliary conditioning signals nor task-specific backbone modifications. With lightweight adaptation and no distillation, it supports fast encoding and one-step decoding across multiple bitrates. We validate Gen2-IC on SD-2.1, SANA-1.5, FLUX.1-dev, and Qwen-Image-2512, spanning U-Net and Transformer architectures as well as diffusion and flow-matching formulations. With stronger priors, Gen2-IC delivers gains below 0.05 bpp: the Qwen variant leads diffusion-based generative codecs in reconstruction fidelity (PSNR), perceptual similarity (LPIPS and DISTS), and recognizer-based semantic fidelity (OCR CER/WER and face-ROI similarity) across four benchmarks.

Yinhuan Huang, Hao Cao, Pu chen +3

Tsinghua University

Learned Image Compression Generative Image Compression Diffusion Model Compression

Ask PDF

Abstract

Generative Image Compression

Explore similar work

ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization

SDGIC: A Semantic Disambiguation-Guided Generative Image Compression Method for Ultra-Low Bitrates

Gen2-IC: Bridging Generative Models and Image Codecs through Latent Transport