Efficient Generative Image Compression with Quantized Latents Refinement

Hao Cao; Yinhuan Huang; Wenqi Guo; Zhijin Qin; Jungong Han

cs.CVOct 21, 2026

Efficient Generative Image Compression with Quantized Latents Refinement

Authors: Hao Cao, Yinhuan Huang, Wenqi Guo, Zhijin Qin, Jungong Han

Organizations: Department of Electronic Engineering, Tsinghua University · Department of Automation, Tsinghua University · State Key Laboratory of Space Network and Communications · Beijing National Research Center for Information Science and Technology

Abstract

Generative image compression (GIC) delivers strong perceptual quality at low bitrates. However, GIC methods with strong perceptual quality often rely on pretrained diffusion models with high latency, while recent lightweight GIC methods reduce latency but deliver lower perceptual quality. We observe that, for lightweight GIC methods at low bitrates, coarse deterministic quantization can collapse nearby encoder latents corresponding to different fine details to the same point, making these details difficult to recover. Motivated by this, we propose GR-GIC, an efficient GIC that delivers competitive reconstruction performance with fast inference. For better performance, we introduce Guided Refinement (GR), which combines latent refinement guided by entropy context with adaptive perturbations for training regularization, jointly improving reconstruction at low bitrates. For fast inference, we introduce reorganized projected residual vector quantization (P-RVQ), which equivalently reorganizes residual updates into low-dimensional lookups to expose greater parallelism and reduce sequential computation. Experiments on Kodak show that for performance, compared with OneDC, GR-GIC achieves BD-rate savings of 23.19%, 37.71%, 22.30%, and 22.14% for LPIPS, DISTS, PSNR, and FID, respectively. For fast inference, the highly optimized implementation of GR-GIC encodes and decodes in 4.60 and 5.98 ms on an NVIDIA A100. Specifically, the reorganized P-RVQ implementation achieves a $3.14\times$ average speedup over the original implementation under the same FP32 setting. On an NVIDIA Jetson Orin NX, GR-GIC achieves latency comparable to lightweight GIC methods.

Figures & tables

Figure 1: Motivation and overview of GR-GIC . (a) Images with similar coarse content but different fine details are mapped to nearby points in the continuous latent space. (b) At low bitrates, coarse deterministic quantization can collapse these distinct latents to the same quantized point, leaving their fine detail differences unspecified. (c) GR uses entropy context to define a Gaussian refinement distribution around the recovered latent and introduces adaptive perturbations during training. (d) GR-GIC combines competitive reconstruction performance with fast inference.

Figure 2: Overview of GR-GIC . The encoder $g_{a}$ produces the latent $\bm{y}$ , while the hyper encoder $h_{a}$ and hyper decoder $h_{s}$ provide the context feature $\bm{\phi}$ . The context model predicts $(\bm{\mu},\bm{\sigma})$ for the groupwise affine projections (APs) and P-RVQ. The GR head refines the recovered latent $\hat{\bm{y}}$ into $\tilde{\bm{y}}$ before the decoder $g_{s}$ reconstructs $\hat{\bm{x}}$ .

Figure 3: Reorganized P-RVQ. (a) Original P-RVQ alternates high dimensional residual updates, projections, and greedy search. (b) The equivalent form stacks source projections and uses precomputed low dimensional lookups. Red arrows mark the critical dependency path.

Figure 4: Rate and reconstruction quality across four datasets. Lower LPIPS, DISTS, and FID and higher PSNR indicate better quality. FID uses base-2 logarithmic vertical axes.

Family	Method	Complexity				BD-rate (%)
Family	Method	Enc.	Dec.	FLOPs	Params	LPIPS	DISTS	PSNR	FID
Pretrained diffusion	ResULIC	96.24	428.32	8549.01	1390.00	206.40	260.74	10.77	304.92
	StableCodec	100.18	200.03	6820.28	1065.81	8.03	6.77	-17.10	-4.45
	RDEIC	157.25	426.68	7767.46	1380.27	69.79	166.61	-12.14	80.26
	OneDC	76.98	204.47	8557.18	1406.42	0.00	0.00	0.00	0.00
	OSCAR	53.04	167.56	6485.61	1009.30	127.15	131.50	154.09	159.92

Table 1: Results on the Kodak dataset, with latency measured on one NVIDIA A100 GPU. Encoding and decoding latencies are in milliseconds (ms), FLOPs in billions (G), parameters in millions (M), and BD-rate in percent. OneDC is the BD-rate anchor, and lower values are better for every reported measure. Bold and underlined values indicate the best and second best results, respectively.

Figure 5: Qualitative comparisons of GR-GIC with high-capacity diffusion and lightweight codecs at similar rates. Boxes mark local details. Labels report LPIPS and BPP, where lower is better.

Model	GR configuration				BPP $=0.036$		BPP $=0.071$		BPP $=0.107$		BPP $=0.143$
Model	Offset	Ctx.	Stoch.	Adapt.	LPIPS	FID	LPIPS	FID	LPIPS	FID	LPIPS	FID
$\mathrm{M}_{0}$	✗	✗	✗	✗	0.277	66.2	0.214	51.2	0.184	43.6	0.165	38.1
$\mathrm{M}_{1}$	✓	✗	✗	✗	0.271	63.6	0.212	46.8	0.184	39.2	0.164	35.4
$\mathrm{M}_{2}$	✓	✓	✗	✗	0.264	62.4	0.208	41.2	0.182	36.5	0.162	31.9
$\mathrm{M}_{3}$	✓	✓	✓	✗	0.264	59.5	0.207	37.9	0.182	33.8	0.162	29.5
$\mathrm{M}_{4}$	✓	✓	✓	✓	0.261	54.6	0.206	36.0	0.180	32.3	0.160	27.6

Table 2: Guided Refinement configurations on Kodak at four nominal rates. The GR columns indicate a learned offset, decoder context, stochastic scale, and adaptive scale.

Method	$512\!\times\!768$		1080p		2K		4K
Method	Enc.	Dec.	Enc.	Dec.	Enc.	Dec.	Enc.	Dec.
EF-LIC	71.253	143.040	347.490	633.721	623.651	1137.985	1441.824	2628.537
ProGIC	83.204	164.517	413.956	781.443	748.899	1412.819	1670.957	3182.561
AEIC	96.784	1290.483	–	$>10$ s	–	$>10$ s	OOM
CoD-Lite	114.456	239.371	564.422	1348.999	974.139	2489.412	–	$>10$ s
OneDC	773.388	1981.831	OOM		OOM		OOM

Table 3: Encoding (Enc.) and decoding (Dec.) latency (ms) on an NVIDIA Jetson Orin NX. Bold and underline mark the two best reported values in each column.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Resolution	$M$	Original FP32	Reorganized FP32	Reorganized FP16	CUDA FP16
$512\!\times\!768$	1	$3.84/1.00\times$	$1.54/2.50\times$	$1.54/2.49\times$	$0.93/4.12\times$
	2	$5.38/1.00\times$	$1.74/3.09\times$	$1.74/3.09\times$	$0.64/8.43\times$
	3	$10.80/1.00\times$	$3.29/3.29\times$	$3.33/3.24\times$	$0.87/12.38\times$
	4	$14.09/1.00\times$	$4.03/3.49\times$	$4.15/3.40\times$	$0.87/16.12\times$
1080p	1	$2.82/1.00\times$	$1.14/2.47\times$	$1.14/2.48\times$	$0.67/4.18\times$
	2	$5.42/1.00\times$	$1.77/3.07\times$	$1.83/2.97\times$	$0.66/8.26\times$

Appendix

Table 4: P-RVQ latency on an NVIDIA A100. Each entry reports the median CUDA event latency in ms followed by speedup over the corresponding original FP32 result. All implementations use weights converted from the same checkpoints.

Module	Implemented topology	Output or role
Analysis transform	$8\times$ pixel unshuffle, $192\!\to\!368$ entry DCB, six 368-channel body DCBs, $3\!\times\!3$ Conv with stride 2 to 256 channels	Main latent at $1/16$ resolution
Hyper analysis	$1\!\times\!1$ Conv $256\!\to\!512$ , two 512-channel DCBs, two residual downsampling stages with stride 2, final 128-channel projection	Hyper latent at $1/64$ resolution
Hyper synthesis	SP $128\!\to\!512$ , one DCB, SP $512\!\to\!512$ , two DCBs, $1\!\times\!1$ Conv to 512 channels	256-channel normalization scale $\bm{q}_{\mathrm{dec}}$ and 256-channel context feature $\bm{\phi}$
Context path	Quadtree support projection, projection of previously decoded groups, one group adaptor DCB, three shared 512-channel DCBs, final $1\!\times\!1$ Conv	256-channel entropy context mean $\bm{\mu}$ and scale $\bm{\sigma}$
GR head	$1\!\times\!1$ Conv $768\!\to\!768$ , three 768-channel DCBs, $1\!\times\!1$ Conv $768\!\to\!512$	256-channel offset $\bm{\delta}$ and adaptive perturbation scale $\bm{\beta}$
Synthesis transform	SP $256\!\to\!368$ , twelve 368-channel body DCBs, one final refinement DCB, $1\!\times\!1$ Conv $368\!\to\!192$ , $8\times$ pixel shuffle	RGB reconstruction

Appendix

Table 5: Detailed GR-GIC architecture. Conv, DCB, and SP denote convolution, DepthConvBlock, and subpixel upsampling, respectively. The analysis and synthesis depths count the repeated body DCBs, while the entry and final refinement blocks are listed separately.

Stream	$y_{1}$	$y_{2}$	$y_{3}$	$y_{4}$	$z$
Input dim.	256	256	256	256	128
Code dim.	16	16	16	16	8
Entries	1024	512	256	128	1024
Bits per stage	10	9	8	7	10

Appendix

Table 6: P-RVQ configuration. The four main latent quantizers operate on quadtree groups. The fifth quantizes the hyper latent. “Bits per stage” is the fixed index width.

Resolution	BPP	Params	FLOPs	Enc.	Dec.	Mem.
$512\!\times\!768$	0.0356	79.389	456.491	4.475	5.949	0.183
	0.0713	79.389	456.531	4.523	5.947	0.185
	0.1069	79.389	456.571	4.717	6.028	0.187
	0.1426	79.389	456.610	4.674	6.009	0.189
1080p	0.0356	79.389	2425.111	13.817	21.428	0.250
	0.0713	79.389	2425.321	14.111	21.483	0.250

Appendix

Table 7: Per operating point GR-GIC complexity, CUDA latency, and peak memory on one A100. FLOPs are reported in G for one encoder and decoder pass under the profiler-supported operator scope. Latency is in ms and peak GPU memory is in GiB.

Resolution	Backend	Enc.	Dec.	Online E2E	Mem.
$512\!\times\!768$	PyTorch FP32	30.453	44.708	66.519	0.438
	PyTorch FP16	15.439	20.298	37.784	0.222
	CUDA FP16	4.597	5.983	10.353	0.186
1080p	PyTorch FP32	109.967	176.403	282.883	0.965
	PyTorch FP16	32.895	57.600	86.817	0.465
	CUDA FP16	14.097	21.442	35.567	0.252

Appendix

Table 8: GR-GIC backend comparison on one A100. Latency is in ms and peak GPU memory is in GiB. Online E2E denotes one consecutive pair of in-memory encode and decode calls.

Figure 6: Rate and reconstruction quality of GR-GIC and lightweight codecs on four datasets. GR-GIC uses four operating points, while EF-LIC uses five. Lower LPIPS, DISTS, and FID and higher PSNR indicate better reconstruction. FID uses a base-2 logarithmic vertical axis.

Metric	Family	Method	Kodak	Tecnick	DIV2K	CLIC 2020
LPIPS	Diffusion	OneDC	0.00	0.00	0.00	0.00
		StableCodec	8.03	32.28	27.06	36.13
		AEIC	41.65	34.58	25.27	32.18
		DiT-IC	35.99	61.79	49.18	52.00
		OSCAR	127.15	225.78	201.57	–
	VAE-based	GLC	21.53	47.87	33.19	39.53

Appendix

Table 9: Pairwise OneDC-anchored BD-rate (%). Bold and underline mark the two lowest displayed values per dataset.

Method	1080p			2K			4K
Method	Enc.	Dec.	Mem.	Enc.	Dec.	Mem.	Enc.	Dec.	Mem.
ResULIC	481.75	3702.45	20.38	–	$>10$ s	–	–	$>10$ s	–
StableCodec	641.76	1657.43	21.80	1085.39	3161.54	30.65	–	$>10$ s	–
RDEIC	683.20	2296.58	13.55	1436.00	5963.77	20.17	–	$>10$ s	–
OneDC	303.33	742.06	12.81	585.72	1554.82	18.86	1542.18	4846.63	35.38
OSCAR	513.20	1123.38	24.75	2382.55	2719.79	79.44	–	–	$>80$

Appendix

Table 10: High-resolution encoding and decoding latency (ms) and peak GPU memory (GiB) on one A100. Bold and underline mark the two best results.

Figure 7: Additional Kodak visuals with LPIPS and BPP labels, where lower is better.

Panel of Figure 8 — Figure 8: Additional Tecnick visuals with LPIPS and BPP labels, where lower is better.

Panel of Figure 11 — Figure 11: Additional Tecnick visuals with LPIPS and BPP labels, where lower is better.

Generative Image Compression

Generative image compression uses compact codes and learned generative priors to reconstruct plausible images at very low bitrates.

See all 35 papers

ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization
Mar 3, 2026Hao Cao, Chengbin Liang, Wenqi Guo +2Generative Image Compression Learned Image Compression
Tsinghua University · State Key Laboratory of Space Network and Communications
SDGIC: A Semantic Disambiguation-Guided Generative Image Compression Method for Ultra-Low Bitrates
Dec 6, 2025Kaile Wang, Lijun He, Haisheng Fu +2Diffusion Model Guidance Learned Image Compression
School of Information and Communications Engineering, Xi’an Jiaotong University, Xi’an 710049, China · Department of Electrical and Computer Engineering, Faculty of Applied Science, The University of British Columbia, Vancouver, BC, Canada
Gen2-IC: Bridging Generative Models and Image Codecs through Latent Transport
Jun 19, 2026Yinhuan Huang, Hao Cao, Pu chen +3Learned Image Compression Generative Image Compression
Tsinghua University

Mar 3, 2026J/K move · Enter open · S save

ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization

Hao Cao, Chengbin Liang, Wenqi Guo, Zhijin Qin+1

Tsinghua University · State Key Laboratory of Space Network and Communications

Generative Image Compression Learned Image Compression Residual VQ

Open paper AskPDF arXiv

Recent advances in generative image compression (GIC) have delivered remarkable improvements in perceptual quality. However, many GICs rely on large-scale and rigid models, which severely constrain their utility for flexible transmission and practical deployment in low-bitrate scenarios. To address these issues, we propose Progressive Generative Image Compression (ProGIC), a compact codec built on residual vector quantization (RVQ). In RVQ, a sequence of vector quantizers encodes the residuals stage by stage, each with its own codebook. The resulting codewords sum to a coarse-to-fine reconstruction and a progressive bitstream, enabling previews from partial data. We pair this with a lightweight backbone based on depthwise-separable convolutions and small attention blocks, enabling practical deployment on both GPUs and CPU-only devices. Experimental results show that ProGIC attains comparable compression performance compared with previous methods. It achieves bitrate savings of up to 57.57% on DISTS and 58.83% on LPIPS compared to MS-ILLM on the Kodak dataset. Beyond perceptual quality, ProGIC enables progressive transmission for flexibility, and also delivers over 10 times faster encoding and decoding compared with MS-ILLM on GPUs for efficiency.

Abstract

Generative Image Compression

Explore similar work

ProGIC: Progressive and Lightweight Generative Image Compression with Residual Vector Quantization

SDGIC: A Semantic Disambiguation-Guided Generative Image Compression Method for Ultra-Low Bitrates

Gen2-IC: Bridging Generative Models and Image Codecs through Latent Transport