Semantic-Guided Frequency Modulation for Learned Image Compression

Gangyi Tian; Wen Ji; Alireza Furutanpey; Mingxuan Li

eess.IVOct 24, 2026

Semantic-Guided Frequency Modulation for Learned Image Compression

Authors: Gangyi Tian, Wen Ji, Alireza Furutanpey, Mingxuan Li

Organizations: State Key Laboratory of Processors, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Institute of AI for Industries, Chinese Academy of Sciences, Nanjing, China · Distributed Systems Group, Vienna University of Technology, Vienna, Austria

Abstract

High-frequency attenuation can reduce the rate of learned image codecs but also remove object boundaries and fine detail. We propose Semantic-Guided Adaptive Latent-Frequency Modulation (SG-AFM), which uses a semantic structural prior to guide attenuation of encoder latents. SG-AFM applies a Haar wavelet transform and combines a fixed semantic-frequency multiplier with a bounded learned correction to modulate high-frequency subbands. The semantic prior is computed only at the encoder. No semantic map is transmitted, and no decoder modules are added. Across six datasets, SG-AFM reduces PSNR-based BD-rate relative to TCM by an average of 13.17%. It achieves the best BD-rate among the compared methods on ExDark, NightCity, and OnePlus, with a mean reduction of 16.80%. An ablation on OnePlus demonstrates the benefit of spatially aligned semantic guidance, and visual comparisons illustrate improved reconstruction of local structures.

Figures & tables

Figure 1: Semantic guidance for latent-frequency modulation. Examples from ExDark, NightCity, and OnePlus show the input, original high-frequency (HF) response $H_{y}$ , semantic structural prior $S$ , modulated response, and reconstruction. Green boxes mark structural regions and yellow boxes mark other HF regions. White dashed boxes locate examples of modulation in the response maps.

Figure 2: SG-AFM within the TCM codec. We derive the semantic prior $S$ from the input image and combine it with wavelet responses of the encoder latent $y$ . We compute the modulation weights from a fixed multiplier and a bounded learned correction. We multiply each signed high-frequency subband by these weights and retain the low-frequency subband before IDWT reconstruction of $y^{\prime}$ . The aggregate $H_{y}^{\prime}$ visualizes the modulated response. Entropy coding retains the TCM main-latent and hyperlatent streams and transmits no semantic map.

Figure 3: Rate-distortion curves on six datasets, measured by PSNR and MS-SSIM against bits per pixel. Blue boxes highlight ExDark, NightCity, and OnePlus, the three complex-illumination benchmarks. Kodak and Tecnick provide normal-light comparisons, and VELoL provides a low-light comparison.

Method	Normal Light		Complex Illumination			Low Light	Average
Method	Kodak	Tecnick	ExDark	NightCity	OnePlus	VELoL	N-Avg.	C-Avg.
BPG [ 19 ]	+37.84	+42.59	+31.51	+61.81	+21.01	+42.60	+40.22	+38.11
VTM-23.3 [ 20 ]	+16.09	+11.63	+13.37	+36.09	+6.23	+21.00	+13.86	+18.56
Cheng-attn [ 21 ]	+14.73	+20.53	+11.86	+11.97	+10.66	+1.63	+17.63	+11.50
TCM-AuxT [ 22 ]	-0.87	-1.94	-0.38	-0.04	-1.52	-0.97	-1.41	-0.65
DCAE [ 23 ]	-8.63	-13.24	-12.30	-11.60	-13.97	-19.26	-10.94	-12.62

Table 1: PSNR-based BD-rate (%) relative to TCM. Negative values indicate rate savings. More negative values are better. N-Avg. and C-Avg. denote averages over the normal-light and complex-illumination datasets, respectively. Best results are highlighted in bold.

Variant	PSNR $\uparrow$	MS-SSIM(dB) $\uparrow$	bpp $\downarrow$	RoI-PSNR $\uparrow$	RoI-L1 $\downarrow$	RoI-Grad $\downarrow$
TCM Anchor	37.7183	17.3199	0.687595	36.6200	0.011387	0.046202
Blind	37.6466	17.2660	0.654422	36.5456	0.011484	0.046752
Random-shuffle	37.6400	17.2701	0.657338	36.5604	0.011477	0.046650
Ours	38.4718	17.8835	0.656493	36.6994	0.011322	0.046068

Table 2: Ablation study on OnePlus at

\lambda=0.0483

. All modulation variants jointly fine-tune the synthesis transform without changing its architecture. RoI denotes the semantic region of interest. Best results are highlighted in bold.

Figure 4: Visual and region-wise reconstruction quality in a complex-illumination scene. For each method, the reconstructed image is followed by an enlargement of the highlighted structural region. Labels report PSNR/RoI-PSNR/bpp. SG-AFM preserves more structural detail in the enlarged region at a comparable rate.

Learned Image Compression

Learned image compression uses neural encoders, quantization, and entropy models to reduce image bitrates while balancing fidelity, perceptual quality, and task utility.

See all 120 papers

FLM: Frequency-Aware Language Models for Generative Image Compression
Aug 26, 2026Jiarun Chen, Kejun Wu, Li Li +3Learned Image Compression Generative Image Compression
School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · MOE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition, University of Science and Technology of China, Hefei 230093, China · College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China +2
Multiple Scale Latents for Learned Image Compression
Aug 11, 2026Jonas Brenig, Radu TimofteLearned Image Compression Representation Learning
Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany
Adaptive Fused Prior Transfer for Controllable Generative Image Compression
May 16, 2026Yifei Pei, Ying Liu, Nam LingLearned Image Compression Generative Image Compression
Department of Computer Science and Engineering, Santa Clara University, Santa Clara, CA 95053 USA

Aug 26, 2026J/K move · Enter open · S save

FLM: Frequency-Aware Language Models for Generative Image Compression

Jiarun Chen, Kejun Wu, Li Li, Chengtao Cai+2

School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · MOE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition, University of Science and Technology of China, Hefei 230093, China · College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China+2

Learned Image Compression Generative Image Compression

Open paper AskPDF arXiv

Generative models have significantly improved the performance ceiling of image lossy compression at low bitrates by exploiting learned priors. However, the generated textures and semantic details may deviate from the source content, thereby affecting the fidelity of image reconstruction. To solve these challenges, we propose FLM, a frequency-aware language model that improves compression efficiency through frequency-domain probabilistic modeling while retaining deterministic reconstruction. At the encoder, the input image is transformed into quantized DCT coefficients, which are organized into discrete sequences using macroblock-based coefficient tokenization. FLM then performs next-coefficient prediction to autoregressively estimate token-wise conditional probability distributions for arithmetic coding, thereby generating a compact bitstream. At the decoder, the LLM and arithmetic decoder jointly recover the frequency-domain data, followed by inverse transformations for image reconstruction. A task-specific frequency-domain dataset and a two-stage fine-tuning strategy are further developed to enable the model to operate across multiple bitrate settings. FLM is a versatile compressor that is compatible with both lossy compression and lossless JPEG recompression frameworks. Experiments show that FLM exceeds conventional and generative lossy compression methods in rate-distortion performance. FLM achieves BD-PSNR gains of 3.30 dB, 3.83 dB, and 3.80 dB than JPEG baseline on Kodak, Tecnick, and CLIC2020, respectively. Better qualitative quality of FLM can be achieved in improving semantically high fidelity and suppressing blocking artifacts. FLM is also validated to be applicable to the lossless recompression task with competitive performance.

Abstract

Learned Image Compression

Explore similar work

FLM: Frequency-Aware Language Models for Generative Image Compression

Multiple Scale Latents for Learned Image Compression

Adaptive Fused Prior Transfer for Controllable Generative Image Compression