FLM: Frequency-Aware Language Models for Generative Image Compression
Authors: Jiarun Chen, Kejun Wu, Li Li, Chengtao Cai, Zhengguo Li, Chia-Wen Lin
Organizations: School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · MOE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition, University of Science and Technology of China, Hefei 230093, China · College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China · VI Department, Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore · Department of Electrical Engineering and the Institute of Communications Engineering, National Tsing Hua University, Hsinchu 30013, Taiwan
Generative models have significantly improved the performance ceiling of image lossy compression at low bitrates by exploiting learned priors. However, the generated textures and semantic details may deviate from the source content, thereby affecting the fidelity of image reconstruction. To solve these challenges, we propose FLM, a frequency-aware language model that improves compression efficiency through frequency-domain probabilistic modeling while retaining deterministic reconstruction. At the encoder, the input image is transformed into quantized DCT coefficients, which are organized into discrete sequences using macroblock-based coefficient tokenization. FLM then performs next-coefficient prediction to autoregressively estimate token-wise conditional probability distributions for arithmetic coding, thereby generating a compact bitstream. At the decoder, the LLM and arithmetic decoder jointly recover the frequency-domain data, followed by inverse transformations for image reconstruction. A task-specific frequency-domain dataset and a two-stage fine-tuning strategy are further developed to enable the model to operate across multiple bitrate settings. FLM is a versatile compressor that is compatible with both lossy compression and lossless JPEG recompression frameworks. Experiments show that FLM exceeds conventional and generative lossy compression methods in rate-distortion performance. FLM achieves BD-PSNR gains of 3.30 dB, 3.83 dB, and 3.80 dB than JPEG baseline on Kodak, Tecnick, and CLIC2020, respectively. Better qualitative quality of FLM can be achieved in improving semantically high fidelity and suppressing blocking artifacts. FLM is also validated to be applicable to the lossless recompression task with competitive performance.
Figures & tables
Fig. 1: Conceptual comparison of representative image compression methods. Diffusion-based methods compromise reconstruction fidelity, whereas JPEG requires a higher bitrate. Our FLM method achieves both high fidelity and low bitrate.
Fig. 2: The overall framework of FLM. The input image is converted into quantized DCT coefficients by a JPEG-style transform, and the proposed Macroblock-Based DCT Coefficient Tokenization organizes the coefficients into token sequences. The task-prompt tokens and coefficient tokens are concatenated and fed into the LLM to predict the conditional distribution of the next token. The proposed Range-Constrained Probability Modeling restricts predictions to valid coefficient tokens to improve coding efficiency, and the resulting probability distribution is then used by an arithmetic encoder to generate the bitstream.
Fig. 3: Prompt template used by FLM. For each macroblock, the template is instantiated with the corresponding channel, macroblock dimensions, quality factor, and minimum and maximum DC and AC coefficient values.
Hyperparameter
Stage I
Stage II
Training paradigm
SFT
SFT
Trainable parameters
Embedding/output head
LoRA adapters
LoRA targets
–
All linear projections
LoRA rank
–
32
Learning rate
5×10−4
1×10−4
Training epochs
1
1
TABLE I: Hyperparameters for two-stage fine-tuning.
Fig. 4: Rate-distortion performance comparison with existing methods on the Kodak, Tecnick, and CLIC2020 datasets.
Model
BD-PSNR (dB) ↑
BD-Rate ↓
Kodak
Tecnick
CLIC2020
Kodak
Tecnick
CLIC2020
JPEG
0
0
0
0%
0%
0%
DiffEIC [ 32 ]
2.24
3.14
2.49
−47.66%
−55.38%
−49.51%
PerCo * [ 37 ]
/
/
/
/
/
/
Control-GIC [ 36 ]
0.95
1.78
1.67
−32.17%
−42.30%
−40.49%
Diff-ICMH [ 38 ]
2.39
3.73
2.98
−50.14%
−55.92%
−52.39%
TABLE II: Comparison of BD-PSNR and BD-Rate than JPEG baseline on Kodak, Tecnick, and CLIC2020 datasets.
Fig. 5: Subjective visual quality comparison of reconstructions produced by different methods for the Kodak image kodim09.
Method
Q=65
Q=75
Q=85
Q=95
JPEG [ 8 ]
1.127
1.369
1.859
3.401
JPEG-XL [ 9 ]
0.960
1.173
1.595
2.849
Lepton [ 40 ]
0.896
1.102
1.520
2.786
LLJPEG [ 10 ]
0.894
1.107
1.536
2.853
CMIX [ 41 ]
0.853
1.054
1.452
2.648
Guo et al. [ 42 ]
0.778
0.965
1.341
2.500
TABLE III: Comparison with JPEG lossless recompression methods in different quality factors.
Fig. 6: Subjective visual quality comparison of reconstructions produced by different methods for the Kodak image kodim20.
Experiment
Model / Setting
Quality Factor
Δ BPP avg
5
35
55
75
BPP
Δ BPP
BPP
Δ BPP
BPP
Δ BPP
BPP
Δ BPP
Fine-tuning
Baseline
0.194
–
1.008
–
1.359
–
1.945
–
–
Fine-tune
0.070
−64.026%
0.433
−57.054%
0.605
−55.461%
0.897
−53.872%
−57.603%
Macroblock size
Baseline ( 16×8 )
0.194
–
1.008
–
1.359
–
1.945
–
–
8×4
0.204
+5.007%
1.054
+4.624%
1.431
+5.297%
2.058
+5.796%
+5.181%
TABLE IV: Ablation studies on fine-tuning strategies, macroblock sizes, and functional module designs.
University of Science and Technology of China, Hefei, China · MOE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition, University of Science and Technology of China, Hefei, China · Nanyang Technological University, Singapore, Singapore +2