cs.CVAug 26, 2026

FLM: Frequency-Aware Language Models for Generative Image Compression

Authors: Jiarun Chen, Kejun Wu, Li Li, Chengtao Cai, Zhengguo Li, Chia-Wen Lin

Organizations: School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China · MOE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition, University of Science and Technology of China, Hefei 230093, China · College of Intelligent Systems Science and Engineering, Harbin Engineering University, Harbin 150001, China · VI Department, Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Generative models have significantly improved the performance ceiling of image lossy compression at low bitrates by exploiting learned priors. However, the generated textures and semantic details may deviate from the source content, thereby affecting the fidelity of image reconstruction. To solve these challenges, we propose FLM, a frequency-aware language model that improves compression efficiency through frequency-domain probabilistic modeling while retaining deterministic reconstruction. At the encoder, the input image is transformed into quantized DCT coefficients, which are organized into discrete sequences using macroblock-based coefficient tokenization. FLM then performs next-coefficient prediction to autoregressively estimate token-wise conditional probability distributions for arithmetic coding, thereby generating a compact bitstream. At the decoder, the LLM and arithmetic decoder jointly recover the frequency-domain data, followed by inverse transformations for image reconstruction. A task-specific frequency-domain dataset and a two-stage fine-tuning strategy are further developed to enable the model to operate across multiple bitrate settings. FLM is a versatile compressor that is compatible with both lossy compression and lossless JPEG recompression frameworks. Experiments show that FLM exceeds conventional and generative lossy compression methods in rate-distortion performance. FLM achieves BD-PSNR gains of 3.30 dB, 3.83 dB, and 3.80 dB than JPEG baseline on Kodak, Tecnick, and CLIC2020, respectively. Better qualitative quality of FLM can be achieved in improving semantically high fidelity and suppressing blocking artifacts. FLM is also validated to be applicable to the lossless recompression task with competitive performance.

Figures & tables

Fig. 1: Conceptual comparison of representative image compression methods. Diffusion-based methods compromise reconstruction fidelity, whereas JPEG requires a higher bitrate. Our FLM method achieves both high fidelity and low bitrate.
Fig. 2: The overall framework of FLM. The input image is converted into quantized DCT coefficients by a JPEG-style transform, and the proposed Macroblock-Based DCT Coefficient Tokenization organizes the coefficients into token sequences. The task-prompt tokens and coefficient tokens are concatenated and fed into the LLM to predict the conditional distribution of the next token. The proposed Range-Constrained Probability Modeling restricts predictions to valid coefficient tokens to improve coding efficiency, and the resulting probability distribution is then used by an arithmetic encoder to generate the bitstream.
Fig. 3: Prompt template used by FLM. For each macroblock, the template is instantiated with the corresponding channel, macroblock dimensions, quality factor, and minimum and maximum DC and AC coefficient values.
HyperparameterStage IStage II
Training paradigmSFTSFT
Trainable parametersEmbedding/output headLoRA adapters
LoRA targets–All linear projections
LoRA rank–32
Learning rate5×10−45\times 10^{-4}1×10−41\times 10^{-4}
Training epochs11
TABLE I: Hyperparameters for two-stage fine-tuning.
Fig. 4: Rate-distortion performance comparison with existing methods on the Kodak, Tecnick, and CLIC2020 datasets.
ModelBD-PSNR (dB) ↑\uparrowBD-Rate ↓\downarrow
KodakTecnickCLIC2020KodakTecnickCLIC2020
JPEG0000%0\%0%0\%0%0\%
DiffEIC [ 32 ]2.243.142.49−47.66%-47.66\%−55.38%-55.38\%−49.51%-49.51\%
PerCo * [ 37 ]//////
Control-GIC [ 36 ]0.951.781.67−32.17%-32.17\%−42.30%-42.30\%−40.49%-40.49\%
Diff-ICMH [ 38 ]2.393.732.98−50.14%-50.14\%−55.92%\mathbf{-55.92\%}−52.39%-52.39\%
TABLE II: Comparison of BD-PSNR and BD-Rate than JPEG baseline on Kodak, Tecnick, and CLIC2020 datasets.
Fig. 5: Subjective visual quality comparison of reconstructions produced by different methods for the Kodak image kodim09.
MethodQ=65Q=65Q=75Q=75Q=85Q=85Q=95Q=95
JPEG [ 8 ]1.1271.3691.8593.401
JPEG-XL [ 9 ]0.9601.1731.5952.849
Lepton [ 40 ]0.8961.1021.5202.786
LLJPEG [ 10 ]0.8941.1071.5362.853
CMIX [ 41 ]0.8531.0541.4522.648
Guo et al. [ 42 ]0.7780.9651.3412.500
TABLE III: Comparison with JPEG lossless recompression methods in different quality factors.
Fig. 6: Subjective visual quality comparison of reconstructions produced by different methods for the Kodak image kodim20.
ExperimentModel / SettingQuality FactorΔ\Delta BPP avg
  5355575
  BPPΔ\Delta BPPBPPΔ\Delta BPPBPPΔ\Delta BPPBPPΔ\Delta BPP
Fine-tuningBaseline0.194–1.008–1.359–1.945––
 Fine-tune0.070−64.026%-64.026\%0.433−57.054%-57.054\%0.605−55.461%-55.461\%0.897−53.872%-53.872\%−57.603%-57.603\%
Macroblock sizeBaseline ( 16×816\times 8 )0.194–1.008–1.359–1.945––
 8×48\times 40.204+5.007%+5.007\%1.054+4.624%+4.624\%1.431+5.297%+5.297\%2.058+5.796%+5.796\%+5.181%+5.181\%
TABLE IV: Ablation studies on fine-tuning strategies, macroblock sizes, and functional module designs.
More in this topic

Learned Image Compression

Learned image compression uses neural encoders, quantization, and entropy models to reduce image bitrates while balancing fidelity, perceptual quality, and task utility.

See all 120 papers

Explore similar work

CardsList
Jun 19, 2026

Gen2-IC: Bridging Generative Models and Image Codecs through Latent Transport

Diffusion-based image compression has achieved strong perceptual quality at ultra-low bitrates. However, existing codecs are often tied to specific backbones and specialized components, making diverse, rapidly evolving generative models difficult to reuse. This raises a natural question: Can modern generative foundation models be connected to image compression through a simple and extensible interface? Two insights guide our design: stronger generative priors make a simpler codec interface viable, and generation and compression can be intrinsically linked through latent transport. We therefore propose Gen2-IC with two stages: (1) Latent Compression maps clean image latents to entropy-constrained latents; and (2) Latent Transport refines them with one near-terminal update based on the pretrained model. Gen2-IC requires neither auxiliary conditioning signals nor task-specific backbone modifications. With lightweight adaptation and no distillation, it supports fast encoding and one-step decoding across multiple bitrates. We validate Gen2-IC on SD-2.1, SANA-1.5, FLUX.1-dev, and Qwen-Image-2512, spanning U-Net and Transformer architectures as well as diffusion and flow-matching formulations. With stronger priors, Gen2-IC delivers gains below 0.05 bpp: the Qwen variant leads diffusion-based generative codecs in reconstruction fidelity (PSNR), perceptual similarity (LPIPS and DISTS), and recognizer-based semantic fidelity (OCR CER/WER and face-ROI similarity) across four benchmarks.

Yinhuan Huang, Hao Cao, Pu chen +3

Tsinghua University

Learned Image CompressionGenerative Image CompressionDiffusion Model Compression
AskPDF
Sep 30, 2026

Rethinking Generative Image Compression at Extremely Low Bitrates

Generative image compression produces visually plausible reconstructions at low bitrates, yet their behavior as the rate approaches zero remains largely unexplored. When pushed below normal operating rates, representative codecs undergo semantic collapse: rather than gracefully losing source-specific detail, they produce malformed or unrecognizable content. Our analysis identifies two factors. As the bitrate decreases, reconstruction losses increasingly conflict with semantic objectives on gradients and visual results, while pixel-space and reconstruction-oriented VAE diffusion models become less efficient on semantic preservation. Guided by these findings, we introduce RAE-CoD, a compression-oriented diffusion (CoD) built in a representation autoencoder (RAE) space with direct alignment between compressed and source representations, preserving recognizable, naturally structured content for a 256×256256\times256 image with as few as 16 bits. We evaluate this framework using five vision foundation models (VFM) and a blinded vision-language model protocol. On MSCOCO-30K, RAE-CoD stands out from all evaluation. At 0.001-0.008 bpp, it reduces relative VFM feature MSE and Fréchet Distance ratio by at least 25.7% and 69.1% over the best competitors. Meanwhile, semantic recognizability and quality of the reconstructions remain nearly constant while source consistency falls smoothly, replacing abrupt semantic collapse with a graceful transition toward unconditional generation. Code will be released at https://github.com/LuizScarlet/RAE-CoD.

Tianyu Zhang, Zhaoyang Jia, Houqiang Li +1

University of Science and Technology of China

Learned Image CompressionAutoencodersGenerative Image CompressionDiffusion Model Compression
AskPDF
Oct 15, 2026

WaveLLM: Wavelet-Domain LLM Entropy Modeling for Lossless Image Compression

Lossless image compression fundamentally relies on accurate entropy modeling, driving a recent shift toward large language model (LLM)-based methods. However, existing approaches mainly emphasize the modeling capacity of LLMs while directly serializing images into pixel or residual sequences, leaving heterogeneous signal patterns entangled within a single prediction stream. In this paper, we argue that effective entropy modeling depends not only on model capacity, but also on the predictability of the input representation. Motivated by this insight, we propose WaveLLM, a wavelet-domain LLM-based entropy model for lossless image compression. By decomposing an image into multi-level wavelet subbands, WaveLLM converts a heterogeneous prediction problem into a set of more structured and statistically homogeneous subproblems that are more amenable to autoregressive probability estimation. To bridge numerical coding symbols with the LLM's native text token space, we introduce a dedicated numerical interface featuring value embeddings and a coefficient prediction head. Furthermore, a task-adaptive prompting strategy is designed to enable a single shared model to adapt to diverse subbands within a unified framework. Built on Llama-3.2-3B with LoRA adaptation, WaveLLM achieves 2.51, 2.24, and 2.27 bpsp on Kodak, CLIC Pro, and DIV2K, respectively, outperforming the PixelCNN-based codec using the same wavelet transform by 0.43, 0.36, and 0.46 bpsp, and surpassing recent LLM-based lossless compressors on all three benchmarks.

Yue Yu, Bowei Kang, Changsheng Gao +2

University of Science and Technology of China, Hefei, China · MOE Key Laboratory of Brain-Inspired Intelligent Perception and Cognition, University of Science and Technology of China, Hefei, China · Nanyang Technological University, Singapore, Singapore +2

Entropy ModelingLossless Data CompressionLearned Image Compression
AskPDF