Source-Anchored Visual Autoregression for Extreme Image Compression
Organizations: Department of Mathematics, HKUST · Huawei Hong Kong Research Center · Tsinghua University
Abstract
In the extreme-low bitrate range below bpp, image compression becomes a constrained generation problem: the reconstruction depends on inferring plausible missing details while staying close to the input image. Autoregressive token models provide an efficient coarse-to-fine prior for this setting, but the transmitted early-scale tokens often leave object identity and texture underdetermined, which can lead to source-inconsistent reconstructions. We introduce Source-Anchored Visual Autoregression (SAVAR), a generative codec built on Visual Autoregressive (VAR) modeling. SAVAR transmits early-scale tokens and a compact caption. The tokens provide image-specific layout information, while the caption supplies global semantics. SAVAR reconstructs a partial VQ-VAE latent from the retained scales, maps it to the start-of-sequence initialization, and uses the same latent to condition the caption embedding before adaptive layer normalization, separating two roles that are typically driven by a shared condition in VAR-style generation. On the Kodak and CLIC2020 datasets in the extreme-low range, SAVAR reduces perceptual distortion relative to matched ARPC retained-scale operating points, lowering LPIPS by about -- and DISTS by up to at a lower bpp. It reconstructs a image in about ms on one NVIDIA L20X GPU, retaining fast autoregressive decoding in the evaluated setting.
Figures & tables
| Kodak (24 imgs) Architecture Method Operating point BPP PSNR MS-SSIM LPIPS DISTS FID KID Autoregressive ARPC [ 35 ] 4 retained scales 0.002710 14.99 0.376 0.530 0.278 – – SAVAR 4 retained scales 0.002615 15.87 0.381 0.482 0.219 – – ARPC [ 35 ] 5 retained scales 0.005690 16.47 0.449 0.470 0.213 – – SAVAR 5 retained scales 0.004263 16.52 0.419 0.427 0.187 – – Generative/perceptual PerCo [ 6 ] 0.003630 15.14 0.377 0.560 0.242 – – MS-ILLM [ 25 ] 0.007899 20.23 0.593 0.502 0.339 245.01 0.0426 DLF [ 31 ] 0.008928 18.65 0.587 0.297 0.160 110.78 – DiffC [ 33 ] 0.004684 16.91 0.451 0.512 0.228 – – DiffC [ 33 ] 0.007014 18.31 0.545 0.428 0.199 – – |
| CLIC2020 (428 imgs) Architecture Method Operating point BPP PSNR MS-SSIM LPIPS DISTS FID KID Autoregressive ARPC [ 35 ] 4 retained scales 0.002670 13.94 0.374 0.506 0.274 116.44 0.0062 SAVAR 4 retained scales 0.002547 14.82 0.396 0.465 0.214 97.10 0.0017 ARPC [ 35 ] 5 retained scales 0.005530 15.55 0.458 0.445 0.211 102.52 0.0042 SAVAR 5 retained scales 0.004194 15.59 0.443 0.408 0.188 90.87 0.0007 Generative/perceptual PerCo [ 6 ] 0.003640 14.14 0.402 0.546 0.251 103.86 0.0011 MS-ILLM [ 25 ] 0.008757 20.38 0.654 0.461 0.309 97.73 0.0356 DLF [ 31 ] 0.009380 18.40 0.644 0.266 0.156 32.20 0.0024 DiffC [ 33 ] 0.004700 16.80 0.519 0.469 0.223 108.57 0.0028 DiffC [ 33 ] 0.006910 18.11 0.607 0.383 0.190 89.98 0.0005 |
| Variant | source | AdaLN source | PSNR | LPIPS | MS-SSIM | LPIPS vs. full |
|---|---|---|---|---|---|---|
| SAVAR | tokens | retained-scale | 16.52 | 0.4269 | 0.4191 | 0.0000 |
| w/o Token-Based Initialization | text | retained-scale | 11.9847 | 0.6964 | 0.1451 | +0.2695 |
| w/o Retained-Scale AdaLN | tokens | text | 15.7596 | 0.5006 | 0.3592 | +0.0737 |
| w/o Both | text | text | 11.4489 | 0.7164 | 0.1393 | +0.2895 |
| Method | Total Enc. (s) | Dec. Time (s) | BD-Rate |
|---|---|---|---|
| MS-ILLM [ 25 ] | |||
| PerCo [ 6 ] | |||
| DLF [ 31 ] | |||
| DiffC [ 33 ] | – | – | |
| ARPC [ 35 ] | – | ||
| SAVAR |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Variant | Caption | BPP | PSNR | LPIPS | MS-SSIM | LPIPS vs. full-correct |
|---|---|---|---|---|---|---|
| Full | correct | 0.004263 | 16.52 | 0.4269 | 0.4191 | 0.0000 |
| Full | no caption | 0.003021 | 15.9850 | 0.4480 | 0.3895 | +0.0211 |
| Full | generic caption | 0.004263 | 15.9751 | 0.4488 | 0.3898 | +0.0219 |
| Full | shuffled caption | 0.004263 | 16.0246 | 0.4504 | 0.3901 | +0.0235 |
| only | correct | 0.004263 | 15.7596 | 0.5006 | 0.3592 | +0.0737 |
| only | no caption | 0.003021 | 15.7575 | 0.4910 | 0.3631 | +0.0641 |
| Architecture | Method | Operating point | BPP | PSNR | MS-SSIM | LPIPS | DISTS | FID | KID | CLIP-I |
|---|---|---|---|---|---|---|---|---|---|---|
| Autoregressive | ARPC [ 35 ] | 4 retained scales | 0.00394 | 12.41 | 0.3333 | 0.5642 | 0.3336 | 84.73 | 0.05367 | 0.5757 |
| SAVAR | 4 retained scales | 0.00295 | 13.55 | 0.2963 | 0.5606 | 0.2653 | 54.41 | 0.03172 | 0.6370 | |
| ARPC [ 35 ] | 5 retained scales | 0.00684 | 13.86 | 0.4048 | 0.5024 | 0.2524 | 58.80 | 0.04678 | 0.6581 | |
| SAVAR | 5 retained scales | 0.00460 | 14.78 | 0.3880 | 0.4359 | 0.1839 | 4.21 | 0.00090 | 0.8422 | |
| Generative/perceptual | PerCo [ 6 ] | 0.00364 | 13.99 | 0.3832 | 0.5538 | 0.2452 | 5.24 | 0.00203 | 0.8084 | |
| MS-ILLM [ 25 ] | 0.00516 | 19.90 | 0.6320 | 0.4630 | 0.3016 | 54.23 | 0.04439 | 0.6800 |
| Dataset | Method | Operating point | BPP | DISTS | CLIP-I | FID | KID |
|---|---|---|---|---|---|---|---|
| Kodak-24 | SAVAR | 4 retained scales | 0.002615 | 0.2190 | 0.8088 | – | – |
| SAVAR | 5 retained scales | 0.004263 | 0.1867 | 0.8478 | – | – | |
| ARPC | 4 retained scales | 0.002710 | 0.2780 | 0.7940 | – | – | |
| ARPC | 5 retained scales | 0.005690 | 0.2130 | 0.8424 | – | – | |
| PerCo | low | 0.003630 | 0.2421 | 0.8183 | – | – | |
| PerCo | mid | 0.032930 | 0.1645 | 0.9100 | – | – |
| Operating point | Token count | Token bpp | Caption bpp | Total bpp |
|---|---|---|---|---|
| 4 retained scales | 30 | 0.001373 | 0.001242 | 0.002615 |
| 5 retained scales | 66 | 0.003021 | 0.001242 | 0.004263 |
| Method | Operating points used | Notes |
|---|---|---|
| MS-ILLM | , , , , | Anchor curve |
| PerCo | , | Includes the adjacent above- bpp point |
| DLF | , | Includes the adjacent above- bpp point |
| DiffC | , | Both points are below bpp |
| ARPC | , | Matched retained-scale points |
| SAVAR | , | Token-plus-caption total bpp |
| Item | Setting |
|---|---|
| Image split | ImageNet ILSVRC train / val |
| Text source | BLIP-2 captions generated from each image |
| Image preprocessing | Short-side resize and center crop to |
| Epochs / steps | 15 epochs / 37,245 optimizer steps |
| Global batch size | 512 |
| Optimizer | AdamW, betas , fused implementation |
Learned Image Compression
Learned image compression uses neural encoders, quantization, and entropy models to reduce image bitrates while balancing fidelity, perceptual quality, and task utility.
See all 133 papers