Diffusion-based image compression has recently shown outstanding perceptual fidelity, yet its practicality is hindered by prohibitive sampling overhead and high memory usage. Most existing diffusion codecs employ U-Net architectures, where hierarchical downsampling forces diffusion to operate in shallow latent spaces (typically with only 8x spatial downscaling), resulting in excessive computation. In contrast, conventional VAE-based codecs work in much deeper latent domains (16x - 64x downscaled), motivating a key question: Can diffusion operate effectively in such compact latent spaces without compromising reconstruction quality? To address this, we introduce DiT-IC, an Aligned Diffusion Transformer for Image Compression, which replaces the U-Net with a Diffusion Transformer capable of performing diffusion in latent space entirely at 32x downscaled resolution. DiT-IC adapts a pretrained text-to-image multi-step DiT into a single-step reconstruction model through three key alignment mechanisms: (1) a variance-guided reconstruction flow that adapts denoising strength to latent uncertainty for efficient reconstruction; (2) a self-distillation alignment that enforces consistency with encoder-defined latent geometry to enable one-step diffusion; and (3) a latent-conditioned guidance that replaces text prompts with semantically aligned latent conditions, enabling text-free inference. With these designs, DiT-IC achieves state-of-the-art perceptual quality while offering up to 30x faster decoding and drastically lower memory usage than existing diffusion-based codecs. Remarkably, it can reconstruct 2048x2048 images on a 16 GB laptop GPU.
Figures & tables
Figure 2: Architectural comparison. The left panel illustrates the overall diffusion-based image compression framework. U-Net-based diffusers perform multi-stage downsampling, while DiTs maintain a constant spatial resolution throughout the denoising process, making them naturally compatible with deeply compressed latent inputs.
Figure 3: Overview of the proposed DiT-IC framework. Following StableCodec [ 64 ] , we adopt ELIC [ 16 ] as our auxiliary encoder.
Figure 4: Variance-Guided Flow Matching. Unlike standard diffusion that starts from Gaussian noise, compression reconstruction begins from a quantized latent yt containing structured noise. The local variance σ(yt) measures spatial uncertainty, which we map to pseudo-timesteps t=F(σ) for spatially adaptive one-step flow matching.
Figure 5: Ablation study of variance-guided reconstruction flow.
Figure 6: Self-Distillation Alignment. DiT-IC distills the multi-step diffusion process into a single forward pass by aligning its denoised latent with the frozen encoder representation, while jointly optimizing the diffusion transformer and decoder. yθ(⋅) denotes the denoised output, and Lrd indicates the rate–distortion loss.
Figure 7: Ablation of self-distillation alignment.
Figure 8: Latent-Conditioned Guidance. We replace text-based guidance in DiT with a latent-conditioned projection derived from the compressed representation by aligning projected latent and text embeddings, enabling text-free conditioning at inference.
Figure 9: Ablation of latent-conditioned guidance.
Figure 10: Visualization comparison. MSE-optimized ELIC [ 16 ] suffers from high-frequency detail loss, whereas diffusion-based codecs such as StableCodec [ 64 ] and OSCAR [ 15 ] produce inconsistent semantic content, e.g., incorrect numbers or window panes. In contrast, DiT-IC achieves a more favorable balance between perceptual quality and semantic consistency.
Figure 11: Rate-distortion-perception curve comparisons of different methods on the Kodak, CLIC2020 and DIV2K dataset.
Methods
Diff.
Code
Diff.
Params
Latency
LPIPS ↓
DISTS ↓
Average
Reso.
Reso.
Steps
Kodak
CLIC
DIV2K
Kodak
CLIC
DIV2K
LPIPS
DISTS
INR-based
C3-WD (CVPR’25) [ 2 ]
-
-
-
-
-
12.90
-47.14
-
62.25
-46.62
-
-17.12
7.82
VAE-based
MS-ILLM (ICML’23) [ 39 ]
-
f16d256
-
181M
0.17s
-38.13
-46.75
-39.52
11.54
-21.10
17.17
-41.47
2.54
EGIC (ECCV’24) [ 25 ]
-
f16d320
-
37M
-47.93
-67.75
-61.75
0.07
-60.69
-61.19
-59.14
-40.60
Table 1: Comprehensive comparison with state-of-the-art methods in terms of BD-rate ( ↓ ) [ 4 ] . “Diff. Reso.” and “Code Reso.” denote the latent resolutions used in the diffusion and coding stages, respectively, where f indicates the spatial downsampling factor relative to the pixel domain, and d denotes the number of channels. Latency is measured as the per-image decoding time (for 1024×1024 resolution) on a single A100 GPU; ♣ marks FP16 inference and ♠ marks FP32. “DiT-IC (baseline)” represents the variant without the proposed alignment strategies. The best results are highlighted in red , and the second-best in blue .
PSNR
MS-SSIM
LPIPS
DISTS
DiT-IC
0.00%
0.00%
0.00%
0.00%
Loss function
-w/o Ladv
-37.10%
-21.54%
-2.27%
-1.80%
-w/o DISTS
-2.15%
-1.30%
-1.83%
5.69%
Training Strategies
DiT from scratch
16.80%
13.41%
22.00%
32.45%
Table 2: Ablation study results measured by BD-rate ↓ [ 4 ] .
Reso.
VAE
Codec
Diffusion
StableCodec
-
SD-VAE [ 51 ]
f8 → f64
U-Net
DiT-IC
-
SANA-VAE [ 58 ]
f32 → f64
DiT
StableCodec
10242
0.19
0.04
0.11
DiT-IC
10242
0.21 (+11%)
0.008 (-30%)
0.055 (-50%)
StableCodec
20482
0.82
0.05
0.8
DiT-IC
20482
0.85 (+4%)
0.012 (-76%)
0.12 (-85%)
Table 3: Runtime latency (s) comparison in FP32 precision.
Figure 12: Overall architecture of our model. The entropy model is based on the hyperprior framework and an autoregressive context model similar to StableCodec [ 64 ] , but replaces heavy components with lightweight DepthConvBlocks [ 20 ] .
Figure 13: Illustrative VLM-generated captions used for semantic conditioning.
Datasets
Kodak
CLIC 2020
DIV2K
Average
Avg. Difference
Methods
DiT-IC
StableCodec
DiT-IC
StableCodec
DiT-IC
StableCodec
DiT-IC
StableCodec
∣Δ∣↑
∣Δ∣(%)↑
FID ↓
-
-
3.750
3.940
8.650
10.350
6.200
7.145
0.945
13.23%
KID ↓
-
-
0.00083
0.00066
0.00060
0.00080
0.00072
0.00073
0.00002
2.06%
NIQE ↓
3.099
3.557
3.833
4.459
3.270
3.603
3.400
3.873
0.473
12.21%
CLIPIQA ↑
0.735
0.716
0.582
0.531
0.626
0.570
0.648
0.606
0.042
6.92%
MUSIQ ↑
74.494
73.177
60.606
58.663
65.818
63.822
66.972
65.221
1.752
2.69%
Table 4: Quantitative perceptual comparison between DiT-IC and StableCodec ( λ=2.0 ) at similar bitrates ( ∼ 0.03–0.04 bpp). We report a suite of perceptual metrics including FID, KID, NIQE, CLIPIQA and MUSIQ. Lower is better for FID/KID/NIQE, and higher is better for the remaining metrics. DiT-IC consistently outperforms StableCodec across most datasets and metrics, with a slight exception on KID for CLIC2020. The best results are highlighted in red .
Figure 14: Detailed Rate-distortion-perception curve comparisons of different methods on the Kodak, CLIC2020 and DIV2K dataset.
Figure 15: DiT-IC achieves superior FID and Semantic accuracy.