Latent diffusion models decode approximately correct latents, making decoder sensitivity to perturbations relevant to image quality. After whitening, forward diffusion corrupts every direction at the same signal-to-noise ratio (SNR), despite unequal decoder sensitivities. We study an invertible linear layer between the encoder and diffusion process that shifts each direction's log-SNR. Its linear minimum mean-square-error (LMMSE) recovery risk has an exact spectral expression for any whitened latent distribution. Under an energy budget and a floor ensuring invertibility, minimizing decoder-sensitivity-weighted LMMSE risk integrated over the diffusion schedule yields a unique optimal allocation. The layer aligns with the decoder's expected pullback metric and equalizes marginal risk reduction across directions, computed by a deterministic spectral solver. At large budgets, the allocation follows a power law in sensitivity whose exponent is determined by the low-SNR tail of the loss weighting: one half for noise prediction and two thirds for data prediction under the cosine schedule, with a crossover on truncated schedules. We instantiate this principle as RB-LDC using Jacobian-based sensitivity estimates. On ImageNet at 256 x 256 resolution with SD-VAE and VA-VAE, trained RB-LDC improves reconstruction from noisy latents over budget-matched isotropic coding in all four metrics and five coding configurations, averaged over 15 SNRs and three pipeline seeds. It reduces average perturbed FID by 17.1 for VA-VAE block-local coding while preserving clean reconstruction. We also report intermediate SNRs where perturbed FID worsens. With a frozen diffusion model, RB-LDC yields lower FID point estimates in all 12 tested combinations of sampling progress and channel SNR under corruption during sampling.
Figures & tables
Image MSE ↓
t
Removed energy (%)
Original
High
Low
0.10
3.19
0.238641
0.156636
0.240446
0.25
3.09
0.168840
0.121811
0.168581
0.50
2.72
0.099320
0.080486
0.098988
0.75
2.18
0.042703
0.038128
0.042549
0.90
1.86
0.012755
0.011873
0.012677
Table 1: Oracle correction of DiT predictions on 256 images (seed 42). High and Low correct the top and bottom 16 eigendirections of the primary identity- Φ sensitivity estimate. Removed energy is the mean percentage of total whitened error energy removed per image, matched between corrections to numerical precision. Image MSE is averaged over images against their clean reconstructions; lower is better.
Figure 1: Slope of logλi⋆ against loghi for hi∝i−1 and k=256 on the discretized cosine schedule (1,000 steps, ρ∈[10−5.6,108] ), as a function of the budget per direction. SNR and min-SNR weighting approach the square-root law; uniform weighting stays near 2/3 until B/k exceeds ρmin−1 (Theorem 3.3 , Corollary 3.4 ).
Tokenizer, coding
k
Δ pFID
Δ PSNR
Δ SSIM
Δ LPIPS
SD-VAE, channel-mix
4
−1.74±0.20
+0.286±0.051
+0.0120±0.0021
−0.0099±0.0033
SD-VAE, block-local
16
−6.97±1.18
+0.700±0.063
+0.0292±0.0024
−0.0244±0.0032
VA-VAE, channel-mix
32
−2.59±1.28
+0.313±0.013
+0.0134±0.0008
−0.0142±0.0019
VA-VAE, block-local
128
−17.07±1.60
+0.855±0.040
+0.0337±0.0021
−0.0346±0.0022
VA-VAE, block-local, b=4
512
−22.31±4.57
+1.303±0.075
+0.0481±0.0035
−0.0420±0.0023
Table 2: Mean ± standard deviation across seeds 42–44 of the 15-SNR-average metric differences ( log10ρ∈[−4,3] , Φ equal to the identity). Differences are RB-LDC minus whitened isotropic coding. Lower is better for pFID and LPIPS; higher is better for PSNR (dB) and SSIM.
Figure 2: Seed-42 per-SNR differences between RB-LDC and isotropic coding on the primary configuration (VA-VAE, block-local, k=128 ). Arrows indicate the direction of improvement. The shaded range marks the intermediate SNRs at which pFID is higher for RB-LDC (Section 5 , Appendix K ).
average over 15 SNRs
log10ρ=+1
Variant
pFID
PSNR
SSIM
pFID
PSNR
SSIM
Δλ
A8
A: identity code, no whitening
0.00
0.00
0.0000
0.00
0.00
0.0000
–
–
B: whitening, isotropic code
−44.87
+1.22
+0.0401
−94.69
+2.35
+0.1159
–
–
C: whitening, dense G for R
−52.65
+1.66
+0.0576
−159.07
+3.97
+0.1849
0.322
0.698
D: RB-LDC
−60.27
+2.04
+0.0714
−200.48
+5.29
+0.2389
0.005
0.99994
Table 3: Seed-42 component analysis on the primary configuration. Average and log10ρ=+1 values are differences relative to variant A (lower pFID and higher PSNR and SSIM are better). Δλ is the relative error of the learned spectrum with respect to the allocation of Theorem 3.2 , and A8 is the alignment of the leading eight-dimensional eigenspaces (1 for identical subspaces). LPIPS is omitted for the reason given in Appendix L .
log10ρch
Progress
−1
0
+1
+2
25%
241.938 / 191.122
121.488 / 106.438
14.056 / 13.395
9.853 / 9.850
50%
323.545 / 235.477
147.836 / 124.612
13.438 / 12.440
9.770 / 9.710
75%
335.773 / 246.106
179.753 / 144.848
14.120 / 11.337
9.804 / 9.676
Table 4: FID after a single corruption during sampling with the frozen 400k-update LightningDiT-XL/1 model and the learned identity- Φ code ( k=B=128 ). Each entry is isotropic / RB-LDC for 5,000 samples. The matched no-injection FID is 9.612 . Progress is the fraction of sampling intervals completed; columns specify the added channel SNR.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Weighting
10−1
1
10
102
103
104
106
108
predicted
uniform ( z0 -prediction)
1.389
1.003
0.790
0.708
0.680
0.671
0.581
0.501
2/3
SNR ( ε -prediction)
0.641
0.593
0.543
0.516
0.505
0.502
0.500
0.500
1/2
min-SNR ( γms=5 )
1.047
0.681
0.561
0.520
0.507
0.502
0.500
0.500
1/2
Appendix
Table 5: Least-squares slope of logλi⋆ against loghi as a function of the budget per direction B/k . The last column is the exponent predicted by Corollary 3.4 for the untruncated schedule.
k
Φ
Δλ
A8
64
identity
0.00278
0.99997
64
random CNN
0.00247
0.99999
128
identity
0.00426
0.99982
128
random CNN
0.00346
0.99997
Appendix
Table 6: Recovery of the optimal allocation by the trained parameterization on a small VAE: relative spectral error Δλ and leading subspace alignment A8 ( r=8 ) after 3,000 steps.
Tokenizer, coding
k
Δ rFID
Δ pFID
Δ PSNR
Δ SSIM
Δ LPIPS
best Δ pFID ( log10ρ )
SD-VAE, channel-mix
4
−0.0002
−1.51
+0.23
+0.0101
−0.0061
−10.53 ( −2.5 )
SD-VAE, block-local
16
+0.0001
−8.33
+0.63
+0.0268
−0.0208
−38.44 ( −3.5 )
VA-VAE, channel-mix
32
+0.0007
−1.20
+0.32
+0.0139
−0.0164
−56.85 ( +1.5 )
VA-VAE, block-local
128
−0.0001
−15.40
+0.81
+0.0312
−0.0372
−105.79 ( +1.0 )
VA-VAE, block-local, b=4
512
−0.0002
−17.04
+1.22
+0.0442
−0.0447
−128.82 ( +0.5 )
Appendix
Table 7: Seed-42 differences between RB-LDC and isotropic coding, averaged over 15 SNRs ( log10ρ∈[−4,3] ), and the largest pFID improvement over the grid with the SNR at which it occurs. Lower is better for pFID and LPIPS; higher is better for PSNR (dB) and SSIM. Δ rFID is computed on clean latents.
Tokenizer, coding
k
cond(H^)
galign
gminimax
gpert
Δ pFID
SD-VAE, channel-mix
4
4.0
1.21
1.99
2.00
−1.74±0.20
SD-VAE, block-local
16
19.8
1.50
6.37
6.58
−6.97±1.18
VA-VAE, channel-mix
32
114.4
2.16
5.22
5.42
−2.59±1.28
VA-VAE, block-local
128
408.5
3.21
17.16
18.46
−17.07±1.60
VA-VAE, block-local, b=4
512
1013.2
3.39
48.38
53.38
−22.31±4.57
Appendix
Table 8: Measured sensitivity spectra from the seed-42 identity- Φ estimates. The pFID column repeats the 15-SNR-average differences over seeds 42–44 (mean ± standard deviation) from Table 2 . The three ratios are defined above.
log10ρ
Δ pFID
Δ PSNR (dB)
Δ SSIM
Δ LPIPS
+3.0
−0.121
+0.320
+0.0106
−0.0034
+2.5
−0.657
+0.819
+0.0268
−0.0156
+2.0
−3.707
+1.657
+0.0575
−0.0475
+1.5
−28.740
+2.571
+0.1011
−0.0983
+1.0
−105.791
+2.946
+0.1230
−0.1308
+0.5
−70.275
+2.303
+0.0770
−0.0799
Appendix
Table 9: Seed-42 per-SNR differences between RB-LDC and isotropic coding on the primary configuration (VA-VAE, block-local, k=128 , Φ equal to the identity). Entries marked with † favor isotropic coding (14 of 60).
Tokenizer and coding
k
Δ pFID
Δ PSNR (dB)
Δ SSIM
Δ LPIPS
SD-VAE, channel-mix
4
−10.53 ( −2.5 )
+0.80 ( +0.5 )
+0.032 ( 0.0 )
−0.037 ( −1.0 )
SD-VAE, block-local
16
−38.44 ( −3.5 )
+2.10 ( +0.5 )
+0.079 ( 0.0 )
−0.113 ( −1.0 )
VA-VAE, channel-mix
32
−56.85 ( +1.5 )
+1.66 ( +1.5 )
+0.070 ( +1.5 )
−0.097 ( −2.0 )
VA-VAE, block-local
128
−105.79 ( +1.0 )
+2.95 ( +1.0 )
+0.123 ( +1.0 )
−0.131 ( +1.0 )
VA-VAE, block-local ( b=4 )
512
−128.82 ( +0.5 )
+3.47 ( +1.0 )
+0.135 ( +1.0 )
−0.141 ( +1.0 )
Appendix
Table 10: Largest seed-42 improvement of RB-LDC over isotropic coding across the 15 SNRs, with the SNR ( log10ρ ) at which it occurs, for each configuration and metric ( Φ equal to the identity).
Figure 3: Seed-42 absolute pFID of isotropic coding and RB-LDC ( Φ equal to the identity) for block-local coding on SD-VAE ( k=16 , left) and VA-VAE ( k=128 , right), as a function of log10ρ on a logarithmic vertical axis.
Figure 4: Six validation images with the largest per-image PSNR improvement of RB-LDC over isotropic coding at log10ρ=+1 (VA-VAE, block-local, b=4 , k=512 , Φ equal to the identity). Rows show the originals, the isotropic reconstructions, the RB-LDC reconstructions, and the absolute errors of both reconstructions. Examples are selected by largest PSNR gain rather than to represent average performance.
log10ρ
Isotropic
RB-LDC
Difference
−4.0
379.867
375.764
−4.103
−3.5
371.459
366.420
−5.040
−3.0
364.674
355.354
−9.319
−2.5
362.612
326.848
−35.764
−2.0
343.666
307.511
−36.155
−1.5
329.293
307.834
−21.458
Appendix
Table 11: REPA-E reconstruction pFID on the fixed 5,000-image validation subset, which also serves as the reference set. The learned identity- Φ code and isotropic baseline use paired channel noise. Differences are RB-LDC minus isotropic.
Arithmetic
Isotropic
RB-LDC
Difference
fp32
1.976481
1.976924
+0.000442
fp64 coding path
1.976725
1.976725
0
Appendix
Table 12: REPA-E clean-rFID check on the same fixed 5,000-image subset. The precision refers to the whitening and coding path; the decoder is fp32 throughout.
FID
Paired CLIP
Progress
log10ρch
Isotropic
RB-LDC
Δ
Isotropic
RB-LDC
Δ±SE
25%
−1
241.938
191.122
−50.817
0.56400
0.59052
+0.02651±0.00068
25%
+0
121.488
106.438
−15.050
0.63151
0.64974
+0.01822±0.00099
25%
+1
14.056
13.395
−0.660
0.84321
0.87109
+0.02788±0.00101
25%
+2
9.853
9.850
−0.003
0.94012
0.95636
+0.01624±0.00066
50%
−1
323.545
235.477
−88.068
0.61354
0.61058
−0.00296±0.00047
Appendix
Table 13: Single-injection results with the learned identity- Φ code, frozen 400k-update generator, and 5,000 paired samples per cell. The no-injection FID on the matched sample set is 9.612 . CLIP compares each output with its no-injection counterpart; its difference is reported with the paired standard error (SE). Lower FID and higher CLIP are better.
FID
Paired CLIP
log10ρch
Isotropic
RB-LDC
Δ
Isotropic
RB-LDC
Δ
−4.0
337.85
331.19
−6.66
0.51603
0.51620
+0.00017
−3.5
323.74
317.66
−6.08
0.51567
0.51629
+0.00061
−3.0
314.10
313.62
−0.48
0.51560
0.51870
+0.00310
−2.5
315.75
312.67
−3.08
0.52085
0.53321
+0.01236
−2.0
321.18
319.19
−1.98
0.54413
0.55998
+0.01585
Appendix
Table 14: Post-generation latent transmission of 10,000 samples from the frozen 400k-update generator, using the analytic DINOv2-based code with eigenvalue ratio capped at 4 and weighted-Procrustes orientation ( k=B=128 ). The original clean generated sample set has FID 6.4284 . CLIP is paired image similarity to that set. Differences use unrounded values before display rounding.
Representation
DiT updates
Samples
gFID
Vanilla, no whitening or code
150k
10K
9.07
Whitened isotropic
150k
10K
12.25
Learned CNN- Φ RB-LDC
150k
10K
17.93
Dense G for R
150k
10K
21.06
Vanilla, no whitening or code
400k
50K
3.87
Whitened isotropic
400k
50K
6.45
Appendix
Table 15: Clean-generation FID with VA-VAE, LightningDiT-XL/1, training seed 42, and CFG 1.0. The panels distinguish the main generation comparison, analytic capped-spectrum variants with different orthogonal factors Q , and the separate 50k-update denoising-loss comparison. Sample counts are generated images; the reference contains 50K images throughout.
Metric
difference (RB-LDC − isotropic)
standard deviation across images
pFID
+33.170
set-level statistic
PSNR (dB)
+00.120
4.62
SSIM
+00.010
0.143
LPIPS
+00.008
0.029
Appendix
Table 16: Seed-42 differences at log10ρ=−1 on the primary configuration. The last column is the standard deviation of the metric across images.
Metric
difference across seeds
maximum p across seeds
pFID
+29.61±1.27
–
PSNR (dB)
+0.169±0.029
1.22×10−18
SSIM
+0.0138±0.0004
1.77×10−266
LPIPS
+0.00766±0.00069
2.93×10−10
Appendix
Table 17: Intermediate-SNR differences (RB-LDC minus isotropic) at log10ρ=−1 on the primary configuration with identity- Φ . Values are mean ± standard deviation across seeds 43–45. Each seed uses the same 5,000 images and noise for both methods. For per-image metrics, the last column is the largest two-sided paired Wilcoxon p -value across the three runs. pFID is a set-level statistic, summarized across seeds without a per-image test.
Quantity
Isotropic
D -
D
Spectrum error Δλ
–
0.007080
0.005537
Subspace alignment A8
–
0.999857
0.999941
Clean rFID
1.8206
1.8208
1.8208
15-SNR average Δ pFID
–
−16.251
−16.285
Peak Δ pFID ( log10ρ=+1 )
–
−105.848
−106.809
Appendix
Table 18: Balance-penalty ablation on the primary configuration. D - and D use training seed 42; all three variants use the same 5,000-image evaluation subset and channel-noise seed 12345. This subset and noise differ from the main evaluation in Table 3 , which uses noise seed 0. pFID differences are relative to the isotropic baseline within this evaluation.
log10ρ
Isotropic
Denoising-loss G
Difference
+3.0
2.228
321.094
+318.866
+2.5
3.319
315.333
+312.014
+2.0
8.996
310.966
+301.970
+1.5
47.984
316.357
+268.373
+1.0
194.020
339.710
+145.691
+0.5
316.654
346.315
+029.660
Appendix
Table 19: Reconstruction pFID of the denoising-loss code after 3,000 joint coding/DiT updates. Both variants use 5K images and paired noise within this evaluation. Its isotropic values are reported separately from the main evaluation in Table 9 .