PEEK: One-Step-Look-Ahead Exposure Bias Correction for Diffusion Sampling
Organizations: National University of Singapore
Abstract
Diffusion image generators owe much of their practical success to accurate denoising networks and strong numerical samplers. However, during sampling, the denoiser predicts the next step from flawed inputs that differ substantially from real data. This train-test gap, or exposure bias, caps image generation quality (i.e., blurry images, missing details, or distorted shapes). To mitigate the problem, we propose PEEK, a training-free deterministic sampler plugin. At each eligible denoising step, PEEK first performs a provisional one-step update, evaluates the denoiser once at the look-ahead state, and interpolates the current clean prediction with this one-step-later prediction to form the next noisy state. By taking the additional step, we obtain a generally more image like prediction from the higher-signal noisy state, mitigating exposure bias. To optimize the interpolation positions, PEEK searches for an interpolation schedule with a greedy metric-driven search made efficient by the PEEK prefix and baseline noise-schedule-rebasing caches. Across pixel-space and latent-space diffusion models with unconditional, label-guided, and text-guided settings on CIFAR-10, LSUNBedroom,I mageNet, and MS-COCO,the look-ahead correction consistently improves over baselines matched by number of function evaluations (NFE), with Fréchet Inception Distance (FID) reductions up to 49.3%. The one-time per-model calibration, the only extra cost, takes at most 2.2 minutes in our experiments. Project website is available at https://dachengruoque.github.io/PEEK-Video/.
Figures & tables
| Method | LSUN FID (Base NFE 6) | Recall | Coverage |
|---|---|---|---|
| DEIS, NFE-Matched | 20.57 | 0.521 | 0.558 |
| DPM-Solver++, NFE-Matched | 15.95 | 0.567 | 0.656 |
| UniPC, NFE-Matched | 12.11 | 0.601 | 0.773 |
| Heun’s 2nd, NFE-Matched | 51.69 | 0.369 | 0.254 |
| Restart-style repeat | 30.97 | 0.330 | 0.400 |
| PEEK, DEIS | 11.38 (44.7%) | 0.614 | 0.744 |
| (a) Additional Dataset FID | |||
|---|---|---|---|
| Dataset | Base NFE | NFE-Matched FID | PEEK FID |
| CIFAR-10 (Model A, 9 steps) | 9 | 25.80 | 20.23 (21.6%) |
| CIFAR-10 (Model A, 20 steps) | 20 | 18.16 | 15.05 (17.1%) |
| CIFAR-10 (Model B, 9 steps) | 9 | 19.87 | 16.46 (17.2%) |
| CIFAR-10 (Model B, 20 steps) | 20 | 12.23 | 10.04 (17.9%) |
| ImageNet | 9 | 20.40 | 19.64 |
| Dataset / model | Resolution | Solver | Base NFE | NFE-Matched FID | PEEK FID |
| ImageNet-EDM | DPM-Solver++ | 6 | 19.71 | 9.99 (49.3%) | |
| ImageNet-EDM | DEIS | 6 | 10.13 | 7.29 (28.0%) | |
| LSUN-Bedroom-EDM | DPM-Solver++ | 5 | 22.89 | 11.77 (48.6%) | |
| LSUN-Bedroom-EDM | DEIS | 5 | 12.10 | 8.79 (27.4%) | |
| LSUN-Bedroom-EDM | iPNDM | 5 | 15.86 | 10.69 (32.6%) |
| Dataset | Base NFE | NFE-Matched FID | PEEK-G FID |
|---|---|---|---|
| CIFAR-10 (Model B) | 9 | 17.48 | 13.18 (24.6%) |
| Basic LSUN model | 6 | 15.95 | 15.60 |
| ImageNet | 9 | 20.40 | 19.68 |
| MS-COCO | 9 | 14.41 | 10.66 (26.0%) |
| Method/ | NFE | DPM-Solver++ | DEIS |
|---|---|---|---|
| PEEK 01111 (Greedy Search Schedule) | 10 | 11.59 | 11.38 |
| PEEK 10111 | 10 | 12.63 | 13.34 |
| PEEK 11011 | 10 | 13.82 | 14.39 |
| PEEK 11101 | 10 | 13.50 | 13.51 |
| PEEK 11110 | 10 | 13.90 | 13.36 |
| NFE-Matched Baseline | 11 | 14.43 | 18.45 |
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Evaluation sampler | Base NFE | Total NFE | |
| PEEK | ||||
| Unconditional Pixel-Space Generation on CIFAR-10 | DPM-Solver++ | 9 | 17 | |
| Unconditional Pixel-Space Generation on CIFAR-10 (EMA, PEEK) | DPM-Solver++ | 9 | 11 | |
| Unconditional Pixel-Space Generation on CIFAR-10 (EMA, PEEK) | DPM-Solver++ | 20 | 23 | \begin{array}[t]{@{}l@{}}(0,0,0,0,0,0,0,\\ 0,0,0,0,0,0,0,\\ 0,0,1,1,1)\end{array} |
| Unconditional Pixel-Space Generation on Basic LSUN Bedroom Model | DPM-Solver++ / DEIS | 6 | 10 | |
| Class-Conditional Latent Generation on ImageNet | DPM-Solver++ | 9 | 13 | |
Diffusion Model Sampling
Diffusion model sampling studies how to generate high-quality samples efficiently by designing reverse processes, solvers, and inference-time controls.
See all 90 papers