Learning Clean Generative Processes from Noisy Measurements
Organizations: Department of Mathematics and Computer Science, University of Basel
Abstract
We study how to learn a generative model of clean images using only noisy observations. Diffusion processes and their generalizations to non-Gaussian noise require posterior predictions at every noise level. We first show that it is possible to learn these predictions from independent noisy views using Noise2Noise regression, but only at or above the noise level of the training data. To learn the predictions below that level, needed to generate clean images, we impose consistency: a prediction should equal its expectation after continuing the sampling process. We derive conditions under which consistency uniquely determines the continuation to the clean image distribution and verify them for Poisson, binomial, Gamma, and negative-binomial measurements. Experiments on CIFAR-10, photon-counting microscopy, and synthetic aperture radar show that the required posterior predictions can be learned from noisy views alone and that the consistency relation repairs the continuation beyond the noise level of the data, which severely deteriorates without consistency.
Figures & tables
| Measurement | given | Dynamics given | Dynamics given |
|---|---|---|---|
| Gaussian | drift | ||
| Poisson | pixel gains a count at rate | ||
| Binomial ( ) | pixel gains a count at rate | ||
| Gamma | pixel jumps by at rate | ||
| Negative binomial | pixel jumps by at rate |
| Continue past | ||||
| Process | Training Measurement | Stop at | No Consistency | Consistency |
| Poisson | 613.1 counts | |||
| Poisson | 178.4 counts | |||
| Poisson | 16.1 counts | |||
| Binomial | 180.1 counts | |||
| Binomial | 100.0 counts | |||
| Method | FID | FID |
|---|---|---|
| GAP ( ) | 98.1 | 58.4 |
| GAP ( ) | 97.2 | 55.8 |
| N2N, stop at one frame | 114.1 | 55.7 |
| N2N, continue to 512 frames | 267.4 | 339.4 |
| N2N consistency loss, continue to 512 frames |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Result | Location |
|---|---|
| Theorem 3.1 | Item 1: Corollary D.5 . Item 2: Corollary E.4 on |
| Noisy-target regression | Lemma C.4 and Proposition C.6 |
| Injectivity of degradation | Proposition C.2 |
| Identification from a positive-level marginal | Theorem C.3 |
| Continuation from a common density ratio | Corollary D.5 |
| Uniqueness for direct predictions | Theorem E.1 and Corollary E.4 |
| Symbol | Meaning |
|---|---|
| Clean image and its distribution | |
| , | Number of pixels, and the image that is one at pixel and zero elsewhere |
| ; | Generic information level; Poisson exposure, binomial detection probability, Gamma shape, and negative-binomial shape |
| , , | Observation at level , its conditional law given , and its marginal law |
| , | The observation for count and for Gamma observations |
| Known degradation kernel from to |
| Model | Common density ratio | Direct predictions | Training from one level |
|---|---|---|---|
| Gaussian | Any image law, under Assumption D.1 | Not treated here | Not treated here; see Remark F.1 |
| Poisson | Any image law | Bounded rates and bounded images; alternatively, linearly growing rates | Bounded rates, Corollary F.3 |
| Binomial | Any image law, with | Bounded counts; alternatively, linearly growing rates | Bounded rates on , Corollary F.3 |
| Gamma | Joint Laplace representation and Assumption D.1 | Bounded positive image values, Lipschitz dependence on the state, and a bound on large jumps | In addition, the integrability and Lipschitz conditions in Corollary F.5 |
| Negative binomial | Any image law | and geometrically bounded posterior moments | Every moment index is covered, Corollary F.3 |
| Poisson | Binomial | Neg. binomial | Gamma | Conv-PC | SAR | |
|---|---|---|---|---|---|---|
| N2N loss | I-divergence, -weighted | I-divergence | squared, variance-weighted | BCE squared mean | I-divergence, -weighted | I-divergence kernel, Bregman mean with |
| N2N levels | uniform , atom | uniform , atom | power law, atom | uniform ; refinement adds atom | ||
| Path steps | 32 | 4 | 32 | 16 | 4 | 24 |
| Path endpoints | uniform | uniform | log-uniform | log-uniform | uniform | log-uniform |
| Path starts per device batch | 4 | 4, every third step | ||||
| 8 | 1 | 8 | 0.3 | 0.003 | 1 |
| Process | Timestep | Law parameter | Counts at a white pixel |
| Poisson | |||
| Poisson | |||
| Poisson | |||
| Binomial | |||
| Binomial | |||
| Binomial |
| Counts at a white pixel | |||
|---|---|---|---|
| Law assumed for the measurements | 613.1 | 178.4 | 16.1 |
| Poisson (ours) | |||
| Gaussian, one variance | |||
| Gaussian after Anscombe | |||
| Method | PSNR (dB) |
|---|---|
| GAP, split | |
| N2N, split, three seeds | |
| N2N, 511-frame target, 256-pixel crops | |
| GAP, 511-frame target |
| Seed | Sampler | Pixel | |||
|---|---|---|---|---|---|
| 1 | Joint sampler | ||||
| 1 | Independent sampler | ||||
| 0 | Joint sampler | ||||
| 0 | Independent sampler |
| Refinement | Seed 1 | Seed 0 |
|---|---|---|
| None, unrefined checkpoint | ||
| Neither term, same budget | ||
| Rollout regression only | ||
| Consistency loss only | ||
| Both |
| Estimator of | residual mean | residual KS |
|---|---|---|
| Ours, posterior mean | ||
| Lee |
Generative Modeling
Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.
See all 542 papers