We study how to learn a generative model of clean images using only noisy observations. Diffusion processes and their generalizations to non-Gaussian noise require posterior predictions at every noise level. We first show that it is possible to learn these predictions from independent noisy views using Noise2Noise regression, but only at or above the noise level of the training data. To learn the predictions below that level, needed to generate clean images, we impose consistency: a prediction should equal its expectation after continuing the sampling process. We derive conditions under which consistency uniquely determines the continuation to the clean image distribution and verify them for Poisson, binomial, Gamma, and negative-binomial measurements. Experiments on CIFAR-10, photon-counting microscopy, and synthetic aperture radar show that the required posterior predictions can be learned from noisy views alone and that the consistency relation repairs the continuation beyond the noise level of the data, which severely deteriorates without consistency.
Figures & tables
Figure 1: The Poisson observation process of one image. The training data are counts Yin at one level ain . Below ain , the known kernels degrade them into N2N inputs (blue). Generation runs with the learned predictions from the empty state at a=0 (green arrows), and only the learned process reaches levels above ain . There, the consistency loss matches the prediction at ain with its average along simulated paths (green, Section 2.2 ). The tiles above ain show the true continuation of this image, and the three at a=64 are independent continuations of the same measurement.
Measurement
Za given X
Dynamics given X
Dynamics given Za
Gaussian
N(aX,aId)
drift X
E[X∣Za]
Poisson
Pois(aX)
pixel i gains a count at rate Xi
E[Xi∣Na]
Binomial ( a<1 )
Binomial(X,a)
pixel i gains a count at rate 1−aXi−Na,i
1−aE[Xi∣Na]−Na,i
Gamma
Gamma(a,1/X)
pixel i jumps by w>0 at rate e−w/Xidw/w
E[e−w/Xi∣Sa]dw/w
Negative binomial
NB(a,q(X))
pixel i jumps by k≥1 at rate qi(X)k/k
E[qi(X)k∣Na]/k
Table 1: Measurement laws and the dynamics they induce. Gamma distributions are parameterized by shape and rate, and NB(a,q) has shape a and mean aq/(1−q) . The noise acts independently across pixels given X , and the level a is written λ , p , L , r in the appendices. The last column replaces the function of the clean image in the dynamics by its posterior prediction. The binomial image is integer-valued, and q(X) is a known pixelwise encoding with values in [0,1) .
Continue past ain
Process
Training Measurement
Stop at ain
No Consistency
Consistency
Poisson
613.1 counts
7.30±0.60
30.66±0.03
3.98±0.13
Poisson
178.4 counts
9.06±0.06
63.80±9.24
4.34±0.03
Poisson
16.1 counts
17.23±0.44
109.14±11.19
8.19±0.30
Binomial
180.1 counts
9.41±0.06
61.17±2.48
11.65±0.12
Binomial
100.0 counts
11.68±0.51
95.23±7.47
11.63±1.65
Table 2: CIFAR-10 FID from 50,000 EMA samples against the clean training set, mean ± standard deviation over two training seeds. All models train for 2000 epochs, where runs with the consistency loss use positive weight ( λcons>0 ) for the last 200. The values for stopping at ain are the predicted posterior means of the λcons=0 model, listed for reference. Counts are expected counts at a white pixel. Rescoring one checkpoint with a fresh set of 50,000 samples moves FID by 0.143 . Without the consistency loss the Gamma trajectory diverges (—).
Method
FID 642
FID 2562
GAP ( β=0.1 )
98.1
58.4
GAP ( β=0.05 )
97.2
55.8
N2N, stop at one frame
114.1
55.7
N2N, continue to 512 frames
267.4
339.4
N2N + consistency loss, continue to 512 frames
93.3±2.5
44.7±2.5
Table 3: Generation on Conv-PC : FID from 10,000 images of 256×256 pixels per method, and from one 64×64 crop of each, against crops of the same sizes from the 512-frame sums. Every method returns a predicted image. N2N sampling starts from zero counts and stops at the indicated exposure. GAP follows its own accumulation rule. The consistency row gives mean ± standard deviation over three trainings started from one checkpoint and supervised by the consistency loss. Other rows are single runs.
Figure 2: Reconstructions and samples from one measured microscopy frame. (a) The input: 5,020 photons in 256×256 pixels. (b, c) Predicted images from N2N and GAP. (d) The 512-frame reference. (e) N2N continuation to the reference exposure without the consistency loss. (f) GAP continuation under its own accumulation rule, which stops at 0.32 photons per pixel. (g) N2N continuation to the reference exposure after training with the consistency loss. (h) Pixelwise standard deviation divided by mean over 500 continuations sampled after training with the consistency loss, on a color scale from 0 to 0.66. Panels (b)–(g) are divided by their means and clipped at four.
Figure 3: Conditional radar samples at L=16 ( 32Lobs ). (a) Observed half-look. (d) Posterior mean. (b, e) An independent and a joint sample. (c, f) Standard deviations over 50 draws, with a shared color bar. The lower speckle level makes the sampled spatial structure visible. Independent draws vary from pixel to pixel, whereas the joint samples vary in patches. The joint sampler underestimates the absolute spread beyond 16Lobs (Appendix H.4 ). Images are shown at ground aspect, with range along the short axis.
Bounded positive image values, Lipschitz dependence on the state, and a bound on large jumps
In addition, the integrability and Lipschitz conditions in Corollary F.5
Negative binomial
Any image law
qi(X)≤qˉ<1 and geometrically bounded posterior moments
Every moment index is covered, Corollary F.3
Appendix
Table 6: Scope of the continuation results. These are exact statements about the true posterior predictions and expected losses. Starting the Gamma process from zero additionally requires the initialization assumption in Appendix C.1.4 .
Poisson
Binomial
Neg. binomial
Gamma
Conv-PC
SAR
N2N loss
I-divergence, δτ -weighted
I-divergence
squared, variance-weighted
BCE + squared mean
I-divergence, δτ -weighted
I-divergence kernel, Bregman mean with ϕ′′(v)=v−1/2
N2N levels
uniform τ , atom τobs/T
uniform τ , atom τobs/T
power law, atom 0.5
Lobsu3
uniform τ≥τmin ; refinement adds atom τmin/T
Lobsu3/2
Path steps
32
4
32
16
4
24
Path endpoints
uniform τ<τobs
uniform τ<τobs
log-uniform (robs,10robs]
log-uniform (Lobs,50Lobs]
uniform τ<τmin
log-uniform (Lobs,50Lobs]
Path starts per device batch
≤1
≤8
≤1
4
≤8
4, every third step
λcons
8
1
8
0.3
0.003
1
Appendix
Table 7: Training and sampling settings. The CIFAR-10 models share the U-Net, batch size 128, learning rate 2⋅10−4 and EMA decay 0.9999 . Paths start at ain in every experiment. In the N2N rows, δτ is the exposure added by the sampling step at timestep τ , T=1000 is the number of timesteps, τobs is the timestep of ain , and u is uniform on (0,1) . The radar model simulates four paths and averages the residuals in pairs before taking their inner product. The Conv-PC model is a 35M-parameter U-Net trained for 100 epochs of 500 steps on 128-pixel crops of single frames, with batch size 32, learning rate 10−4 and EMA decay 0.9999 . For this model, ain lies at timestep τmin=634 of 1000, where the input holds 98% of a frame.
Process
Timestep
Law parameter
Counts at a white pixel
Poisson
τ=440
α=1.3⋅10−2
610
Poisson
τ=500
α=3.8⋅10−3
180
Poisson
τ=600
α=3.5⋅10−4
16
Binomial
τ=132
p=0.71
180
Binomial
τ=226
p=0.39
100
Binomial
τ=364
p=0.098
25
Appendix
Table 8: The training levels ain of Table 2 , as parameters of the observation law: the counts are λmaxα with λmax=4.65⋅104 for the Poisson process and 255p for the binomial one. Table 2 quotes the exact values to one decimal. Here they are rounded. The negative-binomial and Gamma settings have no count axis and are placed on an equal-accuracy scale in the text.
Counts at a white pixel
Law assumed for the measurements
613.1
178.4
16.1
Poisson (ours)
6.50±0.42
8.62±0.03
16.74±0.18
Gaussian, one variance
17.49±0.77
25.58±2.73
110.02±1.91
Gaussian after Anscombe
21.13±0.74
26.16±0.83
111.03±13.47
Appendix
Table 9: A misspecified measurement law. All models stop at the noise level of the training measurements and return the predicted posterior mean. FID from 50,000 samples, mean ± standard deviation over two training seeds. All three models are 1800-epoch checkpoints.
Method
PSNR (dB)
GAP, split
24.917±0.005
N2N, split, three seeds
25.037±0.044
N2N, 511-frame target, 256-pixel crops
26.836±0.010
GAP, 511-frame target
25.805±0.009
Appendix
Table 10: Single-frame Conv-PC restoration under transductive training. The row with three training seeds reports the standard deviation across seeds. The other rows report the standard error across the 512 evaluated frames. The published GAP result for the split setting is 24.84 dB.
Seed
Sampler
Pixel
4×4
8×8
16×16
1
Joint sampler
0.004/89.9
0.013/89.9
0.026/89.6
0.048/88.6
1
Independent sampler
0.002/90.3
0.029/88.3
0.056/85.9
0.100/82.5
0
Joint sampler
0.004/89.8
0.013/89.5
0.019/89.4
0.030/89.2
0
Independent sampler
0.002/90.3
0.031/88.3
0.057/86.1
0.088/83.2
Appendix
Table 11: Prediction of held-out Sentinel-1 second-view block sums. Each entry reports KS distance / empirical coverage of the nominal 90% interval, in percent. For each refinement seed, both samplers use 32 draws on 256 patches drawn from the training run’s own held-out split, disjoint from every patch the model was trained on.
Refinement
Seed 1
Seed 0
None, unrefined checkpoint
0.947
0.947
Neither term, same budget
0.969
0.964
Rollout regression only
0.916
0.934
Consistency loss only
0.990
0.992
Both
0.986
0.984
Appendix
Table 12: Fraction of the network’s own per-pixel posterior spread that the sampled trajectory realizes at the reserved reference band, as a ratio of log-variances over 512 held-out patches. A value of 1 means the trajectory expresses exactly the spread the network predicts. All refined rows branch from the same unrefined checkpoint and use the reported model’s budget of 40 epochs, run in a single stage. The second row is the zero-weight control, whose seed-0 run stopped early at 30 of 40 epochs on the training criterion.
Estimator of X
residual mean
residual KS
Ours, posterior mean
1.003
0.014
Lee 7×7
0.707
0.141
Appendix
Table 13: Despeckling on held-out Sentinel-1 patches. The residual ratio Ytar/(LobsX) has mean 1 for a calibrated estimate. Its 5% null KS floor is 0.0007 .
Generative models learn the statistical properties of their training data, so high-quality generation depends on clean and representative datasets. In scientific imaging, acquisition often yields noisy measurements, while collecting clean references can be costly, impractical or even unattainable. Training directly on these measurements results in a model that reproduces the corrupted data. This can be circumvented by learning the clean population distribution directly from the noisy data. Conditional flow matching (CFM) combines a simple regression objective with stable training, efficient sampling, and strong image-generation performance, making it a natural framework for this setting. We introduce Noise-Robust Conditional Flow Matching (NR-CFM), an unconditional generator that learns from one corrupted observation per image. NR-CFM provides a closed-form clean endpoint correction for additive white Gaussian noise and learns a data-driven correction for general Gaussian corruptions with more complex covariance structure. Across the evaluated corruption settings, NR-CFM outperforms NR-GAN in most cases and remains competitive with Ambient Diffusion in the high-noise regime. We further evaluate NR-CFM on scientific data at signal-to-noise ratios as low as 0.001, where it generates plausible particle images from severely corrupted measurements.
Adrian Urbański, Gabriel della Maggiora, Artur Yakimovich
Center for Advanced Systems Understanding (CASUS), Görlitz, Germany · Helmholtz-Zentrum Dresden-Rossendorf e. V. (HZDR), Dresden, Germany · Institute of Computer Science, University of Wrocław, Wrocław, Poland +2
Diffusion models trained on noisy datasets often reproduce high-frequency training artifacts, significantly degrading generation quality. To address this, we propose SCoRe (Spectral Cutoff Regeneration), a training-free, generation-time spectral regeneration method for clean image generation from diffusion models trained on noisy images. Leveraging the spectral bias of diffusion models, which infer high-frequency details from low-frequency cues, SCoRe suppresses corrupted high-frequency components of a generated image via a frequency cutoff and regenerates them via SDEdit. Crucially, we derive a theoretical mapping between the cutoff frequency and the SDEdit initialization timestep based on Radially Averaged Power Spectral Density (RAPSD), which prevents excessive noise injection during regeneration. Experiments on synthetic (CIFAR-10) and real-world (SIDD) noisy datasets demonstrate that SCoRe substantially outperforms post-processing and noise-robust baselines, restoring samples closer to clean image distributions without any retraining or fine-tuning.
Discrete diffusion models are a powerful class of generative models with strong performance across many domains. For efficiency, however, discrete diffusion typically parameterizes the generative (reverse) process with factorized distributions, which makes it difficult for the model to learn the target process in a small number of steps and necessitates a long, computationally expensive sampling procedure. To reduce the gap between the target and model distributions and enable few-step generation, we propose Forward-Learned Discrete Diffusion (FLDD), which introduces discrete diffusion with a learnable forward (noising) process. Rather than fixing a Markovian forward chain, we adopt a non-Markovian formulation with learnable marginal and posterior distributions. This allows the generative process to remain factorized while matching the target defined by the noising process. We train all parameters end-to-end under the standard variational objective. Experiments on various benchmarks show that, for a given number of sampling steps, our approach produces a higher quality samples than conventional discrete diffusion models using the same reverse parameterization.