stat.MLOct 21, 2026
SaveInternal Guidance as a Sampling Method for Tilted Diffusion Models
Organizations: Simons Institute, University of California Berkeley · Google DeepMind
Abstract
Reward guidance methods sample an exponential tilt of a base density using a diffusion model trained with only samples from the base distribution. Existing methods use what we call external guidance: additive terms in the drift or multiplicative terms on the transition probabilities. We propose internal guidance which leaves the output of the base denoiser unmodified and instead changes the point at which it is evaluated. For discrete distributions embedded in the hypercube , the denoiser of any linearly tilted density is exactly the base denoiser evaluated at a shifted and rescaled input. In practice the base denoiser is a neural network trained on noised samples of the base distribution. The required evaluation points for sampling the tilted density would be out-of-distribution for basic noising of the base distribution. We modify the training algorithm with random diagonal rescalings of the data, so that the denoiser network learns both in- and out-of-distribution evaluations of the denoiser relative to the base distribution. A single denoiser network samples binary reward tilts, class labels, and infilling with no fine-tuning and no external guidance term.
Figures & tables
Figure 1: Sketch of denoiser symmetry. The training manifold is where the denoiser network has learned the base denoiser. The sampling manifold is the typical vectors that will be encountered when sampling a tilted density. The equivalence relationships can find a point with equal evaluation of the base denoiser near the training manifold, not all equivalence relationships map back to the region of the training manifold.
Figure 2: Reward based focusing Mode 0 maintains norm of hypersphere points while placing more mass in reward coordinate and increasing cosine similarity with the one-hot vector of the reward coordinate at the pole of the hypersphere.
Figure 3: Sudoku 9x9 completions sampled from the tilt-operator network.
Figure 4: Probability of correct grid infilling across input cell numbers. Given no initial cells, should return a valid grid 95% of the time, we measure saturation at 87% without any infilling.
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Sketch of overlap between renormalized training set via focusing and points shifted by tilt along stochastic localization path.
Figure 6: Visualization of score-based diffusion and stochastic localization reverse flow path for an asymmetric mixture of normals target density in one dimension. Marginal densities produced at each stage of the processes are equivalent under a time shift and rescaling.
Figure 7: Ising model covariance matrix ground truth and network sampled.
Figure 8: Comparison of error of learned denoiser for Ising model across tilt strengths.
Figure 9: Ising model log-odds across tilt strengths.
Figure 10: Example valid 4x4 grid.
Figure 11: Comparison of error of learned denoiser of 4x4 Sudoku problem across tilt strengths.
Figure 12: 4x4 Sudoku log-odds across tilt strengths.
Figure 13: Sudoku 4x4 denoiser trajectory and early decoding. Partially complete discrete token sequences fixed when denoiser coordinates pass magnitude threshold.
Figure 14: Example ground truth denoiser and network denoiser in 33 coordinates at sampled points. Samples in a) are untilted sampled and b) tilted samples.
Figure 15:
Figure 16: Example valid 6x6 grid.
Figure 17: error with enumerated ground truth denoiser untilted.
Figure 18: 6x6 Sudoku log-odds across tilt strengths.
Figure 19: Example valid 9x9 grid.
Figure 20: 9x9 Sudoku log-odds across tilt strengths for both self-conditioning modes.
Figure 21: Cumulative distribution function of Hamming distance to nearest valid puzzle for 300 samples.
| Given | Removed | Valid grid | Keeps given cells | Joint success |
|---|---|---|---|---|
| 71 | 10 | |||
| 61 | 20 | |||
| 51 | 30 | |||
| 41 | 40 | |||
| 31 | 50 | |||
| 21 | 60 |
Table 1: Sudoku infilling as a function of the number of given cells. Each row is based on 100 uniformly random valid solutions 20 draws per puzzle ( ), with a fresh random mask per puzzle. Tilt strength for both the reward and the cell-fixing tilts, 1000 Euler steps, self-conditioning mode carry . Brackets give 95% Clopper–Pearson intervals. “Joint” is the fraction of samples that are valid grids and also keep every given cell.
Figure 22: Generated invalid puzzle (right) with constraint violations and nearest valid puzzle (left) in Hamming distance. Nearest valid grid would require 8 bit flips to the binary encoding. The bit flips are not all restricted to the invalid bits of the produced puzzle, but previously valid bits may be flipped to get to a valid puzzle.
Figure 23: CDF and density of nearest distance reward-achieving gird in training set for network generated samples and held out set. Nearly identical distributions, so generated puzzles no more similar to training set than a held out portion. No puzzles are exact match, closet puzzle differ in 39 cells
| Exact | Exact (scored) | Classifier | Reward block | |
|---|---|---|---|---|
| 0.00 | ||||
| 0.26 | ||||
| 0.51 | ||||
| 0.77 | ||||
| 1.03 | ||||
| 1.29 |
Table 2: MNIST pooled class-conditional log-odds, two-pass self-conditioning, samples per ( per class). “Exact” is of the exact tilted class marginal; “Exact (scored)” is the same rate passed through each class’s measured classifier errors and averaged over classes, which is the right reference for the classifier-scored column. “Classifier” is the fraction of samples the CNN assigns to the requested class, with a 95% Clopper–Pearson interval. “Reward block” is the fraction whose reward block has its argmax at the requested class. † Beyond the classifier’s resolution limit.
| Exact | Exact (scored) | Classifier | Reward block | |
|---|---|---|---|---|
| 0.00 | ||||
| 0.26 | ||||
| 0.51 | ||||
| 0.77 | ||||
| 1.03 | ||||
| 1.29 |
Table 3: As Table 2 , for carry self-conditioning, with the same grid, sample sizes and seeds. † Beyond the classifier’s resolution limit.
Figure 24: MNIST class-conditional log-odds against tilt strength for each class, two-pass self-conditioning. Exact marginal passed through each class’s classifier errors (line, with a band from the uncertainty in those errors), classifier-scored rate (points with 95% Clopper–Pearson intervals), and the network’s own reward-block rate.
Figure 25: MNIST log-odds pooled over all ten classes, two-pass self-conditioning (Table 2 ).
Figure 26: As Figure 24 , for carry self-conditioning. The overshoot over appears in every class.
Figure 27: MNIST log-odds pooled over all ten classes, carry self-conditioning (Table 3 ).
Figure 28: 25 class-0 images from the held-out set (a), and network samples under both self-conditioning modes with the same random seed (b, c). Panel titles give the classifier’s label (green if it matches the requested class, red if not). The samples are similar in quality and variety to the held-out digits, with no sign of mode collapse. is set so that the exact class- rate is , so about in samples is expected to be of another class.
Figure 29: Example digit generations tilted towards each class. Leftmost image per digit is a sample of 25 images from hold out set network was not trained on, second image is from sample mode carry and third image is sample mode two-pass. Middle and right grids share the same random seed in their sampling, resulting in often similar samples which is not a form of memorization. Tilt strength is calibrated to 95% reward-achieving in log-odds theory, thus it is reasonable to see a few mis-samples in repeated 25 size sets.
More in this topic
Diffusion Model Guidance
Diffusion model guidance steers denoising toward desired conditions, rewards, constraints, or away from unwanted concepts.
See all 361 papers