Reward alignment for diffusion- and flow-based generative models is often formulated over stochastic trajectories, making it less suitable for few-step deterministic generators. Initial noise optimization provides a trajectory-free alternative, but existing gradient-free methods require costly per-sample search at inference. We introduce LeNO (Learning Noise Optimization), which amortizes reward-guided initial noise optimization into a learned residual transport while keeping the pretrained generator frozen. LeNO learns an anchor-conditioned distribution over noise corrections using Local GRPO and scalar rewards, eliminating iterative test-time optimization. For Gaussian initial noise with norm-preserving projection, we derive a tractable upper bound on the KL divergence between the corrected noise distribution and the original prior, providing a principled regularization objective. We evaluate LeNO on few-step text-to-image and protein structure generation, including expensive black-box designability rewards. Across both domains, LeNO improves target rewards with a single noise correction at inference while largely preserving generation quality and diversity.
Figures & tables
Figure 1: LeNO learns reward-improving corrections directly in initial noise space. Each pair shows generations before (left) and after (right) applying the learned noise correction, with the pretrained generator kept frozen. The corrected noises produce images that better capture the desired visual attributes across diverse prompts.
Figure 2: Toy Experiment. (a) Initial noise samples and the reward landscape of a four-component Gaussian mixture. (b) Samples obtained by directly passing the initial noise through the frozen probability-flow ODE. (c) Per-sample initial noise optimization with 30 step optimization. (d) Our amortized noise space correction with the residual transport. The learned correction successfully redirects samples toward the condition-specific high-reward modes with substantially lower inference time computation. (e) Corrected noise distribution over training.
Figure 3: Qualitative comparison of one-step generations before (left) and after LeNO noise correction (right). Top: SANA-Sprint with compositional multi-reward optimization. Bottom: SDXL-Turbo with PickScore optimization.
Figure 4Figure 5
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
Value
Correction scale η
0.3
Exploration std. σ
0.2
KL regularization weight β
0.01
Global particle group size
32
Prompt anchors per update
16
GRPO epochs per rollout
4
Appendix
Table 4: GRPO training hyperparameters.
Single Reward
Multi Reward
HyperNoise
LeNO
HyperNoise
LeNO
Model
SDXL-Turbo
SDXL-Turbo
SANA-Sprint
SANA-Sprint
Denoising Step
1
1
1
1
Learning rate
1×10−3
5×10−5
1×10−3
5×10−5
GradNorm Clipping
1.0
1.0
1.0
1.0
LoRA rank
16
16
128
128
Appendix
Table 5: Training configurations for HyperNoise and LeNO under single- and multi-reward optimization.
Hyperparameter
Value
Frozen generator
RMF-M / RMF-L
Generator sampling steps
1
Residual transport steps
1
Training protein length
N∼U{60,…,128}
LoRA rank / alpha
32/32
Trainable parameters
7.1M / 17.2M
Appendix
Table 6: Training configuration for protein generation.
Method
LPIPS (ret.) ↑
DINO div. (ret.) ↑
MUSIQ ( Δ ) ↑
CLIP-IQA ( Δ ) ↑
SDXL-Turbo
0.5984 (100.0%)
0.1952 (100.0%)
70.96 (+0.00)
0.6600 (+0.0000)
PickScore (step 800)
0.6054 (101.2%)
0.1942 (99.5%)
72.67 (+1.72)
0.7192 (+0.0592)
Aesthetic (step 800)
0.5938 (99.2%)
0.1947 (99.7%)
72.51 (+1.55)
0.7335 (+0.0734)
ImageReward (step 800)
0.6275 (104.9%)
0.2059 (105.5%)
73.93 (+2.97)
0.7306 (+0.0706)
Appendix
Table 7: Diversity and image quality after 800 training steps. Values in parentheses denote diversity retention relative to SDXL-Turbo for LPIPS and DINO, and absolute improvement over SDXL-Turbo for MUSIQ and CLIP-IQA.
Figure 7: Diversity and no-reference image quality throughout single-reward optimization. LPIPS and DINO diversity are normalized by the unoptimized SDXL-Turbo baseline, while MUSIQ and CLIP-IQA are shown as absolute improvements over the baseline. All methods preserve nearly all of the baseline diversity, while consistently improving perceptual image quality. Shaded regions indicate 95% paired bootstrap confidence intervals over prompts.
Training reward
Evaluation reward
Aesthetic ↑
PickScore ↑
ImageReward ↑
SDXL-Turbo (no training)
6.1174
21.8608
0.9710
Aesthetic
6.1796 (+0.0622)
21.9728 (+0.1120)
1.0297 (+0.0587)
PickScore
6.1688 (+0.0514)
22.3308 (+0.4701)
1.1306 (+0.1596)
ImageReward
6.1558 (+0.0384)
21.9092 (+0.0484)
1.1984 (+0.2274)
Appendix
Table 8: Cross-reward evaluation of single-reward LeNO models on SDXL-Turbo. Rows denote the training reward and columns the evaluation reward. Values in parentheses denote improvement over SDXL-Turbo; shaded cells indicate the target reward.
Figure 8: Ablation for multi-step transport reward gain.
Figure 13
Figure 10: Qualitative comparison of DF-GAN generated samples before (left) and after LeNO noise correction (right).
Figure 11: Additional qualitative comparison of SANA-Sprint optimized with compositional multi-rewards.
Figure 12: Additional qualitative comparison of SDXL-Turbo optimized with Image Rewards.
Figure 13: Additional qualitative comparison of SDXL-Turbo optimized with PickScore.
Figure 14: Additional qualitative self-consistency comparison with RMF at protein length L =96.