cs.LGNov 4, 2026

Learning Noise Optimization for Reward Alignment in Few-Step Generative Models

Authors: Hongeun Kim, Jeongsol Kim, Jong Chul Ye

Organizations: KAIST AI

Abstract

Reward alignment for diffusion- and flow-based generative models is often formulated over stochastic trajectories, making it less suitable for few-step deterministic generators. Initial noise optimization provides a trajectory-free alternative, but existing gradient-free methods require costly per-sample search at inference. We introduce LeNO (Learning Noise Optimization), which amortizes reward-guided initial noise optimization into a learned residual transport while keeping the pretrained generator frozen. LeNO learns an anchor-conditioned distribution over noise corrections using Local GRPO and scalar rewards, eliminating iterative test-time optimization. For Gaussian initial noise with norm-preserving projection, we derive a tractable upper bound on the KL divergence between the corrected noise distribution and the original prior, providing a principled regularization objective. We evaluate LeNO on few-step text-to-image and protein structure generation, including expensive black-box designability rewards. Across both domains, LeNO improves target rewards with a single noise correction at inference while largely preserving generation quality and diversity.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Noise-Space Reward Alignment for Few-Step Generative Models

    Oct 15, 2026Woocheol Shin, Taeyoung Yun, Jiwon Sung +7RL for Generative ModelsFlow-Based Generative Modeling

  2. Manifold-Constrained Initial Noise Optimization for Efficient Generative Model Alignment

    Sep 30, 2026Jinho Chang, Jong Chul YeDiffusion Model AlignmentZeroth-Order Optimization