Reward Gradients

Follow this topic Ask about this topic

Recent momentum

-50%

4 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

39 papers

Latest in Reward Gradients

Browse all topics →

$λ$ -Controlled GRPO: Turning Flow-Matching Ratio Instability into a Budgeted Resource
Sep 18, 2026Yufeng Wang, Parivesh Priye, Meeshawn Marathe +1Reward Gradients Flow-Based Policy
Gradients Know What Outcomes Don't: Unlocking Reinforcement Learning for LLM Reasoning with Gradient-Aligned Rewards
Sep 3, 2026Leqi Zheng, Jinbo Su, Fang Niu +8Reinforcement Learning With Verifiable Reward Large Language Model Reasoning
NoisEasier: Test-Time Noise Optimization for Text-to-Video Generation
Aug 31, 2026Yujiang Pu, Yu KongText-To-Video Models Test-Time Scaling
Reward-guided Fine-Tuning of One-Step Generative Models via Wasserstein Gradient Flow
Aug 30, 2026Hoseong Hwang, Woorim Han, Joungin Chun +2Wasserstein Gradient Flows Reward Gradients
REOPD: Reliability-Adaptive Reward Extrapolation for On-Policy Distillation
Aug 12, 2026Yang Sun, Lichao Ma, Houyuan Qin +5On-Policy Distillation Reward Gradients
Diffusion-Based Data-Driven Assortment Optimization
Aug 11, 2026Junyi Liao, Xiaohui Jiang, Zhengwei Tong +2Data-Driven Reward Gradients
A Unified Framework for Dynamic Reward Shaping in Reinforcement Learning
Aug 8, 2026Fouad BahrpeymaPotential-Based Reward Shaping Reward Gradients
Sample-Adaptive Latent Rewards for Uncertainty-Guided Diffusion Post-Training
Aug 6, 2026Rui Li, Yuanzhi Liang, Ke Hao +4Reward Gradients LLM Post-Training Methods
LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction
Aug 6, 2026Yingqing Guo, Hui Yuan, Zijian He +2Flow-Based Policy Reward Gradients
Latent Reward Registers for Diffusion Preference Alignment
Aug 4, 2026Yuanshen Guan, Zipeng Feng, Chengru Song +2Reward Gradients Latent Representation Alignment
When Do Surrogate Updates Improve Decisions? A Local Theory of Trajectory-Wise Transfer
Aug 2, 2026Yuyang ShenTime-Step-Wise Learnable Surrogate Gradient Function Linear Regret
Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning
Jul 31, 2026Xujun Che, Yuchen Yuan, Weida Zhao +1Reward Gradients Abstention
Selective Timestep Weighting and Advantage-Based Replay for Sample-Efficient Diffusion RLHF
Jul 8, 2026Eric Zhu, Abhinav Shrivastava, Soumik MukhopadhyayReinforcement Learning From Human Feedback Reward Gradients
ReGuide: From Test-Time Guidance to Self-Improving Diffusion Policies
Jun 27, 2026Tzu-Hsiang Lin, Srinivas Shakkottai, Dileep Kalathil +1Diffusion Policies Reward Gradients
NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning
Jun 26, 2026Tianlin Pan, Lianyu Pang, Cheng Da +4Reward Gradients Reinforcement Fine-Tuning
Stochastic Linear Contextual Bandits with Bounded Noise: A Set-Membership Approach
Jun 18, 2026Haonan Xu, Yingying LiStochastic Optimization Reward Gradients
NoiseTilt: Noise-Tilted Reverse Kernels for Diffusion Reward Alignment
Jun 16, 2026Jisung Hwang, Yunhong Min, Jaihoon Kim +2Diffusion Sampling Reward Gradients
Gradient-Guided Reward Optimization for Inference-time Alignment
Jun 8, 2026Hankun Lin, Ruqi ZhangInference-Time Defense LLM Inference Efficiency
Adaptive Loss Balancing for Noise-Robust GRPO in Generative Recommendation
Jun 7, 2026Kewei Xu, Junbo Qi, Yanyan Zou +3Reward Gradients Group Relative Policy Optimization
Socratic-SWE: Self-Evolving Coding Agents via Trace-Derived Agent Skills
Jun 5, 2026Chuan Xiao, Zhengbo Jiao, Shaobo Wang +5Self-Evolving Agents Coding Agents
Reward hacking in physical reinforcement learning revealed by turbulent drag reduction
Jun 4, 2026Giorgio Maria Cavallazzi, Miguel Pérez-Cuadrado, Alfredo PinelliReward Gradients Deep Reinforcement Learning
Are we really tilting? The mechanics of reward guidance in flow and diffusion models
Jun 1, 2026Sanjit Dandapanthula, Nicholas M. BoffiReward Gradients Reward-Guided Diffusion
GDSD: Reinforcement Learning as Guided Denoiser Self-Distillation for Diffusion Language Models
May 28, 2026Xiaohang Tang, Keyue Jiang, Che Liu +4Diffusion Language Models Unsupervised On-Policy Self-Distillation
AdvantageFlow: Advantage-Weighted Least Squares for RL in Flow Models
May 25, 2026Branislav Kveton, Anup Rao, Subhojyoti Mukherjee +2Generative Flow Networks Rectified Flow
Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learning
May 24, 2026Yu Yang, Yihong Guo, Anqi Liu +1Offline Reinforcement Learning Diffusion Policies
Guided Trajectory Optimization with Sparse Scaling for Test-Time Diffusion
May 21, 2026Gang Dai, Yining Huang, Yiming Xia +2Test-Time Scaling Reward Gradients
You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories
May 20, 2026Zhepei Wei, Xinyu Zhu, Wei-Lin Chen +3Reinforcement Learning With Verifiable Reward Reinforcement Learning Post-Training
Efficient Adjoint Matching for Fine-tuning Diffusion Models
May 12, 2026Jeongwoo Shin, Dongsoo Shin, Yuchen Zhu +5Adjoint Matching Reward Gradients
Gradient-Free Noise Optimization for Reward Alignment in Generative Models
May 12, 2026Jeongsol Kim, Hongeun Kim, Jian Wang +1Reward Gradients Gradient-Based Optimization
Reinforce Adjoint Matching: Scaling RL Post-Training of Diffusion and Flow-Matching Models
May 11, 2026Andreas Bergmeister, Stefanie Jegelka, Nikolas Nüsken +2Reinforcement Learning Post-Training Reward Gradients
Verifier-Free RL for LLMs via Intrinsic Gradient-Norm Reward
May 11, 2026Xuexiang Wen, Hang Yu, Linchao Zhu +1Reinforcement Learning With Verifiable Reward Reward Gradients
The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs
May 9, 2026Xin Li, Hao Jiang, Annan Wang +2On-Policy Distillation Large Language Model Post-Training
On Training in Imagination
May 7, 2026Nadav Timor, Ravid Shwartz-Ziv, Micah Goldblum +2Entropy Regularized Reinforcement Learning Reward Gradients
Decoupled Guidance Diffusion for Adaptive Offline Safe Reinforcement Learning
May 4, 2026Rufeng Chen, Zhaofan Zhang, Zhejiang Yang +2Reward Gradients Offline Reinforcement Learning
Stable GFlowNets with Probabilistic Guarantees
May 3, 2026Zengxiang Lei, Ananth Shreekumar, Jonathan Rosenthal +6Generative Flow Networks Distributional Learning
How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidance
Apr 29, 2026Jerry Y. Huang, Justin Lin, Sheel Shah +2Generative Flow Networks Flow Map
LeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectories
Apr 16, 2026Zhanhao Liang, Tao Yang, Jie Wu +2Hierarchical Flow Matching Latent Representation Alignment
Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design
Feb 4, 2026Jaemoo Choi, Yuchen Zhu, Wei Guo +6Reward Gradients Likelihood
Adaptive teachers for amortized samplers
Oct 2, 2024Minsu Kim, Sanghyeok Choi, Taeyoung Yun +7Samplers Adaptive Testing

Sep 18, 2026 · cs.LGJ/K move · Enter open · S save

$λ$ -Controlled GRPO: Turning Flow-Matching Ratio Instability into a Budgeted Resource

Yufeng Wang, Parivesh Priye, Meeshawn Marathe, Ramit Pahwa

Reinforcement learning is increasingly used to align image generators with reward signals, and Flow-GRPO recently extended this paradigm to flow-matching models by treating the denoising sampler as a stochastic policy that can be optimized from reward feedback. Training in this setting is unstable in a way specific to multi-step denoising: the policy update changes systematically across denoising steps, with importance ratios drifting below one, becoming increasingly dispersed, clipping at different rates, and leaving fewer usable samples late in training. Prior work treats these effects as separate failure modes and addresses each with a hand-tuned stabilizer. We show instead that they arise from a single per-step quantity, which we call path variance. This quantity is determined exactly by the sampler's Gaussian transition kernel and can be estimated cheaply during training. This reframes instability as a resource that can be measured and budgeted rather than a collection of symptoms to repair. Our method,

λ

-Controlled GRPO, calibrates importance-ratio behavior from this predicted law rather than from noisy empirical statistics, and allocates gradient effort across denoising steps according to their predicted cost. The two scales governing the update are fixed by standard policy choices rather than introduced as free tuning parameters. On a text-to-image model under two reward settings, rendering difficult target text scored by optical character recognition and matching human preferences scored by a preference model,

λ

-Controlled GRPO improves both text accuracy and preference reward over the strongest empirical stabilizer. It also keeps late-step path variance within its intended budget, precisely where the baseline systematically overshoots. The result is a Flow-GRPO update calibrated by its own transition law rather than stabilized after instability appears.

Reward Gradients Flow-Based Policy

Open paper Ask about it PDF