Credit Assignment in RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

32 papers in the last four weeks, up 220% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 171

All topics

Guidance Contrastive Token Credit Assignment for Discrete Policy Optimization
May 28, 2026Shufan Li, Konstantinos Kallidromitis, Akash Gokul +2Token-Level Credit Assignment RL for Language Model Reasoning
UCLA · Panasonic AI Research · NVIDIA
PRO-CUA: Process-Reward Optimization for Computer Use Agents
May 27, 2026Yifei He, Rui Yang, Hao Bai +2Reinforcement Learning Process Reward Models
University of Illinois Urbana-Champaign
StepOPSD: Step-Aware Online Preference Distillation for Agent Reinforcement Learning
May 26, 2026Yanfei Zhang, Xu Lin, Chenglin WuCredit Assignment in RL Step-Level Credit Assignment in RL
Independent Researcher · Tencent · DeepWisdom
Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learning
May 26, 2026Xin Cheng, Shuo He, Lang Feng +4Group Relative Policy Optimization Credit Assignment in RL
Nanyang Technological University, Singapore · Tongyi Lab, Alibaba Group · Southeast University, China
RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents
May 25, 2026Mingchen Li, Hansi Zeng, Zhuo Qian +3RL for Language Model Reasoning Credit Assignment in RL
University of Massachusetts, Amherst · Texas Tech University · University of Connecticut
Credit Assignment with Resets in Language Model Reasoning
May 25, 2026Ankur Samanta, Akshayaa Magesh, Ayush Jain +7RL for Language Model Reasoning Credit Assignment in RL
1Meta AI · 2Columbia University · *Work done at Meta +2
Not only where, But when: Temporal Scheduling for RLVR
May 25, 2026Jinghao Zhang, Ruilin Li, Feng Zhao +1RL for Language Model Reasoning Credit Assignment in RL
University of Science and Technology of China · Shanghai Innovation Institute · Wuhan University
Scaling up Energy-Aware Multi-Agent Reinforcement Learning for Mission-Oriented Drone Networks with Individual Reward
May 24, 2026Changling Li, Ying LiReinforcement Learning Credit Assignment in RL
Department of Computer Science, ETH Zurich, 8092, Zurich, Switzerland · Department of Computer Science, Colby College, 04901, Waterville, ME, USA
From Reasoning Chains to Verifiable Subproblems: Curriculum Reinforcement Learning Enables Credit Assignment for LLM Reasoning
May 21, 2026Xitai Jiang, Zihan Tang, Wenze Lin +3RL for Language Model Reasoning Credit Assignment in RL
LeapLab, Tsinghua University · Qiuzhen College, Tsinghua University · B Gao Huang
OPPO: Bayesian Value Recursion for Token-Level Credit Assignment in LLM Reasoning
May 21, 2026Yu Li, Rui Miao, Tian Lan +1Value Function Estimation Token-Level Credit Assignment
1George Washington University · 2The University of Texas at Dallas
Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents
May 20, 2026Sikuan Yan, Ahmed Bahloul, Ercong Nie +4Group Relative Policy Optimization Credit Assignment in RL
Ludwig Maximilian University of Munich · Munich Center for Machine Learning · Huawei Heisenberg Research Center (Munich) +1
Rewarding Beliefs, Not Actions: Consistency-Guided Credit Assignment for Long-Horizon Agents
May 19, 2026Wenjie Tang, Minne Li, Sijie Huang +2Reinforcement Learning Partially Observable RL
College of Computer, National University of Defense Technology · 2Intelligent Game and Decision Lab (IGDL) · Institute of Artificial Intelligence, Xiamen University
PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models
May 19, 2026Peizheng Guo, Jingyao Wang, Changwen Zheng +1Robot Policy Learning Credit Assignment in RL
Institute of Software Chinese Academy of Sciences · University of Chinese Academy of Sciences
What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents
May 19, 2026Xiaozhe Li, Tianyi Lyu, Yang Li +6Reinforcement Learning Credit Assignment in RL
1Tongji University · 2Shanghai AI Laboratory · 5Independent +2
PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization
May 18, 2026Wonjoong Kim, Yeonjun In, Sangwu Park +2Reward Modeling Credit Assignment in RL
KAIST · Yonsei University
HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agents
May 18, 2026Woongyeng Yeo, Yumin Choi, Taekyung Ki +1Credit Assignment in RL LLM Agent Training
KAIST
SAPO: Step-Aligned Policy Optimization for Reasoning-Based Generative Recommendation
May 17, 2026Zaiyi Zheng, Guanghui Min, Yaochen Zhu +4RL for Language Model Reasoning Credit Assignment in RL
University of Virginia · 2Nokia
Step-wise Rubric Rewards for LLM Reasoning
May 17, 2026Weichu Xie, Haozhe Zhao, Wenpu Liu +15Reward Modeling Credit Assignment in RL
Peking University · JD Explore Academy · Shanghai Jiao Tong University +2
Mind Dreamer: Untethering Imagination via Active Causal Intervention on Latent Manifolds
May 15, 2026Shaojun Xu, Xiaoling Zhou, Yihan Lin +4Reinforcement Learning Credit Assignment in RL
Center for Brain-Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Pen-Tung Sah Institute of Micro-Nano Science and Technology, Xiamen University, Xiamen, China +1
Addressing Terminal Constraints in Data-Driven Demand Response Scheduling
May 14, 2026Maximilian Bloor, Martha White, Ehecatl Antonio del Rio Chanona +1Hierarchical RL Credit Assignment in RL
Sargent Centre for Process Systems Engineering, Imperial College London, London, SW7 2AZ, UK · Department of Computer Science, University of Alberta, Edmonton, AB, Canada · Department of Computing, Imperial College London, London, SW7 2AZ, UK
Resolving Action Bottleneck: Agentic Reinforcement Learning Informed by Token-Level Energy
May 14, 2026Langzhou He, Junyou Zhu, Yue Zhou +7Agentic RL Token-Level Credit Assignment
University of Illinois Chicago · Potsdam Institute for Climate Impact Research · Technical University of Berlin +3
GAGPO: Generalized Advantage Grouped Policy Optimization
May 13, 2026Siyuan Zhu, Chao Yu, Rongxin Yang +4Reinforcement Learning Group Relative Policy Optimization
School of Computer Science and Engineering, Sun Yat-sen University · Meituan
GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillation
May 12, 2026Sijia Li, Yuchen Huang, Zifan Liu +7RL for Language Model Reasoning Credit Assignment in RL
Hong Kong University of Science and Technology · Microsoft Research Asia
Verifiable Process Rewards for Agentic Reasoning
May 11, 2026Huining Yuan, Zelai Xu, Huaijie Wang +6Process Reward Models Credit Assignment in RL
Tsinghua University
PiCA: Pivot-Based Credit Assignment for Search Agentic Reinforcement Learning
May 10, 2026Dongyi Liu, Yifan Niu, Qinwen Wang +2Reinforcement Learning Reward Shaping
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
BoostAPR: Boosting Automated Program Repair via Execution-Grounded Reinforcement Learning with Dual Reward Models
May 9, 2026Yuanhao Li, Hongbo Wang, Xiaotang Shang +3Automated Program Repair Process Reward Models
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China · University of Luxembourg
Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreaking
May 9, 2026Zhida He, Xiaoyu Wen, Han Qi +7RL for Language Models Credit Assignment in RL
Shanghai AI Laboratory · Fudan University · Shanghai Jiao Tong University
Learning CLI Agents with Structured Action Credit under Selective Observation
May 8, 2026Haoyang Su, Ying WenComputer-Use Agents Credit Assignment in RL
Fudan University · Shanghai Innovation Institute · Shanghai Jiao Tong University
The Reciprocity Gradient
May 8, 2026Yue Lin, Pascal Poupart, Shuhui Zhu +5Credit Assignment in RL Policy Gradient Methods
The Chinese University of Hong Kong, Shenzhen · LIGHTSPEED · University of Waterloo +2
DACA-GRPO: Denoising-Aware Credit Assignment for Reinforcement Learning in Diffusion Language Models
May 8, 2026Amin Karimi Monsefi, Dominic Culver, Nikhil Bhendawade +4Group Relative Policy Optimization Credit Assignment in RL
The Ohio State University · Apple

May 28, 2026, cs.CVJ/K move · Enter open · S save

Guidance Contrastive Token Credit Assignment for Discrete Policy Optimization

Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yuta Kyuragi+1

UCLA · Panasonic AI Research · NVIDIA

Token-Level Credit Assignment RL for Language Model Reasoning Credit Assignment in RL Policy Optimization RL for Image Generation Policy Gradient Methods

Open paper AskPDF arXiv

Group-advantage-based reinforcement learning methods, such as GRPO and DAPO, have demonstrated strong performance across diverse domains, including mathematical reasoning and text-to-image generation. However, their reliance on sample-level rewards introduces a key limitation as uniform credit assignment across all tokens fails to capture fine-grained, token-level contributions. To address this issue, we propose Guidance Contrastive Policy Optimization (GCPO), a novel algorithm that enables per-token credit assignment by contrasting model predictions under positive and negative prompts. Rather than uniformly broadcasting sample-level advantages, GCPO assigns token-level advantages proportional to the difference between these contrastive predictions, allowing more precise and informative learning signals. Empirically, we find that GCPO emphasizes semantically relevant regions such as visual areas aligned with textual prompts in text-to-image generation, and critical keywords within reasoning traces for chain-of-thought tasks. Through extensive experiments, GCPO consistently outperforms GRPO and DAPO baselines on both text-to-image generation and chain-of-thought reasoning benchmarks, demonstrating its effectiveness as a general and scalable optimization strategy for discrete policy learning.

Momentum

Related topics

Latest papers 171