Step-Level Credit Assignment in RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

25 papers in the last four weeks, up 178% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 90

All topics

SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages
Oct 7, 2026Raja Kumar, Rajat Koner, Ritwick Chaudhry +3VLM Reasoning RL for VLMs
University of Southern California · Amazon AGI
Beyond Outcome Rewards: Constructing and Assigning Retrieval Credit for Search Agents
Oct 7, 2026Wenyu Huang, Xinyu Hou, Pavlos Vougiouklis +2Reward Design for RL RL for IR
University of Edinburgh, UK · Huawei Technologies Research & Development (UK) Limited
Convex-Concave Reinforcement Learning
Oct 6, 2026Shripad V. Deshmukh, Yaswanth Chittepu, Dhawal Gupta +2Reinforcement Learning Policy Gradient Methods
University of Massachusetts · Institute of Foundation Models
RELACE: retrospective likelihood-based action credit estimation for long-horizon language agents
Oct 5, 2026Sayak Chakrabarti, Sathish Reddy IndurthiGroup Relative Policy Optimization Credit Assignment in RL
Columbia University · Zoom Communications, Inc.
SCAD: Structured Credit Assignment and Distillation for Long-Horizon Agents
Oct 2, 2026Shangyang Wu, Shuai Zhao, Ziyue Zhu +3Hierarchical Planning Step-Level Credit Assignment in RL
Beijing University of Posts and Telecommunications · Tsinghua University
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
Oct 1, 2026Ziyi Chen, Yan Zhang, Jianhui Wei +2RL for Language Models Agentic RL
University of Illinois Urbana-Champaign · National University of Singapore · Zhejiang University +1
My FAULT: Self-Diagnosis as Credit Assignment in Self-Evolving Agentic Reinforcement Learning
Oct 1, 2026Yihua Zhu, Qianying Liu, Weixu Qiao +10Agentic RL Credit Assignment in RL
Alibaba Group · Kyoto University · NII LLMC +6
Semifactual Credit-Augmented Policy Optimization
Sep 30, 2026Junshu Pan, Zhizhang Fu, Shulin Huang +5Prompt Sensitivity RL for Language Model Reasoning
Zhejiang University · Westlake University · Shanghai Innovation Institute +1
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
Sep 30, 2026Bo-Wen Zhang, Junwei He, Maoqi Liu +6Agentic RL Reward Shaping
State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · ByteDance
Advancing Entropy-Level Credit Assignment in RLVR via Proximal Entropy Policy Optimization
Sep 30, 2026Yun Kim, Nojun KwakRL for Language Model Reasoning Credit Assignment in RL
Seoul National University
PRICE the Action Chunks: Physical Relational Credit Assignment for Embodied Reinforcement Learning
Sep 30, 2026Yangang Zou, Jiajun Lu, Weitao Zhou +6Credit Assignment in RL Step-Level Credit Assignment in RL
School of Data Science, Fudan University · Shanghai Innovation Institute · Beihang University +4
Privy to the Foil: Recasting Value Estimation with a Self-Privileged Critic for RLVR
Sep 29, 2026Kun Liang, Chenming Tang, Clive Bai +5Value Function Estimation RL for Language Model Reasoning
School of Computer Science, Peking University · National Key Laboratory for Multimedia Information Processing, Peking University · Foundation Model Department, Tencent
Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning
Sep 28, 2026Dongwon Jung, Hemanth Neelgund Ramesh, Yifan Wang +7Agentic RL Group Relative Policy Optimization
University of California, Davis · Microsoft · University of Washington
Beyond Saying Less: Fine-Grained Alignment for Informative and Faithful Vision-Language Models
Sep 28, 2026Xingming Long, Jie Zhang, Yuecong Min +2Credit Assignment in RL Step-Level Credit Assignment in RL
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Zhongguancun Academy
SpikeCredit: Temporal Credit Carrier for Reinforcement Learning with Sparse Rewards
Sep 28, 2026Yingchao Yu, Pengfei Sun, Wenxuan Pan +4Credit Assignment in RL Step-Level Credit Assignment in RL
School of Information and Intelligent Science, Donghua University, Shanghai, China · School of Engineering, Westlake University, Hangzhou, China · Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom +1
ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning
Sep 28, 2026Yang Li, Jinhan Yang, hai liu +8Agentic RL Credit Assignment in RL
University of Hong Kong · The Chinese University of Hong Kong · Jiangxi Science and Technology Normal University +1
GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
Sep 28, 2026Haodong Zhu, Yangyang Ren, Changbai Li +4Credit Assignment in RL Step-Level Credit Assignment in RL
Beihang University · Zhongguancun Academy · Communication University of China +1
Cross-Rollout Bellman Closure for Long-Horizon Agentic Reinforcement Learning
Sep 28, 2026Yangyang Ren, Haodong Zhu, Linlin Yang +3Agentic RL Step-Level Credit Assignment in RL
Beihang University · Zhongguancun Academy · Communication University of China +2
Can We Trust the Teacher? Decoupled Credit Direction-Magnitude for Self-Distillation
Sep 28, 2026Yugu Li, Zehong Cao, Peizhen Li +3Token-Level Credit Assignment RL for Language Model Reasoning
School of CSIT, Adelaide University, Adelaide, SA 5000, Australia · CAIAA, University of North Texas, Denton, TX 76203, USA · School of EECMS, Curtin University, Bentley, WA 6102, Australia +1
UniOPSD: Unifying Outcome and Hindsight Feedback for Agentic Reinforcement Learning
Sep 28, 2026Zenghuang Fu, Zhaoyang Li, Qiuyuan Ai +8Agentic RL Credit Assignment in RL
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Peking University +3
Dr.Credit: Rubric-Grounded Process Credit Assignment for Deep Research Agents
Sep 28, 2026Yingjian Zhu, Zhenyi Wang, Jiaxin Guo +6Deep Research Agents Step-Level Credit Assignment in RL
School of Artificial Intelligence, University of Chinese Academy of Sciences · State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · Alibaba Token Hub, Alibaba Group +1
Surprising Success, Repeated Failure: Entropy-Guided Credit Assignment for Exploration in LLM Reasoning
Sep 27, 2026Woongyeong Yeo, Minki Kang, Chanuk Lee +3RL for Language Model Reasoning Credit Assignment in RL
KAIST
Back to the Definition: Estimating Step-Level Advantages via Trajectory Graphs for Agentic Reinforcement Learning
Sep 24, 2026Xincheng Yao, Haobo Fu, Weiming Liu +1Agentic RL Credit Assignment in RL
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University. · Tencent AI Platform Department. · MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University.
ProCredit: From Outcome Rewards to Progress Credit in Agentic Reinforcement Learning
Sep 23, 2026Ming Ma, Yi Zhu, Yiran Zhong +7Agentic RL Credit Assignment in RL
Institute of Neuroscience, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Tongyi Lab, Alibaba Group +2
Giving Credit Where It's Due: Redundancy-Aware Learning for Efficient Reasoning
Sep 22, 2026Yuqing Zhou, Hong Wang, Manqing Mao +8RL for Language Model Reasoning Step-Level Credit Assignment in RL
George Mason University · Amazon, Inc
PACT: From Credit Assignment to Critic Alignment
Sep 22, 2026Jiayan Fu, Hang Xu, Yong Zhang +4Reinforcement Learning Token-Level Credit Assignment
AllSpark Team
Fork Where the Model Changes Its Mind: Belief-Shift Branching for Tree-Structured Reinforcement Learning
Sep 12, 2026Bin Lei, Yu Li, Prafulla Kumar Choubey +7Reinforcement Learning RL for Language Model Reasoning
University of Minnesota · Salesforce AI Research
Granularity-Adaptive Credit Assignment for Long-Horizon LLM Agent Reinforcement Learning
Sep 11, 2026Taoran Liang, Yang Liu, Shang Luo +10Credit Assignment in RL Step-Level Credit Assignment in RL
Nankai University · Peking University · Supply Chain Tech Team Y, JD.com +2
DIA: Denoising Intermediate Advantage for Diffusion Policy Optimization
Sep 10, 2026Arjun Sohal, Yuchi Zhao, Miroslav Bogdanovic +1RL for Robotics Step-Level Credit Assignment in RL
University of Toronto, Toronto, ON, Canada · Vector Institute for Artificial Intelligence, Toronto, ON, Canada · Acceleration Consortium, University of Toronto, Toronto, ON, Canada +2
VERPO: Verified Evidence Regularized Policy Optimization
Sep 5, 2026Haijiang Li, Chengyu Lv, Yi Zhang +8RL for Language Model Reasoning Credit Assignment in RL
Independent · Tongji University · Fudan University +3

Oct 7, 2026, cs.CVJ/K move · Enter open · S save

SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages

Raja Kumar, Rajat Koner, Ritwick Chaudhry, Zhuowei Li+2

University of Southern California · Amazon AGI

VLM Reasoning RL for VLMs Visual Reasoning Step-Level Credit Assignment in RL Group Relative Policy Optimization

Open paper AskPDF arXiv

Vision-Language (VL) reasoning requires a model to both extract relevant and accurate information from an image (visual reasoning, VR), and to infer the answer from it (language reasoning, LR). Reinforcement learning with verifiable rewards typically trains both through a single chain-of-thought with a final-answer reward. This gives every CoT token the same sequence-level advantage, failing to distinguish capability specific errors. We propose SPLIT-RL, a staged post-training approach that trains VR and LR in disjoint phases. Because a group's rollouts differ along one capability at a time, the group-relative advantage isolates it, and each phase is optimized using phase-specific reward. We further introduce Claim-Level Advantage (CLA-GRPO), which decomposes VR-phase rollouts into atomic visual claims and provides a fine-grained advantage at claim level based on visual-type group formation. Although trained in two phases, trained policy is evaluated like GRPO model, with a single CoT call at inference time. Under this protocol, SPLIT-RL improves average accuracy over GRPO by 1.4-6.1 points across Qwen3-VL models from 2B to 30B-A3B and InternVL3.5-8B. Evaluating each capability using an oracle based diagnostic shows that answer-only GRPO leaves perception unchanged, whereas SPLIT-RL improves both VR and LR.

Momentum

Related topics

Latest papers 90