Credit Assignment in RL

RL: Reinforcement Learning

Overview

Credit assignment asks which decisions deserve credit or blame when rewards arrive late or apply to an entire trajectory, a central problem in long-horizon and cooperative tasks. Traditional approaches include temporal-difference learning, eligibility traces, critics and advantage estimation, while reward shaping, counterfactual comparisons, and Shapley values offer ways to make contributions more explicit. Recent work on language-model agents often assigns feedback at the token, step, or turn level using process rewards, hindsight, trajectory graphs, or comparisons between alternative rollouts; related methods address multi-agent control and robotics. The key tension is that denser signals can improve learning but may be noisy, biased, or expensive, and it remains difficult to establish that a local action actually caused progress rather than merely accompanied it.

Latest papers

CardsList

229 papers16 papers in the last four weeks, down 69% on the four weeks before. 0.1% of all new papers.

  1. Depth Before Breadth: Reverse-to-Forward Curriculum with Advantage Decomposition for LLM Agent

    Nov 18, 2026Siyuan Du, Yuxin Liu, Meng Tang +4RL for LLM AgentsLong-Horizon RL

  2. Eyes in the Update? Learning Where Rewards Land in Multimodal Reasoning

    Nov 17, 2026Lixuan He, Tianzhe Chu, Shulin Tian +3RL for VLMsCredit Assignment in RL

  3. Delay-Aware Reinforcement Learning for O-RAN Control under Stochastic Reward Feedback

    Nov 14, 2026Xingqi Wu, Junaid Farooq, Tao Li +2RL ControlDelayed RL

  4. Future-Informed Actor-Critic

    Nov 6, 2026Samy Mokeddem, Gaspard Lambrechts, Adrien Bolland +1Actor-Critic MethodsCredit Assignment in RL

  5. Turns Matter: Advancing Streaming Video RL via Turn-Level Credit

    Nov 4, 2026Shuai Dong, Kanghui Tian, Linghan Chen +20Streaming Video UnderstandingRL for Video Reasoning

  6. Hindsight Credit Assignment over Trees for Agentic RL

    Nov 2, 2026Penghong Zhao, Yihao Chen, Maoheng Yuan +8Reinforcement LearningRL for LLM Agents

  7. TSBPO: Tree-Structured Branching for Policy Optimization in Interactive Language Agents

    Oct 30, 2026Rui Yu, Chong Mu, Jinchuan ZhangRL for LLM AgentsMulti-Turn RL

  8. FOCUS: Active Critic Learning for Long-Horizon Language Model Agents

    Oct 26, 2026Zhengping Jiang, William Jurayj, Liaoyaqi Wang +3RL for LLM AgentsLong-Horizon RL

  9. When to Stop Penalizing Actions? Martingale GRPO for VLA Reinforcement Learning

    Oct 23, 2026Zongyao Zhao, Cheng-Jie Zhao, Ruihang Jiang +3Policy OptimizationGroup Relative Policy Optimization

  10. RECAP: Looking Once Is Not Enough for Vision-Language Reasoning

    Oct 22, 2026Zhaolu Kang, Tailong Luo, Chenxin Li +13Reasoning for VLMsVisual Evidence Sensitivity in VLMs

  11. AtomicRL: Atomic Reinforcement Learning for Long-Horizon Robotic Manipulation

    Oct 21, 2026Likui Zhang, Zhe Zhu, Zhihao Zhan +7RL for RoboticsLong-Horizon Robotic Manipulation

  12. One Act, Many Masks: Effect-Marginal Policy Optimization for Long-Horizon Agents

    Oct 21, 2026Shaojin Ma, Ruitao Wang, Pengyi Li +6Policy OptimizationRL for LLM Agents

  13. TrajEvolve: Trajectory Evolution for Reinforcement Learning with Hindsight Credit Assignment

    Oct 20, 2026Yi Wen, Hao Chen, Wanyu Wang +9RL for Language Model ReasoningRL Post-Training

  14. Cooperative Multi-Agent Reinforcement Learning for Market Making on Prediction-Market Order Books

    Oct 19, 2026Skander LejmiMarket MakingAlgorithmic Trading

  15. Rubrics Beyond Evaluation: Exploiting Rubric Structure in Reinforcement Learning

    Oct 19, 2026Yuxun Huang, Yunyao Zhang, ZeLiang Chen +3Rubric-Based RLCredit Assignment in RL

  16. Grounded Credit Assignment in Reinforcement Learning for Multi-Turn Search Agents

    Oct 16, 2026Qirui Liu, Hao Chen, Yi Wen +7Credit Assignment in RLReinforcement Learning with Verifiable Rewards

  17. What Makes a Social Agent's Interactions Effective? Learning from Verifiable Outcomes in Paired Worlds

    Oct 14, 2026Caixin Kang, Ke Wang, Yuchuan Wu +5LLM Agent EvaluationLLM Agent Training

  18. ForkGRPO: Efficient Exploration and Fine-Grained Credit Assignment for Diffusion Language Models

    Oct 14, 2026Zhaoyan Jia, Chengwei Zhou, Gourav Datta +1RL for Language Model ReasoningRL for Diffusion Models

  19. SkillScaffold: Scaffolding Agentic Reinforcement Learning with Policy-Adaptive Skill Credit Assignment and Selective Injection

    Oct 13, 2026Wenlin Zhong, Hao Wang, Kun Yuan +6Reinforcement LearningAgentic RL

  20. FlowCred: Hierarchical Credit Assignment and Functional Coordination for Joint Text-to-Video Post-Training

    Oct 13, 2026Mingxuan Zhu, Yanwei Huang, Shiyuan Liu +4T2V GenerationPrompt Optimization for T2V Generation

  21. CoCA: Criterion-Aware Credit Assignment for Rubric-Based Reinforcement Learning

    Oct 13, 2026Guangcheng Zhu, Yingfan Ma, Xing Zheng +7RL for Language ModelsRubric-Based RL

  22. DeltaTTG: Training Multi-Turn Agents with a Token-to-Go Progress Meter

    Oct 13, 2026Xuyan Ye, Xiyan Fu, Shengda Fan +3RL for LLM AgentsReinforcement Learning

  23. Double-Rollout Reinforcement Learning with Sampling-based Reward Estimation

    Oct 12, 2026Haifeng Qian, David Mosallanezhad, Christian Munley +2Reinforcement LearningRL for Language Model Reasoning

  24. Equal-information evaluation of credit assignment in modular financial reinforcement learning

    Oct 10, 2026Gennadiy Chistiakov, Yury YanovichCredit Assignment in RLReinforcement Learning