Counterfactual Credit Assignment

Overview

Counterfactual credit assignment studies how to determine which actions, reasoning steps, tokens, or agents actually contributed to a reward, especially when feedback arrives only at the end of a task. It matters in cooperative multi-agent systems and language-model reasoning, where a shared outcome can hide useful contributions or harmful decisions. Common approaches compare paired rollouts, intervene on actions or messages, reset from intermediate states, or estimate marginal and Shapley contributions; these signals can then train policies with methods such as PPO and GRPO. A central challenge is making counterfactuals faithful and affordable: replay noise and interacting contributions can distort attribution, so reliable evaluation against executed counterfactual outcomes remains important.

Latest papers

CardsList

30 papers4 papers in the last four weeks, down 43% on the four weeks before. 0.0% of all new papers.

  1. Causal Signal Credit Assignment for Verilog Code Generation via Counterfactual Hardware Verification

    Nov 9, 2026Yali Du, Ming LiRTL Code GenerationCounterfactual Credit Assignment

  2. Same Luck, Different Choice: Twin-World Credit Assignment for Agentic Reinforcement Learning

    Oct 22, 2026Zeropy Surio, Tobias Engel, Mei-Lin Chou +2Step-Level Credit Assignment in RLCounterfactual Credit Assignment

  3. RoleFlow: Self-Evolving Multi-Role Agentic Collaboration via Structural Adaptation

    Oct 21, 2026Yi Ge, Pan Wang, Xuehao Cui +4Multi-Agent LLM SystemsLLM Agent Workflow Optimization

  4. Learning from Revision Consequences: Hindsight Meta-Experience Distillation for Self-Improving Agents

    Oct 6, 2026Qianhan Feng, Zhongzhen Huang, Yakun Zhu +2LLM Agent Skill LearningSelf-Improving Agents

  5. Beyond Entropy: Self-Diagnostic Multi-Role Token Optimization for Video Reasoning

    Oct 2, 2026Yudong Han, Yong Wang, Zaiquan Yang +4Token-Level Credit AssignmentRL for VLMs

  6. VERA: Verifiable Feasibility Representations with Counterfactual Credit for Constrained Multi-Agent Control

    Sep 30, 2026Bo Yin, Dongbo Li, Hongkai Chen +2Multi-Agent ControlConstrained RL

  7. ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning

    Sep 28, 2026Yang Li, Jinhan Yang, hai liu +8Agentic RLCredit Assignment in RL

  8. Learning What to Skip: Counterfactual Credit Assignment for Efficient Multi-Agent LLM Workflows

    Sep 25, 2026Jinfeng Xu, Zheyu Chen, Ziyue Peng +6LLM Agent Workflow OptimizationMulti-Agent LLM Systems

  9. Not All Tokens Deserve Equal Credit: Counterfactual Sensitivity Credit Reallocation for Long-CoT Reasoning

    Jul 30, 2026Qiangqiang He, Zhongheng Wu, ZiJian WangToken-Level Credit AssignmentRL for Language Model Reasoning

  10. CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization

    Jul 28, 2026Bo-Wen Zhang, Junwei He, Wen Wang +5Token-Level Credit AssignmentRubric-Based RL

  11. A Self-Evolving Default Action for Cooperative Tasks with Continuous Action Space

    Jul 21, 2026Shuangyao HuangPolicy GradientCredit Assignment

  12. Counterfactual Shapley Credit Assignment

    Jul 18, 2026Mingxuan Li, Kai-Zhan Lee, Elias BareinboimCredit Assignment in RLTemporal Credit Assignment

  13. RICE-PO: Turning Retrieval Interactions into Credit Signals for Reasoning Agents

    May 25, 2026Mingchen Li, Hansi Zeng, Zhuo Qian +4RL for Language Model ReasoningCredit Assignment in RL

  14. Credit Assignment with Resets in Language Model Reasoning

    May 25, 2026Ankur Samanta, Akshayaa Magesh, Ayush Jain +7RL for Language Model ReasoningCredit Assignment in RL

  15. Reducing Credit Assignment Variance via Counterfactual Reasoning Paths

    Apr 20, 2026Fei Ding, Yongkang Zhang, Youwei Wang +1RL for Language Model ReasoningCredit Assignment in RL

  16. COSAC: Counterfactual Credit Assignment in Sequential Cooperative Teams

    Apr 20, 2026Shripad V. Deshmukh, Jayakumar Subramanian, Raghavendra Addanki +1Sequential MARLCredit Assignment in RL

  17. Marginal-Contribution Policy Gradients under Filtered Feedback for Multi-Agent LLMs

    Apr 3, 2026Elai Ben-Gal, Stela TongMulti-Agent LLM SystemsMulti-Agent System Optimization

  18. Counterfactual Credit Policy Optimization for Multi-Agent Collaboration

    Mar 23, 2026Zhongyi Li, Wan Tian, Jinju Chen +4Counterfactual Credit AssignmentRL for Language Model Reasoning

  19. The Trace Is the State: Exact Credit Assignment for LLM Agent Teams

    Mar 6, 2026Yanjun Chen, Yirong Sun, Hanlin Wang +5Credit AssignmentCounterfactual Credit Assignment

  20. Transformer-Based Multi-Agent Reinforcement Learning for Networked Systems with Long-Range Interactions

    Nov 17, 2025Vidur Sinha, Muhammed Ustaomeroglu, Guannan QuCounterfactual Credit AssignmentGraph Transformer