Fine-Grained Credit Assignment

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

All topics

Marginal-Contribution Policy Gradients under Filtered Feedback for Multi-Agent LLMs
Apr 3, 2026Elai Ben-Gal, Stela TongMulti-Agent LLM Systems Multi-Agent System Optimization
Department of Mathematics Stanford University · Graduate School of Business Stanford University
Explicit Credit Assignment through Local Rewards and Dependence Graphs in Multi-Agent Reinforcement Learning
Jan 29, 2026Bang Giang Le, Viet Cuong TaCredit Assignment in RL Credit Assignment
Human-Machine Interaction Laboratory VNU University of Engineering and Technology, Hanoi, Vietnam

Apr 3, 2026, cs.LGJ/K move · Enter open · S save

Marginal-Contribution Policy Gradients under Filtered Feedback for Multi-Agent LLMs

Elai Ben-Gal, Stela Tong

Department of Mathematics Stanford University · Graduate School of Business Stanford University

Multi-Agent LLM Systems Multi-Agent System Optimization Credit Assignment Cooperative MARL Counterfactual Credit Assignment Fine-Grained Credit Assignment+1

Open paper AskPDF arXiv

We develop a unified treatment of credit assignment for RL training in multi-agent LLM systems. We show that observed reward alone cannot distinguish an agent that determines it from one that never affects it, and that standard shared-reward training performs exact gradient ascent on each agent's private utility rather than system performance. Moreover, we prove no single scalar per agent can consistently account for joint performance once agents interact. We thus develop the unique background-dependent notion of marginal contribution satisfying natural consistency requirements. From it we derive gradient-correct marginal contribution training signals, identify them from filtered feedback, and optimally allocate a budget of exact counterfactual evaluations against learned-signal error. Instantiated in GRPO, our signal improves routed GSM8K accuracy over winner-take-all training at no extra generation cost.

Momentum

Related topics

Latest papers 32

Marginal-Contribution Policy Gradients under Filtered Feedback for Multi-Agent LLMs

Explicit Credit Assignment through Local Rewards and Dependence Graphs in Multi-Agent Reinforcement Learning