Step-Level Credit Assignment

Step-level credit assignment studies how to distribute a task’s reward across the tokens, reasoning steps, or tool-use turns that contributed to the result, rather than giving every part of a trajectory the same signal. Approaches include process reward models, verifier-based progress scores, comparisons between alternative or counterfactual trajectories, and value estimates that share information across trajectory graphs. These methods are tested in settings such as mathematical reasoning, code generation, search, and tool-using agents, where outcome rewards can be sparse and trajectories long. The main challenge is making dense feedback accurately reflect causal contribution without introducing noisy or misleading rewards, especially when verifiers or intermediate labels are unavailable.