LLM agents are commonly trained with RLVR, but the sparse nature of typical RLVR rewards makes credit assignment challenging, especially for long-horizon tasks. We propose Self-Credit, an end-to-end agentic RL method that creates dense supervision signals from sparse outcome rewards using self-generated credit assignment via intra-group rescaling. Self-Credit builds on group-based RL with three components. First, during its rollouts, the policy model generates summaries with citations to earlier turns containing critical evidence, which serve as self-generated credit. Second, we refine this model-generated credit with intra-group rescaling, which compares cited actions across successful and failed rollouts for the same prompt. This approach uses group-level agreement and disagreement to remove undeserved credit and further emphasize credit assigned to critical actions. Third, advantage reweighting uses the rescaled credit weights to update the advantages of the cited turns. On BrowseComp-Plus, Self-Credit improves over the base models by 37.0 and 36.2 points on Qwen3 8B and 32B respectively, and outperforms the strongest agentic RL baselines by 8.5 and 5.6 points, respectively. Self-Credit also improves over the strongest baselines by 4.9 points on GAIA and 4.7 points on xBench-DeepSearch. Analyses show that self-generated credit provides useful signals for selecting critical actions, and that intra-group rescaling, especially on failed rollouts, is important across model scales.
Figures & tables
Figure 1: Self-Credit leverages self-generated summaries with Intra-group rescaling to provide credit assignment for long-horizon agentic RL. Self-Credit outperforms strong baselines on challenging long-horizon tasks, such as BrowseComp-Plus.
Figure 2: Illustration of Self-Credit . (1) Self-Credit leverages self-compaction with turn citations for credit assignment. (2) Across rollouts for the same prompt, intra-group rescaling removes extra credit from cited turns that appear in both successful and failed rollouts, strengthens positive credit for turns unique to successes, and assigns stronger negative credit to turns that are only cited by failures. (3) The rescaled weights are used for advantage reweighting.
\Block 2-1 Method
\Block 2-1 Long Context Management
\Block [c]1-2 Credit Assignment
Form of credits
Source of credits
GRPO
–
–
–
GiGPO
–
Advantage reweighting
Successful and failed rollouts
SUPO
Summary
–
–
ECHO
Turn Selection
Turn masking
Successful rollouts only
Self-Credit
Summary
Advantage reweighting
Successful and failed rollouts
Table 1: Key differences between Self-Credit and baseline methods. Self-Credit leverages self-generated summaries during training to provide credit assignment via advantage reweighting that is grounded in both successful and failed rollouts via intra-group rescaling.
Table 4
Figure 3: Training dynamics on BrowseComp-Plus on Qwen3-32B models of RL baselines with different credit assignment methods. Self-Credit learns stably across steps without excessive use of tool calls.
Table 4: Ablations of Self-Credit on BrowseComp-Plus. Intra-group rescaling, penalties for repeated failures, and self-generated credits are important for the downstream performance using Self-Credit .
Table 5: Analysis of the quality of tool use on BrowseComp-Plus for different training methods. Gold Searched and Gold Opened measure the percentage of questions for which at least one gold document appears among the search results or is opened, respectively. Answer Recall and Evidence Recall measure the fractions of documents containing the gold answer or supporting evidence that are found through search or page opening.
Table 6: Quality of citations generated by different credit assignment methods on BrowseComp-Plus. Opened and Searched report F1 (%) between cited turns and turns whose Open or Search actions find gold documents. Judge F1 reports F1 (%) against turns judged relevant by a stronger LLM judge.
Figure 4: Firing rate and precision of Self-Credit intra-group rescaling across the three rules at different training stages on BrowseComp-Plus. Firing rate denotes how frequently intra-group rescaling is applied to a cited turn, and firing precision indicates whether rescaling gives the correct credit to the cited turns. Intra-group rescaling is triggered selectively during training for different models.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
HotpotQA
MuSiQue
FRAMES
Bamboogle
Avg.
GRPO
64.5
29.4
48.4
39.2
45.4
GiGPO
63.2
31.6
51.7
32.0
44.6
SUPO
63.7
30.6
47.8
35.2
44.3
ECHO
63.4
31.8
49.2
33.6
44.5
Self-Credit
65.1
28.0
51.7
41.6
46.6
Appendix
Table 7: Zero-shot generalization on out-of-distribution search-augmented QA tasks.
Figure 5: Example output of Self-Credit model on a BrowseComp-Plus question.