cs.AIFeb 2, 2026

From Outcome to Representation: Tracing Reasoning Mechanisms through Integrated Policy Gradient

Authors: Changming Li, Kaixing Zhang, Haoyun Xu, Yingdong Shi, Zheng Zhang, Kaitao Song, Kan Ren

Organizations: ShanghaiTech University · Independent Researcher

Abstract

Large Language Models (LLMs) demonstrate remarkable reasoning capabilities, yet the internal mechanisms driving these multi-step processes remain opaque. Existing mechanistic interpretability approaches often rely on superficial text-pattern co-occurrences or capture only short-term effects, struggling to trace the long-horizon, outcome-driven influence of internal components such as neurons or sparse features in multi-step reasoning.In this paper, we introduce Integrated Policy Gradient (IPG), a novel, training-free framework that brings policy-based attribution into mechanistic interpretability for LLM reasoning. Grounded in outcome-oriented and sequential-influence-aware principles, it backpropagates outcome-level, non-differentiable behavioral signals (e.g., reasoning outcome correctness) through entire inference trajectories, utilizing path integration to isolate outcome-relevant mediators at component level for reliable attribution. This provides a linear approximation of Causal Mediation Analysis (CMA) in reasoning. Empirical evaluations demonstrate that \method achieves precise localization and reveals mechanistic insights by identifying sparse, concentrated subsets of internal components associated with specific reasoning behaviors. Our method also enables modulation of reasoning capabilities, offering a powerful and reliable framework for understanding LLM reasoning. The project page is at https://foundation-model-research.github.io/IPG .

Figures & tables

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention

    May 31, 2026Shuochen Chang, Tong Bai, Xiaofeng Zhang +5LLM InterpretabilityCausal Interventions in Language Models

  2. Integrated and Cross-Architecture Interpretation of LLM Reasoning

    May 27, 2026Leonardo Matthew Yauw, Wei-Bin Kou, Yujiu YangTransformer InterpretabilityLLM Interpretability

  3. AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency

    Apr 17, 2026Max Henning Höth, Kristian Kersting, Björn Deiseroth +1CoT FaithfulnessFeature Attribution