Reasoning Trace Analysis

Ask about this topic

Overview

Reasoning trace analysis examines how language models and agents develop, revise, and use intermediate steps—not just whether their final answers are correct. Researchers study traces with step-level annotations and benchmarks, statistical analyses of text and hidden-state trajectories, and causal tools such as activation patching; they use these methods to detect errors, shortcuts, harmful behavior, and opportunities to stop or compress reasoning. A central tension is that a plausible trace may be invalid or unrelated to the answer, while internal signals can reveal useful information that the visible text omits. Current work therefore seeks evaluations that test trace validity and causal influence while balancing faithfulness, safety, and inference cost.

Latest papers

CardsList

125 papers6 papers in the last four weeks, down 14% on the four weeks before. 0.0% of all new papers.

  1. Knowing When to Escalate: Disagreement Patterns for Uncertainty-Aware Content Moderation

    Oct 26, 2026Michał Wawer, Jarosław A. ChudziakContent ModerationReasoning Trace Analysis

    Laboratory of The New Ethos, Warsaw University of Technology, Warsaw, Poland · Institute of Computer Science, Warsaw University of Technology, Warsaw, Poland

  2. ARBOR: Locating Reasoning Failures in Long Chains of Thought through Reasoning State Tracking

    Oct 15, 2026Huihan Li, Siyuan Wang, Naga Vamsi Ramana Dinavahi +2Reasoning Trace AnalysisLarge Reasoning Models

    University of Southern California · The Chinese University of Hong Kong · New York University

  3. Effectiveness of Hidden-State Probes for Localizing Reasoning Failures

    Oct 15, 2026Shubham Subhnil, Yannis Belkhiter, Erik Miehling +2Hidden-State ProbingReasoning Trace Analysis

    Trinity College Dublin, Ireland · IBM Research, Dublin, Ireland · Umeå University, Sweden

  4. When Reasoning Traces Conflict with Evidence: How Answer Configuration Shapes Evidence Following

    Oct 14, 2026Shaoang Li, Yanhang Shi, Jian LiReasoning Trace AnalysisEvidence-Grounded QA

    Stony Brook University

  5. Efficient Safety Guardrails through Latent Dual-Hypothesis Reasoning with On-Demand Trace Reconstruction

    Oct 13, 2026Md Asiful Islam, Fahmida Alam, Mihai SurdeanuLLM GuardrailsLLM Safety

    Department of Computer Science University of Arizona, USA

  6. Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes

    Oct 8, 2026Hao Li, Jinye Zhang, Bobo Li +5Affective ComputingMultimodal Emotion Recognition

    School of Computer Science, National Engineering Research Center for Multimedia Software, Wuhan University, Wuhan 430072, China

  7. Correct Answers, Invalid Traces: What Verifiable Grade-School Math Reveals About Chain-of-Thought Traces

    Sep 29, 2026Ratish Puduppully, Pranabendu Misra, Paarth Iyer +3Faithfulness of Language Model ExplanationsCoT Evaluation

    IT University of Copenhagen · Chennai Mathematical Institute · Indian Institute of Technology Jammu +1

  8. On the Token Value Inequality in Efficient Reasoning

    Sep 27, 2026Runjia Zeng, Hang Hua, Yiyang Liu +5RL for Language Model ReasoningEfficient Language Model Reasoning

    Rochester Institute of Technology · Purdue University · MIT-IBM Watson AI Lab +3

  9. From Anomalies to Failures: Constructing Causal Error Graphs for Agentic Trace Diagnosis

    Sep 26, 2026Shu-Xun Yang, Yidong Wang, Zhuoer Feng +7Agent Failure AnalysisReasoning Trace Analysis

    Beijing Institute of Technology, Beijing, China · Zhipu AI, Beijing, China · Tsinghua University, Beijing, China

  10. Externalizing Requirement-to-Repair Artifacts as Observable Traces for LLM-Based Program Repair

    Sep 14, 2026Zewen Tao, Shin-nosuke IshikawaAutomated Program RepairReasoning Trace Analysis

    Rikkyo University

  11. Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

    Sep 8, 2026Mar Gonzàlez I Català, Haitz Sáez de Ocáriz Borde, Davide Murari +3LLM Uncertainty EstimationLLM Reasoning

    University of Cambridge · Harvey Mudd College

  12. Constructing and Evaluating Clinical Reasoning Trajectories for Medical Agent

    Sep 4, 2026Yunqi Zhu, Wen Tang, Jinghao Niu +2Reasoning EvaluationClinical Reasoning

    Guangzhou University, Guangzhou, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China

  13. Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning

    Sep 3, 2026Kevin Du, Alexander Hoyle, Laura Ruis +1CoT FaithfulnessProcess Reward Models

    ETH Zürich · Work done during an internship at Cohere. · MIT +1

  14. The Answer Is Not the Argument

    Aug 31, 2026Will Yeadon, Sergio Juárez, Paul Mackay +5LLM Answer VerificationCoT Monitoring

    Department of Physics, Durham University, Durham DH1 3LE, UK · Escuela de Ingenier´ıa de Telecomunicaci´on, Department of Signal Theory and Communications, University of Vigo, Vigo E-36310, Spain · Independent Researcher

  15. Cognitive Profiling of LRMs' Reasoning Traces Using Bloom's Taxonomy

    Aug 24, 2026Maria-Eleni Zoumpoulidi, Georgios Paraskevopoulos, Alexandros PotamianosLarge Reasoning ModelsReasoning Trace Analysis

    Institute for Language and Speech Processing, Athena Research Center, Greece · Speech and Language Processing Group, National Technical University of Athens, Greece

  16. Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing

    Aug 18, 2026Kang Chen, Sihan Zhao, Yixin Cao +1Mixture-of-Experts InterpretabilityLLM Interpretability

    Fudan University · Shanghai Innovation Institute

  17. The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics

    Aug 4, 2026Shashwat Sourav, Aishwarya BalwaniLLM ReliabilityBoolean Satisfiability

    Department of Physics, Washington University in St. Louis · Department of Developmental Neurobiology, St. Jude Children’s Research Hospital

  18. HPFA: Hypergraph-Based Paired Failure Attribution for LLM Reasoning

    Aug 3, 2026Runchuan Zhu, Hongbin Lai, Bowen Jiang +4LLM ReasoningReasoning Trace Analysis

    National University of Singapore · Peking University · University of Texas at Austin +1

  19. A False Average: Pooled CoT-Monitor Accuracy Conceals a Reasoning-Dependent Fragility

    Aug 1, 2026Shikhar Shiromani, Leo Richter, Noah SiegelReward HackingAdversarial Attacks on LLMs

    Pivotal Research · University College London

  20. TrAC: Trace-Conditioned Answer Consistency for Efficient Uncertainty Quantification in LLMs

    Aug 1, 2026Dahai Yu, Lin Jiang, Rongchao Xu +1LLM EvaluationReasoning Consistency in Language Models

    Florida State University Tallahassee, Florida, USA

  21. A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models

    Jul 28, 2026Vivek Shukla, Varun Shukla, Atul +2CoT FaithfulnessMathematical Reasoning Benchmarks

    Allenhouse Institute of Technology Kanpur, India

  22. How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories

    Jul 28, 2026Hui Wei, Junda Wu, Sheldon Yu +8LLM InterpretabilityRepresentation Geometry in Language Models

    UC Merced · UC San Diego · UIUC +1

  23. Reason Popper-ly: Patching In-Context Reasoning with Inductive Logic Programming

    Jul 25, 2026Zirong Chen, Meiyi MaReasoning Trace AnalysisReasoning Consistency in Language Models

    Department of Computer Science, Vanderbilt University, USA

  24. Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models

    Jul 24, 2026Junlin Fang, Do Nguyen-Thanh, Xiaogang Xu +2LLM Hallucination DetectionLarge Reasoning Models

    College of Computing and Data Science, Nanyang Technological University · Zhejiang University · Australian Artificial Intelligence Institute, University of Technology Sydney +1

  25. How to Build Marcus's Algebraic Mind: From Minsky's Emotion-Machine Viewpoint

    Jul 18, 2026Hiroyuki Chuma, Kanji Otsuka, Yoichi SatoComputational Cognitive ModelingVector Symbolic Architectures

    Institute of Innovation Research, Hitotsubashi University (Professor Emeritus) · Meisei University (Professor Emeritus) · Shuhari System

  26. How Far Can Root Cause Analysis Go on Real-World Telemetry Data?

    Jul 15, 2026Athira Gopal, Ashwanth KrishnanAgent Failure AnalysisRoot Cause Analysis

    QPIAI India

  27. TRACE: An Operational Reasoning Schema for Auditable Agentic Commitments

    Jul 14, 2026Edward Y. Chang, Emily J. ChangData ProvenanceReasoning Evaluation

    Stanford University · Quadrivium AI

  28. Length Penalties Make Chain-of-Thought Less Monitorable

    Jul 8, 2026Bryce LittleCoT FaithfulnessOverthinking in Language Models

  29. Detecting Answer-Driven Reasoning in LLM-Based Educational Tutors via Truncated Chain-of-Thought Auditing

    Jul 6, 2026Bonan Shen, Dingyan Shang, Youting Wang +1CoT FaithfulnessIntelligent Tutoring Systems

    Independent Researcher United States · Northeastern University Boston, MA, USA · Syracuse University Syracuse, NY, USA

  30. Reading Between the Dots: Decoding Hidden Computation across Filler Tokens

    Jul 3, 2026Kaley Brauer, Claudio Mayrink Verdun, Samuel MarksLLM AuditingLLM Interpretability

    Harvard University · 4Anthropic