cs.CLNov 12, 2026

Tracing Knowledge and Reasoning in LLMs

Authors: Yang Wang, Xiao Wang, Yiqi Liu, Chenghao Xiao, Eleni Vasilaki, Goran Nenadic, Chenghua Lin

Organizations: University of Manchester · Shanghai University of Finance and Economics · University of Sheffield

Abstract

Current evaluations of Large Language Models (LLMs) report performance at the dataset or task level, revealing what a model achieves but offering little insight into the internal processing associated with its performance. We propose a task-centric framework for characterising the model-internal processing elicited by a benchmark. It combines attention- and activation-based signals with representation-based measures to characterise benchmarks spanning a continuum from retrieval-dominant to reasoning-dominant and to surface failure modes that task-level scores conceal. We evaluate 16 LLMs across 24 diverse benchmarks and conduct targeted internal-state analyses on representative models and tasks, observing systematic differences in processing between knowledge-centric and reasoning-centric tasks. We shift evaluation from asking what score a model achieves toward characterising the processing patterns associated with that score, potentially supporting more targeted benchmark selection and more informed model choice. We open source our code at https://github.com/extraordinarylab/trace-kr.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Decomposing and Steering Functional Metacognition in Large Language Models

    May 9, 2026Yanshi Li, Xueru Bai, Shuman Liu +2LLM EvaluationLLM Interpretability

  2. Do Knowledge and Reasoning Oriented Benchmarks Induce Stable Representation Signatures in LLMs?

    Oct 16, 2026Parsa Rahimi, Azim Dehghani AmirabadLLM InterpretabilityLanguage Model Probing

  3. Probing the Trajectories of Reasoning Traces in Large Language Models

    Jan 30, 2026Marthe Ballon, Brecht F. Verbeken, Vincent Ginis +1Reasoning Trace AnalysisLanguage Model Reasoning Evaluation