cs.AINov 18, 2026

Modeling LLM Reasoning Dynamics with Structural Representations

Authors: Ilia Karpov, Andrey Atamanyuk, Dmitry Ilvovsky, Aleksandr Anokhin, Alex Elizov, Dmitry Zagorulkin, Sergei O. Kuznetsov

Organizations: National Research University Higher School of Economics Moscow, Russia · Independent researcher Moscow, Russia

Abstract

Language-model reasoning alternates between sustained activities and local operations. We study how this temporal structure predicts continuation and the computation needed to analyze it. T/E/S combines text, local events, and spans of activity. Across reasoning and information-seeking tasks, events and spans improve next-outcome prediction in 70 of 76 evaluation groups for a Transformer and 74 of 76 for Qwen with LoRA. Span duration and preceding events are associated with transitions, and retaining event history across span boundaries improves prediction. We formulate budget-allocation counting and action-conditional continuation queries, prove counting hardness under explicit encodings, and derive conditions for affordable decision certification. In finite-process experiments, bounds that cancel shared continuation mass recover the same optimal actions as full dynamic programming while reducing mean inference CPU time by 77.7%. On 71 held-out HotpotQA and MuSiQue questions, a compact structural observer improves prediction of whether a fixed inference procedure finishes within a CPU allowance: Brier loss falls from 0.0836 to 0.0756. An annotation audit identifies substantial errors in that observer, separating this computational result from semantic validation of its features.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ROI-Reasoning: Rational Optimization for Inference via Pre-Computation Meta-Cognition

    Jan 7, 2026Muyang Zhao, Chunhao Tian, Qi Qi +1Adaptive Reasoning Resource Allocation for Language ModelsCost-Aware Inference

  2. Tyler: Typed Latent Reasoning for Language Models -- When to Think, What to Compute, and How Much to Allocate

    Jun 15, 2026Hanyu Lin, Min Cai, Jiawei Wen +1Latent ReasoningLLM Reasoning

  3. Structured Thoughts For Improved Reasoning And Context Pruning

    Jul 11, 2026Zain Sarwar, Supriyo Chakraborty, Berkcan Kapusuzoglu +5Efficient Language Model ReasoningStructured CoT Reasoning