Language-model reasoning alternates between sustained activities and local operations. We study how this temporal structure predicts continuation and the computation needed to analyze it. T/E/S combines text, local events, and spans of activity. Across reasoning and information-seeking tasks, events and spans improve next-outcome prediction in 70 of 76 evaluation groups for a Transformer and 74 of 76 for Qwen with LoRA. Span duration and preceding events are associated with transitions, and retaining event history across span boundaries improves prediction. We formulate budget-allocation counting and action-conditional continuation queries, prove counting hardness under explicit encodings, and derive conditions for affordable decision certification. In finite-process experiments, bounds that cancel shared continuation mass recover the same optimal actions as full dynamic programming while reducing mean inference CPU time by 77.7%. On 71 held-out HotpotQA and MuSiQue questions, a compact structural observer improves prediction of whether a fixed inference procedure finishes within a CPU allowance: Brier loss falls from 0.0836 to 0.0756. An annotation audit identifies substantial errors in that observer, separating this computational result from semantic validation of its features.
Figures & tables
Figure 1. T/E/S separates a local check from the surrounding derivation. Modes can recur; only the observed prefix is available for prediction.
Inputs
Transformer
Qwen
Text
0.851
0.698
Text+events
0.805
0.685
Text+spans
0.734
0.603
Text+events+spans
0.719
0.598
Table 1. Input ablation across corpus solvers: 282 validation questions. Weighted multitask loss; lower is better.
Method
CPU time, ms ↓
Full probability computation
12.94
Early selection with separate bounds
5.62
Early selection with shared continuations
2.88
Table 2. Action selection on 404 queries to finite-process models. All methods select the same actions as full computation.
Predictor inputs
Brier ↓
Counters and length
0.0926
Text + counters
0.0836
Text + counters + observed spans
0.0767
Text + counters + observed E/S
0.0756
Table 3. Predicting inference availability for Qwen3.6 retrieval: 71 HotpotQA/MuSiQue questions. Lower Brier loss is better.
Rule
F1 ↑
Tokens
CPU, ms
Always answer
0.4339
561
0.0
Always retrieve
0.5337
1410
0.0
Direct text control
0.5187
1369
271.8
Inference selected by T
0.5184
1372
276.8
Inference selected by T/E/S
0.5192
1372
278.8
Matched random calls
0.5192
1371
281.2
Table 4. Qwen3.6 retrieval on 71 HotpotQA/MuSiQue questions: F1, full output tokens, controller CPU. Averages include four budgets.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Model
NLL
Struct.
Length/END
Semi-Markov
0.550
0.159
0.289
Explicit memory
0.546
0.159
0.315
GRU
0.529
0.160
0.272
Neural Hawkes
0.533
0.154
0.269
Appendix
Table 5. Process approximations: MATH500, GPQA Diamond, BBH. NLL in nats/step; two energy scores. Lower is better.
Large language models (LLMs) can achieve strong reasoning performance with sufficient computation, but they do not inherently know how much computation a task requires. We study budgeted inference-time reasoning for multiple problems under a strict global token constraint and formulate it as an Ordered Stochastic Multiple-Choice Knapsack Problem (OS-MCKP). This perspective highlights a meta-cognitive requirement---anticipating problem difficulty and allocating computation strategically. We propose ROI-Reasoning, a two-stage framework that endows LLMs with intrinsic, budget-aware rationality. First, Meta-Cognitive Fine-Tuning teaches models to predict reasoning cost levels and decide whether a problem is worth attempting before generation. Next, Rationality-Aware Reinforcement Learning optimizes sequential decision making under a hard token budget, allowing models to learn long-horizon allocation strategies. Experiments across mathematical reasoning, multiple-choice question answering, and code generation show that ROI-Reasoning improves scores and generally reduces reference-order regret under tight budgets, with up to a 417% relative score improvement over the strongest same-backbone baseline in mathematics.
Muyang Zhao, Chunhao Tian, Qi Qi +1
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead. Latent reasoning offers a promising alternative by carrying part of the computation in continuous representations. However, existing methods typically predefine when latent computation is invoked and how it is allocated during decoding, leaving a key problem unresolved: when to invoke latent computation, what type of computation to perform, and how much budget to allocate. We propose Typed Latent Reasoning (Tyler), a typed and budget-aware framework for latent reasoning during autoregressive decoding. Tyler learns a policy that, at each decoding step, chooses between emitting a text token and switching to a latent computation module specialized for a particular reasoning function. Once invoked, an operator maps the current reasoning state into latent tokens that support global planning, local state updates, or reusable procedural abstraction. Across extensive experiments on three backbone LLMs, Tyler improves accuracy by up to 14.49 points over CoT and by up to 4.30 points over the strongest competing baseline. It further generalizes across diverse reasoning domains and achieves the best final-stage performance with the lowest forgetting.
Large language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured Thoughts, a framework that organizes reasoning into alternating <try> and <outcome> blocks: <try> captures exploratory scratch work, while <outcome> contains the distilled conclusion of that step. We construct a dataset of structured thoughts by segmenting reasoning traces into <try> blocks and prompting an LLM to summarize each step into its corresponding <outcome>. Fine-tuning pretrained foundation models on this reformatted data produces models that adopt the structured reasoning style, leading to performance gains of up to 8.08% on reasoning benchmarks compared to standard SFT. The explicit structure also enables context pruning: after each <try>/<outcome> pair, the <try> can be pruned, allowing the model to retain conclusions without keeping the full scratch work in the context. A proof-of-concept pruning implementation achieves an average of 85% memory / context savings with an 8.67% performance drop across mathematical tasks.