Behaviour Is an Incomplete Measure of Reasoning Development: Cross-surface pre-arrival accessibility and the limits of developmental inference in a recurrent-depth reasoner

Simon Lam-Muir

cs.LGAug 17, 2026

Behaviour Is an Incomplete Measure of Reasoning Development: Cross-surface pre-arrival accessibility and the limits of developmental inference in a recurrent-depth reasoner

Authors: Simon Lam-Muir

Organizations: Prime Calibre, Australia

Abstract

Capability development is routinely inferred from behavioural thresholds, from final checkpoints, or from what a decoder can read out of a hidden state. These quantities need not identify the same event. We study a 30M-parameter recurrent-depth relational reasoner in a closed, oracle-defined world, using dense behavioural trajectories, two training surfaces, preregistered pre-arrival hidden-state probes, prospectively checked evaluability, and explicit untrained and negative controls, holding the training-time and inference-time axes separate throughout. Behaviour first: under one frozen acquisition criterion, three-hop competence cost 70 logical epochs on the symbolic surface and 13,055 on the verbal surface, a 186.5-fold contrast, after which verbal four-hop competence cleared in 8 logical epochs. Across the 13,055-epoch grind, four-hop held-out behaviour never exceeded 3/40 and ended at 0/40. Internal measurement next: on the verbal surface a linear probe recovered future-answer identity before behavioural arrival at 0.056159 against uniform chance 0.025, an untrained control of 0.024758 and a population frequency baseline of 0.048309 (p = 0.012987; 16/40 answer classes contributing). Analogous pre-arrival accessibility survived the surface change, reaching 0.1020 against a zero-step control of 0.0460 (p = 0.000999) at the upstream structural position and 0.0618 at the readout comparator (p = 0.004), with 21/40 classes contributing. Finally, the natural attempt to track that accessibility across training was not cleanly evaluable: probe eligibility is defined by behavioural arrival, so the measured population changes with the measurand. Behavioural competence, internal accessibility, and training-time development are distinct observables, and neither behaviour nor decoder accessibility identifies the computation training acquired; causal intervention is the necessary next step.

Figures & tables

surface	stage $k$	clearing epoch	in-stage cost	boundary status
symbolic	2	2,808	—	bracketed lower bound
symbolic	3	2,878	70	exact
symbolic	4	2,884	6	exact
symbolic	5	3,983	1,099	exact
symbolic	6	3,985	2	exact
symbolic	7	3,986	1	exact

Table 1: Acquisition-cost staircase, both realizations, one frozen convention (strict

>0.95

held-out accuracy; filename-epoch coordinate). In-stage cost is the difference between successive clearing epochs. Both

k=2

rows are not reportable : no predecessor clearance checkpoint survives and epoch

0

is not assumed. Symbolic rows through

k=10

are shown here for tabular readability; Figure 1 A plots the complete symbolic series through

k=20

from the same authoritative staircase source. Verbal

k=5

is right-censored at the manuscript snapshot. Generated from T1-acquisition-staircase.csv .

Figure 1: Behavioural development. (A) In-stage acquisition cost by curriculum stage for both realizations, logarithmic scale. Verbal

k=3

costs

13{,}055

logical epochs against symbolic

70

at the same depth, a derived contrast of

186.5\times

; verbal

k=4

then costs

8

. Verbal

k=5

is plotted at its manuscript-snapshot lower bound of

{\geq}6{,}400

logical epochs using an open marker and upward arrow to denote right-censoring; this is not a completed stage cost. The

k=2

costs are not plotted because their lower boundaries are bracketed and exact in-stage costs are not reportable. The symbolic series is shown through

k=20

; Table 1 prints the

k\leq 10

subset for compactness, and both are generated from the same authoritative staircase source. (B) Held-out four- and five-hop accuracy across

1{,}643

frames of the verbal

k=3

grind, against the

0.95

criterion. Four-hop peaks at

3/40

and ends at

0/40

; five-hop peaks at

2/40

and ends at

0/40

. Both are at floor scale throughout. Plotted directly from the registered per-frame series. Panels use different developmental coordinates: panel A reports logical-epoch stage costs, whereas panel B uses the registered frame coordinate; horizontal positions are not directly comparable across panels.

Figure 2: Pre-arrival answer accessibility on both surfaces. Trained held-out probe accuracy against all three registered references — untrained zero-step control, population frequency baseline, and uniform 40-way chance — with the maximum of the

B=1000

permutation null marked. (A) Verbal surface. (B) Symbolic surface, primary position. Class heterogeneity is printed with each panel. The two panels are shown together for readability; the verbal–symbolic contrast is descriptive and no registered cross-population effect-size test was performed.

Figure 3: Why the developmental accessibility curve is not identifiable. Largest fixed panel of probe-eligible items achievable within a contiguous checkpoint window, by window width, against the registered requirement of forty classes with at least two items each. No window at any width satisfies it, and the whole-stage intersection is empty. Item count alone is necessary but not sufficient: the widest achievable panel reaches

84

items yet still covers too few classes, so zero windows qualify at any width. Items leave the eligible population by improving, so eligibility is coupled to the measured event.

Explore similar work

Aug 4, 2026

The Ignition Is Real, and It Lives at the Readout: Latent composition, difficulty-clocked ignition, and the interface-constituted commit in a recurrent-depth reasoner

We test whether the "compositional ignition" reported in latent-reasoning models is real computation, an instrument artifact, or inherited from verbal training data. We grow an independent realization of a published 30M-parameter recurrent-depth reasoner from scratch (same recipe and seed), film its development, certify fidelity through a pre-registered whole-signature gate, and measure resolution in two channels at once: the vocabulary readout and the hidden state. The ignition is real and lives at the readout: arrival time rises lawfully with problem depth, resolution is sharp and holds, and the signature reproduces across two same-seed realizations with divergent training trajectories. At commitment the decision margin jumps 5.8-8.0 logits in one iteration, exceeding the 90th percentile of near-threshold non-event steps in 96% of cases; the signed margin's zero-crossing there is definitional and carries no evidential weight, so the evidence is that conditioned magnitude. The hidden-state direction snaps in raw geometry, meeting its pre-registered criterion (in the decoder's LayerNorm coordinates it attenuates just below our bar, so the composite decoder-coordinate claim is not confirmed), and then freezes in both (descriptively so in decoder coordinates; angular steps 52.9 to 1.2 degrees over eight iterations), while subsequent displacement is predominantly radial (0.961 of squared-norm) and readout-null to a measured bound (radial logit effect <=5.7e-6). An earlier velocity-trough claim is withdrawn: pre-registered normalization controls showed it coordinate-dependent. Intermediates were never recoverable through the tied readout (relay 0.00). All criteria were frozen before their data; the predictions ledger, including this paper's own withdrawn headline, ships in the companion repository.

Simon Lam-Muir

Prime Calibre, Australia

Compositional Reasoning Representation Geometry in Language Models Latent Reasoning

Ask PDF

Aug 18, 2026

Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depth

Recurrent-depth reasoners aim to solve harder problems by iterating their update longer at test time, but additional iterations can improve, preserve, or degrade an answer. We show that a measurable property of the trained operator, its finite-time dynamical regime (estimated as settling, marginal, or drifting), indicates which of these occurs. We give a sufficient condition for depth-safety: once an operator's per-step displacement is small relative to the decoder margin, the decoded answer cannot change under further iterations. Empirically, on algorithmic tasks trained from

800

unaugmented examples per difficulty tier, settling operators do not degrade with added depth, and on some tasks convert it into higher accuracy on harder unseen instances (Sudoku,

0.19

0.34

past the training horizon). A single terminal fixed-point objective moves the regime and the depth behavior together: removing it induces drift and removes the gains, and adding it to a generic recurrence yields depth-safe extrapolation on carry propagation. We give four operational criteria for useful test-time depth, use them to catalogue failure modes, and, as a consistency check, apply the same measurements to Huginn-3.5B, which falls in the non-settling family.

Ivan Viakhirev, Kirill Borodin, Amirah Almutairi +3

FRC RAS, Saint-Petersburg, Russia · ITMO, Saint-Petersburg, Russia · lab260, Moscow, Russia +4

Test-Time Scaling Recurrent Neural Networks Neural Algorithmic Reasoning Algorithmic Reasoning

Ask PDF

Jun 28, 2026

The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling

We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays as the number of required sequential steps grows. CCB fixes the semantic content of a task and varies only its depth N in {5,...,50} across three structurally distinct regimes: grounded spatial state-tracking, abstract symbolic pointer manipulation, and transitive relational inference. Across 6,000 trials over five frontier and open-weight LLMs we find a consistent pattern of geometric per-step decay with widely separated domain ceilings: on the first two regimes the strongest models retain pd>0.92 across N=50; on the third every model collapses by N=5, with the best model's 50%-success horizon at H0.5~4.7 steps despite pd=0.863. A trace-level metric (TFBC) shows that 14.5% of correct answers across the benchmark are reached via incorrect intermediate reasoning. Forced verbose state-tracking does not move the ceiling (McNemar p=1.000), and the mean step at which reasoning first diverges, k*, predicts within-domain accuracy better than parameter count. CCB and the geometric decay model together reduce a model's long-horizon reasoning profile to one interpretable number per task family.

Shubh Chapra, Dhruv Kumar, Murari Mandal +1

BITS Pilani, Pilani Campus · KIIT Bhubaneswar

LLM Evaluation Structured Reasoning LLM Reasoning

Ask PDF