Capable systems often show jagged competence: low average error alongside failures on particular inputs. We ask where it comes from in a task built from two known layers. A lower layer A computes five per-slot sums from records; an upper layer B uses the slot-1 sum and a mode carried over from earlier boards to predict the next board's category, so B cannot be computed from the current A alone: B is a strict extension of A. We train small recurrent networks on B and measure, against exact ground truth, what they acquire of A. The central finding is that learning B gives the network a jagged version of A, measured through probe readability and supervised outputs. It is readable where B needs it (slot 1, 97-98% by a linear probe) and becomes less readable where B does not (the other slots, 5-10%); category misreads concentrate near the slot-1 cutoffs; and when A is trained explicitly it comes out only approximately right. The networks show jagged competence in B: natural KL below 3×10−4 bits coexists with a maximum law TV of about 0.25 on constructed histories. Matched experiments show that even a good A is not enough: connecting a learned A to B cuts misreads 1.3 to 17-fold, the same exact A gives fewer misreads as one-hot inputs (0-2) than as numerical inputs (10-317), one B update makes an exact A inexact unless the B gradient is blocked, and exact A still leaves some B failures. In this task, uneven acquisition, access and preservation of the lower theory explain part of the jagged competence of a network that learns the theory built on it; some failures remain unexplained. Whether the same holds in larger systems is a test to run.
Figures & tables
Figure 1: Two task layers and their history witness. (a) A maps four records to five per-slot sums. B uses their slot-1 category and the preceding mode to update the mode and determine the next-category law. The annotations identify training conditions, not evidence that a network learned the depicted decomposition. (b) The histories end on the same neutral sum board but retain different modes and require different laws. Values are integer twelfths; law entries are ordered L/N/H. The task has 24,435 completed sum boards and 1,555 local histories. Everything shown is exact task ground truth, not model output.
Figure 2: Recoverability, computation and access are distinct measurements, not interchangeable routes. (a) The sums output head and its loss attach to the shared carrier; a dashed optional connection feeds sums output to prediction in the connected variants. (b) The matrix identifies the routes and losses in the three versions and r13 variants; probability targets retain their base version’s routes and replace sampled targets with the exact law. The probe fits a reader on frozen states; the swap replaces an aligned donor block, with selectivity checked separately. No measurement, seed average or control result is implied by the icons; procedures and their calibration are in Appendix C.
condition
recoverability
computation
access
preservation
reliable prediction
r8: records → prediction
✓ 1/5 ⋅ 1/5 ⋅ 1/5
–
–
–
× 0/3
r10: records → prediction
✓ 5/5 ⋅ 5/5 ⋅ 5/5
–
–
–
× 0/3
r10: records → prediction+sums
✓ 5/5 ⋅ 5/5 ⋅ 5/5
–
–
–
× 0/3
r11: records → prediction (Erosion continuation)
✓ 1/5 ⋅ 1/5 ⋅ 1/5
× 0/3
–
×
× 0/3
r11: records → prediction+sums (Erosion continuation)
✓ 3/5 ⋅ 1/5 ⋅ 1/5
✓ 1/3
–
×
✓ 1/3
r12: records → prediction+sums
✓ 5/5 ⋅ 5/5 ⋅ 5/5
× 0/3
–
–
× 0/3
Table 1: Five achievements, original law. ✓ supported; × not supported; – not established: available evidence does not establish this achievement. Fractions are seeds passing out of three, except recoverability: supported slots out of five in seed order 0/1/2. Access: D by design (exact sums supplied); I demonstrated by a matched connection intervention. Complete scope, counts and controls: Supplement Table S1.
Figure 3: Recoverable sums and low average error coexist with specific failures. (a,b) r8 linear/MLP; (c) r10 linear: slots 1–5, 512 held-out boards/slot, majority ticks, shuffled floors and oracle ceiling. Open/filled: update 0/20k (a–c), r10/r8 (d,e); shapes: seeds 0/1/2. (d) Natural KL: 60,000 positions, board-only/mode-blind floors; the log axis omits the oracle’s 0 bits. (e) Law TV: 576 r8 histories/180 r10 rows, zero oracle, r8 bar 0.02 and r10 original-law 0.05 bar (its equal-law bar is 0.02). (f) r12 uniform categorical misreads: 24,435 boards, rendering 0, untrained floor, shaded cutoff bands. Compare seeds within panels; r8/r10 comparisons are historical and tasks differ. Paired probe intervals: Appendix E.
Figure 4: Target noise, access and encoding affect measured errors without guaranteeing prediction closure. Matched r13 comparisons: targets within versions (a,b), numbered access variants (c–e), encodings of exact sums (f). Shapes: seeds 0/1/2; grey crosses: matched update 0; dashed line: zero-error oracle except in (d), whose 0-bit oracle lies outside the logarithmic axis. Categorical misreads: 24,435 boards, rendering 0; natural KL: 60,000 positions; maximum long-panel TV: 384 N3–N8 endpoints, bar 0.02. No pooled means or bands. Controls are matched within panels.
Figure 5: Prediction gradients erode exact sums at update 1, not uniquely under AdamW. Columns: seeds 0–2. (a–c) Compare r11 losses: exact starts on 24,435 boards; updates 1/5/10/50/100 use a fixed 128-board subset. Symlog separates 0/1. (d–f) r13, all 1,555 local histories: every tested active optimizer loses exactness at update 1. Update-1 correct counts, seeds 0/1/2, each out of 1,555: AdamW: 393 / 5 / 518; reduced-lr AdamW: 1,554 / 1,468 / 1,553; SGD: 1,440 / 89 / 1,495; blocked prediction gradient: 1,555 / 1,555 / 1,555. Blocked prediction gradient is a separate weight-decay control; dashed line: exact start/oracle. Movement-matched points: fig. D.2 . r11/r13 is historical; no pooling or bands.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
study
recorded compute
version/review records
r8
CPU-era; processor model not recorded in saved per-run summaries
11
r9
CPU-era; processor model not recorded in saved per-run summaries
1
r10
GPU 3 / CUDA fp32 execution recorded in summaries and host verification; GPU model and driver not recorded in the public study artifacts
20
r11
GPU 3 / CUDA fp32 execution recorded in summaries and host verification; GPU model and driver not recorded in the public study artifacts
13
r12
GPU 3 / CUDA fp32 execution recorded in summaries and host verification; GPU model and driver not recorded in the public study artifacts
14
r13
GPU 3 / CUDA fp32 execution recorded in summaries and host verification; GPU model and driver not recorded in the public study artifacts
44
Appendix
Table A.1: Analytical appendix A. Exact recorded counts or decisions; no pooled reader accuracies. Complete support, settings, controls and per-seed records are in the numbered supplementary tables and released data.
study
registered passes
original-law runs
law TV bar / support
r8
5
9
0.02 (576 history endpoints)
r10
0
9
0.05 (180 law rows)
r11
1
24
0.02 (576 history endpoints)
r12
2
15
0.02 (576 history endpoints)
r13
1
27
0.02 (576 history endpoints)
Appendix
Table B.1: Analytical appendix B. Exact recorded counts or decisions; no pooled reader accuracies. Complete support, settings, controls and per-seed records are in the numbered supplementary tables and released data.
condition
natural KL (bits)
law TV (maximum)
B passes
r8: records → prediction
0.00029829 7.047e-05 0.00021024
0.25 0.2548 0.25649
0/3
r8: records+sums → prediction (sums route only)
0.00033865 6.7874e-05 6.5306e-05
0.017721 0.011403 0.020775
2/3
r8: records+sums → prediction
0.00025483 6.8052e-05 8.0944e-05
0.018665 0.0051108 0.0059933
3/3
r10: records → prediction
0.016034 0.019138 0.016491
0.3219 0.28963 0.30706
0/3
r10: records → prediction+sums
0.032325 0.033959 0.035928
0.27737 0.25519 0.23295
0/3
r10: records+sums → prediction (sums route only)
0.0070603 0.0059933 0.0057117
0.12242 0.079414 0.10517
0/3
Appendix
Table B.2: Access does not guarantee registered reliability. Each vertically ordered triple is seeds 0/1/2, never a pooled mean. Compare within study; cross-study comparisons are historical. r8 full history maximum, r12/r13 long N3–N8 maximum: bar 0.02. r10 is a different task: 180 law rows, original bar 0.05 (equal bar 0.02). Natural panels: 5,000 episodes x 12 rounds. Complete per-seed failed bars, census support and labels: Supplement Table S2.
Figure C.1: Separate probe (a) and swap-test (b) procedures. Readers and alignment fit on training identities only; scoring identities are held out. Positive, no-op and shuffled controls calibrate the procedure, while donor-follow and selectivity concern the evaluated carrier. This schematic has no empirical seed average or accuracy; full controls and support are in Appendix C.
study
law
positive / null
deciding quantity
pass
support
measured value
r8
original
current_board_null
B deciding criteria
no
576
0.1875
r8
original
forced_A
B deciding criteria
yes
576
0
r8
original
forced_raw
B deciding criteria
yes
576
0
r10
original
exact_oracle
B deciding criteria
yes
180 law rows
4.4703e-08
r10
original
mode_blind
B deciding criteria
no
180 law rows
0.48635
r10
original
untrained
B deciding criteria
no
180 law rows
0.42896
Appendix
Table C.1: Analytical appendix C. Exact recorded counts or decisions; no pooled reader accuracies. Complete support, settings, controls and per-seed records are in the numbered supplementary tables and released data.
update-1 condition
correct counts (seeds 0/1/2)
histories per seed
AdamW
393 / 5 / 518
1,555
reduced-lr AdamW
1554 / 1468 / 1553
1,555
SGD
1440 / 89 / 1495
1,555
blocked prediction gradient
1555 / 1555 / 1555
1,555
Appendix
Table D.1: Analytical appendix D. Exact recorded counts or decisions; no pooled reader accuracies. Complete support, settings, controls and per-seed records are in the numbered supplementary tables and released data.
Figure D.1: Recorded r12 learning curves show changes in average error, not sufficient evidence of reliable competence. Every seed and condition is retained; (a–c) original and (d–f) equal laws are separate rows, in seed order 0–2. Open points mark each condition’s own update-0 control; dashed zero is the exact oracle. The quantity is natural KL in bits on 5,000 natural episodes of 12 rounds (60,000 scored positions), not a loss or a census frequency. No smoothing, selected checkpoint or across-seed band is used. Compare conditions within the same seed and law only; cross-study comparisons are historical. Natural KL alone does not establish reliability.
Figure D.2: Exact sums erode across active optimizers at common recorded movement. (a–c) Seeds 0–2, all 1,555 local histories. Compare AdamW, reduced-rate AdamW and SGD within each seed’s three-active-arm common range; points are observed, not interpolated. Blocked prediction gradient is excluded from that range and remains a separate control in fig. 5 . Dashed line: exact start/oracle. No pooled means or confidence bands.
Figure D.3: Recorded r13 noise learning curves show changes in average error, not sufficient evidence of reliable competence. Every seed and condition is retained; (a–c) original; no equal-law arms were run, in seed order 0–2. Open points mark each condition’s own update-0 control; dashed zero is the exact oracle. The quantity is natural KL in bits on 5,000 natural episodes of 12 rounds (60,000 scored positions), not a loss or a census frequency. No smoothing, selected checkpoint or across-seed band is used. Compare conditions within the same seed and law only; cross-study comparisons are historical. Natural KL alone does not establish reliability.
raw linear slot-1 sums reader
seed
correct
boards
gain CI95
converged
r11 records → prediction+sums (sums dose 100%)
0
484
512
0.5332/0.62695
yes
r11 records → prediction+sums (sums dose 100%)
1
491
512
0.60938/0.69336
yes
r11 records → prediction+sums (sums dose 100%)
2
500
512
0.68555/0.76172
yes
r12 records → prediction+sums
0
483
512
0.53315/0.62305
yes
r12 records → prediction+sums
1
509
512
0.64648/0.72852
yes
r12 records → prediction+sums
2
505
512
0.69722/0.77148
yes
Appendix
Table E.1: Analytical appendix E. Exact recorded counts or decisions; no pooled reader accuracies. Complete support, settings, controls and per-seed records are in the numbered supplementary tables and released data.
Figure E.1: Cutoff discrimination can succeed while prediction still fails. The supplementary panel was bound before states were read, with disjoint fit/evaluation identities and fixed renderings. Three seed columns show saved 0/5k/20k sampled- and probability-target prediction-loss checkpoints only, with no network updates: (a–c) accuracy and (d–f) cross-entropy, in seed order 0–2. Compare sampled- and probability-target readers for the same seed and pair; circles/solid lines are 12/13 (135 fitting pairs, 150 evaluation pairs; 300 endpoint labels), squares/dashed lines are 23/24 (12/13 pairs; 26 labels). Grey crosses are the saved shuffled-label floors; balanced constant floors are 0.5 accuracy and ln 2 loss, and the exact ceiling is 1 accuracy / 0 loss. No confidence bands are implied. Distances and rendering variation remain descriptive in Appendix E’s support tables; successful discrimination is not proof of predictive use or of a precision ceiling.
study
condition (all seeds)
seconds (0/1/2)
execution scope
aggregation
r13
AdamW
21.962/19.645/20.531
200-update erosion; recorded execution
examples, not totals
r13
SGD
22.219/18.44/18.835
200-update erosion; recorded execution
examples, not totals
r13
blocked prediction gradient
19.329/17.844/17.231
200-update erosion; recorded execution
examples, not totals
r13
reduced-lr AdamW
19.241/19.387/20.737
200-update erosion; recorded execution
examples, not totals
r8
records → prediction
1855.1/1783.3/1895.1
CPU endpoint
examples, not totals
r8
records+sums → prediction
1765.2/1810.8/1837.4
CPU endpoint
examples, not totals
Appendix
Table F.1: Analytical appendix F. Exact recorded counts or decisions; no pooled reader accuracies. Complete support, settings, controls and per-seed records are in the numbered supplementary tables and released data.
Grokking is a delayed transition from memorisation to generalisation. A network first fits observed examples while remaining near chance on held-out inputs, then generalises abruptly. Prior work has focused mainly on this delay, finding that the network's main loss becomes weight decay after the fitting phase. We name the solution to this optimization problem as the minimum-norm endpoint, and study which tasks make all minimum-norm endpoints generalise from partial observations. Specifically, we model pairwise input tasks as edge colourings of Kq,q, with Latin colourings as our structured family, and classify each observed instance as all-good, mixed, or no-good by its endpoint set. Furthermore, we prove two contrasting results: (1) Finite-abelian-group families are asymptotically all-good from partial observations; (2) Under a uniform random Latin square, all-good tasks become asymptotically rare for observation-based learners, leaving mixed or no-good behaviour. Moreover, experiments show that relabellings of addition generalise despite their dense polynomial representations. Finally, on tasks of composite order, finite-training results suggest that predictors learn a coarse part of the output before the full label.
Zihan Gu, Lulu Zhao, Tianyi Ma +4
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · Alibaba Group +1
How does compositional reasoning emerge during learning? We address this question by mathematically analyzing the learning dynamics of deep linear networks. We train these networks in structured synthetic environments and derive a theory linking the structure of experience to compositional learning. Our theory predicts when compositional inferences emerge, whether they are identifiable from the available evidence, and how new linking evidence can rapidly unlock previously unavailable inferences. These results provide a qualitative explanation for several phenomena observed in human cognition. They account for why a composition can fail despite knowing its premises, why similar compositions can emerge at different times, and how a single linking fact can suddenly enable many new inferences. Taken together, these findings establish a mathematical link between the statistical structure of experience and the development of compositional reasoning.
Compositional generalization\unicodex2013the ability to systematically process novel combinations of known components\unicodex2013is a hallmark of human intelligence; however, its theoretical foundation in neural networks is not yet well understood. This paper establishes a necessary and sufficient condition for provable compositional generalization, precisely characterizing its boundary. Conceptually, the condition consists of two principles: (i) structural alignment, where a model's computational graph aligns with a task's true compositional hierarchy, and (ii) unambiguous minimized representations, where each component encodes adequate but not redundant information on the training data. The result is fully proved and machine-verified in Lean 4 and holds even in few-shot and one-shot regimes. The necessity direction establishes that provable compositional generalization cannot circumvent these requirements, while the sufficiency direction yields a unified inductive bias that jointly governs architectural design, training data properties, and regularization strategies. Building on this condition, we develop an example algorithmic approach, illustrate it through a controlled minimal example, and further demonstrate the condition on the SCAN jump task. All conclusions are derived mathematically without reliance on empirical validation. Our work provides a theoretical characterization of provable compositional generalization.