cs.LGSep 27, 2026
SaveA Spectral Theory of Compositional Learning
Organizations: University of Manchester
Abstract
How does compositional reasoning emerge during learning? We address this question by mathematically analyzing the learning dynamics of deep linear networks. We train these networks in structured synthetic environments and derive a theory linking the structure of experience to compositional learning. Our theory predicts when compositional inferences emerge, whether they are identifiable from the available evidence, and how new linking evidence can rapidly unlock previously unavailable inferences. These results provide a qualitative explanation for several phenomena observed in human cognition. They account for why a composition can fail despite knowing its premises, why similar compositions can emerge at different times, and how a single linking fact can suddenly enable many new inferences. Taken together, these findings establish a mathematical link between the statistical structure of experience and the development of compositional reasoning.
Figures & tables
Figure 1: Example relational world.
Figure 2: Experimental manipulations of compositional learning.
Figure 3: Fast and slow laws differ in spectral support. (a) Contribution of each spectral mode to the initial error of the fast and slow laws, normalized within each law and plotted against its relative decay rate, . (b) Predicted geometric error for the same laws in one world. The fast law places more weight on faster modes and therefore converges earlier than the slow law.
Figure 4: Compositional laws emerge at law-specific times predicted by theory. Three laws are shown. a–c , instantaneous accuracy on held-out queries . Shading is s.e.m. across 200 worlds. The first 3,000 epochs are shown; Appendix C.1 gives full-run event times. The corresponding geometric trajectories are shown in Figure C.3 a–c .
Figure 5: A single linking fact makes an underdetermined composition learnable. a–c , held-out accuracy. Shading is s.e.m. across 200 worlds. The corresponding geometric trajectories are shown in Figure C.3 d–f .
Figure 6: One linking fact unlocks many cross-structure inferences. a–c , absolute number of correctly answered held-out cross-structure comparisons, averaged across worlds. Shading is s.e.m. across 200 worlds. The corresponding geometric trajectories are shown in Figure C.3 g–i .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Experiment | Setting | |||
| 1 | 0.03 | 0.03 | 0.03 | |
| Total epochs | 80,000 | 10,000 | 10,000 | |
| Original evaluation interval | 25 | 25 | 25 | |
| RK4 substeps per epoch | — | 4 | 8 | |
| 2 | 0.03 | 0.03 | 0.03 | |
| Pre-intervention epochs | 50,000 | 6,000 | 3,500 |
Table B.1: Run settings for the selected 200 worlds per depth. One epoch visits every constraint row once, including repeated facts and anchors. For staged runs, is set using the pre-intervention matrix and retained after the intervention. Shallow predictions use the discrete closed form. The original intervals are supplemented by denser replay evaluations for Experiment 3 and the retention controls ( Appendix B.5 ).
Figure C.1: Emergence times across all 16 laws and 200 worlds. a–c , predicted versus observed emergence epochs for all 3,200 law-worlds at each depth; the diagonal denotes equality. Points overlap when event times coincide on the evaluation grid. d–f , median observed event time for each fixed law, with interquartile ranges across worlds. All timing axes are logarithmic; limits differ across depths. Error bars describe between-world variation, not uncertainty in the median.
Figure C.2: Linking two structures can temporarily disrupt retrieval of original premises. Mean retrieval of the 24 original premise facts in Experiment 3, scored within each fact’s own lattice, over the same 200 worlds per depth as Figure 6 . The ordinate refers to these premises, not all 32 original training facts. Blue and red curves show linked and no-link networks; dashed curves show theory predictions without trajectory-specific fitting. Shading is s.e.m. across worlds. Time is linear below ten epochs and logarithmic above. The first 30,000 post-intervention epochs are shown; recovery assessments use the full 40,000-epoch run. Curves connect sampled evaluations. Mean retention can approach 100% while an individual world still has an error.
Figure C.3: Geometric error in the three experiments. a–c , Experiment 1, and d–f , Experiment 2: query-to-target distance normalized by median nearest-neighbour spacing among candidates. g–i , Experiment 3: distance between each cross-structure query and its correct target, normalized by median nearest-neighbour spacing among the nine destination entities. Curves are geometric means across 200 worlds, shaded by log-space s.e.m.; dashed lines show predictions.
| Experiment | Measure | |||
|---|---|---|---|---|
| 1 | Accuracy, mean | 0.9999 | 0.9997 | 0.9989 |
| Log geometric error, mean | 1.0000 | 1.0000 | 0.9999 | |
| 2 | Accuracy, mean | 0.9978 | 0.9808 | 0.9359 |
| Accuracy, median absolute error (pp) | 0.00 | 0.22 | 0.25 | |
| Log geometric error, mean | 1.0000 | 0.9994 | 0.9977 | |
| Geometric error, median ratio | 1.000 | 1.020 | 1.027 |
Table C.1: Agreement between predicted and observed trajectories. Values are means over worlds (over law-worlds in Experiment 1) of the per-trajectory , and for Experiment 2 also medians over worlds of the mean absolute accuracy difference and of the geometric-error ratio. Experiments 2 and 3 report the linked condition; no-link trajectories are nearly flat, and at the 95th percentile their predictions differ by at most 0.17 percentage points in accuracy and a factor of 1.002 in geometric error. Horizons are 80,000 epochs at and 10,000 at in Experiment 1, and 2,000 and 30,000 epochs after the linking fact in Experiments 2 and 3.