A learner's present predictions do not fully determine how it will learn next. We study the dynamics of hidden learning structure through two related operations. Fiber Regulation Theory (FRT) describes protected repair of existing function and the learning responses left available after repair. Fiber Acquisition Theory (FAT) describes the formation and renewal of trainable structure, distinguishing neutral installation, access to a learning signal, and additional functional directions. The two operations also constrain one another. A repair obligation can force a change in a learning channel, while acquired features can make new protected repairs feasible or reduce the minimum coefficient cost of existing ones. We characterize joint admissibility and response reachability, and give a finite supervised-learning construction in which the two operations together produce a favorable response absent from either operation alone. We implement protected head repair and fixed-capacity LoRA renewal in Qwen2.5-32B with AdamW, under replay and output distillation. Across two training trajectories differing in seed and within-task example order, the coordinated policy achieves higher teacher-forced Exact than either separate policy in all four comparisons. It also has the lowest macro NLL under both learning rules on the second trajectory. The implementations realize structure-conditioned self-regulation within specified information and action constraints: accessible state readouts and arrived evidence guide repairs and structural renewals, and ordinary learning continues from the modified state.
Figures & tables
Figure 1: Regulation and acquisition within one evolving learner. The complete state st is distinct from the available information it . Regulation repairs existing function; acquisition forms or renews trainable structure. Using It , a policy selects an admissible pair ξt=(ft,at)∈K(st) through available certificates or a valid robust rule. Identity components allow either intervention, both, or no extra write. The state-transition marginal Kˉξt uses the declared execution order. Ordinary learning consumes the resulting state and supplies readouts to later decisions. The FRT–FAT arrow denotes possible cross-effects in admissibility, response and repair cost.
Figure 2: Acquisition-adjusted repair geometry. Two independent exact affine examples of Theorem 5.3 are shown in obligation-response coordinates. The solid old set and dashed enlarged set contain responses attainable with squared coefficient cost at most one. In both examples, q=1 , B=Q=1 and W=2 . (a) Set C=(1,0) , H=(1110) and Ψ=(1,2)⊤ . With v=1/2 , the raw acquired response S=Ψv receives the compensation response −HC†Bv , yielding Lv=D . The target D=(0,1/2)⊤ lies outside the old reachable space. (b) Set C=(1,0,0) , H=(111001) and Ψ=(2,1)⊤ . Both interfaces span the response plane, yet the minimum cost of D=(1,0)⊤ falls from 1 to 2/3 . The arrows in (a) express vector addition, not a learning trajectory.
Original trajectory
Changed trajectory
Learning rule
Policy
Exact
Macro NLL
Exact
Macro NLL
Replay
Baseline
102/128
0.279321
110/128
0.249136
FRT
113/128
0.200758
112/128
0.268962
FAT
114/128
0.216516
110/128
0.263532
Joint FRT–FAT
115/128
0.222998
113/128
0.240527
LwF
Baseline
85/128
0.491530
75/128
0.547257
Table 1: Full-policy outcomes on the same 128-item MASSIVE-derived panel. Exact is teacher-forced label-plus-EOS correctness. Bold marks the highest Exact and lowest macro NLL within each learning rule and trajectory. These are complete-policy comparisons, not fixed-action factorial experiments.
PyTorch 2.10.0, CUDA wheel 12.8, Transformers 4.57.6, PEFT 0.18.1, Accelerate 1.12.0, NumPy 2.3.5; math SDPA, nonreentrant checkpointing, TF32 disabled
Appendix
Table 2: Execution settings of the tested model and learning rules.
Learning rule
Policy
FRT commits
FAT commits
Replay redirections
Replay
Baseline
0
0
0
Regulation
3
0
95
Acquisition
0
3
0
Joint
3
3
96
Distillation
Baseline
0
0
0
Regulation
5
0
0
Appendix
Table 3: Committed structural operations in the changed training trajectory. Repair consultations may decline to write. Replay redirections count individual examples substituted during subsequent batches.
Policy
Teacher forwards per trajectory
Extra calls
Baseline
771
0
Regulation
771
0
Acquisition
783
12
Joint
803
32
Appendix
Table 4: Teacher forward calls in distillation. Both training trajectories have the same count for each policy. Extra calls are relative to the corresponding baseline.
Ordinary update
Before FRT
After FRT
ΔFQC
96
0.016320877
0.016660973
+0.000340097
160
0.009440793
0.010511544
+0.001070751
192
0
0
0
224
0.014273338
0.014275268
+0.000001930
256
0.004792862
0.004691567
−0.000101294
Appendix
Table 5: Matched-physical-proposal finite acquisition opportunity (nats per current example). All entries use arrived training samples and the same teacher within each pair.
Trajectory
Learning rule
Exact contrast (count)
Macro-NLL contrast
Original
Replay
−10
+0.085045
Original
Distillation
−1
+0.072506
Changed
Replay
+1
−0.042830
Changed
Distillation
0
−0.020493
Appendix
Table 6: Readout-specific complete-policy contrasts. The policies may follow different later states and actions, so these values are not fixed-action causal factorial effects.
Neural network training dynamics studies how parameters and representations change during optimization and how those changes shape learning, stability, and generalization.
A learning system can occupy execution states that are indistinguishable under every declared present-behavior readout yet respond differently to future training. We formalize this through fiber fingerprints: controlled future-learning response laws restricted to present-behavior equivalence classes. Prefix-compatible finite probes induce a predictive quotient functor, a Nerode-type minimal recursively sufficient representation, and a canonical set-level predictive fiber without assuming smoothness, reversibility, finite rank, or a manifold. Under an explicit finite-dimensional Hilbert realization, response decomposes into visible, visible-mode-reuse, and irreducible-new sectors; a history-reachability bridge retains only distinctions generated by natural training histories. Conditional mechanism results then identify a graph-Hodge chronology decomposition, a regular switching class with root-mean-square scale pη3/2 and finite-scale corrections, and an exact Adam moment section whose immediate adaptive field is constant while common future gradients can reveal hidden moment differences. Frozen Transformer--LoRA--AdamW studies with Qwen2.5-7B and Mistral-7B-v0.3 support a local action backbone, longer-horizon first-return non-closure, and fresh visible-relative completion with output-range reuse and a low-rank irreducible sector. Stronger claims remain bounded by preregistered negative or mixed results: re-anchored transport is unresolved above its measurement floor; the strict finite-grid Hodge--3/2 conjunction is unmet despite prospective contraction; Qwen accessibility is not established in the frozen raw moment chart; and Mistral revelation is future-context dependent rather than bank invariant. Within these support-, scale-, metric-, and context-resolved boundaries, present behavior is not a sufficient statistic for declared future learning.
Learning must acquire new capabilities while preserving both prior responsibilities and the capacity to learn again. We introduce Generalized Residual Closure (GRC), a framework for learning as recursive closure of future-relevant discrepancies: closing a residual establishes the conditions for subsequent prediction, interaction, and learning. Under a complete representation-relation description at a fixed learner-world boundary, persistent internal learning has two primitive modes: Transformation within a representation and revision of the Representation itself. We establish a local tangent decomposition under regularity assumptions and a criterion for when representation revision is necessary. In an affine model, we derive a necessary-and-sufficient condition for stability-plasticity compatibility and the unique solution of a constrained quadratic update problem, which preserves registered old responsibilities while reducing residuals with an effective safe response. We prove that reconstructive semantic protection weakly enlarges the safe-response operator relative to preserving an exact historical realization. Dynamic sufficiency and future-closure viability extend representation adequacy from current prediction to lawful future updating and continued learning. Growth Learning expands the lawful closure domain or lowers optimal closure cost without regression of the registered capability-cost frontier; a conditional commit rule maintains this order. Restricted-sector recoveries and a conditional representation theorem connect the framework to optimization, machine learning, and control. Together, these results organize adaptation, representation revision, and reusable capability within a common account of continued learning.
Dongxu Li, Yinuo Zhang, Hongyu Zhang +1
Xi’an Jiaotong University · Xi’an University of Architecture and Technology
A learner's current observations need not determine its response to further training. We formulate intrinsic-extrinsic coupling through the continuation-conditioned value of a constrained learning-state intervention, with observation-relative fibers describing present agreement. An executable finite-frame classifier-head write protects current logits while repairing specified historical margins under finite-precision acceptance checks. We distinguish local admissibility, continuation-conditioned intervention value, and complete-policy performance. A matched four-cell contrast identifies readout-specific non-additivity between the same intrinsic intervention and alternative external continuations. In a CLINC-derived class-incremental setting, replay changes the write's 32-update contribution from five correct predictions to zero. Nonzero interactions also occur under output distillation, with a RoBERTa backbone, and under optimizer-native SGDW dynamics. Under SGDW, correct-count interactions are negative in all three activated roots at 128 updates, showing that coupling need not imply positive synergy. The mathematical analysis distinguishes feasible local repairs and favorable terminal outputs from training-reachable repair regions. Separate coordination tests show that content controls match or exceed the development gain, while a five-root fresh-test comparison with Fiber present in every arm shows root-dependent rather than uniformly beneficial correct-count effects. On the secondary cross-entropy readout, guided allocation yields lower mean loss than standard replay in all five pairs. Together, these results make intrinsic-extrinsic coupling operational by connecting executable state geometry to continuation-conditioned value, matched interaction identification, and closed-loop coordination, while separating identified coupling from complete-policy performance.