cs.LGAug 17, 2026

Fiber Fingerprints of Hidden Learning-State Dynamics

Authors: Qinyou Wang

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

A learning system can occupy execution states that are indistinguishable under every declared present-behavior readout yet respond differently to future training. We formalize this through fiber fingerprints: controlled future-learning response laws restricted to present-behavior equivalence classes. Prefix-compatible finite probes induce a predictive quotient functor, a Nerode-type minimal recursively sufficient representation, and a canonical set-level predictive fiber without assuming smoothness, reversibility, finite rank, or a manifold. Under an explicit finite-dimensional Hilbert realization, response decomposes into visible, visible-mode-reuse, and irreducible-new sectors; a history-reachability bridge retains only distinctions generated by natural training histories. Conditional mechanism results then identify a graph-Hodge chronology decomposition, a regular switching class with root-mean-square scale pη3/2\sqrt{p}η^{3/2} and finite-scale corrections, and an exact Adam moment section whose immediate adaptive field is constant while common future gradients can reveal hidden moment differences. Frozen Transformer--LoRA--AdamW studies with Qwen2.5-7B and Mistral-7B-v0.3 support a local action backbone, longer-horizon first-return non-closure, and fresh visible-relative completion with output-range reuse and a low-rank irreducible sector. Stronger claims remain bounded by preregistered negative or mixed results: re-anchored transport is unresolved above its measurement floor; the strict finite-grid Hodge--3/23/2 conjunction is unmet despite prospective contraction; Qwen accessibility is not established in the frozen raw moment chart; and Mistral revelation is future-context dependent rather than bank invariant. Within these support-, scale-, metric-, and context-resolved boundaries, present behavior is not a sufficient statistic for declared future learning.

Figures & tables

Figure 1: Finite controlled core and conditional realizations.
Figure 2: Complete Transformer–LoRA–AdamW execution state.
Figure 3: Visible-relative completion on history-supported directions.
Figure 4: Chronology interaction and the regular switching class.
Figure 5: Adam matched-section cancellation and future-gradient revelation.
Empirical objectEvidence supported by the frozen studiesStatus
Local action and finite returnAction-specific one-step backbone; longer-horizon first-return non-closure.Supported, local
Visible-relative completionFresh histories reproduce visible response, output-range reuse, and a low-rank irreducible-new sector in both models.Supported
Re-anchored path transportNo path effect is resolved above the independent-bank floor; most ambient relative transports are ill conditioned.Not resolved
Hard-switching chronologyCrossing, Hodge, bridge, and smooth-control diagnostics support the mechanism and prospective contraction.Strict conjunction unmet
Adam accessibilityMistral matched sections are accessible in the frozen chart; joint Qwen–Mistral panel membership is not established.Context/metric dependent
Future-gradient revelationNonzero responses, zero-gradient controls, amplitude scaling, and analytic recurrence support de-cancellation.Supported in Mistral
Table 1: Empirical membership of the conditional theory.
Figure 6: Fresh visible-relative completion at H5 and shifted H6.
Figure 7: Contraction of finite-grid exponent errors after prospective refinement.
Figure 8: Matched-section response scale and cross-bank direction; panel (b) is exploratory.
Premise Not implied
Present-behavior equality⇏\not\Rightarrowfuture-learning equality
Set-level quotient transport⇏\not\Rightarrowstable ambient linear transport
Asymptotic 3/23/2 membership⇏\not\Rightarrowexact finite-grid exponent
Algebraic moment reachability⇏\not\Rightarrowneural-data accessibility
Surjective control differential⇏\not\Rightarrowarbitrary raw-coordinate condition threshold
Positive future observability⇏\not\Rightarrowpositive cross-bank directional coherence
Table 2: Key non-implications retained by the theory.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Formal study nameRole in the paper
Local action and return structureEstablishes an action-specific local backbone, first-return non-closure, low local return rank, and the boundary of a fixed context-independent propagator.
Fresh visible-relative completion replicationIndependently replicates visible/reuse/irreducible-new response completion in both models.
Re-anchored path-transport studyTests a finite linear transport/monodromy membership after exact endpoint re-anchoring.
Controlled hard-switching scaling studyTests crossing counts, marks, graph-Hodge cycle scaling, future bridges, smooth controls, and prospective finite-scale convergence.
Joint matched-section accessibility screenTests natural tangent alignment and joint Qwen–Mistral membership in a frozen two-step, three-coordinate control chart.
Conditional matched-section revelation studyTests future-gradient de-cancellation in the empirically accessible Mistral subclass.
Appendix
Table 3: Formal study names and roles in the paper.
StudyFrozen outcomeStatus
Local action and return structureFresh Qwen/Mistral histories support action-specific local maps. The one-step law extends, but a fixed closed three-dimensional two-step cocycle is not supported. The full 9×99\times 9 first-return system has r90,r95≈4r_{90},r_{95}\approx 4 ; shifted-H6 low rank replicates.Supported / boundary
Fresh visible-relative completion replicationAt H5, median response energy is 60.29% visible, 23.99% reuse, and 15.72% irreducible-new. At shifted H6 it is 67.15%, 19.73%, and 13.13%. Both models satisfy the frozen replication without pooling rescue.Supported
Re-anchored path-transport studyMean cross-bank energy 3.57×10−63.57\times 10^{-6} ; one-sided lower bound −2.996×10−6-2.996\times 10^{-6} ; synchronized sign p=0.171875p=0.171875 ; median SNR 0.0016510.001651 ; one of twelve principal relative-transport condition checks passes.Not supported
Controlled hard-switching scaling studyInitial cycle approximately p0.5113η1.5163p^{0.5113}\eta^{1.5163} and softplus η2.0095\eta^{2.0095} . Refined cycle p0.497334η1.506284p^{0.497334}\eta^{1.506284} , future bridges near η1.5061\eta^{1.5061} , and softplus η2.000554\eta^{2.000554} . Primary cycle equivalence holds, but four narrow conjunction criteria remain outside their frozen bands.Mixed
Joint matched-section accessibility screenNatural tangent alignment is high. Mistral satisfies the frozen first-slot screen for all six candidate seeds; Qwen satisfies none under the declared raw (m,v)(m,v) Euclidean conditioning criterion. No future outcome is computed.Criterion not met
Conditional matched-section revelation studySix Mistral chronology slots are dynamically reconstructed; twelve of twelve full-amplitude responses are nonzero; full zero-gradient controls are zero; median full/half norm ratio 1.998946771.99894677 ; median full/half cosine 0.9999989420.999998942 . A bank-invariant direction is not supported.Mixed
Appendix
Table 4: Frozen outcomes stated in formal language.
HorizonVisibleReuseIrreducible-newMedian r95(S)r_{95}(S)
Source H560.29%23.99%15.72%3
Shifted H667.15%19.73%13.13%3
Appendix
Table 5: Median sector fractions in the fresh completion replication.
TopicLicensed interpretationNot licensed
Predictive statePresent behavior is not a sufficient statistic for declared future-learning responses.A finite fingerprint recovers the complete physical neural state.
Global fiberThe predictive quotient functor has a canonical set-level category of elements; regular bundles are conditional.The measured Transformer system is unconditionally a smooth fiber bundle.
CompletionFresh response contains visible, reuse, and irreducible-new sectors relative to a frozen projector and metrics.The irreducible rank is the true neural hidden dimension.
TransportQuotient transport is canonical; empirical finite linear transport is support- and resolution-conditioned.The negative path study proves path independence or identity monodromy.
SwitchingThe controlled hard-ReLU system supports the proposed Hodge– 3/23/2 mechanism and second-order approach while missing its full finite-grid conjunction.Native Qwen or Mistral training universally obeys a 3/23/2 law.
AdamAn exact algebraic matched section exists; accessible Mistral sections show future-gradient-conditioned revelation.Generic natural reachability or a bank-invariant revelation direction.
Appendix
Table 6: Interpretive boundaries.
ResultRequiredNot required
Predictive quotient and global opfibrationComplete composition and prefix-compatible probesTopology, probability, smoothness
Prefix-depth filtrationDeclared prefix action readoutsDifferentiability
Visible-relative completionFinite-dimensional Hilbert metrics and a declared visible projectorGlobal bundle
History bridgeHistory chart and covarianceStationarity
Hodge interaction isolationFinite task graph and common-affine memory channelSwitching assumptions
Graph-projected 3/23/2 lawCrossing covariance, centering, and aggregate dependence conditionsTransformer or ReLU by name
Appendix
Table 7: Dependencies of the principal results.
More in this topic

Hidden-State Probing

Hidden-state probing reads model activations to study what they represent and whether those signals support monitoring or control.

See all 49 papers

Explore similar work

CardsList
Oct 21, 2026

Dynamics of Hidden Learning Structure

A learner's present predictions do not fully determine how it will learn next. We study the dynamics of hidden learning structure through two related operations. Fiber Regulation Theory (FRT) describes protected repair of existing function and the learning responses left available after repair. Fiber Acquisition Theory (FAT) describes the formation and renewal of trainable structure, distinguishing neutral installation, access to a learning signal, and additional functional directions. The two operations also constrain one another. A repair obligation can force a change in a learning channel, while acquired features can make new protected repairs feasible or reduce the minimum coefficient cost of existing ones. We characterize joint admissibility and response reachability, and give a finite supervised-learning construction in which the two operations together produce a favorable response absent from either operation alone. We implement protected head repair and fixed-capacity LoRA renewal in Qwen2.5-32B with AdamW, under replay and output distillation. Across two training trajectories differing in seed and within-task example order, the coordinated policy achieves higher teacher-forced Exact than either separate policy in all four comparisons. It also has the lowest macro NLL under both learning rules on the second trajectory. The implementations realize structure-conditioned self-regulation within specified information and action constraints: accessible state readouts and arrived evidence guide repairs and structural renewals, and ordinary learning continues from the modified state.

Qinyou Wang

Neural Network Training DynamicsDynamic Neural NetworksContinual Learning for LLMs
AskPDF
Oct 7, 2026

When does a network's training history predict its future learning better than its current state? Evidence from a response probe and a forecasting screen

Networks that behave alike now can still learn differently when training continues. Work on loss of plasticity and critical periods shows that the path to a state shapes what follows; it does not show whether the path carries information that a measurement of the state itself misses. We ask when the training history of a network predicts its future learning better than its current state. In a main study, small multilayer perceptrons were trained under three history regimes (42 histories), and future learning was measured at four checkpoints by a short probe: a copy of the network trained for 100 updates on a new task. Before the prediction result was read, the protocol checked the probe. It responded monotonically to a function-preserving rescaling of hidden units, repeated measurements agreed (intraclass correlation 0.940, [0.903, 0.997], in the least reliable class, mean of three repeats), and a re-initialisation of units was visible directly after it but not 100 to 200 updates later. A history state of at most four dimensions did not improve on a calibrated model of the current state (gain -21.4%, 90% interval [-91.9, 8.1]; required in advance: 10%). A companion screen on 1,560 synthetic regression runs asked the same question for a target further away, the final error of the run. There, history models forecast better than the current validation error after 12 of up to 240 epochs (compact state 30.3%, [15.8, 39.4], a contextual comparison) and were not distinguishable from it after 48. In both studies the history was informative only while the current state was not yet informative about the target; this reading was formed after the results.

Martin Hofmann, Patrick Mäder

Data-intensive Systems and Visualization Group (dAI.SY), Technische Universität Ilmenau, Max-Planck-Ring 14, 98693 Ilmenau, Thuringia, Germany · German Centre for Integrative Biodiversity Research (iDiv) Halle–Jena–Leipzig, Deutscher Platz 5e, 04103 Leipzig, Saxony, Germany · Faculty of Biological Sciences, Friedrich Schiller University, Fürstengraben 1, 07745 Jena, Thuringia, Germany

Neural Network Training Dynamics
AskPDF
Aug 21, 2026

Training, learning and inference: unified dynamics of neural systems

We define an atomic generation fact f=(u,tau,omega,z;rho), recording the origin, realized transformation, concrete occurrence, generated result and relation role. Compiled into a Generation-Fact Graph (GFG), these facts provide an AI-native, compilable scientific fact substrate preserving generation histories. We establish a GFG-based recursive scientific process in which analysis, intervention, replay and validation form facts for later cycles. Using nanoGPT, we establish unified training-learning dynamics. Training is the evolution of a parameter-optimizer system with state and memory: each actual training action enters the receiving state and produces a finite-amplitude nonlinear functional response conditioned by that state and target-specific update geometry. Learning is the persistent reorganization of distributed functional support by these responses; capability formation, maintenance, decline or recovery becomes observable when target-specific states are evaluated against their readout boundaries. Three primary coordinates - target-boundary state, target-specific update geometry and parameter-Adam receiving state - yield a second-order predictor operating before post-update outputs are read. On held-out runs, it achieved 91.43% accuracy and 91.49% macro-averaged recall across four transitions. We further establish inference as a frozen projection of training-learning dynamics. Component gating and rollback show causal recruitment and non-additive combination of query-conditioned support formed during training, deriving organizational conditions realized by Attention. Controlled feedback indicates possible double-edged reinforcement effects. ResNet/CIFAR-100 and diffusion/CIFAR-10 experiments confirm receiving-state-conditioned responses, persistent support reorganization and frozen inference projection beyond nanoGPT.

Mian Wang

Wuhan Polytechnic University, Wuhan 430023, Hubei, China

Neural Network Training Dynamics
AskPDF