Delayed Optimizer-State Transport Shapes Short-Horizon Training Decisions

Jinhui Guo

cs.LGAug 25, 2026

Delayed Optimizer-State Transport Shapes Short-Horizon Training Decisions

Authors: Jinhui Guo

Organizations: School of Physics, Beihang University, Beijing 100191, China

Abstract

Adaptive optimizers retain gradient history in moment variables, allowing a local change in loss weighting to alter later updates. We examine whether this delayed transport is large enough to change prospective short-horizon decisions. On committed future-minibatch sequences, we differentiate eight-step AdamW trajectories through the complete model--optimizer state and select exposure-matched Math--Code loss schedules before independent evaluation. Across 12 unused 0.3M Transformer histories, full transport lowers token-disjoint loss relative to an optimizer-aware immediate derivative in 10/12 histories (mean benefit $4.71\times10^{-4}$ ; exact one-sided sign test, $p=0.0193$ ). The two controllers act equally often but select different schedules in 60/96 windows. Crossed checkpoint--future-path tests attribute this reordering to the interaction between optimizer state and near-future data, while an independent Ising--CNN experiment shows that deleting moment-state transport destroys accurate response prediction. Full-transport scores also concentrate exact-rollout winners in larger candidate libraries, focusing finite-amplitude evaluation on a shortlist. On these committed short paths, optimizer memory and near-future data order are therefore actionable components of the training state, providing a mechanism-based criterion for when finite-horizon rather than one-step intervention is required.

Figures & tables

Panel of Figure 1 — Figure 1: Pathwise training response separates source, transport, and readout. A local loss-weight perturbation enters the augmented model–optimizer state through $B_{s}$ , propagates through the finite-time Jacobian product $\Phi$ , and is projected by the observable-specific readout $C_{y,t}$ . The same decomposition supports the three tests used below: deleting moment-state tangent channels, comparing full transport with an optimizer-aware immediate response, and ranking finite temporal schedules before exact reranking in large candidate libraries. Actions and terminal states are locked before the independent readout.

Figure 2: Delayed transport changes finite actions and improves the independent readout. (a) Token-disjoint history-level benefit of full eight-step transport relative to the optimizer-aware immediate derivative, VGA, and neutral training. (b) Paired benefit relative to neutral training. (c) Both full and immediate controllers intervene in all 96 windows, yet select the same temporal schedule in only 36. Actions and terminal states are locked before the test readout.

Figure 3: The preferred short-horizon action depends on the future minibatch path. Here a tape is a pre-generated sequence of eight paired future minibatches. (a) Schedule-score variation is dominated by checkpoint–tape interaction. (b) On the realized tape, full transport retains positive decision value relative to memory-deleted and immediate controllers. (c) The value collapses when actions are predicted from other tapes, exposing the boundary of state-only control.

Figure 4: Optimizer moments carry delayed response across Transformer scales. (a) Direct pathwise tangent reconstruction remains accurate at approximately 0.3M and 1M parameters. (b) Mean signed block-projection fractions quantify contributions to the next parameter tangent; the definition and closure check are given in Supp. A.2 . (c,d) Individual-path transport remains accurate even when an ensemble-mean response profile transfers weakly across scales.

Figure 5: The optimizer-state mechanism transfers to a Binder-anchored Ising benchmark. (a) Binder crossings define the physical anchor before neural action selection; the final confirmation reuses this anchor while replacing neural paths and test chains. (b) Candidate-contrast NRMSE, cosine similarity, and discrete action agreement across the staged tests. The elevated absolute-calibration NRMSE marks a calibration boundary, while action agreement remains high. (c) The selected action reduces the absolute deviation of the neural crossing from the Binder anchor on new test chains. (d) Full transport outperforms the immediate controller relative to neutral training.

Figure 6: Full transport concentrates finite-amplitude winners at equal exact-rollout budget. (a) Each colored point is the winner recall obtained after holding out one Ising path, Ising history, or Math–Code history cluster; black segments denote the mean within each cluster type. (b) With neutral plus

q=8

nonneutral rollouts, full transport has the highest exhaustive-winner recall in both systems, followed by memory deletion, immediate ranking, and random selection. (c) Charging the one-time development calibration gives break-even points of 7.32 Ising and 12.22 Math–Code future-path bundles for repeated reuse of the same library structures. This accounting counts candidate rollouts rather than total training time.

Figure S1: Receding-horizon behavior in the independent 0.3M full-versus-neutral cohort. (a) Terminal test benefit

\bar{e}_{0}-\bar{e}_{*}

by training history; the dashed line marks the history mean. (b) Controller-readout benefit over the eight replanning windows; thin lines show histories, and the dark curve and band show the mean and its standard error. (c) Selection counts for the schedules in Eq. ( S10 ).

Figure S2: Math–Code scale and duration observations. (a) History-level 1M benefits on the locked and dense test readouts. (b) Mean 64-step benefit at 0.3M and 1M; error bars are history-bootstrap 95% intervals. (c) Controller, audit, and test benefits after 64 and 224 controlled steps in one nested 1M development trajectory.

Figure S3: Ising shortlist calibration and fresh-path evaluation. (a) The 50th of 54 ordered development gaps fixes

c_{0.90}

. (b) Applying this coefficient on fresh paths gives the median and range of the adaptive shortlist size for each

K

and library. (c) Adaptive reranking agrees with the exhaustive action in 18/18 settings per library; fixed

q=16

misses one

K=128

action in each mixed library.

Figure S4: Lag-resolved Ising–CNN response and moment-tangent deletion for (a) the antisymmetric readout

m

and (b) the symmetric readout

e

. The full-state tangent follows the exact finite response in both panels. Deleting perturbations in the AdamW moment coordinates leaves the first lag unchanged but removes most of the response transmitted to later updates. Curves show path averages; NRMSE is computed pathwise.

Figure S5: Same-state decision geometry in the four-arm Math–Code experiment. (a) Gray points retain the immediate winner, orange points change it, and the dashed line is the exact multicandidate boundary

D=-M

. (b) Centered full–immediate score cosines for the retained- and changed-winner groups; box lines give medians and quartiles. (c) The two rankings disagree at 60/96 states along each replayed trajectory and at 6/12 common initial states.

Figure S6: Specified-future-path confirmation. (a) Validation scores preserve useful local action ordering despite imperfect calibration of the full test-contrast vector; red points denote the locked actions. (b) Full transport gives the largest mean independent-test benefit among the four policies. (c) Full exceeds memory deletion and the immediate derivative in five of six history means. Error bars in (b) show standard errors across histories.

Figure S7: Math–Code transfer of transport-guided exact reranking. (a) Ordered normalized winner gaps for development and fresh settings, together with the frozen Math–Code calibration coefficient. (b) Fixed

q=16

recovers every exhaustive winner; the adaptive rule gives 88.9%, 94.4%, and 94.4% agreement in the three libraries. (c) Every adaptive action improves token-disjoint test loss; the horizontal line and band show the history mean and its 95% bootstrap interval.

Method	differentiation	screen	action	readout	total	RSS
Ordinary training	0	0	0.115	0.031	0.145	0.433
VGA	0.930	5.269	0.123	0.031	6.360	0.785
Adam-aware immediate	1.540	5.382	0.123	0.031	7.073	0.560
Full, forward	4.676	4.603	0.106	0.029	9.402	0.566
Full, adjoint	0.600	4.848	0.107	0.030	5.657	1.522

Table S1: Direct 0.3M four-arm CPU cost. Entries are medians of 12 Apple-M4 CPU measurements in seconds; RSS is peak resident memory in GiB from fresh worker processes. “Screen” is the branch-compatibility evaluation, “action” is the selected eight-step update, and “readout” is the held-out evaluation. Component medians need not sum to the median total.

Scale	train	readout	adjoint	exact	action	control	$T_{\mathrm{ctrl}}/T_{0}$
0.3M	0.0937	0.0275	0.4859	0.7031	0.0945	1.3110	10.82
1M	0.1751	0.0494	0.9201	1.2380	0.1806	2.3881	10.64

Table S2: Complete measured Math–Code CPU cost per eight-step window. Times are seconds.

Figure S8: External comparator, complete cost, and candidate-count sensitivity. (a) History-level test-loss benefits in the early full, VGA, and neutral comparison; squares and intervals show means and 95% history-bootstrap intervals. (b) Measured CPU time per eight-step window, decomposed into adjoint, exact screening, selected-action training, and readout. Labels give the ratio to ordinary training and readout. (c) Fresh-winner recall after separately calibrating three candidate-count factors; the dashed line marks 90% recall.

Dimension	Observed boundary	Supported here	Source of boundary
Scale and duration	1M: accurate local response; 9/12 positive histories; one path drifts further by 224 steps	One-window reconstruction extends to 1M	Scale and suffix length vary together; repeated actions change later states and Jacobians
Unknown future	Leave-one-tape-out benefit $-1.61\times 10^{-5}$ ; 1/4 histories positive	Utility on committed future paths	State–tape–schedule interaction contributes 71.9% of score variation
Cross-readout	NRMSE/cosine $=0.474/0.892$ ; calibration gates missed	Locked actions can improve test loss	Action choice uses local ordering; NRMSE weights all contrast amplitudes
Optimizer family	SGD response is inaccurate at the common amplitude	Moment transport in the tested momentum and AdamW settings	The same pulse gives $11.5\times$ the AdamW symmetric-response RMS
External comparator	Early full–VGA difference is positive in 3/6 histories	VGA mixes current-update geometry with delayed transport	The VGA score omits AdamW preconditioning and moment-state updates
Shortlist calibration	Sphere A gives 32/36; recalibrated candidate-count forms tie	Fixed $q=16$ exact reranking gives 108/108	Winner-gap geometry changes across libraries; recalibration absorbs the tested $K$ factors

Table S3: Empirical boundaries of the reported results and their observed sources.

Neural Network Optimization

Neural network optimization studies algorithms and training dynamics that help models learn efficiently, reliably, and generalize well.

See all 321 papers

Finite-Horizon Input-Output Dynamics of Minibatch Perturbations in AdamW
Aug 20, 2026Kang Liu, Suyan LiNeural Network Optimization Neural Network Training Dynamics
School of Future Technology, Xi’an Jiaotong University, Xi’an, China, 710049 · Department of Electrical and Computer Engineering, National University of Singapore, Singapore, 119077.
Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfer
Aug 20, 2026Qinyang XuSubliminal Learning Causal Interventions in Language Models
Xiamen University
Optimizer Memory Schedules for Outscaling the Overtraining Axis
Sep 4, 2026Katie Everett, Shikai QiuNeural Network Optimization Muon Optimizer
MIT CSAIL

Aug 20, 2026J/K move · Enter open · S save

Finite-Horizon Input-Output Dynamics of Minibatch Perturbations in AdamW

Kang Liu, Suyan Li

School of Future Technology, Xi’an Jiaotong University, Xi’an, China, 710049 · Department of Electrical and Computer Engineering, National University of Singapore, Singapore, 119077.

Neural Network Optimization Neural Network Training Dynamics Adaptive Gradient Methods Gradient Descent Dynamics

Open paper AskPDF arXiv

A minibatch can influence training beyond the update at which it is observed because AdamW stores past gradient information in its optimizer states. We study this delayed effect through paired trajectories that differ only in one gradient update and share the same subsequent training sequence. We formulate AdamW as a finite-horizon input--state--output (ISO) system whose state contains the model parameters and first- and second-moment estimates. Linearizing the joint dynamics yields a signed response operator that maps a localized gradient perturbation to its future loss effects, revealing how optimizer memory shapes their magnitude, timing, and sign. We further derive an exact multistep error decomposition and establish first-order finite-horizon accuracy under local smoothness and controlled activation switching. Experiments validate the response mechanism and optimizer-state effects, while repeated-future analyses reveal substantial prospective structure in delayed influence that can be partially recovered from ISO approximations. Code is available at https://github.com/Kanyooo/Loss_ISO.

Abstract

Neural Network Optimization

Explore similar work

Finite-Horizon Input-Output Dynamics of Minibatch Perturbations in AdamW

Stored in Optimizer State, Valued by Later Training: A Causal Account of Subliminal Trait Transfer

Optimizer Memory Schedules for Outscaling the Overtraining Axis