Finite-Horizon Fisher Memory in Two-Sided Power-Bounded Recurrent Systems

Jeonghoon Lee

cs.LGSep 30, 2026

Finite-Horizon Fisher Memory in Two-Sided Power-Bounded Recurrent Systems

Authors: Jeonghoon Lee

Organizations: Attractor Dynamics Inc.

Abstract

We analyse allocation, admission and post-write retention in finite-horizon linear-Gaussian noisy recurrent memories. At every horizon, the directional Fisher memory $M_n$ satisfies $\operatorname{tr}M_n=N$ : non-normality redistributes information but cannot raise its spherical average, while normal carriers satisfy $M_n=I$ . For bi-power-bounded carriers, we derive uniform $1/n$ lag bounds, identify the limit of $M_n$ with the inverse of the classical Cesàro asymptotic limit of $W^\top$ , and give finite-horizon error bounds. A time-varying coupling defines an end-to-end store operator. The writer-optimal direction need not be store-optimal. After writing ends, an invertible hold preserves the full stored Fisher matrix. Additive contamination bounded by $α$ times the closure covariance retains at least $1/(1+α)$ of that matrix; a covariance-aware decoder attains the corresponding accuracy. With recurrent carriers held fixed, training input masks and linear readouts approached the task-specific optimum in 160 runs, with median normalized Rayleigh efficiency above $0.998$ . Binary accuracy matched the Gaussian prediction to mean absolute error below $0.002$ over more than four orders of magnitude in $J$ . In a separate pre-specified study of 320 runs, trained masks followed the designated input-time objective in both carrier types, in 16 of 16 draws. These studies used development-seen carriers and are pre-specified validations, not blind holdouts. The same fixed design reproduced the objective-specific result in 16 of 16 draws on carriers unused before run commitment. Exact isolation preserved information, while a decoder fixed at its training horizon fell to chance; inverse-adjoint transport restored its sampled decisions to numerical precision.

Figures & tables

Carrier	cond	$\sigma_{4096}=\max_{j\leq 4096}\\|W^{j}\\|_{2}$ , med	$\lambda_{max}$ (oracle) med [min, max]	$\lambda_{min}$ med	$nJ_{n}(n-1)$ at $n=4096$ , med
Haar orthogonal	1	1.00	1.000 [1.000, 1.000]	1.000	1.000
$SQS^{-1}$	2	1.82	1.831 [1.774, 1.852]	0.470	1.831
	5	4.14	3.139 [3.040, 3.298]	0.133	3.138
	10	7.88	4.312 [4.098, 4.639]	0.045	4.310
	20	15.0	5.491 [5.246, 5.637]	0.014	5.488
	35 (held-out)	25.6	6.107 [5.892, 6.331]	0.0054	6.105

Table 1

write carrier	storage	direction	$J^{(s)}_{\mathrm{tot}}$	oldest stored input
normal $Q$	open	conditioned-writer oracle	0.405	0.00008
normal $Q$	isolated	conditioned-writer oracle	0.421	0.03375
conditioned $SQS^{-1}$	open	conditioned-writer oracle	2.290	0.00056
conditioned $SQS^{-1}$	isolated	conditioned-writer oracle	2.509	0.18823

Table 2

field	value
writer dimension $N$ , store dimension $d$	32, 16
write interval $T_{w}$ ; designated input times $t_{0}$	24; $0$ and $12$
training horizon $H_{\text{train}}$ ; query horizons	128; $64,256,1024,4096$
input amplitude $a$ ; class priors	$1.5$ ; equal
initial state	$x_{w}(0)=x_{s}(0)=0$
noise schedule	writer innovations $z_{t}\sim\mathcal{N}(0,I_{N})$ enter the write block at every step. Under isolation the coupling closes at $t=T_{w}=24$ : only innovations inside the write interval reach the store, and the endpoint is carried to the query horizon by $A=U^{H-T_{w}}$ with no further noise. Under open coupling the same recursion runs to the query horizon, so write-path innovations keep entering the store for all $H$ steps. Disturbance added directly to the store covariance occurs only in the declared approximate-isolation conditions

Table 3

	non-normal	normal
median $R_{J}$	0.99936	0.99891
median $A_{\text{sub}}$	0.99861	0.99614
median $R_{J}$ of an isotropic random direction	0.1444	0.2621

Table 4

	$t_{0}=0$ NN	$t_{0}=12$ NN	$t_{0}=0$ N	$t_{0}=12$ N
median own-objective $r_{j}(v^{\text{learn}}_{j})$	0.99945	0.99945	0.99885	0.99874
median available separation $G^{\star}_{j}$	0.2393	0.2090	0.5095	0.4455
median achieved separation $G^{\text{learn}}_{j}$	0.2353	0.2126	0.5114	0.4412
median contrast recovery $C_{j}$	0.9990	1.0010	0.9997	0.9987

Table 5

	$t_{0}=0$ NN	$t_{0}=12$ NN	$t_{0}=0$ N	$t_{0}=12$ N
median own-objective $r_{j}(v^{\text{learn}}_{j})$	0.9994	0.9995	0.9989	0.9987
median available separation $G^{\star}_{j}$	0.2584	0.2922	0.4411	0.4581
median achieved separation $G^{\text{learn}}_{j}$	0.2597	0.2904	0.4509	0.4613
median contrast recovery $C_{j}$	1.0005	0.9984	0.9975	0.9969

Table 6

comparison	non-normal	normal
open store, $J(H{=}64)/J(H{=}4096)$ , decline along the horizon	118.3	119.3
$J(\text{isolated})/J(\text{open})$ , both at $H=4096$ , isolated versus open at a fixed horizon	520.4	486.9

Table 7

	fixed decoder at $H_{\text{train}}$	covariance-aware at $H_{\text{train}}$	fixed decoder away	covariance-aware away
non-normal	0.7718	0.7760	0.5029	0.7746
normal	0.7114	0.7136	0.5136	0.7128

Table 8

category	items in this section	what a reader may conclude
standard identities	$\operatorname{tr}M_{n}=N$ ; $M_{n}=I$ for a normal carrier; $\operatorname{Acc}^{\star}=\Phi(a\sqrt{J})$ ; post-write Fisher invariance under an invertible hold	properties of the model, established analytically
derived theorem claims	store trace bound; approximate-isolation floor $J_{0}/(1+\alpha)$ ; the $G^{\text{learn}}$ accounting identity	proved in the appendices, verified numerically
numerical implementation tests	sequential-versus-block parity; scan parity; CPU-versus-CUDA endpoint parity; empirical-versus-analytic covariance and Fisher; decoder transport; calibration error and slope	the implementation realises the analytic results to its stated numerical and statistical accuracy
finite-budget optimization results	$R_{J}$ , $A_{\text{sub}}$ , own-objective $r_{j}$ , $G^{\text{learn}}$ , $C_{j}$	behavioural training reaches the analytic optimum under the stated budget
exploratory observations	the preliminary target-time probe; preliminary oracle separations; runtime benchmarks	hypothesis-generating only, not confirmatory

Table 9

purpose	command
after creating the two links above, recompute the verdicts and manuscript tables from the released outputs	python code/nc_verdict.py --out NC1_VERDICT_RECOMPUTED.json ; python code/nc3c_verdict.py --out NC3C_VERDICT_RECOMPUTED.json ; python code/make_claim_ledger.py
check the Cesàro identification ( Classical ingredients , before §3.3)	python code/check_cesaro_identification.py
regenerate all manuscript figures and supplementary plots from the released CSVs into new directories	python code/make_public_figures.py --out reproduced/public_figures ; python code/make_nc_figures.py --out reproduced/nc_figures ; python code/make_section7_figures.py --out reproduced/section7_figures
rerun NC-1/NC-2 and exploratory training in a disposable extraction without input links	python code/run_stage.py smoke ; python code/run_stage.py pilot ; python code/run_stage.py confirmatory ; python code/run_exploratory.py --amplitude 1.5 --train-horizon 128
rerun the second carrier block of §7.4.1 in another disposable extraction without input links	python code/run_nc3c.py --block 16-31 --results-dir results/nc3c_block2_20260920 ; python code/nc3c_verdict.py --results results/nc3c_block2_20260920 --out results/nc3c_block2_20260920/VERDICT.json
verify that the released files remain unchanged	shasum -a 256 -c SHA256SUMS

Table 10

configuration	metric	median	Q1	Q3	min	max	n
A0	lambda_max	1	1	1	1	1	8
A0	lambda_min	1	1	1	1	1	8
A0	trace_over_N	1	1	1	1	1	8
A0	K_plus	1	1	1	1	1	8
A0	K_minus	1	1	1	1	1	8
SQS-c2	lambda_max	1.83134	1.78732	1.83805	1.77414	1.85209	8

Table 11

configuration	n=32 med [min,max]	n=128 med [min,max]	n=512 med [min,max]	n=2048 med [min,max]	n=4096 med [min,max]
A0	1.0000 [1.0000, 1.0000]	1.0000 [1.0000, 1.0000]	1.0000 [1.0000, 1.0000]	1.0000 [1.0000, 1.0000]	1.0000 [1.0000, 1.0000]
SQS-c2	1.8087 [1.7671, 1.8399]	1.8216 [1.7681, 1.8480]	1.8298 [1.7728, 1.8505]	1.8309 [1.7740, 1.8517]	1.8311 [1.7740, 1.8519]
SQS-c5	3.0782 [2.9180, 3.1654]	3.1047 [3.0034, 3.2607]	3.1297 [3.0303, 3.2881]	3.1379 [3.0377, 3.2961]	3.1379 [3.0385, 3.2972]
SQS-c10	4.1283 [3.8057, 4.4853]	4.2511 [3.9791, 4.5919]	4.2641 [4.1207, 4.6274]	4.3168 [4.0891, 4.6366]	4.3102 [4.0976, 4.6372]
SQS-c20	5.0889 [4.9661, 5.4788]	5.4363 [5.1671, 5.6192]	5.4808 [5.2417, 5.6027]	5.4862 [5.2396, 5.6330]	5.4883 [5.2425, 5.6335]
SQS-c50	6.1324 [5.9256, 6.5722]	6.4079 [6.2504, 6.8209]	6.5396 [6.3195, 6.8635]	6.5519 [6.3343, 6.8720]	6.5594 [6.3398, 6.8768]

Table 12

reading	threshold	value	label
paired ratio, conditioned/normal, isolated, store-only $J_{tot}$ , $n=4096$	>1.05	median 5.429 [3.926, 11.800]	store concentration above one: yes
normal isolated, oldest-lag relative spread over horizons	<1e-6	1.6e-15; medians 0.0337536, 0.0337536, 0.0337536, 0.0337536	oldest-lag store info constant: yes
normal open, oldest-lag $n{=}64$ / $n{=}4096$	>= 10	118.8	open-cell oldest-lag decays: yes
normal isolated, reach (inputs with $J^{(s)}>10^{-6}$ )	report	23 of 24 in every draw	REACH_COUNT
nonnormal isolated, oldest-lag relative spread over horizons	<1e-6	1.3e-15; medians 0.1882299, 0.1882299, 0.1882299, 0.1882299	oldest-lag store info constant: yes
nonnormal open, oldest-lag $n{=}64$ / $n{=}4096$	>= 10	88.0	open-cell oldest-lag decays: yes

Table 13

write	storage	direction	$n=64$	$n=256$	$n=1024$	$n=4096$
normal	open	oracle	0.413 / 0.00994 / 1.385	0.397 / 0.00199 / 1.354	0.405 / 0.00044 / 1.374	0.405 / 0.00008 / 1.376
normal	open	random	0.626 / 0.01550 / 1.719	0.630 / 0.00259 / 1.708	0.622 / 0.00057 / 1.711	0.621 / 0.00016 / 1.711
normal	isolated	oracle	0.421 / 0.03375 / 1.417	0.421 / 0.03375 / 1.420	0.421 / 0.03375 / 1.421	0.421 / 0.03375 / 1.421
normal	isolated	random	0.597 / 0.04447 / 1.597	0.597 / 0.04447 / 1.597	0.597 / 0.04447 / 1.597	0.597 / 0.04447 / 1.597
nonnormal	open	oracle	2.263 / 0.04914 / 6.578	2.268 / 0.01055 / 6.770	2.287 / 0.00170 / 6.800	2.290 / 0.00056 / 6.800
nonnormal	open	random	0.456 / 0.01244 / 1.449	0.471 / 0.00204 / 1.409	0.469 / 0.00043 / 1.407	0.469 / 0.00007 / 1.407

Table 14

Explore similar work

Jul 3, 2026cs.LG

SHiPPO: Recurrent Memory with Transported Polynomial Projections

HiPPO gives recurrent states memory semantics as coefficients of online polynomial projections, but in fixed channel coordinates. Modern selective SSMs, by contrast, rely on token-dependent control and channel interaction. We introduce SHiPPO (Sylvester HiPPO), a transported projection-memory prior that lifts HiPPO coefficient memories into a moving channel frame. For any fixed or realized right-transport path, SHiPPO transports the approximation family and channel metric together; conditional on that path, the state is ordinary HiPPO in a tied moving frame and follows Sylvester coefficient dynamics, preserving the left online-memory operator while adding right-action transport. For selective-SSM execution, we derive a restricted group-local realization with controller-compatible right actions, exponential-adjusted updates, exact block-affine scan, and recurrent decoding. We also give a simultaneous-reducibility criterion identifying when right transports collapse to static mixing plus independent scalar or blockwise banks. Controlled diagnostics show that larger current-token write rank improves ordinary prediction error but cannot recover order-sensitive changes to already-written memory; transported-memory variants recover this signal, which disappears when the transport pathway is removed. A finite-field associative-recall diagnostic with interleaved bindings, operations, and queries provides complementary autoregressive evidence while leaving the preferred right-action realization open. Taken together, these results support SHiPPO as a mechanistically grounded transported-memory prior, with evidence focused on memory mechanisms rather than broad sequence-modeling dominance.

Tomoya Mizuguchi, Bum Jun Kim

Kyoto University, Japan · The University of Tokyo, Japan

Recurrent Model Recurrent State

Ask PDF

May 6, 2026stat.ML

Sharp Capacity Thresholds in Linear Associative Memory: From Winner-Take-All to Listwise Retrieval

How many key-value associations can a

d\times d

linear memory store? We show that the answer depends not only on the

d^2

degrees of freedom in the memory matrix, but also on the retrieval criterion. In an isotropic Gaussian model for the stored pairs, we show that top-1 retrieval, where every signal must beat its largest distractor, requires the logarithmic model-size scale

d^2\asymp n\log n

. We prove that the correlation matrix memory construction, which stores associations by superposing key-target outer products, achieves this scale through a sharp phase transition, and that the same scaling is necessary for any linear memory. Thus the logarithm is the intrinsic extreme-value price of winner-take-all decoding. We next consider listwise retrieval, where the correct target need not be the unique top-scoring item but should remain among the strongest candidates. To formalize this regime, we propose the Tail-Average Margin (TAM), a convex upper-tail criterion that certifies inclusion of the correct target in a controlled candidate list. Under this listwise retrieval criterion, the capacity follows the quadratic scale

d^2\asymp n

. At load

n/d^2\toα

, we develop an exact asymptotic theory for the TAM empirical-risk minimizer through a two-parameter scalar variational principle. The theory has a rich phenomenology: in the ridgeless limit it yields a closed-form critical load separating satisfiable and unsatisfiable phases, and it predicts the limiting laws of true scores, competitor scores, margins, and percentile profiles. Finally, a small-tail extrapolation further leads to the conjectural sharp top-1 threshold

d^2\sim 2n\log n

Nicholas Barnfield, Juno Kim, Eshaan Nichani +2

Harvard University · University of California, Berkeley · Princeton University

Dense Associative Memory Full-Rank Correlation Matrices Capacity Keys And Value

Ask PDF

May 23, 2026nlin.AO

Memory Uncertainty Relation and Harmonic Memory in Random Recurrent Networks

We present an inequality that bounds the short-term memory capability of dynamical systems from below. It can be interpreted as an uncertainty relation between a measure of short-term memory and that of the size of state fluctuations induced by input signals. The lower bound can be achieved by a readout weight and thus represents a suboptimal memory called harmonic memory. We examine analytically and numerically the inequality in a number of reservoir systems subject to input noise. We illustrate cases in which equality is achieved exactly, equality holds asymptotically, and the inequality is strict. We also study the effect of a state-space regularization to elucidate the inequality in terms of the fluctuation structure of the state-space. We find that a certain strength of input noise induces extra memory under the regularization, and we refer to this phenomenon as noise-induced memory. We observe that the memory uncertainty relation does not hold in general for the regularized memory and harmonic memory. This fact is explained in terms of the mechanism of noise-induced memory.

Taichi Haruna, Kohei Nakajima

Department of Information and Mathematical Sciences, School of Arts and Sciences, Tokyo Woman’s Christian University, 2-6-1 Zempukuji, Suginami-ku, Tokyo 167-8585, Japan · Graduate School of Information Science and Technology, The University of Tokyo, Bunkyo-ku, Tokyo 113-8656, Japan

Fluctuation-Dissipation Theorem Recurrent Neural Networks Harmonization

Ask PDF