Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence

Wentao Wang; Hengyu Zhong; Yunhan Jiang; Jialiang An; Meng Lu

cs.LGSep 30, 2026

Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence

Authors: Wentao Wang, Hengyu Zhong, Yunhan Jiang, Jialiang An, Meng Lu

Organizations: Peking University · GigaAI

Abstract

As new evidence arrives, a sequence model must update what it remembers and how memory influences predictions. While Transformers incur computation and cache costs scaling with context length, fixed-state recurrent models offer constant-memory inference. However, linear and spectral recurrences traditionally rely on static transitions, failing to dynamically revise how stored representations decay or rotate. While recent selective architectures introduce input-dependent transitions, they assign independent controls to every memory mode, coupling control cost to state capacity. We show that high-dimensional spectral memory does not require high-dimensional control, and introduce Shared Phase and Retention Control for Efficient Adaptive Spectral Recurrence (SPARC). SPARC employs just two input-dependent scalar signals to coordinate memory retention and phase rotation across heterogeneous complex modes, while preserving mode-specific baseline timescales and frequencies. Its diagonal affine recurrence supports parallel associative scans for sequence-level BPTT as well as exact structured Real-Time Recurrent Learning (RTRL) for online credit assignment. Across partially observable continuous control, POPGym, and sequence classification, SPARC achieves a 9.09% relative return improvement on Walker-P and a 1.36% relative accuracy gain on FordA over second-best methods. On an NVIDIA Blackwell GPU, our implementation reduces recurrent-mixer training latency by 18.2%-34.2% in fixed-token workloads and accelerates scans by 3.1x-4.7x over an optimized RG-LRU baseline. These results show that two shared control signals can efficiently govern adaptive spectral memory across online and full-sequence settings. Code is available at https://github.com/Botwwt/sparc.

Figures & tables

Figure 1: Two complementary memory requirements. (a) Recall relevant events in order despite distractors. (b) Track event-driven changes in contraction and rotation. Task details are in Appendix C.3 .

Figure 2: Overview of SPARC. (a) The layer input supplies a shared control block and the recurrent block, which produces the updated state. (b) Two scalar control heads coordinate retention and phase across spectral modes. The Linear–SiLU pathway illustrates a possible controller architecture; Eq. ( 31 ) specifies the tanh-projection parameterization used in this paper. (c) Mode

j

multiplies its previous state by an input-dependent complex transition and adds the current write contribution.

Method	Ant-P	Walker-P	Hopper-P	Cheetah-P
RTU	3823.8	899.8	1266.7	2547.3
LRU	4476.5	655.5	1363.4	2688.9
RG-LRU	1953.7	865.5	1224.4	2539.5
GateLoop	4105.6	911.9	1325.1	2466.1
Mamba-3	1355.7	680.2	863.6	2019.9
GRU	1921.1	760.1	631.3	2174.1

Table 1: Partially observable continuous control.

Figure 3: Learning curves for Autoencode, CountRecall, RepeatFirst, RepeatPrevious, Noisy Pendulum, and HigherLower. Curves and shading use the same three-seed averaging and sample-standard-deviation convention as Figure 5 .

Method	FordA	SLC	UWave	pMNIST	CIFAR-10
RTU	95.38	99.67	93.33	96.75	59.32
LRU	95.38	99.67	92.22	96.82	60.11
RG-LRU	92.89	99.67	98.52	85.48	51.01
GateLoop	91.97	99.00	91.85	62.31	52.46
Mamba-3	90.58	99.33	98.15	88.51	48.65
GRU	93.54	99.00	94.81	90.92	62.69

Table 2: Full-sequence classification.

Figure 4: GPU efficiency on a single NVIDIA RTX PRO 6000 Blackwell Server Edition GPU. (a) Scan-only timing after coefficient generation. (b,c) Recurrent-mixer forward-plus-backward timing at fixed token count, for widths

2048

and

2560

. (d) Runtime and peak allocated-memory ratios, SPARC divided by RG-LRU, as batch size varies. Ratios below one favor SPARC.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Variant	Intervention and scope
Fixed shared controls	Set $r_{t}=p_{t}=0$ in every pathway; retain the learned modal spectrum and the selected content activation.
No adaptive retention	Set $e_{j,t}=\nu_{j}$ only in the transition, including its phase clock; retain the original adaptive write normalization and gate.
No adaptive phase	Set $\lambda_{j,t}=\rho_{j,t}e^{i\theta_{j}}$ ; retain $p_{t}$ in the write gate.
No phase clock	Replace $(1-\rho_{j,t})\kappa_{p}p_{t}$ by $\kappa_{p}p_{t}$ ; keep retention and all write terms.
Static phase clock	Use $(1-e^{-\nu_{j}})\kappa_{p}p_{t}$ ; retain adaptive retention and writing.
Static transition	Set $\lambda_{j}=e^{-\nu_{j}+i\theta_{j}}$ ; retain the original adaptive write pathway.

Appendix

Table 3: Component-removal definitions.

Setting	Continuous control	POPGym
Environment interaction steps	$4{,}999{,}168$	$3{,}999{,}744$
Physical rollout length	$2048$	$2048$
PPO epochs per rollout	$4$	$10$
Minibatches per epoch	$32$	$32$
Real-valued recurrent state coordinates	$384$	$128$
SPARC complex modes	$192$	$64$

Appendix

Table 4: Shared reinforcement-learning hyperparameters.

Dataset	Length	Classes	Fit	Validation
FordA	$500$	$2$	$3240$	$361$
StarLightCurves	$1024$	$3$	$900$	$100$
UWaveGestureLibraryAll	$945$	$8$	$806$	$90$
Permuted MNIST	$784$	$10$	$54{,}001$	$5999$
Seq. CIFAR-10 Gray	$1024$	$10$	$45{,}000$	$5000$

Appendix

Table 5: Classification data configurations.

Method	Autoenc.	CountRec.	Repeat1st	RepeatPrev.	NoisyPend.	HigherLow.
RTU	-0.427	-0.565	0.448	0.891	0.244	0.501
LRU	-0.431	-0.597	0.242	0.908	0.246	0.498
RG-LRU	-0.476	-0.644	0.550	-0.004	0.246	0.503
GateLoop	-0.438	-0.647	0.727	0.268	0.245	0.498
Mamba-3	-0.499	-0.811	0.471	-0.428	0.269	0.487
GRU	-0.469	-0.835	0.811	0.744	0.311	0.495

Appendix

Table 6: Partially observable memory tasks in POPGym.

Figure 5: Learning curves on Ant-P, Walker-P, Hopper-P, and Cheetah-P. Each curve averages three seed trajectories after applying a $100$ -rollout moving average within each seed. Shading denotes one sample standard deviation across the smoothed trajectories.

Variant	RepeatFirst	Walker-P	FordA	CIFAR-10
SPARC	$\mathbf{0.829}$	$\mathbf{994.79}$	$\mathbf{96.68}$	$\mathbf{59.92}$
Neutral write gate	$0.663$	$909.56$	$\mathbf{96.68}$	$59.35$
Linear content	$0.123$	$921.90$	$96.12$	$59.46$
Fixed write normalization	$0.628$	$\underline{943.45}$	$96.40$	$58.73$
No phase clock	$0.288$	$928.64$	$\underline{96.49}$	$49.03$
Retention-only transition	$\underline{0.756}$	$837.70$	$96.03$	$\underline{59.79}$

Appendix

Table 7: Selected component ablations.

Task	SPARC	Variant	$\Delta$ (variant $-$ SPARC)
Autoencode	$-0.431$ , $\pm,0.003$	$\mathbf{-0.424}$ , $\pm,0.023$	$0.007$ , $\pm,0.021$
CountRecall	$-0.515$ , $\pm,0.098$	$\mathbf{-0.493}$ , $\pm,0.148$	$0.022$ , $\pm,0.051$
RepeatFirst	$\mathbf{0.829}$ , $\pm,0.008$	$0.756$ , $\pm,0.051$	$-0.073$ , $\pm,0.058$
RepeatPrevious	$0.855$ , $\pm,0.033$	$\mathbf{0.938}$ , $\pm,0.051$	$0.083$ , $\pm,0.019$
Noisy Pendulum	$0.461$ , $\pm,0.039$	$\mathbf{0.504}$ , $\pm,0.053$	$0.043$ , $\pm,0.020$
CartPole	$\mathbf{0.959}$ , $\pm,0.026$	$0.921$ , $\pm,0.016$	$-0.038$ , $\pm,0.010$

Appendix

Table 8: Retention-only transition.

Task	SPARC	Variant	$\Delta$ (variant $-$ SPARC)
Autoencode	$-0.431$ , $\pm,0.003$	$\mathbf{-0.424}$ , $\pm,0.023$	$0.007$ , $\pm,0.021$
CountRecall	$\mathbf{-0.515}$ , $\pm,0.098$	$-0.568$ , $\pm,0.229$	$-0.053$ , $\pm,0.131$
RepeatFirst	$\mathbf{0.829}$ , $\pm,0.008$	$0.326$ , $\pm,0.515$	$-0.503$ , $\pm,0.522$
RepeatPrevious	$0.855$ , $\pm,0.033$	$\mathbf{0.902}$ , $\pm,0.024$	$0.047$ , $\pm,0.056$
Noisy Pendulum	$\mathbf{0.461}$ , $\pm,0.039$	$0.457$ , $\pm,0.097$	$-0.004$ , $\pm,0.102$
CartPole	$\mathbf{0.959}$ , $\pm,0.026$	$0.947$ , $\pm,0.005$	$-0.012$ , $\pm,0.032$

Appendix

Table 9: Phase-only transition.

Task	SPARC	Variant	$\Delta$ (variant $-$ SPARC)
Autoencode	$\mathbf{-0.431}$ , $\pm,0.003$	$-0.441$ , $\pm,0.019$	$-0.009$ , $\pm,0.016$
CountRecall	$-0.515$ , $\pm,0.098$	$\mathbf{-0.397}$ , $\pm,0.161$	$0.119$ , $\pm,0.064$
RepeatFirst	$\mathbf{0.829}$ , $\pm,0.008$	$0.534$ , $\pm,0.196$	$-0.294$ , $\pm,0.189$
RepeatPrevious	$0.855$ , $\pm,0.033$	$\mathbf{0.900}$ , $\pm,0.077$	$0.045$ , $\pm,0.061$
Noisy Pendulum	$0.461$ , $\pm,0.039$	$\mathbf{0.532}$ , $\pm,0.031$	$0.070$ , $\pm,0.050$
CartPole	$\mathbf{0.959}$ , $\pm,0.026$	$0.949$ , $\pm,0.012$	$-0.010$ , $\pm,0.038$

Appendix

Table 10: Static transition.

Task	SPARC	Variant	$\Delta$ (variant $-$ SPARC)
Autoencode	$\mathbf{-0.431}$ , $\pm,0.003$	$-0.454$ , $\pm,0.029$	$-0.022$ , $\pm,0.027$
CountRecall	$\mathbf{-0.515}$ , $\pm,0.098$	$-0.627$ , $\pm,0.151$	$-0.112$ , $\pm,0.145$
RepeatFirst	$\mathbf{0.829}$ , $\pm,0.008$	$0.288$ , $\pm,0.242$	$-0.541$ , $\pm,0.234$
RepeatPrevious	$\mathbf{0.855}$ , $\pm,0.033$	$0.661$ , $\pm,0.120$	$-0.194$ , $\pm,0.136$
Noisy Pendulum	$\mathbf{0.461}$ , $\pm,0.039$	$0.384$ , $\pm,0.017$	$-0.077$ , $\pm,0.056$
CartPole	$\mathbf{0.959}$ , $\pm,0.026$	$0.945$ , $\pm,0.008$	$-0.013$ , $\pm,0.018$

Appendix

Table 11: No phase clock.

Task	SPARC	Variant	$\Delta$ (variant $-$ SPARC)
Autoencode	$\mathbf{-0.431}$ , $\pm,0.003$	$-0.452$ , $\pm,0.015$	$-0.021$ , $\pm,0.017$
CountRecall	$-0.515$ , $\pm,0.098$	$\mathbf{-0.389}$ , $\pm,0.068$	$0.126$ , $\pm,0.081$
RepeatFirst	$\mathbf{0.829}$ , $\pm,0.008$	$0.383$ , $\pm,0.547$	$-0.446$ , $\pm,0.555$
RepeatPrevious	$\mathbf{0.855}$ , $\pm,0.033$	$0.845$ , $\pm,0.053$	$-0.010$ , $\pm,0.071$
Noisy Pendulum	$\mathbf{0.461}$ , $\pm,0.039$	$0.413$ , $\pm,0.087$	$-0.048$ , $\pm,0.069$
CartPole	$\mathbf{0.959}$ , $\pm,0.026$	$0.811$ , $\pm,0.016$	$-0.148$ , $\pm,0.042$

Appendix

Table 12: Static phase clock.

Task	SPARC	Variant	$\Delta$ (variant $-$ SPARC)
Autoencode	$-0.431$ , $\pm,0.003$	$\mathbf{-0.430}$ , $\pm,0.026$	$0.001$ , $\pm,0.025$
CountRecall	$-0.515$ , $\pm,0.098$	$\mathbf{-0.511}$ , $\pm,0.059$	$0.004$ , $\pm,0.073$
RepeatFirst	$\mathbf{0.829}$ , $\pm,0.008$	$0.663$ , $\pm,0.067$	$-0.166$ , $\pm,0.074$
RepeatPrevious	$0.855$ , $\pm,0.033$	$\mathbf{0.867}$ , $\pm,0.033$	$0.012$ , $\pm,0.001$
Noisy Pendulum	$\mathbf{0.461}$ , $\pm,0.039$	$0.425$ , $\pm,0.030$	$-0.036$ , $\pm,0.022$
CartPole	$\mathbf{0.959}$ , $\pm,0.026$	$0.915$ , $\pm,0.078$	$-0.044$ , $\pm,0.104$

Appendix

Table 13: Neutral write gate.

Task	SPARC	Variant	$\Delta$ (variant $-$ SPARC)
Autoencode	$\mathbf{-0.431}$ , $\pm,0.003$	$-0.447$ , $\pm,0.024$	$-0.016$ , $\pm,0.021$
CountRecall	$\mathbf{-0.515}$ , $\pm,0.098$	$-0.613$ , $\pm,0.114$	$-0.098$ , $\pm,0.051$
RepeatFirst	$\mathbf{0.829}$ , $\pm,0.008$	$0.721$ , $\pm,0.016$	$-0.107$ , $\pm,0.024$
RepeatPrevious	$0.855$ , $\pm,0.033$	$\mathbf{0.907}$ , $\pm,0.031$	$0.052$ , $\pm,0.052$
Noisy Pendulum	$0.461$ , $\pm,0.039$	$\mathbf{0.464}$ , $\pm,0.031$	$0.003$ , $\pm,0.041$
CartPole	$\mathbf{0.959}$ , $\pm,0.026$	$0.950$ , $\pm,0.027$	$-0.009$ , $\pm,0.000$

Appendix

Table 14: Frozen reference gain.

Task	SPARC	Variant	$\Delta$ (variant $-$ SPARC)
Autoencode	$\mathbf{-0.431}$ , $\pm,0.003$	$-0.477$ , $\pm,0.021$	$-0.045$ , $\pm,0.019$
CountRecall	$-0.515$ , $\pm,0.098$	$\mathbf{-0.452}$ , $\pm,0.036$	$0.063$ , $\pm,0.077$
RepeatFirst	$\mathbf{0.829}$ , $\pm,0.008$	$0.628$ , $\pm,0.049$	$-0.200$ , $\pm,0.057$
RepeatPrevious	$0.855$ , $\pm,0.033$	$\mathbf{0.878}$ , $\pm,0.065$	$0.023$ , $\pm,0.037$
Noisy Pendulum	$\mathbf{0.461}$ , $\pm,0.039$	$0.366$ , $\pm,0.145$	$-0.095$ , $\pm,0.133$
CartPole	$\mathbf{0.959}$ , $\pm,0.026$	$0.891$ , $\pm,0.065$	$-0.067$ , $\pm,0.039$

Appendix

Table 15: Fixed write normalization.

Task	SPARC	Variant	$\Delta$ (variant $-$ SPARC)
Autoencode	$\mathbf{-0.431}$ , $\pm,0.003$	$-0.442$ , $\pm,0.012$	$-0.011$ , $\pm,0.009$
CountRecall	$-0.515$ , $\pm,0.098$	$\mathbf{-0.465}$ , $\pm,0.060$	$0.050$ , $\pm,0.081$
RepeatFirst	$\mathbf{0.829}$ , $\pm,0.008$	$0.123$ , $\pm,0.857$	$-0.705$ , $\pm,0.865$
RepeatPrevious	$0.855$ , $\pm,0.033$	$\mathbf{0.867}$ , $\pm,0.070$	$0.012$ , $\pm,0.041$
Noisy Pendulum	$\mathbf{0.461}$ , $\pm,0.039$	$0.425$ , $\pm,0.060$	$-0.037$ , $\pm,0.079$
CartPole	$\mathbf{0.959}$ , $\pm,0.026$	$0.944$ , $\pm,0.001$	$-0.015$ , $\pm,0.027$

Appendix

Table 16: Linear content.

Task	Gate SD	Saturation (%)	Gate ratio	Content (%)
Autoencode	$0.0913,\text{$ \pm,0.0299 $}$	$32.67,\text{$ \pm,3.70 $}$	1.171	15.49
CountRecall	$0.0426,\text{$ \pm,0.0274 $}$	$23.50,\text{$ \pm,5.50 $}$	1.252	25.38
RepeatFirst	$0.4508,\text{$ \pm,0.0382 $}$	$95.73,\text{$ \pm,1.79 $}$	1.816	0.76
RepeatPrevious	$0.0081,\text{$ \pm,0.0013 $}$	$31.52,\text{$ \pm,4.34 $}$	1.244	16.43
Noisy Pendulum	$0.0890,\text{$ \pm,0.0203 $}$	$75.65,\text{$ \pm,1.48 $}$	1.079	4.23
CartPole	$0.0072,\text{$ \pm,0.0015 $}$	$28.33,\text{$ \pm,5.32 $}$	1.149	26.49

Appendix

Table 17: Write-gate activity and content saturation.

Task	Decay scale	Surv. (%)	Rel. surv.	Phase (deg)	Clock (%)
Autoencode	2.769	63.99	1.039	30.553	38.05
CountRecall	0.708	80.46	1.114	17.901	25.30
RepeatFirst	0.255	99.12	1.091	2.963	3.29
RepeatPrevious	0.223	84.66	1.584	6.026	9.74
Noisy Pendulum	4.416	54.92	0.967	30.287	41.12
CartPole	0.623	67.66	1.210	17.442	22.16

Appendix

Table 18: Retention and phase activity.

$B$	$T$	$d_{\mathrm{mix}}$	Naive	Accel.	Speedup	Naive	Accel.	Speedup
			Backward			Forward + backward
4	2048	2048	238.730	1.225	$194.81\times$	358.822	2.617	$137.13\times$
2	4096	2048	479.999	1.289	$372.50\times$	721.780	2.510	$287.54\times$
1	8192	2048	967.711	0.911	$1061.91\times$	1469.467	1.690	$869.62\times$
4	2048	2560	236.191	1.081	$218.45\times$	358.953	1.824	$196.78\times$
2	4096	2560	489.609	1.105	$442.89\times$	740.985	1.846	$401.39\times$

Appendix

Table 19: SPARC versus its naive implementation at a fixed budget of

8192

tokens.

$B$	Naive	Accel.	Speedup	Naive	Accel.	Speedup
	Backward			Forward + backward
1	228.853	0.853	$268.35\times$	344.970	1.900	$181.56\times$
2	240.696	0.942	$255.42\times$	361.919	2.026	$178.67\times$
4	236.798	0.890	$265.99\times$	355.549	1.541	$230.69\times$
8	239.711	1.710	$140.21\times$	365.954	2.548	$143.62\times$
16	244.437	3.044	$80.31\times$	366.240	4.570	$80.15\times$

Appendix

Table 20: SPARC versus its naive implementation across batch sizes, with

T=2048

and

d_{\mathrm{mix}}=2048

Quantity	Maximum absolute error	Normalized error
Forward output	$5.66\times 10^{-7}$	$5.66\times 10^{-7}$
Input gradient	$2.03\times 10^{-6}$	$5.28\times 10^{-7}$
Log-decay parameter gradient	$4.77\times 10^{-6}$	$1.17\times 10^{-6}$
Log-phase parameter gradient	$7.63\times 10^{-5}$	$6.58\times 10^{-7}$
Phase-controller gradient	$6.68\times 10^{-6}$	$7.09\times 10^{-7}$
Retention-controller gradient	$3.05\times 10^{-5}$	$1.43\times 10^{-6}$

Appendix

Table 21: Numerical agreement between the accelerated SPARC mixer and the naive reference.

Explore similar work

SchemaMem: Schema-Indexed Recurrent Memory for Delayed State Retrieval
Sep 27, 2026Sungwoo Goo, Hwi-yeol Yun, Sangkeun JungRecurrent Model Persistent State
College of Pharmacy, Chungnam National University Daejeon, Republic of Korea · Department of Computer Science & Engineering, Chungnam National University Daejeon, Republic of Korea
Structured Recurrent Mixers for Massively Parallelized Sequence Generation
May 9, 2026Benjamin L. BadgerAsymmetric Grouped-Attention Variate Mixer Recurrent Model
IBM
The Recurrent Transformer: Greater Effective Depth and Efficient Decoding
Apr 23, 2026Costin-Andrei Oncescu, Depen Morwani, Samy Jelassi +3Transformer Architectures Kv-Cache Management
Harvard University

Sep 27, 2026, cs.LGJ/K move · Enter open · S save

SchemaMem: Schema-Indexed Recurrent Memory for Delayed State Retrieval

Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung

College of Pharmacy, Chungnam National University Daejeon, Republic of Korea · Department of Computer Science & Engineering, Chungnam National University Daejeon, Republic of Korea

Recurrent Model Persistent State Retention Linear Attention Bidirectional Long Short-Term Memory

Open paper AskPDF arXiv

Attention provides direct access to past representations, but retaining an ever-growing history is costly. Recurrent models bound persistent state, yet must preserve selected information while processing subsequent inputs. We introduce SchemaMem, an attention-based recurrent memory architecture combining chunk-local attention with a persistent, schema-indexed phase state. Learned schema embeddings provide a shared representational reference for reading and writing. Reads use the current state, whereas writes use the layer input and static schema embeddings, excluding direct feedback from that layer's own state. Chunk-boundary commits aggregate bounded phase increments through forward computation. The same parameters also support full-history attention training before and during recurrent training. We studied selective updates, preservation, and delayed retrieval in a controlled address--value task, comparing three-layer models with approximately matched parameter counts and persistent-state dimensions. Across nine address/value settings and three training seeds, SchemaMem has higher mean written-value retention at four times the maximum training delay than both baselines, which are trained toward a higher in-range accuracy target. Updated-value recovery favors SchemaMem in all nine settings against Mamba-3 and seven against Gated DeltaNet. Defaults consistently favor Gated DeltaNet over SchemaMem at that delay, and SchemaMem requires substantially more optimization steps. These results identify a promising retention--optimization trade-off in schema-indexed recurrence.

Abstract

Explore similar work

SchemaMem: Schema-Indexed Recurrent Memory for Delayed State Retrieval

Structured Recurrent Mixers for Massively Parallelized Sequence Generation

The Recurrent Transformer: Greater Effective Depth and Efficient Decoding