Beyond Episodic AI: Cognitive Field Networks for Biologically Inspired Persistent Cognition

Byung Gyu Chae

cs.AISep 15, 2026

Beyond Episodic AI: Cognitive Field Networks for Biologically Inspired Persistent Cognition

Authors: Byung Gyu Chae

Organizations: Electronics and Telecommunications Research Institute, 218 Gajeong-ro, Yuseong-gu, Daejeon 34129, Republic of Korea

Abstract

Cognitive Field Theory describes cognition as memory-dressed collective dynamics, in which slow relaxation modes support a persistent cognitive field. By connecting non-Markovian memory dynamics to discrete recurrent state evolution through an extended state, we develop a Cognitive Field Network (CFN), a recurrent Transformer architecture with hidden-field re-entry, $Φ_{n+1} = F_θ(X_{n+1},Φ_n)$ . Rather than prescribing an explicit memory operation, the CFN allows the full token-resolved hidden field to re-enter the next computational cycle. We find that learning organizes this pathway into a functional channel for persistent, content-dependent internal states, with temporal persistence systematically controlled by the recurrent training horizon. The learned dynamics generalize beyond this horizon: semantic continuation sustains the recurrent state without explicit answer replay, whereas unsupported states exhibit finite passive relaxation. Periodic re-exposure to relevant input renews the surviving field, producing an approximately stationary nonzero regime over extended recurrent trajectories. Unrelated-input and recurrence-off controls fail to reproduce this behavior, while near-paraphrased input produces weaker renewal, revealing the dependence of persistence on both field re-entry and input relevance. These findings connect memory-dressed field formation, input-driven reorganization, and recurrent state continuity within a unified dynamical framework. The CFN thus provides an experimentally controllable platform for studying persistent, history-dependent cognition without a separately prescribed memory system.

Figures & tables

Figure 1: Cognitive Field Network (CFN): self-reentrant architecture and learning for persistent cognition. (a) Transition from episodic inference to self-reentrant cognitive dynamics. The hidden field $\Phi_{t}$ generated during one computational cycle is retained and re-entered into the subsequent cycle, yielding the state-dependent evolution $\Phi_{t+1}=F_{\theta}(X_{t+1},\Phi_{t})$ . (b) Decoder-style CFN implementation used in this work. The current token sequence follows the ordinary causal Transformer pathway, while the complete token-resolved hidden field $\Phi_{t-1}$ from the preceding cycle is supplied as unmasked cross-attention memory to every Transformer layer, with learnable recurrent coupling $g$ . After the final layer, $\Phi_{t}=H_{t}^{(L)}$ provides both the internal field retained for the next cycle and the hidden representation from which the language-model output is read out. (c) Learning and experimental analysis of self-reentrant cognitive dynamics. The architecture makes field re-entry available without prescribing its functional role. Task-driven learning can organize this pathway into persistent, content-dependent recurrent computation. The recurrent hidden field can be directly accessed and experimentally manipulated, enabling causal tests of memory persistence and the functional contribution of self-reentry across successive computational cycles. The framework also provides a basis for quantitative dynamical analysis through Jacobian spectra, relaxation-mode distributions, memory kernels, and collective response observables.

Component	Specification
Backbone	GPT-NeoX decoder Transformer
Number of Transformer blocks	$6$
Hidden dimension $d$	$512$
Attention heads	$8$
Feed-forward dimension	$2048$
Positional representation	Rotary positional embedding (RoPE)

Table 1: Architecture of the decoder-style Cognitive Field Network used in this work. The final token-resolved hidden field from each computational cycle is retained and supplied as unmasked recurrent cross-attention memory to every Transformer layer of the subsequent cycle.

Figure 2: Learning dynamics of recurrent field re-entry during language-model pretraining. (a) Training and validation cross-entropy losses for the GPT-NeoX-based CFN trained on WikiText-103. Both losses decrease rapidly during early training, followed by a slower late-training regime. (b) Evolution of the learnable re-entry coupling $g$ (green) and the recurrent-field strength $R_{\rm field}$ measured on the training (blue) and validation (orange) sets. The recurrent quantities exhibit an initial decrease, followed by recovery and gradual stabilization.

Figure 3: Learning-induced reorganization of the relaxation spectrum. Time-scale density of states (TDOS) obtained from the local Jacobian of the $H^{(4)}\rightarrow H^{(5)}$ layer map at successive language-model pretraining checkpoints. A fixed probe sequence of length $T=8$ yields a $4096\times 4096$ Jacobian, with relaxation rates defined as $\lambda_{\alpha}=-\ln|\mu_{\alpha}|$ , where $\mu_{\alpha}$ are the Jacobian eigenvalues. For each checkpoint, the upper panel shows the full stable relaxation spectrum, and the lower panel shows the infrared region $0<\lambda<0.2$ . The dashed vertical line indicates the reference cutoff $\lambda_{c}=0.05$ . The infrared spectral weight increases rapidly during early training and subsequently decreases as the spectrum broadens.

Figure 4: Infrared self-energy and inverse-gap proxies during learning. (a) Normalized infrared self-energy proxy $\Sigma_{\epsilon}^{\rm norm}(0)$ . (b) Corresponding inverse-self-energy proxy $r_{\rm gap}^{\rm proxy}=1/\Sigma_{\epsilon}^{\rm norm}(0)$ . Both quantities are computed from the stable relaxation spectrum of the $H^{(4)}\rightarrow H^{(5)}$ Jacobian at successive pretraining checkpoints, using a fixed probe sequence of length $T=8$ and an infrared regulator $\epsilon=10^{-3}$ .

Figure 5: Cross-cycle protocol for learning content-specific memory and selective retrieval. During the source (WRITE) cycle

n

, the CFN receives a sequence containing

N

randomly generated entity–attribute bindings (illustrated for

N=2

) and produces the token-resolved recurrent field

\Phi_{n}=H_{n}^{(L)}

. During the subsequent QUERY cycle

n+1

, the source text is no longer available, and the field

\Phi_{n}

is re-entered through recurrent cross-attention. Independent single-query inputs retrieve different attributes from the same preceding field

\Phi_{n}

. Training uses full backpropagation through both cycles, with answer supervision applied only during the QUERY cycle.

Figure 6: Learning dynamics of selective retrieval from the re-entered cognitive field. Validation exact-match accuracy during cross-cycle retrieval training for memory loads $N=1,2,4,8,$ and $16$ , where $N$ denotes the number of entity–attribute bindings presented during the source cycle. The $N=1$ task reaches near-perfect accuracy rapidly, whereas larger memory loads exhibit distinct learning trajectories. For $N=2$ , accuracy remains near $0.5$ before rising sharply toward unity. The $N=8$ and $N=16$ conditions also show abrupt improvements, while $N=4$ exhibits greater fluctuations before reaching high retrieval accuracy.

Memory load	$C_{\rm EM}$	$W_{\rm EM}$	$O_{\rm EM}$	$C-W$	$C-O$
$N=1$	1.0000	0.0275	0.0350	0.9725	0.9650
$N=2$	0.9975	0.0150	0.0354	0.9825	0.9621
$N=4$	0.9760	0.0131	0.0185	0.9629	0.9575
$N=8$	0.9988	0.0160	0.0167	0.9827	0.9821
$N=16$	0.9976	0.0140	0.0193	0.9835	0.9783

Table 2: Causal intervention on the recurrent field during selective retrieval.

C

W

, and

O

denote the correct-field, wrong-field, and state-off conditions, respectively. Subscript EM denotes exact-match accuracy. The differences

C-W

and

C-O

quantify the retrieval advantage of the correct field over the controls.

Figure 7: Experimental protocol for persistent memory through recurrent cognitive-field dynamics. (a) Cross-cycle memory protocol without context replay. During the WRITE cycle, the CFN receives an entity–attribute binding and generates the recurrent field $\Phi_{0}$ . The original context is then removed, while the field propagates through $d-1$ bridge cycles with unrelated inputs $X_{1},\ldots,X_{d-1}$ . At cycle distance $d$ , a QUERY retrieves the original attribute from the propagated field $\Phi_{d-1}$ . (b) Causal intervention on the recurrent field. For the same query and matched intervening inputs, retrieval is evaluated using the correct field $\Phi_{C}$ , an unrelated field $\Phi_{W}$ , or disabled re-entry $\Phi_{\rm off}$ . Content-specific retrieval is assessed through $P(A\mid Q,\Phi_{C})\gg P(A\mid Q,\Phi_{W})\simeq P(A\mid Q,\Phi_{\rm off})$ . (c) Schematic illustration of learning-dependent memory persistence for training distances $d_{\rm train}=1,2,4,$ and $8$ . Larger training distances correspond to longer illustrated retention horizons.

Figure 8: Long-horizon persistence of content-specific information in the recurrent cognitive field. Exact-match retrieval for the CFN trained at cycle distance $d_{\rm train}=8$ and evaluated without further optimization or replay of the original WRITE context. The vertical dashed line indicates the training distance. (a) Teacher-forced exact-match accuracy under the correct-field ( $C$ ), wrong-field ( $W$ ), and state-off ( $O$ ) conditions. Correct-field retrieval remains near perfect beyond the trained distance before gradually declining, while the wrong-field and state-off controls remain near background levels. (b) Content-specific retrieval signals measured by the exact-match differences $C-W$ and $C-O$ . Both differences remain large beyond the trained distance and decrease at longer cycle distances.

Figure 9: Learning-dependent temporal persistence of the recurrent cognitive field. (a) Content-specific retention measured by the correct–wrong exact-match difference $\mathcal{M}(d)$ for training distances $d_{\rm train}=1,2,4,$ and $8$ . The vertical dashed lines indicate the corresponding training distances. The retention curves shift toward longer cycle distances as $d_{\rm train}$ increases. (b) Retention half-distance $d_{1/2}$ , defined by $\mathcal{M}(d_{1/2})=\mathcal{M}_{0}/2$ , as a function of $d_{\rm train}$ . The measured values are $d_{1/2}\simeq 1.52,\,4.55,\,11.59,$ and $23.02$ for $d_{\rm train}=1,2,4,$ and $8$ , respectively. The dashed line indicates the reference $d_{1/2}=d_{\rm train}$ .

Figure 10: Learning-dependent displacement of long-horizon memory retention under semantic continuation. Teacher-forced exact-match accuracy as a function of cycle distance $d$ for CFN models trained at $d_{\rm train}=2,4,$ and $8$ using semantic-continuation bridges. Symbols represent measured retention values, and solid curves denote four-parameter logistic fits. The fitted midpoints are $d_{50}=6.84$ , $11.43$ , and $26.63$ for $d_{\rm train}=2,4,$ and $8$ , respectively. The retention curves shift toward longer cycle distances as the training distance increases.

Figure 11: Approximate temporal scaling collapse of learned recurrent-memory retention under semantic continuation. Normalized correct-field retention, $\widetilde{P}=(P-P_{\infty})/(P_{0}-P_{\infty})$ , is plotted against the rescaled cycle distance $u=d/d_{\rm train}$ for CFN models trained at $d_{\rm train}=2,4,$ and $8$ . Symbols represent measured teacher-forced exact-match retention, and solid curves denote the corresponding logistic fits. The three retention profiles exhibit an approximate collapse in the transition region, with modest differences in the rescaled midpoints $d_{50}/d_{\rm train}$ .

$d_{\rm train}$	$d_{50}$	$s$	$\Delta d_{90\rightarrow 10}$	$d_{50}/d_{\rm train}$	$s/d_{\rm train}$
2	6.84	1.21	5.32	3.42	0.605
4	11.43	2.40	10.54	2.86	0.600
8	26.63	4.99	21.95	3.33	0.624

Table 3: Logistic characterization of long-horizon correct-field retention under semantic continuation. The midpoint

d_{50}

and transition scale

s

are obtained from Eq. ( 72 ). The transition width is

\Delta d_{90\rightarrow 10}=2s\ln 9

. The last two columns show the fitted midpoint and transition scale normalized by the training distance.

Figure 12: Representation-sensitive renewal of recurrent-memory retention under periodic source re-exposure. Teacher-forced exact-match accuracy as a function of cycle distance $d$ for a CFN trained with semantic continuation at $d_{\rm train}=4$ . Periodic re-exposure presents either the exact original source or a near-paraphrased version of the same episode, without supplying the target answer through intervening bridges. Exact source re-exposure produces repeated recovery of correct-field retrieval, whereas near-paraphrased re-exposure yields substantially weaker recovery. Wrong-field and recurrent-field-off controls remain near background accuracy.

Figure 13: Long-horizon recurrent-memory retention under periodic content-matched source re-exposure. Teacher-forced exact-match accuracy for a CFN trained at $d_{\rm train}=4$ , with the original source re-presented every eight recurrent cycles. The vertical dotted lines indicate source re-exposure events. Correct-field retrieval exhibits repeated recovery followed by gradual decay between successive re-exposure events, producing a sawtooth trajectory. After an initial transient, the peak–trough envelope approaches an approximately stationary nonzero regime over the 128-cycle evaluation window. Wrong-field and recurrent-field-off controls remain near background accuracy.

Computational Cognitive Modeling

Computational cognitive modeling uses formal models and simulations to explain how people perceive, learn, remember, reason, and choose.

See all 126 papers

Oct 6, 2026

Continuous Memory Machines

Recurrent neural networks typically compress information into a single vector-valued recurrent state, forcing short-term computation and long-term retention to share the same representation. Past extensions alleviate this bottleneck by increasing the memory capacity or separating timescales, but lack the combination of rapid neuron-level processing and longer-term retention found in biology. To that end, we introduce the Continuous Memory Machine (CMM), a recurrent architecture with matrix-valued short- and long-term memory states serving distinct functional roles. Building on the Continuous Thought Machine (CTM), the CMM's short-term memory tracks recent neural activity, with uniquely parameterized neuron-level models learning to use these activity patterns for computation. A persistent long-term memory stores information for later use, with a Transformer jointly updating both memory stores, providing an expressive bidirectional read--write mechanism such that each store can reorganize its own contents and both read from and write to the other. Across algorithmic, in-context learning, and recurrent reasoning tasks, the CMM outperforms a broad suite of baselines, exhibiting stronger generalization than prior memory-augmented networks while preserving the CTM's interpretable attention patterns. Code is available at https://github.com/SakanaAI/continuous-memory-machines.

Ciaran Regan, Kai Arulkumaran, Luke Darlow +3

University of Tsukuba · Sakana AI

Memory-Augmented Neural Networks Recurrent Neural Networks

Ask PDF

Mar 28, 2026

Persistent Memory Through Triple-Loop Consolidation Under Stochastic Unit Turnover

Dissipative cognitive architectures maintain computation through continuous energy expenditure, where units that exhaust their energy are stochastically replaced with fresh random state. This creates a fundamental challenge: how can persistent, context-specific memory survive when all learnable state is periodically destroyed? Existing memory mechanisms -- including elastic weight consolidation, synaptic intelligence, and surprise-driven gating -- rely on gradient computation and are inapplicable to systems that do not perform it. We introduce Deep Memory (DM), a backpropagation-free persistent memory mechanism operating through a triple-loop consolidation cycle: (1) recording of expert-specific content centroids, (2) seeding of replaced units with stored representations, and (3) stabilization through continuous re-entry. Discrete expert routing via Mixture-of-Experts (MoE) gating is required, in the regimes tested, to prevent the centroid convergence that would render stored memories identical. We derive a Foster-Lyapunov drift bound for the full triple loop, showing that seeding rescales the turnover noise floor. Across

1{,}007

simulation runs over thirteen blocks: (i) removing stable context-expert binding removes specialization (

\mathrm{MI}=1.10

vs.

0.001

;

n=91

); (ii) DM achieves

R=0.984

vs.

0.385

without memory (

n=16

); (iii) continuous seeding reconstructs representations after interference (

R_\mathrm{recon}=0.978

; one-shot fails;

n=30

); (iv) the mechanism operates within a characterized

(K,p)

envelope (

n=350

); (v) recording

\times

seeding is the minimal critical dyad (

n=40

); (vi) associative and reservoir baselines (Hopfield, ESN) are compared under matched turnover (

n=370

). DM is thus a falsifiable, bounded mechanism for persistent memory in backpropagation-free cognitive systems, with functional parallels to hippocampal consolidation.

Jianwei Lou

RailMind Systems, Neuss, Germany

Computational Cognitive Modeling Memory Consolidation

Ask PDF

Apr 8, 2026

Information as Structural Alignment: A Dynamical Theory of Continual Learning

Catastrophic forgetting is not an engineering failure. It is a mathematical consequence of storing knowledge as global parameter superposition. Existing methods, such as regularization, replay, and frozen subnetworks, add external mechanisms to a shared-parameter substrate. None derives retention from the learning dynamics themselves. This paper introduces the Informational Buildup Framework (IBF), an alternative substrate for continual learning, based on the premise that information is the achievement of structural alignment rather than stored content. In IBF, two equations govern the dynamics: a Law of Motion that drives configuration toward higher coherence, and Modification Dynamics that persistently deform the coherence landscape in response to localized discrepancies. Memory, agency, and self-correction arise from these dynamics rather than being added as separate modules. We first demonstrate the full lifecycle in a transparent two-dimensional toy model, then validate across three domains: a controlled non-stationary world, chess evaluated independently by Stockfish, and Split-CIFAR-100 with a frozen ViT encoder. Across all three, IBF achieves replay-superior retention without storing raw data. We observe near-zero forgetting on CIFAR-100 (BT = -0.004), positive backward transfer in chess (+38.5 cp), and 43% less forgetting than replay in the controlled domain. In chess, the framework achieves a mean behavioral advantage of +88.9 +/- 2.8 cp under independent evaluation, exceeding MLP and replay baselines.

Radu Negulescu

The Informational Buildup Foundation

Continual Learning Catastrophic Forgetting Online Continual Learning

Ask PDF

Abstract

Computational Cognitive Modeling

Explore similar work

Continuous Memory Machines

Persistent Memory Through Triple-Loop Consolidation Under Stochastic Unit Turnover

Information as Structural Alignment: A Dynamical Theory of Continual Learning