cs.NEMay 5, 2026

Symmetry, Defects, and Diffusion in Continuous-memory Recurrent Networks

Authors: Hanson Hanxuan Mo

Abstract

Continuous-memory recurrent networks must preserve phase, position, or orientation despite model imperfections and state noise. We develop a geometric framework that separates three questions: how many memory coordinates are neutrally transported, how deterministic perturbations alter their finite-horizon stability, and how ambient noise is decoded along them. Exact per-input equivariance transports analytical group tangents pathwise and, on a compact nondegenerate orbit stratum, yields at least q=dim(G/H)q=\dim(G/H) zero group-tangent Lyapunov exponents under stationary ergodic driving. For imperfect dynamics, a four-block tangent/normal decomposition gives local and finite-horizon bounds on tangent growth and subspace rotation, distinguishing first-order direct damage from second-order leakage through contracting normal directions. For noisy dynamics, a specified decoder maps ambient covariance QQ to coordinate covariance ZQZZQZ^\top; under isotropic noise and fixed tangent energy, least-squares decoding and scaled-isometric action geometry minimize local diffusion. Local and finite-horizon evaluations include cases both within and outside the sufficient conditions. In a fresh twenty-seed T2T^2 replication, a decoder-covariance objective improves noisy horizon-256 memory in every pair while meeting a prespecified clean-error equivalence margin. Direct noise training also improves noisy memory but incurs a clean-error tradeoff. In coupled T4/T8T^4/T^8 integrators, the advantage of anisotropic-covariance over isotropic regularization reverses when evaluation noise becomes isotropic. These results connect continuous symmetry to measurable limits and design choices for recurrent memory under explicit dynamical, decoder, and noise assumptions.

Explore similar work

Jun 16, 2026cs.LG

Task-Restricted Symmetries in Recurrent Weight Space

Recurrent networks can contain substantial functional redundancy in weight space: changing a recurrent matrix may leave the input-output rollout nearly unchanged on a task distribution, while similar-scale changes can destroy the same behavior. We study this redundancy in one-layer tanh RNNs using ordered real Schur coordinates. The Schur form separates spectral blocks from directed nonnormal couplings, giving a diagnostic basis for structured ablations that keep the input and readout maps fixed. In a fixed-length copy task, selected nonnormal Schur couplings can be removed with little loss in some trained solutions, whereas other couplings are necessary for accurate autonomous replay. Across flip-flop, sine generation, and context-dependent integration, the loss-preserving ablation profile varies across tasks and trained solutions. These results identify candidate approximate functional invariances, not universal symmetries of recurrent weight space. Schur-coordinate ablations provide a practical diagnostic for which structured perturbations preserve a trained recurrent solution and which ones disrupt its computation.
Simon Dräger
Jun 2, 2026cs.LG

Exact equivariance, kept through training, buys zero-shot generalisation across the symmetry group

A latent world model built from an equivariant encoder and predictor inherits a provable symmetry of its training loss: when the dynamics carries a group GG acting on latents by an orthogonal representation ρ(g)ρ(g), the one-step prediction relMSE is exactly invariant across the whole group, so fitting a restricted slice of orientations mathematically determines it on the entire orbit. The symmetry survives a real Muon/AdamW+EMA+VICReg run -- composed residual 106\sim 10^{-6} after training, under any optimiser (intrinsic Vector-Neuron/e3nn parametrisation) -- and one-step error is flat across the group (5-seed medians: equivariant ×1.00\times 1.00 vs a higher-capacity non-equivariant baseline ×12.7\times 12.7 in 2D, ×17.2\times 17.2 in 3D), while that baseline fits the slice but breaks out-of-distribution. The flatness is not a synthetic artefact: on real-robot DROID end-effector trajectories the equivariant model stays flat across the orbit (×1.000\times 1.000, rotation residual 1.5×10161.5\times 10^{-16}) while a 4.5×4.5\times-larger baseline degrades ×11\times 11. One caution is load-bearing: flatness is necessary, not sufficient -- the theorem transports the in-distribution error level unchanged but does not lower it (3D relMSE 0.43\approx 0.43): across-group error is constant, not low. The same isometry lifts to a closed-loop corollary: under a matching equivariant planner the control error is invariant across the group -- float-floor-exact in 2D/SO(2), statistically flat in 3D/SE(3). Stress-tested against Sutton's Bitter Lesson (augmentation, scale, soft-equivariance), each closes at most the across-group task metric, never the float-floor exactness. This is the generalisation-side foundation of a certified-world-models programme (arXiv:2606.13092, 2606.24945, 2606.24946): flatness transports competence, and the trust bounds built on it are downstream products.
Hongbo Wang
Jun 11, 2026cs.LG

Certified World Models: Predictability Across Configuration, Horizon, and Resolution

Scale buys interpolation; structure buys certifiable transfer. A world model's average error does not say whether a particular rollout can be trusted, or for how long. For equivariant latent world models we give a predictability certificate: a computable region spanning configuration, horizon, and resolution. Under exact equivariance, rollout error is invariant over the monoid generated by k primitive symmetries and is certified from the k generators (Theorem A); universal orbit-flatness over equivariant targets characterizes equivariance at the function level (Lemma 2), so an unconstrained architecture cannot certify the property by construction. Approximate orbit-transfer defects propagate by the finite-time Lyapunov spectrum (Theorem B): expanding channels give a logarithmic horizon Tj(ε)log(1/ε)/λjT_j(ε)\sim\log(1/ε)/λ_j, neutral channels accumulate recurrent defect linearly, and contracting channels accumulate a bounded nonzero floor. Exact conserved charge values are certified to all horizons only at zero defect; with one-step defect ηη, charge-value error grows at most as Tη. Empirically, on a 40-dimensional learned model a ZN\mathbb{Z}_N-equivariant network recovers the full Lyapunov spectrum (R2=0.98R^2=0.98-0.990.99) where dense and recurrent baselines fail. A cone/adapted-metric certificate reads an a-priori horizon off the model's own Jacobian, tight on uniformly hyperbolic dynamics and self-abstaining elsewhere; the resulting horizon improves a budgeted re-observation decision. For public non-equivariant world models the tangent spectrum gives a training-free candidate horizon, paired with a held-out divergence cross-check that abstains or corrects when the learned loop over-promises.
Hongbo Wang