We study the dynamical behavior of tokens in transformers from a control-theoretic perspective. Our model includes the feed-forward layer present after the self-attention mechanism, with the self-attention mechanism interpreted as an interacting particle system and the feed-forward layer as an independent control. Our main theoretical result establishes that the feed-forward network can steer the tokens arbitrarily close to consensus regardless of the key, query, and value matrices. Our result are easily extended to convergence to many clusters and to multi-head attention. We conduct numerical experiments to verify our results, and compare thresholding behavior from our theory to real-world LLMs.
Figures & tables
Figure 1: Setup 1 with A=I and V=J under softmax-normalized attention. Left: mean pairwise similarity R1 as a function of k/k∗ . Middle: mean distance to the target x0 . Right: zoomed in mean distance to the target x0 around k/k∗∈[0.9,1.1] . The vertical line marks k/k∗=1 . Experiment parameters: d=10 , n=100 , tfinal=30 , β=1 , dt=0.001 .
Figure 2: Setup 2, random Gaussian A,V . Left: mean pairwise similarity R1 as a function of k/k∗ . Middle: mean distance to the target x0 . Right: zoomed in mean distance to the target x0 around k/k∗∈[0.9,1.1] . The vertical line marks k/k∗=1 . Experiment parameters: d=10 , n=100 , tfinal=10 , β=1 , dt=0.001 .
Figure 3: nanoGPT: measured feed-forward amplitude k^ and softmax attention scale kℓ,raw∗ and accounting for LayerNorm kℓ,norm∗ by layer. White regions signify when the maximum FFN amplitude exceeds attention bound.
Figure 4: Pythia-410M: measured feed-forward amplitude k^ and softmax attention scale kℓ,raw∗ and accounting for LayerNorm kℓ,norm∗ by layer. White regions signify when the maximum FFN amplitude exceeds attention bound..
Figure 5: TinyLlama-1.1B: measured feed-forward amplitude k^ and softmax attention scale kℓ,raw∗ and accounting for RMSNorm kℓ,norm∗ by layer. White regions signify when the maximum FFN amplitude exceeds attention bound.
Attention-only dynamical theories model Transformer residual directions as particles aggregating on a sphere. We extend this framework by incorporating the feed-forward network (FFN) term as a local steering field acting on each token state. The resulting theory predicts that the tangential component of the FFN field is necessary for motion in residual-direction space, that critical residual directions correspond to nonlinear projective equilibria, and that a commutator defect determines when a finite attention--FFN block can be accurately approximated by a parallel, additive flow. Across GPT-2, Pythia, Mistral, and Llama models, the extended theory improves one-step angular prediction relative to an attention-only baseline, with the contribution of the FFN increasing from GPT-2 to Llama-3-8B. Intervention experiments show that retaining only the tangential FFN component preserves most model quality, whereas retaining only the radial component causes performance to collapse. The tangential component also preserves output diversity under aggregation pressure. As a practical application, layers with small commutator defects can be approximately parallelized with only a modest increase in loss, whereas layers with large defects degrade rapidly. These findings support the interpretation of FFN layers as directional steering fields that shape Transformer residual geometry and govern the feasibility of block-level interventions.
Timur Mudarisov, Mikhail Burtsev, Radu State
University of Luxembourg · 2London Institute of Mathematical Sciences
In machine learning, a self-attention dynamics describes the attention mechanism of transformers, and is obtained treating the layer index of a transformer as time index. Even though naturally discrete-time, this dynamical system is most often studied in continuous-time. Scope of this paper is to provide a thorough investigation of the discrete-time version of the self-attention dynamics: a complete classification of its equilibria is given, and their stability properties are studied, in both the single-head and the multihead case. We show that in both cases the main attractor of the dynamics is a consensus equilibrium aligned with the principal eigenvector of the value matrix (in the single-head case, a corresponding matrix of weights in the multihead case), even though other locally asymptotically stable bipartite equilibria can be show to exist at least in the single-head situation. The behavior is confirmed by looking at the learned weight matrices of a real transformer.
Claudio Altafini
Division of Automatic Control, Dept. of Electrical Engineering, Linköping University, SE-58183, Linköping, Sweden
Transformers perform inference by iteratively transforming token representations across layers. This layerwise computation has been studied empirically, and recent mean-field theories of Transformer dynamics explain how attention can drive token distributions toward clustering. However, existing analyses treat model parameters as prescribed, leaving open how training reshapes this clustering picture. We study this question in a noisy mean-field Transformer in which only a parameter-linear FFN is trained under L2 regularization. We identify a training-induced phase of the inference dynamics across layers: after clustering in the early layers, the token distribution can leave the clustered regime near the final layers. A single gradient-descent update already produces this escape, and minimizers of the training objective satisfy an exponential turnpike bound under which clustering persists through the interior layers. Our analysis is based on an entropy-regularized interaction energy that captures the clustering bias of attention, for which we establish a double-well landscape and a local Polyak-Łojasiewicz inequality. More broadly, our results point toward a training-aware mean-field theory in which training and inference dynamics of Transformers are treated together.
Noboru Isobe, Daisuke Inoue, Masaaki Imaizumi
RIKEN Center for Advanced Intelligence Project · Imperial College London · The University of Tokyo +1