The Role of Feed-Forward Layers in Transformer Dynamics

Thomas Jacob Maranzatto; Semih Akkoc; Sennur Ulukus

cs.LGSep 28, 2026

The Role of Feed-Forward Layers in Transformer Dynamics

Authors: Thomas Jacob Maranzatto, Semih Akkoc, Sennur Ulukus

Organizations: Department of Electrical and Computer Engineering University of Maryland College Park, MD 20742, USA

Abstract

We study the dynamical behavior of tokens in transformers from a control-theoretic perspective. Our model includes the feed-forward layer present after the self-attention mechanism, with the self-attention mechanism interpreted as an interacting particle system and the feed-forward layer as an independent control. Our main theoretical result establishes that the feed-forward network can steer the tokens arbitrarily close to consensus regardless of the key, query, and value matrices. Our result are easily extended to convergence to many clusters and to multi-head attention. We conduct numerical experiments to verify our results, and compare thresholding behavior from our theory to real-world LLMs.

Figures & tables

Figure 1: Setup 1 with $A=I$ and $V=J$ under softmax-normalized attention. Left: mean pairwise similarity $R_{1}$ as a function of $k/k^{*}$ . Middle: mean distance to the target $x_{0}$ . Right: zoomed in mean distance to the target $x_{0}$ around $k/k^{*}\in[0.9,1.1]$ . The vertical line marks $k/k^{*}=1$ . Experiment parameters: $d=10$ , $n=100$ , $t_{\mathrm{final}}=30$ , $\beta=1$ , $dt=0.001$ .

Figure 2: Setup 2, random Gaussian $A,V$ . Left: mean pairwise similarity $R_{1}$ as a function of $k/k^{*}$ . Middle: mean distance to the target $x_{0}$ . Right: zoomed in mean distance to the target $x_{0}$ around $k/k^{*}\in[0.9,1.1]$ . The vertical line marks $k/k^{*}=1$ . Experiment parameters: $d=10$ , $n=100$ , $t_{\mathrm{final}}=10$ , $\beta=1$ , $dt=0.001$ .

Figure 3: nanoGPT: measured feed-forward amplitude $\hat{k}$ and softmax attention scale $k^{*}_{\ell,\mathrm{raw}}$ and accounting for LayerNorm $k^{*}_{\ell,\mathrm{norm}}$ by layer. White regions signify when the maximum FFN amplitude exceeds attention bound.

Figure 4: Pythia-410M: measured feed-forward amplitude $\hat{k}$ and softmax attention scale $k^{*}_{\ell,\mathrm{raw}}$ and accounting for LayerNorm $k^{*}_{\ell,\mathrm{norm}}$ by layer. White regions signify when the maximum FFN amplitude exceeds attention bound..

Figure 5: TinyLlama-1.1B: measured feed-forward amplitude $\hat{k}$ and softmax attention scale $k^{*}_{\ell,\mathrm{raw}}$ and accounting for RMSNorm $k^{*}_{\ell,\mathrm{norm}}$ by layer. White regions signify when the maximum FFN amplitude exceeds attention bound.

Transformer

Transformers are neural architectures that use attention to model relationships among sequence elements in language, vision, and scientific data.

See all 540 papers

Feed-Forward Steering in Transformer Residual Dynamics
Aug 3, 2026Timur Mudarisov, Mikhail Burtsev, Radu StateTransformer Interpretability Transformer FFNs
University of Luxembourg · 2London Institute of Mathematical Sciences
Asymptotic Analysis of Discrete-Time Self-Attention Dynamics in Transformers
Oct 9, 2026Claudio AltafiniSelf-Attention Attention Mechanisms
Division of Automatic Control, Dept. of Electrical Engineering, Linköping University, SE-58183, Linköping, Sweden
Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers
May 8, 2026Noboru Isobe, Daisuke Inoue, Masaaki ImaizumiTransformer Transformer Training Dynamics
RIKEN Center for Advanced Intelligence Project · Imperial College London · The University of Tokyo +1

Aug 3, 2026J/K move · Enter open · S save

Feed-Forward Steering in Transformer Residual Dynamics

Timur Mudarisov, Mikhail Burtsev, Radu State

University of Luxembourg · 2London Institute of Mathematical Sciences

Transformer Interpretability Transformer FFNs Transformer Transformer Attention

Open paper AskPDF arXiv

Attention-only dynamical theories model Transformer residual directions as particles aggregating on a sphere. We extend this framework by incorporating the feed-forward network (FFN) term as a local steering field acting on each token state. The resulting theory predicts that the tangential component of the FFN field is necessary for motion in residual-direction space, that critical residual directions correspond to nonlinear projective equilibria, and that a commutator defect determines when a finite attention--FFN block can be accurately approximated by a parallel, additive flow. Across GPT-2, Pythia, Mistral, and Llama models, the extended theory improves one-step angular prediction relative to an attention-only baseline, with the contribution of the FFN increasing from GPT-2 to Llama-3-8B. Intervention experiments show that retaining only the tangential FFN component preserves most model quality, whereas retaining only the radial component causes performance to collapse. The tangential component also preserves output diversity under aggregation pressure. As a practical application, layers with small commutator defects can be approximately parallelized with only a modest increase in loss, whereas layers with large defects degrade rapidly. These findings support the interpretation of FFN layers as directional steering fields that shape Transformer residual geometry and govern the feasibility of block-level interventions.

Abstract

Transformer

Explore similar work

Feed-Forward Steering in Transformer Residual Dynamics

Asymptotic Analysis of Discrete-Time Self-Attention Dynamics in Transformers

Training-Induced Escape from Token Clustering in a Mean-Field Formulation of Transformers