math.OCNov 14, 2026

Mean-field Token Steering

Authors: Md Zulfiqur Haider, Asmaa Eldesoukey, Abhishek Halder, Yongxin Chen

Organizations: Iowa State University · Georgia Institute of Technology

Abstract

We consider stochastic token steering through a deep stack of transformer layers as a mean-field control problem for an interacting particle system on the unit sphere, where interactions arise from the attention mechanism. In the mean-field limit, the token population evolves via a controlled integro-partial differential equation where the feedback control acts as a corrective vector field guiding the population toward an aggregate configuration of tokens in the representation space. We deduce the minimum effort control to accomplish this task over a fixed time horizon interpreted as normalized layer depth. We show that the resulting controlled dynamics for the token population can be understood as a mean-field Schrödinger bridge adapted to the sphere geometry. Building on this, we propose a novel generalization of the dynamic Sinkhorn recursion to compute the consistent token evolution. Numerical experiments confirm that the proposed approach achieves exact steering of the token population.

Figures & tables

Explore similar work

CardsList
  1. A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models

    Jul 6, 2026Nima Eshraghi, Lovedeep Gondara, Yuqing Huang +5Language Model SteeringSparse Autoencoders

  2. Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime

    May 11, 2026Albert Alcalde, Leon Bungert, Konstantin Riedl +1Transformer InferenceTransformer

  3. The Role of Feed-Forward Layers in Transformer Dynamics

    Sep 28, 2026Thomas Jacob Maranzatto, Semih Akkoc, Sennur UlukusTransformerTransformer FFNs