Competition for attention predicts good-to-bad tipping in AI

Neil F. Johnson; Frank Y. Huo

cs.AIFeb 16, 2026

Competition for attention predicts good-to-bad tipping in AI

Authors: Neil F. Johnson, Frank Y. Huo

Organizations: Physics Department and Dynamic Online Networks Laboratory, George Washington University, Washington D.C. 20052, U.S.A.

Abstract

More than half the global population now carries devices that can run ChatGPT-like language models with no Internet connection and minimal safety oversight -- and hence the potential to promote self-harm, financial losses and extremism among other dangers. Existing safety tools either require cloud connectivity or discover failures only after harm has occurred. Here we show that a large class of potentially dangerous tipping originates at the atomistic scale in such edge AI due to competition for the machinery's attention. This yields a mathematical formula for the dynamical tipping point n*, governed by dot-product competition for attention between the conversation's context and competing output basins, that reveals new control levers. Validated against multiple AI models, the mechanism can be instantiated for different definitions of 'good' and 'bad' and hence in principle applies across domains (e.g. health, law, finance, defense), changing legal landscapes (e.g. EU, UK, US and state level), languages, and cultural settings.

Figures & tables

Figure 1: Good-to-bad tipping during conversation with AI. (a) Schematic of on-device deployment where model runs locally with no Internet connection and no safety oversight. (b) Examples of 1-step conversations across topics with an LLM representative of models now running on modest phones and edge devices. Tipping from good ( $\mathbf{B}$ ) to bad ( $\mathbf{D}$ ) content can be immediate or follow a run of good output. SI Sec. 14 shows many more examples.

Figure 2: Tipping during multi-step conversations can be controlled by conversation so far . Introduction of additional content (e.g. $\mathbf{C}$ ) – and the order in which it is introduced – can steer AI’s response to a given question between good ( $\mathbf{B}$ ) and bad ( $\mathbf{D}$ ). The mechanism is the same competition for attention between ${\bf B}$ and ${\bf D}$ as in Fig. 1(b), now operating across a two-way conversation that evolves over time. See Fig. 3(c)(d) for theoretical demonstrations of how future output can be steered.

Figure 3: Practical implementation . (a) A lite controller. (b) Schematic of competing basins for outputs ${\bf B}$ and ${\bf D}$ given input ${\bf A}$ . Using basin centroids ${\bf A}=(0.4,-0.3)$ , ${\bf B}=(0.8,0)$ , ${\bf D}=(0.9,0.5)$ , panels (c) and (d) show predicted output for a conversation so far ${\bf ACCA}$ in which content ${\bf C}$ has been introduced. (c) Aligning ${\bf C}$ toward ${\bf D}$ decreases the tipping point $n^{*}$ , i.e. it shortens the time before the output tips to ${\bf D}$ . Example ${\bf C}=(0.2,0.2)$ is shown; predicted tipping point $n^{*}=1$ as shown. (d) Aligning ${\bf C}$ away from ${\bf D}$ increases $n^{*}$ , as predicted by Eq. 2, and hence delays the output tipping to ${\bf D}$ . Example ${\bf C}=(-0.2,-0.2)$ is shown. Outputs also shown for examples of other inputs $(\dots)$ .

Figure 4: Quantitative validation of predicted vs. observed tipping points $n^{*}$ across 18 non-control model $\times$ prompt combinations . The two separate scales of observation, which also serve to test the two approximations used to derive Eq. 2, overlap within Clopper–Pearson 95% confidence intervals at every bin. Red diamond at “no ${\bf D}$ ”: 2/18 cases where token-level geometry classifies output as ${\bf D}$ -like but all sentences are semantically ${\bf B}$ , both involving negation — identifying the boundary of the geometric approximation. Statistical tests are reported in the main text.

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 438 papers

Jul 28, 2026

Many-body Tipping Dynamics of ChatGPT-like AIs

Why do ChatGPT-like AIs, despite major architectural and training differences, unexpectedly tip to undesirable content (e.g. harmful, misleading, repetitive) even under deterministic greedy decoding? We show that a broad class of such tippings is caused by the many-body interactions between tokens (spins) as they cross the finite-layer system. Tipping emerges as a dynamical first passage process between competing output basins. Attention disorder controls the transport toward, away from, or along the basins' boundary. A few-basin reduction yields a closed finite-layer threshold, whose coarse-grained predictions show good agreement across ChatGPT-like families. These results suggest that a broad class of AI failures represents 'foreseeable engineering risk' rather than inherently unpredictable behavior, with important implications for legal and societal assessments of AI harm.

Frank Yingjie Huo, Neil F. Johnson

Department of Physics, The George Washington University, Washington, DC, USA

LLM Reliability LLM Safety

Ask PDF

Sep 21, 2026

Indirect tipping: a social attack surface in AI agent populations

As generative AI agents are deployed at scale, safety will depend not only on technical safeguards and individual model design, but also on collective equilibria that determine how agent populations process information, prioritize actions, and respond to uncertainty. Yet the same equilibria that enable agents to coordinate also create a social attack surface. The standard framework to assess this vulnerability is critical mass dynamics: the minimum fraction of adversarial agents required to overturn an equilibrium through direct competition. Here, we show that this approach risks underestimating system vulnerability by reducing the problem to the identification of singular tipping points, and ignoring indirect but potentially more efficient routes through which collective behavior can be redirected. Through experiments with populations of LLM agents and an analytic framework that captures their collective dynamics at scale, we map critical-mass thresholds that define a directed, weighted topology over the space of coordination equilibria, and treat this topology as a navigable landscape. We show that indirect tipping through intermediate stepping-stone equilibria can reduce the committed minority required to reach an alternative state, bypass majority requirements, and make possible transitions inaccessible through direct challenges. The diversity of available alternatives and timing of the attack further reshape this landscape, creating opportunities for control as well as risks of unintended destabilization. These results show that an equilibrium's resistance to committed intervention is not an intrinsic property but a structural feature of its competitive relations with alternative states. Securing populations of interacting AI agents therefore requires mapping this social landscape alongside individual agent capabilities and the technical channels through which they interact.

Ariel Flint, Luca Maria Aiello, Sara M. Constantino +2

Department of Mathematics, City St George’s, University of London, UK. · IT University of Copenhagen, Denmark. · Pioneer Centre for AI, Copenhagen, Denmark. +2

Multi-Agent Coordination AI Agent Security Multi-Agent Systems LLM Agent Safety Emergent Behavior in Multi-Agent Systems

Ask PDF

May 14, 2026

Fusion-fission forecasts when AI will shift to undesirable behavior

The key problem facing ChatGPT-like AI's use across society is that its behavior can shift, unnoticed, from desirable to undesirable -- encouraging self-harm, extremist acts, financial losses, or costly medical and military mistakes -- and no one can yet predict when. Shifts persist in even the newest AI models despite remarkable progress in AI modeling, post-training alignment and safeguards. Here we show that a vector generalization of fusion-fission group dynamics observed in living and active-matter systems drives -- and can forecast -- future shifts in the AI's behavior. The shift condition, which is also derivable mathematically, results from group-level competition between the conversation-so-far (C) and the desirable (B) and undesirable (D) basin dynamics which can be estimated in advance for a given application. It is neither model-specific nor driven by stochastic sampling. We validate it across six independent tests, including: 90 percent correct across seven AI models spanning two orders of magnitude in parameter count (124M-12B); production-scale persistence across ten frontier chatbots; and a priori time-stamped prediction eleven months before the Stanford 'Delusional Spirals' corpus appeared, and independently confirmed by that corpus of 207,443 human-AI exchanges. Because it sits architecturally below the current safety stack, the same formula provides a real-time warning signal that current alignment does not supply, portable across current and future ChatGPT-like AI architectures and instantiable in application domains where competing response classes can be defined.

Neil F. Johnson, Frank Yingjie Huo

Physics Department, The George Washington University, Washington DC 20052, USA

AI Safety Evaluation LLM Safety

Ask PDF

Abstract

LLM Safety

Explore similar work

Many-body Tipping Dynamics of ChatGPT-like AIs

Indirect tipping: a social attack surface in AI agent populations

Fusion-fission forecasts when AI will shift to undesirable behavior